HeadLight

HeadLight: Egocentric Visual Augmentation by Wearable Wide Projector

Shunichi Kasahara
ACM SIGGRAPH 2018 Emerging Technologies

Paper (ACM Digital Library)

Abstract

Visual augmentation to the real environment has potential not only to display information but also to provide a new perception of the physical world. However, the currently available mixed reality technologies could not provide enough angle of view. Thus, we introduce “Headlight”, a wearable projector system that provides wide egocentric visual augmentation. Our system consists of a small laser projector with a fish-eye wider conversion lens, a headphone and a pose tracker. HeadLight provides projection angle with approx. 105 deg. horizontal and 55 deg. vertical from the point of view of the user. In this system, the three-dimensional virtual space that is consistent with the physical environment is rendered with a virtual camera based on tracking information of the device. By processing inverse correction of the lens distortion and projecting the rendered image from the projector, HeadLight performs consistent visual augmentation in the real world. With Headlight, we envision that physical phenomena that human could not perceive will be perceived through visual augmentation.

Augmenting the view from one’s own eyes

Visual augmentation of the real environment can do more than display information: making invisible information visible could extend human perception itself. Optical see-through glasses, the most likely technology for visual augmentation, have a field of view of up to about 35 degrees, enough to show information but not for immersive augmentation. Projection mapping can immerse people in augmentation that is aligned with the real world, but projectors are usually installed in fixed positions and cannot augment the world from the user’s point of view, and earlier wearable projectors covered only a narrow angle. We asked how a wearable system could augment a large part of the user’s view, from the user’s own point of view.

A wearable projector with a fish-eye lens

HeadLight is worn on the head and consists of a small laser projector with a fish-eye wide conversion lens, a headphone and a pose tracker. A laser projector gives a sharp image at any distance, whereas an LED projector needs its focus adjusted to the surface. On its own, the projector we used covers 42.1 degrees horizontally and 23.4 degrees vertically. With the conversion lens, the projection widens to at least 105 degrees horizontally and 55 degrees vertically, and up to 131 degrees diagonally, which covers a large part of the human field of view without covering the eyes.

The lens bends the projected image into a strong pincushion shape and shifts the principal point of the projector. We model this distortion with a fisheye model, calibrate the parameters in advance, and render every output image with the inverse distortion, so that the projection on the surface comes out straight.

A pose tracker (HTC Vive Tracker) follows the rotation and translation of the headset, and a pre-calibrated transformation gives the pose of the projector. The computer graphics scene, which includes three-dimensional data of the real environment such as a 3D model of a building or a point cloud, is rendered from this pose and then corrected for the lens. The virtual space thus stays consistent with the physical environment, and the projection augments the real world in place.

Where the projected image appears

Because HeadLight projects from the user’s own point of view and redraws the image with every head movement, where an image appears to the user no longer depends on the surface the light falls on: the same projection can be seen as a flat screen over uneven surfaces, as graphics lying on the physical surfaces, or as a virtual object in front of or behind them.

These are the three designs of egocentric projection. In the virtual screen mode, the user sees the projected image itself, even when it falls on complex surfaces at different distances. In the aligned rendering mode, HeadLight renders three-dimensional data of the environment the user is looking at, and the graphics appear to lie on the physical surfaces. In the spatial rendering mode, the image changes in step with the head movement, which creates motion parallax. From the depth this parallax conveys, the user perceives a virtual object in front of or behind the surface, whatever the physical arrangement of the actual projection plane.

Perceiving what humans cannot see

Our lives and our environment are designed around human perceptual abilities, while the real world is full of species with different sensory abilities. If technology let people take on such abilities, they could perceive the real world from a new perspective. With immersive first-person projection mapping, HeadLight can express physical phenomena that people cannot perceive: a visualization of aerodynamics could let people sense the flow of air around them, and the 3D data of the surroundings can produce an X-ray-like augmentation, as if seeing through to the other side of a wall.

The current prototype has two limitations. Because the conversion lens is mounted on an existing laser projector, the amount of light is limited, which a projector designed for HeadLight would improve. The tracking also covers only a room-sized area, a restriction that an inside-out tracking device is expected to remove.

Exhibition and credits

• Visual system (collaboration): Satoru Higa (比嘉 了)
• Sound design: Shotaro Hirata
• Hardware design: Tetsuya Konishi (小西 哲哉)

日本語

実世界視覚拡張のための装着型Mixed Reality装置。広角変換レンズ、小型レーザープロジェクタ、ヘッドフォン、ヘッドトラッキング装置等で構成される。リアルタイム映像補正により、ユーザの視点から対角線上で最大131度の広角視覚投影を実現し、目を覆うことなく、人間の大部分の視野をカバーする映像投影を可能にする。さらに、ヘッドトラッキングと３次元空間情報によるプロジェクションマッピングによって、装着者が向いている方向へ投影された映像はその物理環境に存在しているように見える。

HeadLightが可能にする装着型の広視野角映像表示により、時間・空間感覚変化や人間が本来持ち得ない物理量知覚などを可能にする事を目指している

Reference

Shunichi Kasahara. 2018. HeadLight: Egocentric Visual Augmentation by Wearable Wide Projector. In ACM SIGGRAPH 2018 Emerging Technologies (SIGGRAPH ’18). Association for Computing Machinery, New York, NY, USA, Article 10, 1–2. https://doi.org/10.1145/3214907.3214926

@inproceedings{10.1145/3214907.3214926,
  author    = {Kasahara, Shunichi},
  title     = {HeadLight: Egocentric Visual Augmentation by Wearable Wide Projector},
  booktitle = {ACM SIGGRAPH 2018 Emerging Technologies},
  series    = {SIGGRAPH '18},
  year      = {2018},
  articleno = {10},
  pages     = {1--2},
  publisher = {Association for Computing Machinery},
  address   = {New York, NY, USA},
  doi       = {10.1145/3214907.3214926}
}

Figures from the paper (Kasahara, SIGGRAPH ’18 Emerging Technologies). © 2018 Copyright held by the owner/author(s).

• Related project: Fragment Shadow