Active Lighting - Real Illumination from Virtual Environments

  A. Ghosh, M. Trentacoste, H. Seetzen, W. Heidrich

lights white lights grace cathedral user grace cathedral

Left: Room with lights switched on. Center: Lights switched to Grace Cathedral environment. Right: User viewing the Grace Cathedral environment on an HDR display with active lighting.

Abstract

We introduce a method for actively controlling the illumination in a room so that it is consistent with a virtual world. In combination with a high dynamic range display, the system produces both uniform and directional illumination at intensity levels covering a wide range of real-world environments. It thereby allows natural adaptation processes of the human visual system to take place, for example when moving between bright and dark environments. In addition, the directional illumination provides additional information about the environment in the user’s peripheral field of view. We describe both the hardware and the software aspects of our system. We also conducted an informal survey to determine whether users prefer the dynamic illumination over constant room illumination in an entertainment setting.

Active Lighting Setup

In our prototype implementation, we focus on methods that could conceivably be used in entertainment applications, such as gaming environments or home theaters. We use computer-controlled LED lights that are distributed throughout the room. All lights are individually programmable to a 24 bit RGB color. This setup allows us not only to raise or lower the ambient light in the room, but also to create some degree of directional illumination, which results in a low-resolution dynamic room illumination approximating an environment-map for the assumed viewing position. Although the light sources are located outside of the user’s direct field of view, the directional illuminationinteracts with objects inside the field of view, such as the monitor or the wall behind it. The goal of our system is to illuminate the room so that it matches a low-frequency version of the virtual scene.
Floorplan

Room layout for lighting system

We assembled our prototype system in a separate room, approximately 15.5’ long, 9’ wide, and 9.5’ high. The lighting system consists of 24 RGB LED lights (ColorKinetics iColor Cove), each of which can be individually programmed to a 24 bit RGB color value. We used seven poles with stands to mount the 24 light sources. The lights were positioned and oriented such that they predominantly illuminated the ceiling, as well as the walls to the left, right and in front of the viewer.


undiffused
 diffused

Left: Color banding of iColor Cove. Right: Smooth pattern generated using diffuser.


To create a smoothly varying illumination pattern we used strong diffusers at the light sources, which also reduce color separation of the RGB elements. The diffuser for each light consist of 2” diameter transparent acrylic tubing that was cut in half along its axis, and spray-painted lightly on the outside with white plastic paint (Krylon Fusion for plastic). To avoid internal reflection losses we used reflective film to coat the inside of the light source.
light modifications

Opened iColor Cove light source with reflective foil and plastic diffuser.


Calibration

Some calibration steps are necessary in order to control the system in a way consistent with the virtual world and the image shown on the display. Geometric calibration is necessary to determine the positions of the light sources relative to the viewer, and their spread, which is modeled as a Gaussian. Photometric calibration is subsequently performed to match white points and illumination levels between the light sources and the display.
basis images


Light probe images acquired for each of the 24 light sources at the intended viewing position.


To obtain geometric calibration information, we place a reflective ball at the intended viewer position to act as a light probe. We take photographs with a web camera (Creative NX Ultra) while switching on one light at a time. We then model the impact of every light source by fitting a Gaussian to the environment map. It is centered around the direction corresponding to the brightest point in the environment map, and its standard deviation is chosen such as to minimize the RMS error.


Experiments

To test the concept, we conducted an user survey with a set of three experiments. As the evaluation criterion, we chose user preference rather than other possible criteria such as perceived realism. This choice was made due to our primary interest in entertainment applications for the current system.

Uniform Illumination

Tunnel bright Tunnel dark

Left: Uniform lighting corresponding to daylight. Right: Uniform lighting corresponding to tunnel lighting.

We tested the preference of participants for directionally uniform illumination level over constant room illumination with an HDR driving video. All participants preferred or strongly preferred dynamic illumination over a constant brightness level, particulalry compared to a dark room. This stronger preference can be explained by the ability of the HDR display to produce light levels that are starting to be uncomfortable in very dark environments.


Directional Illumination

Grace user 1 Grace User 2

Left: Directional lighting corresponding to bright windows. Right: Directional lighting corresponding to the alter.

We then tested the preference of participants for directionally localized illumination changes  over uniform brightness changes with a rotation interface to explore HDR panoramas. All participants preferred or strongly preferred the directional illumination over the uniform one. With one exception, all participants also felt an improved sense of orientation in the presence of directional lighting.

Low Dynamic Range Footage

NFS user1 NFS user2

NFS driving footage. Left: Lighting corresponding to passing streetlamp. Right: Lighting corresponding to tunnel lighting.

To test the usefulness of the lighting system in combination with low-dynamic range displays, the participants were shown a footage from “Need for Speed Underground 2” on a conventional display. In this experiment, the preference for directional dynamic illumination was very strong. One participant found the dynamic lighting distracting in the presence of a conventional display, but not in the presence of an HDR display. We believe that this ambivalence is in some sense caused by the lighting system overpowering the conventional display, which cannot produce the same intensities as the HDR display.



Conclusions

Our user survey shows overwhelming support for this concept in combination with an HDR display: all of our participants preferred the lighting system over constant room illumination. We believe that this combination of HDR display and lighting system comprises the best setup, since it makes it possible to create similar brightnesses both on the display and in the surrounding room. Even in combination with a conventional low-dynamic range display, the participants were predominantly positive about the lighting system.

We believe that the work presented here also creates a variety of promising directions for future research. One important area is artistic tools for content creation, in particular for augmenting existing film material with information about directional illumination. At the moment, we focus on entertainment-style applications, where user preference is arguably all that matters. An interesting topic for future work is to analyze whether the system can also be helpful in task-oriented applications, for example ones that require navigation in space.

Video

  Active lighting video

                                                                                                                                                  


Publications


Back to the projects page ...