Near-Field Illumination for Mixed Reality with Delta Radiance Fields

Tobias Alexander Franke ACM SIGGRAPH 2013 Posters

Teaser Infinite Head model inserted into a real scene with one reconstructed light source: (a) a synthetic object is inserted without illumination, (b) visible first bounce around the base as well as low resolution shadow (32 propagations, 512^2 VPLs, 11ms per frame), (d) indirect effects without synthetic object for better visualization, (d) visualization of the DLPV (red dots indicate negative values).


Apart from geometric registration, fusing synthetic objects with a real context requires believable interaction of real and virtual light. Where shadows provide the visual cues to locate a synthetic object in a real scene and transferred light from real light sources let it appear in harmony with its surroundings, the mutual indirect interaction of illumination is a necessary detail to convince an observer that the rendered result is not merely augmented but part of the scene. Such a mixed reality (MR) system has applications in movie production, advertisement of unfinished products or cultural heritage visualization. While there are a range of relighting tools available for offline renderers or static scenes (e.g. photos), interactive MR systems usually disregard proper lighting entirely or greatly simplify shading. A method often employed is to merge virtual light and shadows cast from synthetic objects with a real background with Differential Rendering, leaving out any other light interaction such as indirect light bounces. Attempts have been made to resolve this issue with Instant Radiosity [Knecht et al. 2010; Lensing and Broll 2012]. To suppress flickering, a large number of virtual point lights (VPL) is necessary, drastically taxing execution speed. We propose to model all light, virtual and real, as a unified radiance field instead to avoid performance issues from oversampling and to maintain temporal coherence.


Supplemental video


 author = {Franke, Tobias Alexander},
 title = {Near-field illumination for mixed reality with delta radiance fields},
 booktitle = {ACM SIGGRAPH 2013 Posters},
 series = {SIGGRAPH '13},
 year = {2013},
 isbn = {978-1-4503-2342-0},
 location = {Anaheim, California},
 pages = {76:1--76:1},
 articleno = {76},
 numpages = {1},
 url = {},
 doi = {10.1145/2503385.2503468},
 acmid = {2503468},
 publisher = {ACM},
 address = {New York, NY, USA},