Tobias Alexander Franke

The Dirtchamber

My private playground for mixed-reality and rendering tests is the Dirtchamber. It is composed out of several binaries and my D3D11 tool library Dune.

The main development target is a three-component system for real-time relighting consisting of a regular view-camera combined with a depth sensor and an additional hemispherical camera. A marker is tracked to consistently place a virtual object into a real scene. The relighting algorithm uses a deferred render engine and derives its light source information from the hemispherical camera, which is placed right next to the marker. Because depth information is available for the entire image (the renderer has a special GBuffer with the color and depth images from the view-camera), all postprocessing shaders can be applied naturally to the result and combined with the buffer that contains the virtual object.

Main features:

Project page

Voxel Cone Tracing

This demo is a partial implementation of Interactive Indirect Illumination Using Voxel Cone Tracing (Paper) to simulate first bounce indirect light off arbitrary rough surfaces. Voxel Cone Tracing works by injecting VPLs generated from RSMs into large volumetric structures which are then prefiltered (i.e. mip-mapped). Instead of casting many rays to integrate the cone contribution on a surface point, lower resolution mip-map levels of the volume are queried to gather the pre-integrated first bounce.

The project is built with my D3D11 toolchain Dune and is configured with CMake. Additionally, it depends on the Open Asset Import Library to load geometry and a header-only version of Boost. The code comes with documentation which you can view online.

The code is released under the MIT License.

Download source

dirtchamber GitHub page

Light Propagation Volumes

This demo is an implementation of Kaplanyan and Dachsbacher’s Cascaded light propagation volumes for real-time indirect illumination (Paper, Techreport) to simulate first-bounce indirect diffuse illumination. LPVs are small volumetric textures which are injected with VPLs generated from an RSM and then scatter the light from each cell in the volume to their neighbors. After several iterations, the volume contains the diffuse indirect light distribution of the scene covered by it.

The project is built with my D3D11 toolchain Dune and is configured with CMake. Additionally, it depends on the Open Asset Import Library to load geometry and a header-only version of Boost. The code comes with documentation which you can view online.

The code is released under the MIT License.

Download source

dirtchamber GitHub page

Image Diff

GGX importance sampling McGuire environmap sampling 8x squared difference image

This is a little tool to generate difference images of two inputs. The difference of the luminance of two pixels is squared and amplified by a user-supplied factor and then output into a heatmap. The heatmap contains both negative and positive values. I have used this script to generate analysis images for papers.

The above three images show specular reflection from filtered importance sampling with GGX, filtered sampling from Blinn-Phong and the 8x amplified difference between both. The GGX serves as groundtruth in this case, therefore blue-white pixels in the difference image show regions where the Blinn-Phong reflection outputs more energy than the GGX version, while red-yellow regions output less.

Download source

diff-tool GitHub page

Environment Map Sampling

This is an implementation of Paul Debevec’s A Median Cut Algorithm for Light Probe Sampling. The code generates sample positions for 2^n light sources. You will still need to sum up the energy of the respective cut region to get the final flux. To run the sample, please fetch Sean Barrett’s stb_image.

The code is released under the MIT License.

Download source

mediancut GitHub page

PTM Converter

Polynomial Texture Maps are special texture files which contain biquadratic polynomial coefficients per pixel, modelling the reflectance behavior under varying incident radiance. Even though the format is in heavy use in archeology, I found that the available open-source implementations are rather sparse. The following code is a one-file C++ converter which reads (at the moment) LRGB encoded PTMs (either uncompressed or with JPEG compression) and dumps the luminance coefficients and the RGB data into separate PNG images. These images can be used without the need to write a PTM reader for instance in web applications using WebGL. The necessary scale and bias parameters are printed on the console.

A PTM split into RGB, low order and high order coefficients

If you’re looking for a simple reader converting all data into regular char arrays, look no further! The code should be simple enough to parse, just have a look at ptm_dump on how to get to the data. A big Thank You! to Sean Barrett for stb_image, which I use to read JPEG compressed data and write PNG files to the disk.

The code is released under the MIT License.

Download source

ptmconvert GitHub page

Project page

Real-time Mixed Reality Relighting

Augmented and mixed reality try to merge real and virtual worlds, for instance by putting a virtual object into a real image. To do this, one essentially needs two parts: a tracking method to figure out where the object is in real space, and some rendering methods that can transfer real light onto a virtual object and vice versa. I deal with the second problem.

I have implemented different systems to see which options work well for a real-time augmented reality simulation. The first implementation uses precomputed ambient occlusion for rigid models combined with irradiance mapping from SH probes. Glossy reflections are simulated by interpolating between the irradiance map and a regular reflection map. Shadows rendered from VSMs are added with Differential Rendering.

The following video shows the steps from a simple model masked out and projected on top of an image, adding one effect after the other.

In my second test, I used Precomputed Radiance Transfer to precalculate the light transport response on the surface of the object, essentially replacing static ambient occlusion with proper low frequency occlusion and integration of light and materials.

The setup consists of one tracking camera and a 180 degree lens camera for the lighting. Please note that the irradiance map is rotated, so the light shining from the back actually appears on the front side. This was done to prevent my arm to block off the image when lighting the front side.

X3D WowModelViewer Exporter (discontinued)

I was involved with the development group of the WoWModelViewer, where I added an X3D exporting option for pure X3D files that can be played with an X3D-player such as InstantReality. X3D is a successor of VRML and is used for web applications.

The exporter is able to write embedded X3D for XHTML webpages. These in turn are interpreted by a Javacript library called X3DOM, parsing through the dom tree and replacing X3D nodes with WebGL code. Rendering a WOW character then doesn’t require any plugins whatsoever.

The current WowModelViewer doesn’t feature the old exporters as far as I can see, but you can still pull the code from tag 0.7.1 beta2 and get involved to write a plugin for the new code base.

WowModelViewer Bitbucket page

PDF Presenter (discontinued)

PDFPresenter was a tool I started after I switched from a Mac to a Laptop without Keynote and didn’t have access to a presenter screen for PDF based presentations I created with Beamer. There were a couple of other tools similar to PDFPresenter out in the wild, but most of them simply did pretty transition effects for PDFs, and I wanted to see my current slide, the next one and some notes. I no longer develop it, but you can get the last release and the source code from the repository.

The code is released under the GPL 2 License.

Download source

PDFPresenter GitHub page

Deimos (discontinued)

Deimos is my library for images, math and other stuff that was somehow useful when I needed it, and has been used in different projects. All code has been tested with gcc-4 and vc-8. Note that this package is under continuous construction and probably contains errors here and there. The following modules are available:

The code is released under the MIT License.

Download source

deimos GitHub page