Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport

Matthew O'Toole, Felix Heide, Lei Xiao, Matthias B. Hullin, Wolfgang Heidrich, and Kiriakos N. Kutulakos
In: ACM Trans. Graph. (Proc. SIGGRAPH) (Aug. 2014), 33:4
 

Abstract

We analyze light propagation in an unknown scene using projectors and cameras that operate at transient timescales. In this new photography regime, the projector emits a spatio-temporal 3D signal and the camera receives a transformed version of it, determined by the set of all light transport paths through the scene and the time delays they induce. The underlying 3D-to-3D transformation encodes scene geometry and global transport in great detail, but individual transport components (e.g., direct reflections, inter-reflections, caustics, etc.) are coupled nontrivially in both space and time. To overcome this complexity, we observe that transient light transport is always separable in the temporal frequency domain. This makes it possible to analyze transient transport one temporal frequency at a time by trivially adapting techniques from conventional projector-to-camera transport. We use this idea in a prototype that offers three never-seen-before abilities: (1) acquiring time-of-flight depth images that are robust to general indirect transport, such as interreflections and caustics; (2) distinguishing between direct views of objects and their mirror reflection; and (3) using a photonic mixer device to capture sharp, evolving wavefronts of "light in flight".

Images

Bibtex

@ARTICLE{otoole2014,
    author = {O'Toole, Matthew and Heide, Felix and Xiao, Lei and Hullin, Matthias B. and Heidrich, Wolfgang and
              Kutulakos, Kiriakos N.},
     title = {Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport},
   journal = {ACM Trans. Graph. (Proc. SIGGRAPH)},
    volume = {33},
    number = {4},
      year = {2014},
     month = aug,
      note = {to appear},
  abstract = {We analyze light propagation in an unknown scene using projectors and cameras that operate at
              transient timescales. In this new photography regime, the projector emits a spatio-temporal 3D
              signal and the camera receives a transformed version of it, determined by the set of all light
              transport paths through the scene and the time delays they induce. The underlying 3D-to-3D
              transformation encodes scene geometry and global transport in great detail, but individual transport
              components (e.g., direct reflections, inter-reflections, caustics, etc.) are coupled nontrivially in
              both space and time. 
              To overcome this complexity, we observe that transient light transport is always separable in the
              temporal frequency domain. This makes it possible to analyze transient transport one temporal
              frequency at a time by trivially adapting techniques from conventional projector-to-camera
              transport. We use this idea in a prototype that offers three never-seen-before abilities: (1)
              acquiring time-of-flight depth images that are robust to general indirect transport, such as
              interreflections and caustics; (2) distinguishing between direct views of objects and their mirror
              reflection; and (3) using a photonic mixer device to capture sharp, evolving wavefronts of "light in
              flight".}
}