Moment-Based Methods for Real-Time Shadows and Fast Transient Imaging

Dissertation, University of Bonn, 2017
 

Abstract

We apply the theory of moments to develop computationally efficient methods for real-time rendering of shadows and reconstruction of transient images from few measurements. Given moments of an unknown probability distribution, i.e. the expectations of known, real random variables, the theory of moments strives to characterize all distributions that could have led to these moments. Earlier works in computer graphics only use the most basic results of this powerful theory.

When filtering shadows based on shadow maps, the distribution of depth values within the filter region has to be estimated. Variance shadow mapping does this using two power moments. While this linear representation admits direct filtering, it leads to a very coarse reconstruction. We generalize this approach to use an arbitrary set of general moments and benchmark thousands of possible choices. Based on the results, we propose the use of moment shadow mapping which produces high-quality antialiased shadows efficiently by storing four power moments in 64 bits per shadow map texel.

Techniques for shadow map filtering have been applied to a variety of problems. We combine these existing approaches with moment shadow mapping to render shadows of translucent occluders using alpha blending, soft shadows using summed-area tables and prefiltered single scattering using six power moments. All these techniques have a high overhead per texel of the moment shadow map but a low overhead per shaded pixel. Thus, they scale well to the increasingly high resolutions of modern displays.

Transient images help to analyze light transport in scenes. Besides two spatial dimensions, they are resolved in time of flight. Earlier cost-efficient approaches reconstruct them from measurements of amplitude modulated continuous wave lidar systems but they typically take more than a minute of capture time. We pose this reconstruction problem as trigonometric moment problem. The maximum entropy spectral estimate and the Pisarenko estimate are known closed-form solutions to such problems which yield continuous and sparse reconstructions, respectively. By applying them, we reconstruct complex impulse responses with m distinct returns from measurements at as few as m non-zero frequencies. For m=3 our experiments with measured data confirm this. Thus, our techniques are computationally efficient and simultaneously reduce capture times drastically. We successfully capture 18.6 transient images per second which leads to transient video. As an important byproduct, this fast and accurate reconstruction of impulse responses enables removal of multipath interference in range images.

Images

Bibtex

@PHDTHESIS{peters-2017-phd,
    author = {Peters, Christoph},
     title = {Moment-Based Methods for Real-Time Shadows and Fast Transient Imaging},
      type = {Dissertation},
      year = {2017},
    school = {University of Bonn},
      note = {To appear.},
  abstract = {We apply the theory of moments to develop computationally efficient methods for real-time rendering
              of shadows and reconstruction of transient images from few measurements. Given moments of an unknown
              probability distribution, i.e. the expectations of known, real random variables, the theory of
              moments strives to characterize all distributions that could have led to these moments. Earlier
              works in computer graphics only use the most basic results of this powerful theory. 
              
              When filtering shadows based on shadow maps, the distribution of depth values within the filter
              region has to be estimated. Variance shadow mapping does this using two power moments. While this
              linear representation admits direct filtering, it leads to a very coarse reconstruction. We
              generalize this approach to use an arbitrary set of general moments and benchmark thousands of
              possible choices. Based on the results, we propose the use of moment shadow mapping which produces
              high-quality antialiased shadows efficiently by storing four power moments in 64 bits per shadow map
              texel. 
              
              Techniques for shadow map filtering have been applied to a variety of problems. We combine these
              existing approaches with moment shadow mapping to render shadows of translucent occluders using
              alpha blending, soft shadows using summed-area tables and prefiltered single scattering using six
              power moments. All these techniques have a high overhead per texel of the moment shadow map but a
              low overhead per shaded pixel. Thus, they scale well to the increasingly high resolutions of modern
              displays.
              
              Transient images help to analyze light transport in scenes. Besides two spatial dimensions, they are
              resolved in time of flight. Earlier cost-efficient approaches reconstruct them from measurements of
              amplitude modulated continuous wave lidar systems but they typically take more than a minute of
              capture time. We pose this reconstruction problem as trigonometric moment problem. The maximum
              entropy spectral estimate and the Pisarenko estimate are known closed-form solutions to such
              problems which yield continuous and sparse reconstructions, respectively. By applying them, we
              reconstruct complex impulse responses with m distinct returns from measurements at as few as m
              non-zero frequencies. For m=3 our experiments with measured data confirm this. Thus, our techniques
              are computationally efficient and simultaneously reduce capture times drastically. We successfully
              capture 18.6 transient images per second which leads to transient video. As an important byproduct,
              this fast and accurate reconstruction of impulse responses enables removal of multipath interference
              in range images.}
}