Acquisition, Transmission and Rendering of Objects with Optically Complicated Material Appearance

Dissertation, Universität Bonn, 2015
 

Abstract

A major goal in computer graphics is the generation of photorealistic images. Nowadays, the degree of realism is often not restricted by the rendering algorithm but instead mainly depends on the quality of the virtual scene description. Besides manual modeling by artists, the parameters of the virtual scene's objects can also be determined from measurements of real-world exemplars. In this thesis, we will explore the acquisition and faithful representation of whole objects, including their optical material properties. The applications for realistic virtual objects are manifold. They can be used as digital props in special effects in movies and computer games, for the documentation and public dissemination of cultural heritage over the Internet or as product previews in online shops, just to name a few examples. The key in our proposed digitization method is the choice of the bidirectional texture function (BTF) to convey the digital material appearance. The BTF defines the amount of reflected light - the reflectance - in dependence on view and light directions and spatial position. This provides a high degree of realism and allows to faithfully reproduce even tiny details. In contrast to so called "model-driven" methods, which derive the reflectance values from an analytical mathematical function with a few parameters, the BTF is a "data-driven" representation. Here, the reflectance is the result of direct interpolation between densely measured values. However, to describe the BTF with sufficient detail, it is necessary to capture billions of datapoints. For this reason, we first propose setups for the fast automated acquisition of these reflectance samples as well as the objects' 3D geometry. In both cases a precise calibration is mandatory. Hence, we explain the employed calibration algorithms in detail. The final digitized object is the result of a consecutive postprocessing on the measured data. Due to practical limitations of the setups, the sampling of the reflectance data is often incomplete. Hence, we propose to employ a data-driven hole filling approach based on matrix factorization. Our evaluation on 27 different objects with variations in shape and material demonstrates that the proposed digitization approach in general results in a very faithful reproductions of the original appearance. However, we also show the limitations of our method. Even after applying state-of-the-art compression algorithms, one major disadvantage of BTFs with respect to model-based techniques is the tremendous memory requirement. We thus propose two approaches for the transmission of BTF materials over the Internet and real-time rendering on the GPU that cope with the large amounts of data. Our experiments show that by using an additional compression as well as progressive transmission, digital materials can be streamed over the Internet and display a high-quality appearance after just a few seconds. Furthermore, we manage to reduce the GPU memory footprint by up to 97% using a clever level of detail strategy. This way, the GPU's memory bottleneck is mostly avoided and the real-time rendering of virtual scenes containing several digitized objects becomes possible. The different aspects tackled in this thesis complement each other. Together the proposed techniques form an ecosystem for digital object appearance that is practically applicable in many scenarios.

Download: http://hss.ulb.uni-bonn.de/2015/4222/4222.htm

Images

Bibtex

@PHDTHESIS{schwartz-2015-phd,
    author = {Schwartz, Christopher},
     title = {Acquisition, Transmission and Rendering of Objects with Optically Complicated Material Appearance},
      type = {Dissertation},
      year = {2015},
    school = {Universit{\"a}t Bonn},
  abstract = {A major goal in computer graphics is the generation of photorealistic images. Nowadays, the degree
              of realism is often not restricted by the rendering algorithm but instead mainly depends on the
              quality of the virtual scene description. Besides manual modeling by artists, the parameters of the
              virtual scene's objects can also be determined from measurements of real-world exemplars.
              In this thesis, we will explore the acquisition and faithful representation of whole objects,
              including their optical material properties. The applications for realistic virtual objects are
              manifold. They can be used as digital props in special effects in movies and computer games, for the
              documentation and public dissemination of cultural heritage over the Internet or as product previews
              in online shops, just to name a few examples.
              The key in our proposed digitization method is the choice of the bidirectional texture function
              (BTF) to convey the digital material appearance. The BTF defines the amount of reflected light - the
              reflectance - in dependence on view and light directions and spatial position. This provides a high
              degree of realism and allows to faithfully reproduce even tiny details. In contrast to so called
              "model-driven" methods, which derive the reflectance values from an analytical mathematical function
              with a few parameters, the BTF is a "data-driven" representation. Here, the reflectance is the
              result of direct interpolation between densely measured values.
              However, to describe the BTF with sufficient detail, it is necessary to capture billions of
              datapoints. For this reason, we first propose setups for the fast automated acquisition of these
              reflectance samples as well as the objects' 3D geometry. In both cases a precise calibration is
              mandatory. Hence, we explain the employed calibration algorithms in detail.
              The final digitized object is the result of a consecutive postprocessing on the measured data. Due
              to practical limitations of the setups, the sampling of the reflectance data is often incomplete.
              Hence, we propose to employ a data-driven hole filling approach based on matrix factorization.
              Our evaluation on 27 different objects with variations in shape and material demonstrates that the
              proposed digitization approach in general results in a very faithful reproductions of the original
              appearance. However, we also show the limitations of our method.
              Even after applying state-of-the-art compression algorithms, one major disadvantage of BTFs with
              respect to model-based techniques is the tremendous memory requirement. We thus propose two
              approaches for the transmission of BTF materials over the Internet and real-time rendering on the
              GPU that cope with the large amounts of data. Our experiments show that by using an additional
              compression as well as progressive transmission, digital materials can be streamed over the Internet
              and display a high-quality appearance after just a few seconds. Furthermore, we manage to reduce the
              GPU memory footprint by up to 97% using a clever level of detail strategy. This way, the GPU's
              memory bottleneck is mostly avoided and the real-time rendering of virtual scenes containing several
              digitized objects becomes possible.
              The different aspects tackled in this thesis complement each other. Together the proposed techniques
              form an ecosystem for digital object appearance that is practically applicable in many scenarios.},
       url = {http://hss.ulb.uni-bonn.de/2015/4222/4222.htm}
}