Material Recognition Meets 3D Reconstruction: Novel Tools for Efficient, Automatic Acquisition Systems

Dissertation, Universität Bonn, 2016
 

Abstract

For decades, the accurate acquisition of geometry and reflectance properties has represented one of the major objectives in computer vision and computer graphics with many applications in industry, entertainment and cultural heritage. Reproducing even the finest details of surface geometry and surface reflectance has become a ubiquitous prerequisite in visual prototyping, advertisement or digital preservation of objects. However, today's acquisition methods are typically designed for only a rather small range of material types. Furthermore, there is still a lack of accurate reconstruction methods for objects with a more complex surface reflectance behavior beyond diffuse reflectance. In addition to accurate acquisition techniques, the demand for creating large quantities of digital contents also pushes the focus towards fully automatic and highly efficient solutions that allow for masses of objects to be acquired as fast as possible. This thesis is dedicated to the investigation of basic components that allow an efficient, automatic acquisition process. We argue that such an efficient, automatic acquisition can be realized when material recognition "meets" 3D reconstruction and we will demonstrate that reliably recognizing the materials of the considered object allows a more efficient geometry acquisition. Therefore, the main objectives of this thesis are given by the development of novel, robust geometry acquisition techniques for surface materials beyond diffuse surface reflectance, and the development of novel, robust techniques for material recognition. In the context of 3D geometry acquisition, we introduce an improvement of structured light systems, which are capable of robustly acquiring objects ranging from diffuse surface reflectance to even specular surface reflectance with a sufficient diffuse component. We demonstrate that the resolution of the reconstruction can be increased significantly for multi-camera, multi-projector structured light systems by using overlappings of patterns that have been projected under different projector poses. As the reconstructions obtained by applying such triangulation-based techniques still contain high-frequency noise due to inaccurately localized correspondences established for images acquired under different viewpoints, we furthermore introduce a novel geometry acquisition technique that complements the structured light system with additional photometric normals and results in significantly more accurate reconstructions. In addition, we also present a novel method to acquire the 3D shape of mirroring objects with complex surface geometry. The aforementioned investigations on 3D reconstruction are accompanied by the development of novel tools for reliable material recognition which can be used in an initial step to recognize the present surface materials and, hence, to efficiently select the subsequently applied appropriate acquisition techniques based on these classified materials. In the scope of this thesis, we therefore focus on material recognition for scenarios with controlled illumination as given in lab environments as well as scenarios with natural illumination that are given in photographs of typical daily life scenes. Finally, based on the techniques developed in this thesis, we provide novel concepts towards efficient, automatic acquisition systems.

Download: http://hss.ulb.uni-bonn.de/2017/4692/4692.htm

Bilder

Bibtex

@PHDTHESIS{weinmann-2016,
    author = {Weinmann, Michael},
     title = {Material Recognition Meets 3D Reconstruction: Novel Tools for Efficient, Automatic Acquisition
              Systems},
      type = {Dissertation},
      year = {2016},
    school = {Universit{\"a}t Bonn},
  abstract = {For decades, the accurate acquisition of geometry and reflectance properties has represented one of
              the major objectives in computer vision and computer graphics with many applications in industry,
              entertainment and cultural heritage. Reproducing even the finest details of surface geometry and
              surface reflectance has become a ubiquitous prerequisite in visual prototyping, advertisement or
              digital preservation of objects. However, today's acquisition methods are typically designed for
              only a rather small range of material types. Furthermore, there is still a lack of accurate
              reconstruction methods for objects with a more complex surface reflectance behavior beyond diffuse
              reflectance. In addition to accurate acquisition techniques, the demand for creating large
              quantities of digital contents also pushes the focus towards fully automatic and highly efficient
              solutions that allow for masses of objects to be acquired as fast as possible. This thesis is
              dedicated to the investigation of basic components that allow an efficient, automatic acquisition
              process. We argue that such an efficient, automatic acquisition can be realized when material
              recognition "meets" 3D reconstruction and we will demonstrate that reliably recognizing the
              materials of the considered object allows a more efficient geometry acquisition. Therefore, the main
              objectives of this thesis are given by the development of novel, robust geometry acquisition
              techniques for surface materials beyond diffuse surface reflectance, and the development of novel,
              robust techniques for material recognition. In the context of 3D geometry acquisition, we introduce
              an improvement of structured light systems, which are capable of robustly acquiring objects ranging
              from diffuse surface reflectance to even specular surface reflectance with a sufficient diffuse
              component. We demonstrate that the resolution of the reconstruction can be increased significantly
              for multi-camera, multi-projector structured light systems by using overlappings of patterns that
              have been projected under different projector poses. As the reconstructions obtained by applying
              such triangulation-based techniques still contain high-frequency noise due to inaccurately localized
              correspondences established for images acquired under different viewpoints, we furthermore introduce
              a novel geometry acquisition technique that complements the structured light system with additional
              photometric normals and results in significantly more accurate reconstructions. In addition, we also
              present a novel method to acquire the 3D shape of mirroring objects with complex surface geometry.
              The aforementioned investigations on 3D reconstruction are accompanied by the development of novel
              tools for reliable material recognition which can be used in an initial step to recognize the
              present surface materials and, hence, to efficiently select the subsequently applied appropriate
              acquisition techniques based on these classified materials. In the scope of this thesis, we
              therefore focus on material recognition for scenarios with controlled illumination as given in lab
              environments as well as scenarios with natural illumination that are given in photographs of typical
              daily life scenes. Finally, based on the techniques developed in this thesis, we provide novel
              concepts towards efficient, automatic acquisition systems.},
       url = {http://hss.ulb.uni-bonn.de/2017/4692/4692.htm}
}