Albedo estimation for real-time 3D reconstruction using RGB-D and IR data

In: ISPRS Journal of Photogrammetry and Remote Sensing (2019), 150(213-225)
 

Abstract

Re­con­struct­ing scenes in real-time us­ing low-cost sen­sors has gained in­creas­ing at­ten­tion in re­cent re­search and en­abled nu­mer­ous ap­pli­ca­tions in graph­ics, vi­sion, and ro­bot­ics. While cur­rent tech­niques of­fer a subs­tantial im­prove­ment re­gard­ing the qual­ity of the recon­structed geom­e­try, the de­gree of re­al­ism of the over­all appear­ance is still lack­ing as the re­con­struc­tion of ac­cu­rate sur­face ap­pear­ance is highly chal­leng­ing due to the com­plex in­ter­play of sur­face geom­e­try, re­flectance prop­er­ties and sur­round­ing il­lu­mi­na­tion. We pre­sent a novel ap­proach that al­lows the re­con­struc­tion of both the geom­e­try and the spa­tially vary­ing sur­face albedo of a scene from RGB-D and IR data ob­tained via com­mod­ity sen­sors. In com­par­i­son to pre­vi­ous ap­proaches, our ap­proach of­fers an im­proved ro­bust­ness and a sig­nif­i­cant speed-up to even ful­fill the real-time re­quire­ments. For this pur­pose, we ex­ploit the ben­e­fits of scene seg­men­ta­tion to im­prove albedo es­ti­ma­tion due to the re­sult­ing bet­ter seg­ment-wise cou­pling of IR and RGB data that takes into ac­count the wavelength char­ac­ter­is­tics of dif­fer­ent ma­te­ri­als within the scene. The es­ti­mated albedo is di­rectly in­te­grated into the dense vol­u­met­ric re­con­struc­tion frame­work us­ing a novel weight­ing scheme to gen­er­ate high-qual­ity re­sults. In our eval­u­a­tion, we demon­strate that our ap­proach al­lows albedo cap­tur­ing of com­pli­cated sce­nar­ios in­clud­ing com­plex, high-fre­quent and strongly vary­ing light­ing as well as shad­ows.

Bilder

Bibtex

@ARTICLE{stotko2019albedo,
    author = {Stotko, Patrick and Weinmann, Michael and Klein, Reinhard},
     pages = {213--225},
     title = {Albedo estimation for real-time 3D reconstruction using RGB-D and IR data},
   journal = {ISPRS Journal of Photogrammetry and Remote Sensing},
    volume = {150},
      year = {2019},
  abstract = {Re­con­struct­ing scenes in real-time us­ing low-cost sen­sors has gained in­creas­ing
              at­ten­tion in re­cent re­search and en­abled nu­mer­ous ap­pli­ca­tions in graph­ics,
              vi­sion, and ro­bot­ics. While cur­rent tech­niques of­fer a subs­tantial im­prove­ment
              re­gard­ing the qual­ity of the recon­structed geom­e­try, the de­gree of re­al­ism of the
              over­all appear­ance is still lack­ing as the re­con­struc­tion of ac­cu­rate sur­face
              ap­pear­ance is highly chal­leng­ing due to the com­plex in­ter­play of sur­face
              geom­e­try, re­flectance prop­er­ties and sur­round­ing il­lu­mi­na­tion. We pre­sent a
              novel ap­proach that al­lows the re­con­struc­tion of both the geom­e­try and the spa­tially
              vary­ing sur­face albedo of a scene from RGB-D and IR data ob­tained via com­mod­ity sen­sors.
              In com­par­i­son to pre­vi­ous ap­proaches, our ap­proach of­fers an im­proved
              ro­bust­ness and a sig­nif­i­cant speed-up to even ful­fill the real-time re­quire­ments.
              For this pur­pose, we ex­ploit the ben­e­fits of scene seg­men­ta­tion to im­prove albedo
              es­ti­ma­tion due to the re­sult­ing bet­ter seg­ment-wise cou­pling of IR and RGB data that
              takes into ac­count the wavelength char­ac­ter­is­tics of dif­fer­ent ma­te­ri­als within
              the scene. The es­ti­mated albedo is di­rectly in­te­grated into the dense vol­u­met­ric
              re­con­struc­tion frame­work us­ing a novel weight­ing scheme to gen­er­ate high-qual­ity
              re­sults. In our eval­u­a­tion, we demon­strate that our ap­proach al­lows albedo
              cap­tur­ing of com­pli­cated sce­nar­ios in­clud­ing com­plex, high-fre­quent and strongly
              vary­ing light­ing as well as shad­ows.},
      issn = {0924-2716},
       url = {http://www.sciencedirect.com/science/article/pii/S0924271619300279},
       doi = {https://doi.org/10.1016/j.isprsjprs.2019.01.018}
}