Intelligent interaction and uncertainty visualization for efficient drusen and retinal layer segmentation in Optical Coherence Tomography

Shekoufeh Gorgi Zadeh, Maximilian Wintergerst, and Thomas Schultz
In: Computers and Graphics (2019), 83(51-61)
 

Abstract

Convolutional neural networks (CNNs) represent the state of the art for fully automated medical image segmentation. However, few works have combined CNNs with interactive user feedback in order to verify and, where necessary, correct their results. We present an interactive visual system that achieves this for the specific use case of segmenting drusen, which serve as a biomarker of age related macular degeneration, from Optical Coherence Tomography. Our main idea is to exploit the probabilistic nature of CNN-based segmentation. First, we derive two uncertainty measures from it. We demonstrate that they indicate cases in which automated segmentation is likely to have failed, and that visualizing them makes manual verification and correction more efficient. Second, based on the probabilistic information, we design intelligent tools for segmentation correction, which automatically propose the most likely alternative segmentation in agreement with user-specified constraints. In a small user study, uncertainty visualization and intelligent interaction reduced the time required to correct retinal layer segmentation by around 53% and, for drusen segmentation, even by 73%. In the future, we plan to use our system not only for efficient segmentation correction, but also for rapid creation of larger training sets.

Images

Bibtex

@ARTICLE{gorgizadeh:cag19,
    author = {Gorgi Zadeh, Shekoufeh and Wintergerst, Maximilian and Schultz, Thomas},
     pages = {51--61},
     title = {Intelligent interaction and uncertainty visualization for efficient drusen and retinal layer
              segmentation in Optical Coherence Tomography},
   journal = {Computers and Graphics},
    volume = {83},
      year = {2019},
  abstract = {Convolutional neural networks (CNNs) represent the state of the art for fully automated medical
              image segmentation. However, few works have combined CNNs with interactive user feedback in order to
              verify and, where necessary, correct their results. We present an interactive visual system that
              achieves this for the specific use case of segmenting drusen, which serve as a biomarker of age
              related macular degeneration, from Optical Coherence Tomography. Our main idea is to exploit the
              probabilistic nature of CNN-based segmentation. First, we derive two uncertainty measures from it.
              We demonstrate that they indicate cases in which automated segmentation is likely to have failed,
              and that visualizing them makes manual verification and correction more efficient. Second, based on
              the probabilistic information, we design intelligent tools for segmentation correction, which
              automatically propose the most likely alternative segmentation in agreement with user-specified
              constraints. In a small user study, uncertainty visualization and intelligent interaction reduced
              the time required to correct retinal layer segmentation by around 53% and, for drusen segmentation,
              even by 73%. In the future, we plan to use our system not only for efficient segmentation
              correction, but also for rapid creation of larger training sets.},
       doi = {10.1016/j.cag.2019.07.001}
}