Replication and Refinement of an Algorithm for Automated Drusen Segmentation on Optical Coherence Tomography

Maximilian Wintergerst, Shekoufeh Gorgi Zadeh, Vitalis Wiens, Sarah Thiele, Steffen Schmitz-Valckenberg, Frank Holz, Robert Finger und Thomas Schultz
In: Scientific Reports (2020), 10(7395)
 

Abstract

Here, we investigate the extent to which re-implementing a previously published algorithm for OCT-based drusen quantification permits replicating the reported accuracy on an independent dataset. We refined that algorithm so that its accuracy is increased. Following a systematic literature search, an algorithm was selected based on its reported excellent results. Several steps were added to improve its accuracy. The replicated and refined algorithms were evaluated on an independent dataset with the same metrics as in the original publication. Accuracy of the refined algorithm (overlap ratio 36–52%) was significantly greater than the replicated one (overlap ratio 25–39%). In particular, separation of the retinal pigment epithelium and the ellipsoid zone could be improved by the refinement. However, accuracy was still lower than reported previously on different data (overlap ratio 67–76%). This is the first replication study of an algorithm for OCT image analysis. Its results indicate that current standards for algorithm validation do not provide a reliable estimate of algorithm performance on images that differ with respect to patient selection and image quality. In order to contribute to an improved reproducibility in this field, we publish both our replication and the refinement, as well as an exemplary dataset.

Bilder

Paper herunterladen

Paper herunterladen

Bibtex

@ARTICLE{Wintergerst:SREP2020,
    author = {Wintergerst, Maximilian and Gorgi Zadeh, Shekoufeh and Wiens, Vitalis and Thiele, Sarah and
              Schmitz-Valckenberg, Steffen and Holz, Frank and Finger, Robert and Schultz, Thomas},
     pages = {7395},
     title = {Replication and Refinement of an Algorithm for Automated Drusen Segmentation on Optical Coherence
              Tomography},
   journal = {Scientific Reports},
    volume = {10},
      year = {2020},
  abstract = {Here, we investigate the extent to which re-implementing a previously published algorithm for
              OCT-based drusen quantification permits replicating the reported accuracy on an independent dataset.
              We refined that algorithm so that its accuracy is increased. Following a systematic literature
              search, an algorithm was selected based on its reported excellent results. Several steps were added
              to improve its accuracy. The replicated and refined algorithms were evaluated on an independent
              dataset with the same metrics as in the original publication. Accuracy of the refined algorithm
              (overlap ratio 36–52%) was significantly greater than the replicated one (overlap ratio 25–39%).
              In particular, separation of the retinal pigment epithelium and the ellipsoid zone could be improved
              by the refinement. However, accuracy was still lower than reported previously on different data
              (overlap ratio 67–76%). This is the first replication study of an algorithm for OCT image
              analysis. Its results indicate that current standards for algorithm validation do not provide a
              reliable estimate of algorithm performance on images that differ with respect to patient selection
              and image quality. In order to contribute to an improved reproducibility in this field, we publish
              both our replication and the refinement, as well as an exemplary dataset.}
}