Interactive Classification of Multi-Shell Diffusion MRI With Features From a Dual-Branch CNN Autoencoder

In proceedings of EG Workshop on Visual Computing for Biology and Medicine, 2020
 

Abstract

Multi-shell diffusion MRI and Diffusion Spectrum Imaging are modern neuroimaging modalities that acquire diffusion weighted images at a high angular resolution, while also probing varying levels of diffusion weighting (b values). This yields large and intricate data for which very few interactive visualization techniques are currently available. We designed and implemented the first system that permits an interactive, iteratively refined classification of such data, which can serve as a foundation for isosurface visualizations and direct volume rendering.

Our system leverages features learned by a Convolutional Neural Network. CNNs are state of the art for representation learning, but training them is too slow for interactive use. Therefore, we combine a computationally efficient random forest classifier with autoencoder based features that can be pre-computed by the CNN. Since features from existing CNN architectures are not suitable for this purpose, we design a specific dual-branch CNN architecture, and carefully evaluate our design decisions. We demonstrate that our approach produces more accurate classifications compared to learning with raw data, established domain-specific features, or PCA dimensionality reduction.

Images

Additional Material

  • Video (MPEG-4 video, 8.3 MB)

Bibtex

@INPROCEEDINGS{Torayev:VCBM2020,
     author = {Torayev, Agajan and Schultz, Thomas},
      title = {Interactive Classification of Multi-Shell Diffusion MRI With Features From a Dual-Branch CNN
               Autoencoder},
  booktitle = {EG Workshop on Visual Computing for Biology and Medicine},
       year = {2020},
   abstract = {Multi-shell diffusion MRI and Diffusion Spectrum Imaging are modern neuroimaging modalities that
               acquire diffusion weighted images at a high angular resolution, while also probing varying levels of
               diffusion weighting (b values). This yields large and intricate data for which very few interactive
               visualization techniques are currently available. We designed and implemented the first system that
               permits an interactive, iteratively refined classification of such data, which can serve as a
               foundation for isosurface visualizations and direct volume rendering.
               
               Our system leverages features learned by a Convolutional Neural Network. CNNs are state of the art
               for representation learning, but training them is too slow for interactive use. Therefore, we
               combine a computationally efficient random forest classifier with autoencoder based features that
               can be pre-computed by the CNN. Since features from existing CNN architectures are not suitable for
               this purpose, we design a specific dual-branch CNN architecture, and carefully evaluate our design
               decisions. We demonstrate that our approach produces more accurate classifications compared to
               learning with raw data, established domain-specific features, or PCA dimensionality reduction.}
}