Shearlet-Based Descriptors and Deep Learning Approaches for Medical Image Classification

Abstract

In this Ph.D. thesis, we develop effective techniques for medical image classification, particularly, for histopathological and magnetic resonance images (MRI). Our techniques are capable of handling the high variability in the content of such images. Handcrafted techniques based on texture analysis are used for the classification task. We also use deep learning models but training such models from scratch can be a challenging process, instead, we employ deep features and transfer learning.

First, we propose a combined texture-based feature representation that is computed in the complex shearlet domain for histopathological image classification. With complex coefficients, we examine both the magnitude and relative phase of shearlets to form the feature space. Our proposed techniques are successful for histopathological image classification. Furthermore, we investigate their ability to generalize to MRI datasets that present an additional challenge, namely high dimensionality. An MRI sample consists of a large number of slices. Our proposed shearlet-based feature representation for histopathological images cannot be used without adjustment. Therefore, we consider the 3D shearlet transform given the volumetric nature of MRI data. An advantage of the 3D shearlet transform is that it takes into consideration adjacent slices of MRI data.

Secondly, we study the classification of histopathological images using pre-trained deep learning models. A pre-trained deep learning model can act as a starting point for datasets with a limited number of samples. Therefore, we used various models either as unsupervised feature extractors, or weight initializers to classify histopathological images. When it comes to MRI samples, fine-tuning a deep learning model is not straightforward. Pre-trained models are trained on RGB images which have a channel size of 3, but an MRI sample has a larger number of slices. Fine-tuning a convolutional neural network (CNN) requires adjusting a model to work with MRI data. We fine-tune pre-trained models and then use them as feature extractors. Thereafter, we demonstrate the effectiveness of fine-tuned deep features with classical machine learning (ML) classifiers, namely a support vector machine and a decision tree bagger. Furthermore, instead of using a classical ML classifier for the MRI sample, we built a custom CNN that takes both the 3D shearlet descriptors and deep features as an input. This custom network processes our feature representation end-to-end and then classifies an MRI sample. Our custom CNN is more effective in comparison to a classical ML on a hidden MRI dataset. It is an indication that our CNN model is less susceptible to over-fitting.