A Class-Conditioned Deep Neural Network to Reconstruct CT Volumes from X-Ray Images: Depth-Aware Connection and Adaptive Feature Fusion

Abstract

X-ray and Computed Tomography (CT) are two of the most usual imaging systems for the medical domain in order to inspect the body’s inside, check the major organs, and internal injuries. While the Computed Tomography (CT) generates 3D images to check internal body structure for diagnosis, the X-ray has offered a cost-effective imaging technique and incurs lower radiation dose. Therefore, due to the high screening cost and deficiency of screening machines, CT scan appliances are not always accessible. 3D CT volume reconstruction from its 2D X-ray counterpart is a challenging problem. Furthermore, the conventional CT reconstruction algorithms necessitate hundreds of X-ray projections to provide the body's 3D view that cannot be executed on a traditional X-ray apparatus. In spite of the advancement of convolutional neural networks to achieve a reconstruction performance by generating the anatomical structures with realistic textures, semantic features still remain unexplored. In this study, we investigate the novel 3D reconstruction problem by proposing a class-conditioned deep neural network (CCX-rayNet), which is adept at recovering textures and shapes faithful to semantic classes in the real-world. Firstly, we propose to modulate the features conditioned on the categorical prior by generating the affine transformation parameters. Secondly, we enhance the feature representation of the X-ray image by bridging 2D and 3D features. In particular, we estimate a 3D attention mask to be applied on the expanded 3D feature map, where the contextual relationship will be highlighted. In our proposed biplanar view CCX-rayNet, we also include an adaptive feature fusion (AFF) module to resolve a registration problem that happens with uncontrolled input data by using the similarity matrix. To the best of our knowledge, this proposed method is the first work to deal with semantic prior in the 3D CT reconstruction task. Both qualitative and quantitative assessments demonstrate the exceptional performance of our proposed CCX-rayNet, which outperforms state-of-the-art methods. Using this 3D reconstruction process, a medical practitioner can achieve some useful appliances for clinical practices at a minimal cost.