Combination of Graph and Convolutional Networks for Brain Tumor Segmentation from Multi-Modal MR Images In Clinical Applications

Marjan  Vatanpour; Javad  Haddadnia; Shahryar Salmani  Bajestani

doi:10.18502/fbt.v12i3.19178

Marjan Vatanpour Department of Biomedical Engineering, Khayyam University, Mashhad, Iran
Javad Haddadnia Department of Biomedical Engineering, Hakim Sabzevari University, Sabzevar, Iran
Shahryar Salmani Bajestani Department of Biomedical Engineering, Mashhad Branch, Islamic Azad University, Mashhad, Iran

DOI: https://doi.org/10.18502/fbt.v12i3.19178

Keywords: Brain Tumor; Segmentation; Convolutional Neural Network; Graph Neural Network; Magnetic Resonance Imaging.

Abstract

Purpose: Brain tumors are very important for the overall health of humans, which happen due to the uncontrolled increase and duplication of abnormal cells. Therefore, brain tumor segmentation is a very important step in medical diagnosis and can help in early tumor detection, treatment planning, and tumor progression follow-ups. To solve the problems related to manual segmentation such as time-cost, inaccuracy and subjectivity, automatic segmentation with deep learning methods is presented. This study aimed to develop an automatic brain tumor segmentation based on the combination of convolutional and graph neural networks to overcome the shortcomings of each network when they are used individually.

Materials and Methods: The main goal of this study is to propose a novel architecture for brain tumor segmentation from multi-modal MR images and comparison of the results with related SOTA studies. The novel architecture uses a simple Convolutional Neural Network (CNN) and Graph Neural Network (GNN) sequentially. In the first stage, the volumetric 3D image with a combination of all modalities is fed to the simple convolutional network. After retrieving the feature representation of the CNN, a graph model is created and fed to the GNN. The CNN will help to capture local information of patches and GNN will retrieve the global information available in the data which together can provide promising results.

Results: The proposed model used for the segmentation of the BraTS2021 dataset showed the average Dice score of 0.86 and the average Hausdorff of 17.94. The results showed that the combination of CNN and GNN can the performance of the task at hand. Also, the heatmaps extracted can show the importance of adding the GNN into the CNN.

Conclusion: New and creative advancements in artificial intelligence and its applications for medical image segmentation are very promising. We proposed a hybrid network of CNN and GNN to capture local and global information and combine them in a way such that we can recreate an acceptable segmented result which is justified with Dice score and Hausdorff metrics quantitatively. The proposed methodology performed better in comparison with the other related methods. Also, the activation heatmaps confirm the reliability of the approach qualitatively.