BrainDx: a dual-transformer framework using PVT and SegFormer for tumor diagnosis

Abstract

Brain tumor diagnosis is challenging due to their complex morphology, indistinct boundaries, and subtle variations in Magnetic Resonance Imaging (MRI) scans. Manual diagnosis is time-consuming and error-prone, making the need for automated systems crucial. Recent advancements in deep learning, particularly in transformer models, have led to improved accuracy and speed in medical image analysis. Objective: This research aims to develop an Artificial Intelligencee (AI) based framework that integrates the Pyramid Vision Transformer (PVT) for tumor classification and the SegFormer for tumor segmentation, thereby enhancing diagnostic accuracy, speed, and reducing human error in brain tumor detection. Methodology: The proposed framework, BrainDX, utilizes PVT to classify MRI images into tumor types (Gliomas, Meningiomas, Pituitary Tumors, and Healthy Brain), and SegFormer to segment tumor regions in real-time. The dataset consists of annotated MRI images that undergo preprocessing (normalization, resizing, and augmentation). The models are trained and evaluated based on performance metrics, including accuracy, Dice score, Intersection over Union (IoU), and segmentation time. Results: The framework was evaluated across three benchmark MRI datasets, achieving a classification accuracy of 94.0% and a Dice score of 0.87 for tumor segmentation. SegFormer demonstrated real-time segmentation, processing MRI images in under 50 ms. Both models maintained high efficiency while delivering robust performance, even in cases of irregular tumor boundaries. Future Scope: Future work will focus on further optimizing the model for real-time clinical use, improving generalization across diverse tumor types and MRI modalities. This AI-powered system has the potential to enhance diagnostic processes and improve patient outcomes significantly.

Keywords Brain Tumor Classification; Deep Learning; Pyramid Vision Transformer (PVT); SegFormer; Tumor Segmentation