Multi-branch CNNFormer: a novel framework for predicting prostate cancer response to hormonal therapy

Abstract

Purpose: This study aims to accurately predict the effects of hormonal therapy on prostate cancer (PC) lesions by integrating multi-modality magnetic resonance imaging (MRI) and the clinical marker prostate-specific antigen (PSA). It addresses the limitations of Convolutional Neural Networks (CNNs) in capturing long-range spatial relations and the Vision Transformer (ViT)’s deficiency in localization information due to consecutive downsampling. The research question focuses on improving PC response prediction accuracy by combining both approaches. Methods: We propose a 3D multi-branch CNN Transformer (CNNFormer) model, integrating 3D CNN and 3D ViT. Each branch of the model utilizes a 3D CNN to encode volumetric images into high-level feature representations, preserving detailed localization, while the 3D ViT extracts global salient features. The framework was evaluated on a 39-individual patient cohort, stratified by PSA biomarker status. Results: Our framework achieved remarkable performance in differentiating responders and non-responders to hormonal therapy, with an accuracy of 97.50%, sensitivity of 100%, and specificity of 95.83%. These results demonstrate the effectiveness of the CNNFormer model, despite the cohort’s small size. Conclusion: The findings emphasize the framework’s potential in enhancing personalized PC treatment planning and monitoring. By combining the strengths of CNN and ViT, the proposed approach offers robust, accurate prediction of PC response to hormonal therapy, with implications for improving clinical decision-making.

keywords Deep learning, Hormonal therapy, Prostate cancer, Vision transformer