Multimodality Calibration in 3D Multi Input-Multi Output Network for Dementia Diagnosis with Incomplete Acquisitions

Alzheimer’s disease (AD) is one of the leading causes of cognitive decline and death worldwide, necessitating ongoing research in diagnosis, prognosis, and treatment. Advances in Artificial Intelligence, particularly Deep Learning (DL), have enabled the development of sophisticated diagnostic models using medical imaging data. In clinical trials, Magnetic Resonance Imaging (MRI) and Positron Emission Tomography (PET) are predominantly used, offering valuable metabolic and structural insights. Integrating these modalities through Multimodal Deep Learning (MDL) approaches enhances diagnostic accuracy and robustness. However, implementing MDL in medical imaging presents challenges, such as the difficulty in obtaining complete multimodal data for each patient and effectively integrating the characteristics of multiple modalities. To address these issues, we propose a 3D Multi Input-Multi Output (3D-MIMO) neural network that employs an efficient training strategy to manage data absence while using a Transfer Module (TM) to synergistically fuse PET and MRI data. We validate our approach using the Open Access Series of Imaging Studies (OASIS) and Alzheimer’s Disease Neuroimaging Initiative (ADNI) datasets, demonstrating the efficacy of our 3D-MIMO neural network in handling multimodal data.