Segmentation of Pigment Signs in Fundus Images with a Hybrid Approach: A Case Study

Abstract: Retinitis pigmentosa is a retinal disorder leading to a progressive visual field loss and eventually to complete blindness, but an early diagnosis could delay its progression through specific therapies. Retinitis pigmentosa is characterized by typical pigment signs that accumulate in the different regions of the retina. Pigment signs could be detected by a low-cost diagnosis tool, analyzing visual fundus retinal images and applying segmentation algorithms to annotate the pigments, so that, in a telemedicine scenario, the segmented images could be forwarded to an ophthalmologist for a rapid diagnosis.Deep learning approaches might be appropriate for this problem, but they have rarely been used to address it. However, pigment segmentation is a challenging task due to image resolution, small size of pigments and their proximity with blood vessels with which they share similar colors, and inter-patient widely changing image features. Very recently, transformer architectures, based on the self-attention paradigm, have emerged in the deep learning community as a powerful yet not completely explored tool to learn features directly from the data. Nonetheless, they could not be directly exploited on small datasets, as they require a very large amount of data to learn meaningful features. To overcome the need for large training datasets, but also to reduce the high computation effort, hybrid architectures have been proposed, with the aim to combine the long-range relationship detection of transformers with the invariance and short-range detection properties of classical deep learning architectures. Here, we investigate the performances of the Group Transformer U-Net, a hybrid approach for pigment segmentation on fundus images. This hybrid architecture modifies the classical U-Net structure introducing bottleneck multihead self-attention blocks between convolutional layers in both the contracting and expanding paths of the network. We compare the results obtained with this approach with the ones of the standard U-Net, and we describe how these results are affected when using different loss functions for the learning process, or strategies to address class imbalance.