Author: Nikolas Pontikos (United Kingdom)
Co-authors: William Woof, Kristina Hess, Michel Michaelides, Peter Krawitz, Frank Holz, Andrew Webster
Inherited retinal diseases (IRDs) are single-gene disorders caused by genetic mutations in any one of over 270 genes. They are a leading cause of blindness in children and working age adults globally. Identifying the causative gene through genetic testing is crucial for potential gene targeted treatments, recruitment to clinical trials, prognosis and family planning. However the prescription of appropriate genetic tests and the interpretation of genetic results requires phenotype-genotype recognition that only few IRD experts are currently capable of providing. Therefore we aimed to develop Eye2Gene, an AI algorithm, to predict the probable IRD causative gene from the retinal scans of suspected IRD patients.
Eye2Gene was trained and tested on retinal scans of 1,907 IRD patients with a known genetic diagnosis from Moorfields Eye Hospital and externally validated on a cohort of 37 IRD patients with a genetic diagnosis from the University Hospital Bonn. The training was limited to predicting the top 36 most common genes which represents 82% of all IRD cases at Moorfields.
Following quality control, the Moorfields training dataset consisted of 44,817 images from 1,907 IRD patients from Moorfields Eye Hospital, split into three different modalities: Fundus Auto-Flourescence (FAF), Infrared (IR), and Spectral-Domain Optical Coherence Tomography (SD-OCT). For each of the three modalities, five individual Inception v3 convolutional neural networks (CNNs) were trained on different samples of the training data using 5-fold cross validation. This resulted in Eye2Gene, an ensemble of fifteen different neural networks. Each network was trained to identify up to 36 gene classes. Per image predictions were obtained by combining the fifteen CNNs predictions. Per patient predictions were obtained by combining the prediction across multiple images from the same patient. In order to assess generalisability of Eye2Gene, we defined a held-out dataset consisting of 264 patients from Moorfields not used in the training and an external dataset of 37 patients from University Hospital of Bonn. From these a subset 50 FAF scans was extracted for human evaluation by eight ophthalmologists with varying levels of experience in IRDs in order to benchmark Eye2Gene against human performance. Ophthalmologists were asked to pick their top five gene choices per image out of the 36 genes.
Eye2Gene yields a top-5 accuracy of 88% (i.e predicts the correct gene in the top 5 guesses 88% of the time) in the Moorfields held-out dataset and 83% in the external validation University Hospital of Bonn dataset. On the 50 FAF images that were used for the human benchmarking, Eye2Gene achieved a top-5 accuracy of 70% while the ophthalmologists obtained a maximum combined top-5 accuracy of 78% (at least one of the eight ophthalmologists included the correct gene in the top 5 guesses 78% of the time). However, the maximum top-5 accuracy for any single ophthalmologist was not higher than 36%.
We have developed an AI algorithm Eye2Gene, capable of predicting the 36 top most common IRD genes in the UK and German population to a top-5 accuracy of >80%. From our preliminary benchmarking, Eye2Gene achieves performance similar to a consensus of human experts on an external dataset. This makes Eye2Gene the most advanced AI system yet for the recognition of genes from IRD retinal scans. Future work will include increasing the training dataset through aggregation to achieve better performance and predict more gene classes. Additionally, segmentation of retinal features will also be necessary for obtaining interpretatable outputs to satisfy explainability and transparency requirements. We expect Eye2Gene will eventually enable democratisation of IRD expertise, currently available in only a few centres in the world, which will facilitate the prescription and interpretation of genetic tests.
I am a co-founder of Phenopolis Ltd , a software company.