Dr. SWATHI KALIKI

Dr. V S Vijitha, Dr.Rajiv Viriyala, Dr.Krishna Kishore Dhara

Abstract

Purpose: To explore the utility of artificial intelligence (AI) in the diagnosis and grouping of intraocular retinoblastoma (iRB)

Methods: Image review by AI

Results: Of 771 fundus images of 109 eyes, 181 images had no tumor and 590 images had iRB based on review by 2 ocular oncologists (inter-observer variability of <1%). The sensitivity and specificity of the trained AI model for detection of RB in an image was 85% and 99% respectively. Of 109 eyes, the sensitivity and specificity for detection of RB by AI model was 96% and 64% respectively. The eyes were normal (n=31) or harbored group A (n=1), B (n=22), C (n=8), D (n=23), and E (n=24) RB (inter-observer variability of 0%). The sensitivity and specificity of the trained AI model was 100%, 100% for group A; 82%, 98% for group B; 63%, 99% for group C; 78%, 98% for group D, and 92%, 91% for group E respectively. Conclusion: The AI model for iRB is highly sensitive in detection of RB with high specificity for classification of iRB.

Full Text  

Artificial Intelligence and Machine Learning in Ocular Oncology: Retinoblastoma
Swathi Kaliki, M.D.
Vijitha S Vempuluru, M.D
Neha Ghose, M.D
Gaurav Patil, M.D
Rajiv Viriyala, B.Tech
Krishna Kishore Dhara, PhD

From the 1Operation Eyesight Universal Institute for Eye Cancer (SK, VSV, NG, GP), L V Prasad Eye Institute, Hyderabad, India and 2Artificial Intelligence and Machine Learning, TechSophy Inc, Hyderabad, India.

Support provided by The Operation Eyesight Universal Institute for Eye Cancer (SK) and Hyderabad Eye Research Foundation (SK), Hyderabad, India. The funders had no role in the preparation, review or approval of the manuscript

No conflicting relationship exists for any author.

Inquiries to:

Swathi Kaliki, M.D., The Operation Eyesight Universal Institute for Eye Cancer, L V Prasad Eye Institute, Hyderabad-500034
Email: kalikiswathi@yahoo.com
Tel: 91 40 68102502
Fax: 91 40 68108339
ORCID ID: 0000-0002-0800-9961

Keywords:

Eye
Tumor
Retinoblastoma
Artificial intelligence
Machine learning

Précis:

Analysis of 771 fundus images of 109 eyes by a trained artificial intelligence model revealed a sensitivity, specificity, positive predictive value, and negative predictive of 96%, 94%, 97%, and 91% respectively for detection of RB.

ABSTRACT

Purpose:

To explore the utility of AI and machine learning in the diagnosis and grouping of intraocular retinoblastoma (iRB)

Methods:

AI and Machine learning, Computer Vision (OpenCV)

Results:

Of 771 fundus images of 109 eyes, 181 images had no tumor and 590 images displayed iRB based on review by 2 independent ocular oncologists (with an inter-observer variability of <1%). The sensitivity, specificity, positive predictive value, and negative predictive value of the trained AI model was 85%, 99%, 99.6%, and 67% respectively. Of 109 eyes, the sensitivity, specificity, positive predictive value, and negative predictive value for detection of RB by AI model was 96%, 94%, 97%, 91% respectively.

Of these, the eyes were normal (n=31) or belonged to group A (n=1), B (n=22), C (n=8), D (n=23), and E (n=24) RB based on review by 2 independent ocular oncologists (with an inter-observer variability of 0%). The sensitivity, specificity, positive predictive value, and negative predictive value of the trained AI model was 100%, 100%, 100%, and 100% for group A; 82%, 98%, 90%, and 96% for group B; 63%, 99%, 83%, and 97% for group C; 78%, 98%, 90%, and 94% for group D, and 92%, 91%, 73%, and 98% for group E respectively.

Conclusion:

The AI model for iRB is highly sensitive in detection of RB with high specificity for classification of iRB.

Introduction

The concept of artificial intelligence (AI) dates back to 1950 and is based on Alan Turing’s experiment on imitation game,[1] though the term AI was coined by John McCarthy in 1956 at the Dartmouth Summer Research Project on AI. The use of AI in medicine has gained importance in the past few decades. AI in medicine is either in virtual form via machine learning (ML) or deep learning aiding in the development of complex treatment algorithms, predictive models via electronic medical records and big data, or softbots;[2,3] or physical form via sophisticated medical devices and robots or carebots.[2,4]

In the virtual form, there have been various applications of AI and ML in the field of medicine mainly based on numerical data and digital images. The various applications in medicine include use of epidemiology informatics in public health,[5] improvement of decision making in anaesthesiology,[6] improvement of precision medicine in the fields of reproductive medicine[7] and cardiovascular medicine,[8] improvement of diagnostic accuracy and objectivity in pathology with the advent of digital pathology,[9] improvement of technological aspects, image quality and image reading in nuclear medicine,[10] and also aiding in diagnosis, evaluation of prognosis, prediction of epidemic, and drug discovery in the recent COVID-19 pandemic.[11] In the field of cancer, AI and ML has contributed to image analysis, improving diagnosis, multi-layered data analysis with integration of cancer genomics, development of treatment algorithms, estimating prognosis, drug discovery, and ultimately improving the cancer healthcare to precision oncology.[12]

In the recent years, there has been increasing interest for the use of AI and ML in the field of ophthalmology. Ophthalmology is a branch of medicine which uses various imaging modalities such as fundus photographs, optical coherence tomography, computed tomography, magnetic resonance imaging etc; and is based on numerical data such as visual acuity, intraocular pressure, corneal thickness, refractive error, cup to disc ratio etc for diagnosis and prognosis of various ophthalmic diseases. Numerical data and digital images have been used to develop AI and ML models for various diseases in ophthalmology. Till date, there have been no AI models for the diagnosis of retinoblastoma (RB) and herein, we explore the utility of AI and ML for diagnosis and classification of intraocular RB.

Methods

This was a retrospective study conducted at the Operation Eyesight Universal Institute for Eye Cancer, LV Prasad Eye Institute, Hyderabad, India and TechSophy, Hyderabad, India. Approval from the Institutional Review Board was obtained. The study adhered to the tenets of Declaration of Helsinki. Fundus photographs of children diagnosed with treatment-naïve unilateral or bilateral RB from January 2017 to June 2021 were retrieved from the institutional database. All images were captured on Retcam II (Clarity Medical Systems, Inc., Pleasanton, CA, USA).

Poor quality images were excluded. All images were read by two independent ocular oncologists (VSV, NG) and labelled as normal or RB. In images with tumors, a group label was assigned to every image according to the International Classification of Intraocular Retinoblastoma (ICIoR) as Group A, B, C, D, or E.[13] After reviewing all images of a single eye, a cumulative label was assigned as Normal, Group A, B, C, D, or E. In cases of discrepancy of interpretation between the 2 ocular oncologists, label assigned by VSV was taken as the gold standard for comparison with the results from AI model.

The images that were manually labelled were also used to develop and test the AI model for RB by feature extraction and use of RB classifier. These are described in detail below:

Feature extraction:

The features needed for the classification of RB according to ICIoR were extracted using a series of algorithms that involved deep learning object detection, computer vision, and other geometry and intensity gradation algorithms. Open Computer Vision (OpenCV) techniques and deep learning (DL) models were employed to extract, identify or calculate the following features from every image: hough circle (i.e. entire area of fundus captured within an image), optic disc (detection), tumor (detection), maximum tumor size (mm), number of tumors, distance between optic disc and the closest tumor, vitreous seeds, sub-retinal seeds, number of seeds, maximum distance between seeds and the nearest tumor, blood vessels and intraocular hemorrhage. Feature vectors representing the image were used to train the RB classifier of the AI model (Figure 1). Eighty percent of the images in the dataset were used to train the classifier and 20% were used to test the RB classifier.

For detection of optic disc and tumor, Multi-Label Classification Model of Deep Learning (DL) was utilized on training and test images to predict a bounding box and class with a confidence score for each prediction (Figure 2A and 2B). Transfer learning[14] was used for training the AI model in identification of optic disc and tumor

Transfer learning implementation used an efficient Mobile-Net v2 SSD-based deep learning model[15] as a base and removed the last layer with a combined optic disc and tumor detection. To train this model, we manually labelled optic disc and tumors in fundus images with rectangular boxes. A total of 327 images were labelled where 285 images were used for training and 42 images were used for validation. The modified deep learning network was trained on the labelled optic disc and tumor fundus images.

After the optic disc and tumors were identified with a degree of confidence, the relative distance between them and the size of the tumor were estimated. Estimation of tumor size was performed with a reference diameter of 1.5mm for optic disc (Figure 2C) when the latter was detected in an image. In the absence of an optic disc within the image frame, the calculation was based on the hough circular diameter of 12 mm which was averaged from all images containing optic disc (Figure 2D).

Detection of tumor seeds was complex and the tumor seeds were identified by a two-step approach. The region of interest was identified by an OpenCV algorithm which was further processed by DL model to identify the seeds (Figure 2E and 2F). Identification of regions of interest was done in eight directions from the center of the tumor to the edge of the hough circle diameter of the image by calculating the mean of the pixel intensity. A threshold was determined using the minimum of these means.

Once a threshold was established, another pass through the eight directions was performed to determine the areas of interest where there could be tumor seeds that have pixel intensity above the threshold and away from the tumor. Depending on the direction, a directional straight line or a diagonal, either a rectangle or a square area of interest was selected. A trained deep learning object classifier that is trained specifically for seeds detection was used to identify the presence of tumor seeds and when present, the maximum distance of the seeds from the nearest tumor was computed. This model was trained and validated on 132 images.

Localization of intraocular hemorrhage (Figure 2G and 2H) and blood vessels (Figure 2I and 2J) were mapped by OpenCV and combination of OpenCV & DL models respectively.[16] Calculations of distance of tumor from optic disc and size of the tumor were based on OpenCV algorithms and geometry respectively.

Assigning group labels with RB classifier:

Each of the fundus image was run through all the algorithms to obtain a feature vector that consisted of features including detection of optic disc, tumor detection, determination of tumor size (< 3 mm or > 3mm), total number of tumors, detection of intraocular haemorrhage, localization of overlying retinal blood vessels, distance between optic disc and tumor, presence of subretinal/vitreous seeds, number of tumor seeds, and maximum distance between seeds and the nearest tumor (< 3mm or > 3 mm).

Using the group classification labels by experts, a decision tree machine learning model[17] was trained. This RB classifier model was trained on 616 images and tested on 155 images.

The final RB classification was based on an ensemble of the RB classifier results of all the images of a given eye of the patient. Following analysis of all images of one eye, the highest label was assigned as the group label for the eye.

Performance Metrics:

Performance metrics were calculated for the entire data set which included both ‘training’ and ‘test’ images. Parameters assessed included overall accuracy, misclassification as RB, under-classification rates and over-classification rates. Sensitivity, specificity, positive predictive value, negative predictive value for detection of RB and classification of RB were calculated. Performance metrics were calculated for every image and also for every eye since each eye had multiple images ranging between 4 to 20.

Results

Seven hundred and seventy-one fundus photographs from 109 eyes of 62 patients diagnosed with treatment naïve unilateral or bilateral retinoblastoma between January 2017 and June 2021 were retrieved from the institutional database. Of 771 fundus images, 181 images had no tumor and 590 images had iRB based on review by 2 ocular oncologists. The eyes were normal (n=31) or harbored group A (n=1), B (n=22), C (n=8), D (n=23), and E (n=24) RB. Inter-observer variability in detection of RB or classification of RB was 0.98% for individual images and 0% based on cumulative results for each eye (Table 1).

Performance Metrics:

Labels predicted by the RB classifier against true labels assigned by the observers are depicted in Figure 3. Of 590 images with RB, the accuracy for detection of RB was 85%, with RB being detected in 500 images. While 90 images (15%) were misclassified as normal, RB was detected but under-classified in 45 images (8%). Thus, a total of 135 images (23%) were under-classified. Of 181 images with no tumor, RB was detected in 2 (1%). Metrics for grouping of eyes yielded higher accuracy: Of 78 eyes with RB, RB was identified in 75 (96%) with under-classification in 1 (1%) eyes, and misclassified as normal in 3 (4%). Thus a total of 4 eyes (5%) were under-classified. Of 31 normal eyes, 29 (94%) were classified as normal and 2 (6%) were misclassified as RB.

Test characteristics:

The sensitivity and specificity of the trained AI model for detection of RB in an image was 85% and 99% respectively. Of 109 eyes, the sensitivity and specificity for detection of RB by AI model was 96% and 94% respectively. The sensitivity and specificity of RB classification was 100% and 100% for group A, 82% and 98% for group B, 63% and 99% for group C; 78% and 98% for group D, and 92% and 91% for group E respectively. Test characteristics for eyes in each group are summarized in Table 2.

Discussion:

While there has been an increasing use of AI in daily life with Apple Inc.’s Siri and Amazon’s Alexa, there has also been increasing applications of AI and ML in healthcare. Despite increasing discussions about AI and ML in healthcare in the literature, it was estimated that 100% of US healthcare was delivered without any use of AI in 2017.[18] However, it is predicted that there will be an increased use of AI and ML in healthcare in the future, with certain tasks being overpowered by AI-enabled technology.

In the field of ophthalmology, AI and ML has been capable of analysing quantifiable data and images, thus aiding in diagnosis and staging of various ocular pathologies including strabismus, refractive error, keratitis, keratoconus, cataracts, glaucoma, papilledema, optic disc abnormalities, diabetic retinopathy, age-related macular degeneration, retinopathy of prematurity, thyroid eye disease, choroidal nevus and transformation to melanoma.[19-23] AI has also been used in development of treatment algorithms, assessing treatment need, and prognostication of ocular pathologies.[19-21]

Retinoblastoma is the most common malignant intraocular tumor in children with most patients presenting with advanced disease at presentation in lower and lower middle-income countries.[24-26] Advanced disease at presentation results in poor chances of vision, globe and life salvage. Locally advanced disease is mainly related to delayed presentation and delayed diagnosis.[24] The diagnosis of RB is mainly dependent on symptoms like leukocoria and strabismus and red reflex testing, while most countries do not have routine fundus screening protocols in children.[27] In this study, there were more number of group D and E (n=47; 60%) eyes during the study period compared to Groups A to C (n=31; 40%) eyes suggestive of higher proportion of patients with advanced RB at presentation.

Fundus photographs have been the basis for development of AI models for various retinal pathologies including diabetic retinopathy, age-related macular degeneration and retinopathy of prematurity. AI and ML for diabetic retinopathy screening using fundus photographs has shown a sensitivity of 88% to 100% and specificity of 91% to 99% in detecting moderate diabetic retinopathy or worse.[28-30] Fundus image analysis has assisted in AI-related screening for age-related macular degeneration with an accuracy of 88% to 92%.[31] Similarly, AI models have been developed for retinopathy of prematurity for detection of plus disease based on fundus photographs with 93% to 100% sensitivity and 78% to 94% specificity.[32,33] In this study, we used fundus photographs of children with treatment naïve RB to develop an AI model for diagnosis and classification of RB. The most common DL method used for analysis of medical images is convoluted neural network (CNN).[34] In our study, we developed multiple CNN based DL models and developed OpenCV based computer vision algorithms for extracting features and machine learning classification algorithm for RB classification based on the extracted features. With the use of DL and OpenCV algorithm, the AI model achieved a sensitivity of 96% and specificity of 94% for detection of RB.

ICIoR classification is based on quantifiable data such as size of tumor (< or > 3 mm), distance of tumor from optic disc and fovea, distance of tumor seeds from the tumor, and number of seeds.13 Using the DL models, this data could be extracted from the fundus images. This allowed classification of intraocular RB with the AI model which was based on fundus images and the quantitative data. The AI model achieved sensitivity of 63% to 100% and specificity of 91% to 100% for classification of RB based on ICIoR.

The limitations of the study include relatively smaller sample size and unequal distribution of RB in different groups based on ICIoR. Also, we have not tested this model on fundus images of different races or on fundus images captured on different cameras. Thus the applicability of this AI model in these above circumstances is unknown. However, this is the first AI model developed for detection and grouping of RB and there is scope for improving the accuracy of the model with use of more DL algorithms.

In conclusion, this AI tool is a promising screening tool for RB. It has shown high sensitivity and specificity for detection of RB, though the sensitivity and specificity is variable for grouping of intraocular RB. Based on our study, this AI model performs well on images captured on wide-field fundus camera. If this AI model can be used on fundus images captured on an inexpensive non-mydriatic fundus camera, it can facilitate mass screening of children in the community, so as to diagnose RB early before the onset of leukocoria or strabismus.

References:

1. Turing AM. Computing machinery and intelligence. Mind 1950; 59(236):433-460.
2. Hamet P, Tremblay J. Artificial intelligence in medicine. Metabolism 2017; 69S:S36-S40.
3. Luxton DD. Recommendations for the ethical use and design of artificial intelligent care providers. Artif Intell Med. 2014; 62(1):1-10.
4. Cornet G. Robot companions and ethics a pragmatic approach of ethical design. J Int Bioethique. 2013; 24(4):49-58, 179-180.
5. Thiébaut R, Thiessard F; Section Editors for the IMIA Yearbook Section on Public Health and Epidemiology Informatics. Artificial Intelligence in Public Health and Epidemiology. Yearb Med Inform. 2018; 27(1):207-210.
6. Alexander JC, Romito BT, Çobanoğlu MC. The present and future role of artificial intelligence and machine learning in anesthesiology. Int Anesthesiol Clin. 2020; 58(4):7-16.
7. Hajirasouliha I, Elemento O. Precision medicine and artificial intelligence: overview and relevance to reproductive medicine. Fertil Steril. 2020; 114(5):908-913.
8. Haq IU, Haq I, Xu B. Artificial intelligence in personalized cardiovascular medicine and cardiovascular imaging. Cardiovasc Diagn Ther. 2021; 11(3):911-923.
9. Jiang Y, Yang M, Wang S, Li X, Sun Y. Emerging role of deep learning-based artificial intelligence in tumor pathology. Cancer Commun (Lond). 2020; 40(4):154-166.
10. Seifert R, Weber M, Kocakavuk E, Rischpler C, Kersting D. Artificial Intelligence and Machine Learning in Nuclear Medicine: Future Perspectives. Semin Nucl Med. 2021; 51(2):170-177.
11. Wang L, Zhang Y, Wang D, et al. Artificial Intelligence for COVID-19: A Systematic Review. Front Med (Lausanne). 2021; 8:704256.
12. Shimizu H, Nakayama KI. Artificial intelligence in oncology. Cancer Sci. 2020; 111(5):1452-1460.
13. Shields CL, Mashayekhi A, Au AK, et al. The International Classification of Retinoblastoma predicts chemoreduction success. Ophthalmology 2006; 113(12):2276-2280.
14. Tan C, Sun F, Kong T, Zhang W, Yang C, Liu C. A Survey on Deep Transfer Learning. In: Kůrková V, Manolopoulos Y, Hammer B, Iliadis L, Maglogiannis I, editors. Artificial Neural Networks and Machine Learning – ICANN 2018. ICANN 2018. Lecture Notes in Computer Science: Springer, Cham; 2018. vol 11141. Pp 270-279.
15. Yu H, Chen C, Du X, et al. TensorFlow Model Garden. https://github.com/tensorflow/models. Accessed on February 9th 2022.
16. Dubey S, Mittal U, Singh R. Blood Vessel Segmentation and Microaneurysm Detection for Diabetic Retinopathy. https://github.com/getsanjeev/retina-features. Accessed on February 9th 2022.
17. Shaley-Shwartz S, Ben-David S. Decision Trees. In: Shaley-Shwartz S, Ben-David S, editors. Understanding Machine Learning: From Theory to Algorithms: Cambridge: Cambridge University Press;2014. Pp 212-218.
18. Fogel AL, Kvedar JC. Artificial intelligence powers digital medicine. NPJ Digit Med. 2018; 1:5.
19. Armstrong GW, Lorch AC. A(eye): A review of current applications of artificial intelligence and machine learning in ophthalmology. Int Ophthalmol Clin. 2020; 60(1):57-71.
20. Lin WC, Chen JS, Chiang MF, Hribar MR. Applications of artificial intelligence to electronic health record data in ophthalmology. Transl Vis Sci Technol. 2020; 9(2):13.
21. Benet D, Pellicer-Valero OJ. Artificial intelligence: the unstoppable revolution in ophthalmology. Surv Ophthalmol. 2022; 67(1):252-270.
22. Song X, Liu Z, Li L, et al. Artificial intelligence CT screening model for thyroid-associated ophthalmopathy and tests under clinical conditions. Int J Comput Assist Radiol Surg. 2021; 16(2):323-330.
23. Shields CL, Lally SE, Dalvin LA, et al. White paper on ophthalmic imaging for choroidal nevus identification and transformation into melanoma. Transl Vis Sci Technol. 2021; 10(2):24.
24. Global Retinoblastoma Study Group, Fabian ID, Abdallah E, Abdullahi SU, et al. Global retinoblastoma presentation and analysis by national income level. JAMA Oncol. 2020; 6(5):685-695.
25. Kaliki S, Ji X, Zou Y, et al. Lag Time between Onset of First Symptom and Treatment of Retinoblastoma: An International Collaborative Study of 692 Patients from 10 Countries. Cancers (Basel). 2021; 13(8):1956.
26. Kaliki S, Patel A, Iram S, Ramappa G, Mohamed A, Palkonda VAR. Retinoblastoma in India: Clinical presentation and outcome in 1,457 patients (2,074 Eyes). Retina 2019; 39(2):379-391.
27. Vempuluru VS, Kaliki S. Screening for retinoblastoma: A systematic review of current strategies. Asia Pac J Ophthalmol (Phila). 2021; 10(2):192-199.
28. Gulshan V, Peng L, Coram M, et al. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA. 2016; 316(22):2402-2410.
29. Ting DSW, Cheung CY, Lim G, et al. Development and validation of a deep learning system for diabetic retinopathy and related eye diseases using retinal images from multiethnic populations with diabetes. JAMA. 2017; 318(22):2211-2223.
30. Tufail A, Rudisill C, Egan C, et al. Automated diabetic retinopathy image assessment software: Diagnostic accuracy and cost-effectiveness compared with human graders. Ophthalmology 2017; 124(3):343-351.
31. Burlina PM, Joshi N, Pekala M, Pacheco KD, Freund DE, Bressler NM. Automated grading of Age-Related Macular Degeneration from color fundus images using deep convolutional neural networks. JAMA Ophthalmol. 2017; 135(11):1170-1176.
32. Campbell JP, Singh P, Redd TK, et al. Applications of artificial intelligence for retinopathy of prematurity screening. Pediatrics 2021; 147(3):e2020016618.
33. Scruggs BA, Chan RVP, Kalpathy-Cramer J, Chiang MF, Campbell JP. Artificial intelligence in retinopathy of prematurity diagnosis. Transl Vis Sci Technol. 2020; 9(2):5.
34. Jeon S, Liu Y, Li JO, Webster D, Peng L, Ting D. AI papers in ophthalmology made simple. Eye (Lond) 2020; 34(11):1947-1949.

Table 1: Group labels assigned to of 771 images from 109 eyes by observers*

ICIoR Group	Images	Eyes
Normal	181	31
A	6	1
B	204	22
C	70	8
D	182	23
E	128	24
Total	771	109

*Interobserver variability between the two observers was 0.98% for images and 0% for eyes; ICIoR: International Classification for Intraocular Retinoblastoma

Table 2: Performance metrics of RB classifier

	Detection of RB	Normal	A	B	C	D	E
For individual images
Sensitivity	85%	99%	17%	79%	84%	73%	60%
Specificity	99%	85%	100%	96%	97%	97%	99%
PPV	100%	67%	100%	87%	73%	89%	90%
NPV	67%	100%	95%	93%	98%	92%	100%
For all images of an eye
Sensitivity	96%	94%	100%	82%	63%	78%	92%
Specificity	94%	96%	100%	98%	99%	98%	91%
PPV	97%	91%	100%	90%	83%	90%	73%
NPV	91%	98%	100%	96%	97%	94%	98%

RB: retinoblastoma; PPV: positive predictive value; NPV: negative predictive value

Figure 1: Overview of methodology employed in training and testing of retinoblastoma classifier

Figure 2: Algorithms used for identification features from fundus images

(A) & (B): Bounding boxes with confidence score for optic disc and tumor

(C) & (D): Calculation of tumor size with with reference to optic disc and hough circle respectively

(E) & (F): Identification of regions of interest (tumor seeds) and vitreous seeds within mapped areas of interest by OpenCV algorithm and Deep Learning model

(G) & (H): Mapping of hemorrhage overlying the tumor using by combination of OpenCV and Deep Learning methods

(I) to (J): Mapping of blood vessels overlying the tumor by combination of OpenCV and Deep Learning methods

Figure 3: Performance metrics of retinoblastoma (RB) classifier for detection and classification of retinoblastoma

(A) & (B): Grids depicting labels assigned by observers versus labels assigned by RB classifier for 771 images and 109 eyes respectively

(C) & (D): Graphical summary of accurate, over-classification and under-classification by RB classifier in 771 images and 109 eyes respectively