Modeling and mitigating human annotations to design processing systems with human-in-the-loop machine learning for glaucomatous defects: The future in artificial intelligence

Dear Editor,

The field of glaucoma diagnosis is rapidly moving towards artificial intelligence (AI) for improving and enhancing patient care.[1,2,3,4,5] For any AI algorithm to deliver successfully, the basis starts with data annotation.[6] This is where the human interface comes into play. Data annotation is the core ingredient to the success of any AI model. Take, for example, the facial recognition AI; the only way for the image detection AI tool to detect a face in a photo is, if many photos are labeled as “face” exist within the software. If there are no annotated data, then there is no machine learning algorithm in the first place, to detect the image. Annotating data also known as labeling data is the first and most important step in creating a successful AI model.[6] One such image annotating tool, which can be utilized by all, is Microsoft Visual Object Tagging Tool (VoTT) for a comprehensive and customized data labeling [Fig. 1].[7] Customized annotations of the optic nerve head and retinal nerve fiber layer (RNFL) images [Figs. 2-5] can prove useful, in not only identifying glaucomatous discs but also in predicting segmentation of the glaucomatous cup, disc, peripapillary atrophy, and RNFL defect in the background fundus separately, which has never been reported in the literature before, according to our knowledge. This methodology of annotations, though time-consuming, can be utilized by all ophthalmologists to create their own Human-in-the-loop (HITL) AI model.

Figure 1.

The annotation toolbox with tools (green arrow) utilized for labeling the dataset in the Visual Object Tagging Tool (VoTT) software comprising the rectangle tool (red arrow) and polygonal tool (yellow arrow) for various types of labeling

Figure 2.

Sample fundus photograph of an eye with glaucomatous cupping utilized for annotating

Figure 5.

Annotation of the dataset. (a) Customized labeling of the optic cup (green-dotted area). (b) Customized labeling of the optic disc (pink-dotted area). (c) Customized labeling of peri-papillary atrophy (gray-dotted area). (d) Customized annotation of the retinal nerve fiber layer (RNFL) slit defect (blue-dotted area). (e) Customized annotation of the RNFL arcuate defect (gray-dotted area). (f) Complete annotation of an eye with glaucomatous changes in the optic nerve head and RNFL region

Figure 3.

Annotation of the dataset. (a) Customized labeling of the optic cup (red-dotted area). (b) Customized labeling of the optic disc (pink-dotted area). (c) Customized labeling of peripapillary atrophy (gray-dotted area). (d) Complete annotation of an eye with glaucomatous changes in the optic nerve head

Figure 4.

Sample fundus photograph of an eye with glaucomatous cupping and retinal nerve fiber layer defect utilized for annotating

HITL is the process of leveraging the power of machines and human intelligence to create AI models, where humans annotate data. In this loop, with humans help, the machine becomes smarter to take quick and accurate decisions. Customized human-led data annotation process can pave the way for future in AI, where the pairing of humans and machines takes place to yield better results, and not to establish the supremacy of one over the other.

Declaration of patient consent

The authors certify that they have obtained all appropriate patient consent forms. In the form, the patient(s) has/have given his/her/their consent for his/her/their images and other clinical information to be reported in the journal. The patient(s) understand that his/her/their name(s) and initials will not be published and due efforts will be made to conceal their identity, but anonymity cannot be guaranteed.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.