Innovation in neuro-ophthalmology: The validation of smartphone-based self-screening for early detection of disorders

Abstract

PURPOSE:

This diagnosis study evaluated the diagnostic validity of Neuro-ophthalmology Disorder Early Screening (NODES), a telemedicine-based smartphone application that integrates structured anamnesis and simple neuro-ophthalmology examination for early detection of neuro-ophthalmological disorders.

MATERIALS AND METHODS:

This study was conducted at Sardjito Eye Center Neuro-ophthalmology subdivision, assessing the diagnostic validity of the NODES mobile application of 16 neuro-ophthalmology diagnoses compared to confirmative diagnoses of neuro-ophthalmologists. A total of 126 participants meeting inclusion criteria underwent two treatments: simple examination using the NODES application, compared to confirmative examination by neuro-ophthalmologists. Standard protocols were integrated into both treatments to evaluate the diagnosis accuracy of the examination.

RESULTS:

NODES showcase promising results as a new diagnostic tool for these four disorders; myasthenia gravis (MG) with the area under the curve (AUC) =0.951, multicranial nerve palsy orbital apex (MNP OA) with AUC = 0.900, monocranial nerve palsy N.III (MNP N.III) with AUC = 0.824, and monocranial sixth nerve palsy (MNP N.VI) with AUC = 0.764.

CONCLUSION:

The NODES application can be used as a new diagnostic screening tool for several euro-ophthalmology diagnoses including MG, multicranial nerve palsy orbital apex, monocranial nerve palsy N.III, and monocranial sixth nerve palsy. The high specificity of NODES in other neuro-ophthalmology diagnoses can be quite useful to identify individuals without the condition, reducing the occurrence of false positives and crucial in preventing unnecessary stress, further testing, and potential misdiagnoses for those who do not actually have the condition.

Introduction

In the field of ophthalmology, the use of telemedicine has been conducted, especially screening in remote areas, for example, fundus photographs with program analysis to refer patients with eye diseases such as diabetic retinopathy, premature retinopathy, glaucoma, and neuro-ophthalmology.[1]

Incidence of neuro-ophthalmology disease has increased over time, with data showing the incidence increases with age, with a similar ratio of gender incidence among men and women.[2] Indonesia with the sixth largest population in the world as well as a large geographical area consisting of various islands, makes the problem of equitable distribution of health services one of the crucial issues in Indonesia.[3] The current era of the COVID-19 pandemic has caused more difficult patient access to health services.[4] The development of smartphone-based applications is relevant in developing countries, especially Indonesia with a large population, large geographical area, and limited access to health services.[5]

There have been several previous studies on the use of applications as a medium for neuro-ophthalmology examination such as the V home application, ETDRS vision examination,[6] and Phase I Pilot Survey of Smart Optometry accessibility, but there have been no studies that combine structured history algorithms and simple ophthalmology examinations into neuro-ophthalmology disorder early screening detection application tools.[7,8,9]

Neuro-ophthalmology Disorder Early Screening (NODES) was expected to be the pioneer of smartphone-based self-screening tool for neuro-ophthalmology disorder early detection and able to provide access to information exchange quickly and accurately because, in the management of neuro-ophthalmology, the onset of the disease is particularly important for visual prognosis and adequate management.[10] This experimental pilot study evaluates the diagnostic validity of Neuro-ophthalmology Disorder Early Screening (NODES), a telemedicine-based smartphone application that integrates structured anamnesis and simple neuro-ophthalmology examination for early detection of neuro-ophthalmological disorders.

Materials and Methods

Design and subjects

This was an experimental pilot randomized controlled study that investigated 126 patients at Sardjito Eye Center Neuro-ophthalmology subdivision. This study was performed following the stipulation of Helsinki and was approved by the ethical institutional review committee (ID: KE/FK/0215/EC/2023). The subjects were recruited by consecutive sampling using Slovin methods with a formula of n = N/(1+ [N × e²]) with N = population, n = sample, and e = margin of error 5%. The data collected from medical records showed an average population of 163 patients per month, and then, we calculated the minimal sample needed for this study was 116 patients. The study was conducted on patients who visited neuro-ophthalmology clinic from March to August 2023 and voluntarily participated after given informed consent. The patients consisted of one group, receiving two treatments, in the first treatment, patients were examined with a structured history algorithm and a simple examination with the guideline of using a smartphone-based early screening application (NODES) and the second treatment was anamnesis, and confirmative examination by neuro-ophthalmologists. Both patients and examiners were blinded by the results from each examination.

Inclusion criteria are patients of the Neuro-ophthalmology subdivision in Sardjito Eye Center willing to participate after informed consent. Exclusion criteria are patients with diagnoses outside of 16 NODES diagnoses: nonarteritic anterior ischemic optic neuropathy (NAION), giant cell arteritis (GCA), compressive optic neuropathy (CON), infiltrative optic neuropathy (ION), toxic optic neuropathy (TON), optic nerve atrophy (ONA), optic neuritis (ON), monocranial nerve Palsy; N.III, N.IV, N.VI (MNP N.III, N.IV, N.VI), multicranial nerve palsy; orbital apex and sinus cavernous (MNP OA and MNP SC), myasthenia gravis (MG), Horner’s Syndrome (HS), Adie’s tonic pupil (ATP), and brain vascular abnormalities (BVAs), and incomplete data.

Standard protocols

Data from all eligible patients were collected from anamnesis and visual examination. Informed consent is taken after the interviewer explains to patients the purpose and benefit of this study. If the criterion of this study is met, patients will be examined by an independent examiner with the NODES application. Before analysis, the independent examiner who has a medical student graduate qualification had been trained by neuro-ophthalmologists to do the standardized simple neuro-ophthalmology examination.

To avoid bias, the diagnoses that are made by the NODES application will not be told to the neuro-ophthalmologists and we provided questionnaires with the same anamnesis points to neuro-ophthalmologists. Diagnoses that are made by the neuro-ophthalmologists were based on the hospital’s standard protocol anamnesis and neuro-ophthalmology examination including slit-lamp examination, funduscopy, optical coherence tomography, magnetic resonance imaging, and Humphrey visual field analyzer.

Neuro-ophthalmology disorder early screening (NODES)

Neuro-Ophthalmology Disease Early Screening (NODES) is a telemedicine-based smartphone application developed by the Department of Ophthalmology, Faculty of Medicine, Public Health, and Nursing, Gadjah Mada University Yogyakarta. Most common complaints of neuro-ophthalmology are usually related to vision loss, visual field defects, diplopia, ptosis, pupil anisocoria, pain, and photophobia.[11,12,13,14,15] The neuro-ophthalmologists in Sardjito Eye Center brainstormed and searched for a literature review to build anamnesis and examination algorithms used for NODES. The NODES application integrates a structured anamnesis algorithm that includes several important questions related to the early diagnosis of neuro-ophthalmology disorder and simple neuro-ophthalmology examination built into the application such as ptosis evaluation, pupil response, visual acuity, color vision, visual field test, and nine gaze.

Anamnesis point and simple neuro-ophthalmology done by the interviewer later put into the NODES application to calculate the possibility of each diagnosis with the highest point of NODES calculation will be taken as the main diagnosis, as shown in Figure 1.

Figure 1.

(a) Neuro-ophthalmology disorder early screening (NODES) interface (b and c) simple neuro-ophthalmology examination, including Pemeriksaan Visus (Visual Acuity Examination), Pemeriksaan Refleks Pupil (Pupil Reflex Examination), Posisi Kelopak Mata (Ptosis Examination), Pemeriksaan Gerakan Bola Mata (Extraocular Muscle Examination), Pemeriksaan Sensitivitas Warna (Colour Sensitivity Test), and Pemeriksaan Lapang Pandang (Visual Field Examination). (d) diagnosis results from NODES * application is currently available in Indonesian language only

Statistical analyses

Statistical analysis was done in a descriptive and analytical method. Variables analyzed in this study were diagnosis results from the NODES mobile application compared to confirmative diagnoses from neuro-ophthalmologists. Data were analyzed for sensitivity, specificity, accuracy, receiver operating characteristic (ROC) curve, and area under the curve (AUC).

Results

A total of 126 patients participated in this study, with a mean age of 41.30 ± 15.92 (range: 9–79 years old). The distribution of gender among the participants was quite balanced, with 76 female participants (60.3%), and 50 male participants (39.7%). The data collection period spanned over 6 months and was conducted within the neuro-ophthalmology subdivision of the Sardjito Eye Center in Yogyakarta. The results of the diagnostic study showcasing the sensitivity, specificity, and accuracy of various conditions evaluated are presented in Table 1.

Table 1.

Diagnostic evaluation results

Diagnosis	NODES	Neuro-ophthalmologist	Sensitivity (%)	Specificity (%)	Accuracy (%)	CI 95%
MG	3	3	100.00	100.00	100.00	97.11%–100.00%
MNP N. III	5	4	100.00	99.18	99.96	97.00%–100.00%
MNP OA	8	1	100.00	94.40	99.72	96.57%–100.00%
ONA	13	11	81.82	96.52	82.55	74.78%–88.73%
CON	20	25	72.00	98.02	73.30	64.69%–80.79%
ON	35	40	70.00	91.86	71.09	62.35%–78.82%
NAION	22	22	63.64	92.31	65.07	56.07%–73.34%
BVA	6	8	50.00	98.31	52.42	43.33%–61.38%
MNP N. VI	4	7	42.86	99.16	45.67	36.77%–54.78%
TON	10	5	40.00	93.39	42.67	33.90%–51.79%
GCA	-	-	N/A	N/A	N/A	N/A
ION	-	-	N/A	N/A	N/A	N/A
MNP N. IV	-	-	N/A	N/A	N/A	N/A
MNP SC	-	-	N/A	N/A	N/A	N/A
HS	-	-	N/A	N/A	N/A	N/A
ATP	-	-	N/A	N/A	N/A	N/A

NODES=Neuro-ophthalmology disorder early screening, CI=Confidence interval, MG=Myasthenia gravis, MNP N.III=Monocranial nerve third palsy, MNP OA=Multicranial nerve palsy orbital apex, ONA=Optic nerve atrophy, CON=Compressive optic neuropathy, ON=Optic neuritis, NAION=Nonarteritic anterior ischemic optic neuropathy, BVA=Brain vascular abnormality, MNP N. VI=Monocranial sixth nerve palsy, TON=Toxic optic neuropathy, GCA=Giant cell arteritis, ION=Infiltrative optic neuropathy, MNP N. IV=Monocranial fourth nerve palsy, MNP SC=Multicranial nerve palsy sinus cavernous, HS=Horner’s syndrome, ATP=Adie’s tonic pupil, N/A=Not available

The results of sensitivity and specificity of these disorders showcase promising results; MG with 100% both sensitivity and specificity, monocranial nerve palsy N.III (MNP N.III) with 100% and 99.18%, multicranial nerve palsy orbital apex (MNP OA) with 100% and 94.40%, ONA with 81.82% and 96.52%, CON 72.00% and 98.02%, and ON 70.00% and 91.86% for sensitivity and specificity, respectively. The lower sensitivity and specificity rate for other disorders are reported as follows: NAION with 63.64% and 92.31%, BVA with 50.00% and 98.31%, monocranial nerve palsy N.VI (MNP N.VI) with 42.86% and 99.16%, and at last TON with 40.00% and 93.39% for sensitivity and specificity, respectively, and we have no data in six of the diagnoses including GCAs, ION, monocranial fourth nerve palsy (MNP N. IV), multicranial nerve palsy SC (MNP SC), HS, and ATP.

We observed a discrepancy between the NODES group and the neuro-ophthalmologist group in the diagnosis of MNP OA despite NODES demonstrating high sensitivity, specificity, and accuracy. This difference can be attributed to NODES generating seven false positives but no false negatives. This reflects a cautious diagnostic strategy that prioritizes avoiding missed cases by leaning toward overdiagnosis, thereby enhancing sensitivity. The high sensitivity and specificity values indicate that NODES effectively identifies true positives and accurately excludes true negatives in the diagnostic process.

We also analyzed the receiver operating characteristic (ROC) in this study. The ROC curve is a graphical representation that illustrates the performance of a binary classification model or diagnostic test. In this study can see the variation of the ROC curve of significant diagnosis in Figure 2 with the best AUC in MG (AUC = 0.951, P = 0.008), followed by multicranial nerve palsy orbital apex (AUC = 0.900, P = 0.016), monocranial third nerve palsy (AUC = 0.824, P = 0.028), and least acceptable AUC is in monocranial sixth nerve palsy (AUC = 0.764, P = 0.019).

Figure 2.

Receiver of characteristics (ROC) Curves of (a) Myasthenia Gravis (b) Multicranial Nerve Palsy Orbital Apex (MNP OA) (c) Monocranial Third Nerve Palsy (MNP N.III) (d) Monocranial Sixth Nerve Palsy (MNP N.VI)

Discussion

The AUC of the NODES self-screening tool for MG and multicranial nerve palsy orbital apex showed excellent results of 0.951 and 0.900, respectively. This indicated that the diagnostic marker or clinical presentation being evaluated has a remarkably high discriminatory ability in identifying cases of those two specific diagnoses. The AUC 0.824 of monocranial third nerve palsy showed good results and AUC 0.764 of monocranial sixth nerve palsy showed fair results to become the new screening diagnostic tools for neuro-ophthalmology disorders.[16]

In this study, the sensitivity and specificity of nerve palsy disorders such as MG, monocranial nerve palsy N.III (MNP N.III), nd multicranial nerve palsy orbital apex (MNP OA) showed higher results since these disorders mainly affect extraocular motility and easily identified in NODES simple neuro-ophthalmology examinations.

However, disorders involving optic nerve such as ON, NAION, TON, and ONA can indeed present similar symptoms. This similarity in clinical presentation can make it challenging to distinguish between these disorders based solely on symptoms and NODES simple neuro-ophthalmology examinations. The difficulty in distinguishing disorders involving the optic disc arises from the shared anatomical and physiological features of the optic nerve, the susceptibility of the optic disc to various pathologies, and the interconnected nature of visual processing in the brain. Despite differences in underlying etiology, both conditions can manifest with similar symptoms due to their impact on optic nerve function. Any pathology affecting the optic nerve or its adjacent structures including BVA can disrupt optic nerve transmission, leading to common symptoms such as visual acuity loss and visual field defects.[17]

A previous study compared the accuracy of neurology residents versus a mobile medical app (Neurology Dx) in drawing a comprehensive list of differential diagnoses from neuro-ophthalmology clinical vignettes and resulted in the App scored a frequency of 41.2% in identifying the first gold standard differential correctly.[18]

The distribution pattern of diagnosis from the NODES application reveals intriguing insights into the prevalence of various ocular health. Diagnoses such as ON and NAION appear to be of particular significance due to their higher counts n = 35 (27.8%) and n = 22 (17.5%), respectively. Similarly, CON, ONA, and TON demonstrate notable prevalence within the dataset, suggesting their clinical relevance. In contrast, categories represented by only a few instances, such as BVA, monocranial nerve palsy nerve III (MNP N.III), monocranial nerve palsy nerve VI (MNP N.VI), multicranial nerve palsy orbital apex (MNP OA), MG, and no diagnosis of GCA, ION, mononuclear nerve palsy nerve IV (MNP N.IV), HS, and ATP, could be indicative of rare neuro-ophthalmology conditions with less prevalence.[19]

The development of the NODES application involved an integrated team comprising neuro-ophthalmologists, general ophthalmologists, general practitioners, and IT specialists. The application underwent field testing in various healthcare settings in Indonesia, including primary, secondary, and tertiary care facilities. This rigorous process ensured the application’s ease of use for general practitioners and its practical applicability and generalizability in patient care.

The limitations were an insufficient number of cases for certain diagnoses and the current inability of the NODES application to provide emergency parameters and treatment plans for general practitioners. However, the application’s algorithmic features, which generate quantified diagnostic results and analyze cutoff values to identify emergencies requiring referral, show potential for enhancing its utility. Future studies will focus on addressing data gaps, expanding the study population, and further exploring these features to optimize the application’s clinical performance.

Conclusion

The NODES application can be used as new diagnostic screening tools for several neuro-ophthalmology diagnoses including MG, multicranial nerve palsy orbital apex, monocranial nerve palsy N.III, and monocranial sixth nerve palsy. The high specificity of NODES in other neuro-ophthalmology diagnoses can be quite useful to identify individuals without the condition, reducing the occurrence of false positives and crucial in preventing unnecessary stress, further testing, and potential misdiagnoses for those who do not actually have the condition. NODES offers a solution for efficient patient screening, enhancing early detection and intervention strategies. Further research and validation are warranted to assess the smartphone-based application’s performance comprehensively. This includes optimizing a quantified algorithm to identify emergence parameters and referral plans, potentially refining the application based on user feedback and technological advancements, and further testing in a broader population to fill the data gaps.

Data availability statement

The datasets generated during and/or analyzed during the current study are not publicly available, but are available from the corresponding author on reasonable request.

Conflicts of interest

The authors declare that there are no conflicts of interests of this paper.

Funding Statement

Dana Masyarakat Funding Grant 2023 From the Faculty of Medicine, Public Health, and Nursing, Universitas Gadjah Mada.