Application and validation of a novel inflammatory score in the clinical grading of infectious endophthalmitis: The endophthalmitis management study – Report 2

Abstract

Purpose:

This study was conducted to describe and validate a novel inflammatory score (IS) system in the management of infectious endophthalmitis.

Methods:

This was a prospective comparative non-interventional observational study. The study included the patients with clinical signs and symptoms of acute post-cataract surgery endophthalmitis (surgery within 6 weeks) with visual acuity from 6/18 to light perception. IS was scored by the clinical picture at two-levels of four ocular tissues on a scale of 0 (normal) to 4 (severe). Four masked graders of different levels of experience evaluated slit-lamp photographs. The concordance correlation coefficient was assessed between the slit-lamp clinical grading and photographic grading. We measured the concordance correlation coefficient, Pearson’s correlation (indicating precision), and the bias correction factor (indicating the accuracy).

Results:

The study included 43 eyes of 43 patients. The concordance correlation coefficient was 0.99 (95% CI 0.995 to 0.998). Both Pearson’s correlation coefficient and the bias correction were 0.99. The interclass correlation coefficient (ICC) was measured. The intra-rater ICC was 0.833 with good agreement (95% CI, 0.711 to 0.906; P < 0.001). Inter-rater ICC for consistency was 0.92 (95% CI 0.87 to 0.95). Inter-rater ICC for absolute agreement was 0.86 (95% CI 0.66 to 0.93).

Conclusion:

Currently used IS scoring in the study is a reliable, reproducible, and easy-to-apply scale to measure inflammation severity in endophthalmitis. We propose that it can have applications in decision-making for primary treatment and monitoring progression in acute infectious endophthalmitis.

Endophthalmitis is a severe vision-threatening infection affecting the eye’s inner coats with exudation in the vitreous cavity.[1,2] In the initial stages or in mild infections, there is a relatively well-preserved media clarity, allowing a good red reflex, occasionally even allowing a reasonable evaluation of the retinal details.[3] Infection causes inflammation in the vitreous cavity. This leads to the development of a vitreous haze due to the exudation of inflammatory cells and proteins.[4] Endophthalmitis can occur with concurrent keratitis, which also contributes to the additional reduction in media clarity.[5] The quantum of medical and surgical intervention and the duration of therapy in endophthalmitis is based on the approximated estimation of inflammation as a surrogate measure of infection. Thus, quantifying the inflammation essentially guides the clinician in deciding the primary treatment strategy.

Quantifying inflammation is not new. It has been used in uveitis for several decades such as the Standardization of Uveitis Nomenclature (SUN)[6] working group for grading inflammation in anterior and vitreous chambers and the National Institute of Health (NIH) classification.[7] Earlier attempts were made to quantify inflammation in experimental (rabbit) endophthalmitis.[8–10] In human endophthalmitis, Meredith et al.[10] had used a modified version of this grading earlier. We had used it in a clinical prospective randomized trial to evaluate the response to intravitreal dexamethasone in exogenous endophthalmitis.[11] These studies used a direct examination of the eyes, a stereoscopic view with slit-lamp or binocular indirect ophthalmoscope. But one also needs a two-dimensional slit-lamp photographic grading of the eyes for remote consultation.

The aim of the current study was to correlate the inflammatory score generated on the slit-lamp photograph with a biomicroscopic examination of the eye on the day of presentation using our earlier grading system.[11] We tested the inflammatory score (IS) for agreement across multiple observers and between clinical evaluation and photographic documentation.

Methods

The current study was a prospective, comparative, non-interventional, observational study and was part of the larger prospective randomized trial called the Endophthalmitis Management Study (EMS). The Institutional Review Board had approved this study, and it is registered in the CTRI (Clinical Trial Registry of India - # 2019/02/017876). The patient or the parents or guardians of the patient filled out a standard consent form for electronic data privacy at the time of registration. None of the identifiable parameters of the patient were used for analysis of the data. The clinical data of each patient who underwent a comprehensive ophthalmic examination was entered into a browser-based electronic medical records system (eyeSmart EMR) by uniformly trained ophthalmic personnel and supervised by an ophthalmologist using a standardized template. The study adhered to all tenets of the Declaration of Helsinki.

It included patients with clinical signs and symptoms of acute postoperative endophthalmitis with visual acuity ranging from 6/18 to light perception (LP) and who had given their consent to participate in the study. Acute postoperative endophthalmitis was defined as endophthalmitis occurring within 6 weeks of cataract surgery. Exclusion criteria were people younger than 18 years, single-eyed patients, retinal detachment at presentation detected by retinal examination or ultrasonography, unfit for surgery, not willing for surgery, and not giving consent for surgery. A detailed baseline ocular examination included presenting visual acuity, applanation tonometry, slit-lamp examination of the anterior segment, and fundus examination using an indirect ophthalmoscope with the highest illumination. We used a quantitative endophthalmitis scoring scale [Table 1][11] for the IS. The methodology followed in the current study was as per the recently published EMS protocol.[12]

Table 1.

Inflammatory score in endophthalmitis consisting of objective points for different structures

Tissue	Response	Points
		0	1	2	3	4
Cornea	Clarity	Clear	Mild	Moderate (iris visible)	Severe (iris bare details)	Opaque (no iris view)
	Abscess	None	<1 mm	1-2 mm	3-4 mm	>5 mm
Anterior chamber	Flare/Cells	None	Trace	Mild	Moderate	Severe
	Fibrin/Hypopyon	None	Mild <25%	Moderate >25%	Severe <75%	No iris view
Iris	Blood vessels dilated	None	Mild	Moderate	Severe	NVI
	Exudates over iris	None	Mild <25%	Moderate <50%	Severe <75%	Pupil occluded
Vitreous	Flare	None	Trace	Mild	Moderate	Severe
	Opacities	None	Cells	Clumps	Red reflex	Opaque

Additional scoring: Cornea opaque, add 20; AC opaque, add 15; Pupil fully covered with exudates, add 10; Vitreous opaque, add 5.

The photo-documentation included a slit-lamp photograph (Haag-Streit BX-900), diffuse and slit-view, in all eyes, fundus photographs (Zeiss FF 450 IR), where possible, and B-scan ultrasound (10 MHz probe; Accutome) in all eyes. These photographs were taken at presentation and every follow-up visit as per the EMS protocol (EMS report # 1 in press). The slit-lamp photographs were standardized [Table 2].

Table 2.

Standardized slit-lamp photographic settings used in the current study

	Cornea	Anterior Chamber	Iris	Anterior Vitreous
Sensor rating (ISO)	200	200	200	200
Flash intensity	High	High	High	High
Background	100%	0%-25%	0%-10%	0%-10%
Angle	30°-45°	30°	45°	45°
Slit beam	Fully open	0.1-1 mm	Fully open	0.1-1 mm
Filter	Diffused	-	-	-
Magnification	25×	25×	25×	25×
Aperture setting	3	1	4	1

The IS has been described in detail previously.[11] In brief, it is based on the clinical features of four ocular tissues and two levels on a scale of 0 to 4. The ocular tissues and features are cornea (clarity and abscess), anterior chamber (flare/cells; fibrin/hypopyon), iris (dilated blood vessels and exudates over iris), and the vitreous (flare and opacities). There is a provision of adding supplemental IS (5–20) if any ocular tissue does not allow further examination [Table 1].

In this study, the sequence of IS evaluation was as follows:

1. Baseline: The treating faculty (fellowship-trained retina specialist) of the institute made the baseline evaluation, and the score was entered into the electronic medical record (EMR) of the patient. Slit-lamp photograph (slit and diffuse) of the eye was obtained for all patients and the fundus photograph, where possible.

2. Photo grading: Four masked graders evaluated the slit-lamp photographs and fundus photos of patients with different levels of experience in retina services.

   • Grader A: Vitreoretinal faculty; more than 5 years of experience;
   • Grader B: Vitreoretinal faculty; less than 5 years of experience (documented twice);
   • Grader C: Vitreoretinal fellow-in-training; 1.5 years of experience in fellowship;
   • Grader D: Vitreoretinal fellow-in-training; less than 6 months of experience.

All IS scoring was done at two-timepoints: at presentation and one week after treatment. The treatment of the individual patient was as per the approved EMS protocol. Briefly, it was primary vitreous tap + intravitreal antibiotics or core vitrectomy + intravitreal antibiotics.

We measured the intraclass correlation coefficient. The photo-graded IS was compared with the one recorded first on slit-lamp biomicroscopic examination. We measured interobserver variation and agreement. Besides, two gradings of grader B, documented a week after the first IS scoring and masked to the first one, were compared to check intra-rater reliability. The mean score of the multiple graders was averaged and correlated with the baseline score noted in the clinic on the first examination.

Statistical analysis

The IS data was arranged on a Microsoft Excel spreadsheet and analyzed using MedCalc version 19.4.1 (Ostend, Belgium). A concordance correlation coefficient measured the concordance between the slit-lamp clinical grading and the photographic grading. We also measured Pearson’s correlation (a measure of precision) and the bias correction factor (a measure of accuracy). Pearson’s correlation measures the extent to which each observation deviates from the best-fit line. The bias correction factor indicates the extent to which the best-fit line deviates from the 45° line. The interobserver intraclass correlation coefficient was calculated to assess “reliability,” that is, reproducibility between graders—both “consistency” and the “absolute agreement” of the readings. Values < 0.5 indicated poor reliability, values between 0.5 and 0.75 indicated moderate reliability, those between 0.9 and 0.75 indicated good reliability, and values >0.9 indicated excellent reliability.

Results

The study included 43 eyes of 43 patients with acute postoperative endophthalmitis from within the EMS cohort. All of these eyes had received cataract surgery with intraocular lens implantation. There were 20 males, and the average age of the cohort was 60.88 ± 9.09 years (median 61 years). The inflammatory score range was 5 to 24. Examples of IS grading of the slit-lamp photo are shown in Fig. 1.

Figure 1.

Slit-lamp photos of two cases of endophthalmitis with an IS of 7 (a) (calculated as corneal clarity - 2, anterior chamber fibrin - 2, vitreous opacities - 3) and 24 (b) (calculated as cornea - 4, additional points - 20)

The intra-rater agreement was calculated between the two readings of a single grader—grader B, in this study. The ICC intra-rater correlation coefficient was 0.833 (95% Confidence Interval, CI, 0.711 - 0.906), P < 0.001). The concordance correlation coefficient was 0.99 (95% CI 0.995–0.998). Both Pearson’s correlation coefficient and the bias correction factor were 0.99. The inter-rater ICC for consistency was 0.75 (95% C.I. 0.64–0.84) for single measures and 0.92 (95% CI 0.87– 0.95) for average measures. The inter-rater ICC for absolute agreement was 0.6 (95% CI 0.33–0.78) for single measures and 0.86 (95% CI 0.66–0.93) for average measures. The results showed that the graders correlated well between them and the baseline clinical grading [Table 3 and Fig. 2].

Table 3.

Correlation of the baseline clinical grading by four graders

The correlation coefficient, P	Baseline clinical grading	Grader A	Grader B	Grader C	Grader D
Baseline clinical grading	-	0.878, P<0.001	0.862, P<0.001	0.83, P<0.001	0.805, P<0.001
Grader A	0.878, P<0.001	-	0.806, P<0.001	0.749, P<0.001	0.732, P<0.001
Grader B	0.862, P<0.001	0.806, P<0.001	-	0.903, P<0.001	0.778, P<0.001
Grader C	0.83, P<0.001	0.749, P<0.001	0.903, P<0.001	-	0.749, P<0.001
Grader D	0.805, P<0.001	0.732, P<0.001	0.778, P<0.001	0.749, P<0.001	-

Figure 2.

Heat map showing the concordance between the IS noted clinically and that on slit-lamp photos

Discussion

Grading of any disease is important to understand its severity, monitor its progression, and for a quick cross-reference. Quantifying inflammation is necessary for many ophthalmic disorders. Quantification of non-infective uveitis is used in clinical practice.[6] Quantification of inflammation in infectious endophthalmitis, both experimental and in humans, has been used in earlier studies.[8–10] Management of a patient with acute endophthalmitis is an emergency. Typically, the patient presents only a few days after the symptoms (redness, pain, and reduced vision), and hence further delay in instituting appropriate therapy will adversely impact the outcome.

The selection of the first-line treatment in post-cataract surgery endophthalmitis is currently based on the presenting vision.[13] One needs an objective assessment beyond or in addition to the presenting vision for a particular choice of primary treatment of endophthalmitis. In real-world practice, it is rather impractical for many clinicians to examine the same patient to decide or tele-consult for decision-making. But it is possible for many clinicians to examine a standard slit-lamp photograph and make a final recommendation.

Measurement of an IS in endophthalmitis has been attempted in the past. The current study measured the reliability of grading using clinical photographs obtained through the slit-lamp camera. The study showed a high concordance coefficient between baseline slit-lamp grading and the average photographic grading between graders. The concordance correlation coefficient combines precision and accuracy measures to determine how far the observed data deviate from the line of perfect concordance. The high concordance between the baseline clinical grading and the mean grader scores suggests that the inflammation can be measured objectively with reproducible accuracy on high-quality slit-lamp photographs. This valuable inference augurs well for a possible application in tele-ophthalmic consultation. It assumes more importance when a long-distance and remote consultation is necessary. It is also important in the country and regions with a smaller number of retina specialists to the population and disease burden, such as India.[14]

The ICC is a widely used reliability index in test-retest, intra-rater, and inter-rater reliability analyses. Before any assessment tools can be used for research or clinical applications, establishing their reliability is paramount. “Reliability” is defined as the extent to which measurements can be replicated.[15] It reflects both the degree of correlation and the agreement between the measurements.[16,17] Reliability value ranges between 0 and 1, with values closer to 1 representing stronger reliability. Commonly, tests like Pearson’s correlation coefficient, paired t-test, and the Bland–Altman plot have been used to assess reliability.[18-21] The paired t-test and Bland–Altman plot are methods for analyzing agreement, while Pearson’s correlation coefficient is only a correlation measure. Hence, these are not ideal measures of reliability. A more desirable measure of reliability should reflect both degrees of correlation and agreement between measurements. ICC is such an index.

In the current study, the inter-rater ICC for an absolute agreement was good with excellent consistency. An absolute agreement would mean that two graders would score inflammation at a similar value. A consistent agreement would mean that the difference in scoring would be constant even though the absolute scoring could vary. The results in this study show that the current IS system is reproducible across graders for a primary estimation of baseline inflammation (absolute agreement) and is an even better tool for following up care in recording the change in the severity of inflammation with the dispensed treatment (consistency).

In this study, the baseline clinical grading of the IS was correlated to the scoring by four graders on high-resolution slit-lamp photos. The graders were of varying degrees of seniority. There was a trend of increasing the baseline grading with increasing seniority but the difference was negligible. Thus, we infer that the scale is easy to interpret and implement and does not require any specific level of seniority or experience. Vitreous flare is also a part of the IS grading. This was not assessed in the current study as this was a slit-lamp photo-based study in which it would not be possible to assess the vitreous flare. This is a small limitation of the study.

Conclusion

In conclusion, the novel IS system is a reliable and easy-to-apply scale for an appropriate diagnosis of the severity of inflammation in endophthalmitis. The ease of applicability and reproducibility may allow the role of the same in tele-ophthalmic consultation.

Ethics approval and consent to participate

Ethics approval was taken for this study and consent to participate was taken from all patients. The Institutional Review Board of L. V. Prasad Eye Institute had approved this study, and it is registered in the CTRI (Clinical Trial Registry of India - # 2019/02/017876).

Authors’ contributions

VPD, AD conceptualized, collected data, analyzed and wrote the manuscript. TD analyzed data and corrected the manuscript. All the other authors proofread and gave critical inputs to the manuscript.

Financial support and sponsorship

This study was funded by the Hyderabad Eye Research Foundation.

Conflicts of interest

There are no conflicts of interest.