Novel electroretinography devices to detect hydroxychloroquine retinopathy: study protocol for a diagnostic accuracy and feasibility study

Abstract

Introduction

Annual screening for hydroxychloroquine (HCQ) retinopathy is recommended, and electroretinography (ERG) is considered a gold-standard test, but there are screening shortfalls and standard ERG is burdensome and has limited availability. Newer, portable ERG devices using skin-based electrodes may increase screening capacity but need validation. This study aims to determine initial device accuracies and feasibility of further research.

Methods and analysis

Prospective diagnostic device accuracy and feasibility study comparing novel ERG devices to standard screening tests. Three groups of 35 participants on HCQ, categorised by HCQ retinopathy (definite, possible and no retinopathy), and 35 healthy control participants, recruited by consecutive sampling, will have full field and multifocal ERG index tests, delivered using skin-contact electrodes by two devices—RETEval full-field and UTAS multifocal ERG, both manufactured by LKC Technologies (Gaithersburg, Maryland, USA), compared with spectral-domain optical coherence tomography and autofluorescence reference tests graded by two masked, independent retinal specialists. Eligible HCQ participants will either have diagnosed HCQ retinopathy or be eligible for screening per UK guidelines. Healthy control participants will have no prior HCQ exposure and be of similar age and sex to HCQ participants. Primary outcome is device-specific sensitivity and specificity. Secondary outcomes include the effect of dilation on device outputs, analysis of discriminatory waveforms, device acceptability and recruitment rate. Safety outcomes include adverse and serious adverse events and device events.

Ethics and dissemination

Cambridge East ethics committee gave a favourable opinion (24/EE/0011, 23/02/2024). Results will be published in a peer-reviewed ophthalmology journal.

Trial registration number

ClinicalTrials.gov NCT06035887.

WHAT IS KNOWN ON THIS SUBJECT.

WHAT THIS STUDY ADDS

• This study evaluates the preliminary accuracy and feasibility of using two novel ERG devices to detect HCQ retinopathy.

HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY

• If proven valid, these novel devices and methods may significantly change how HCQ retinopathy is screened for in the UK.

Introduction

Chloroquine (CQ) and its derivative hydroxychloroquine (HCQ) are synthetic 4-aminoquinoline (4-AQ) drugs, originally derived from the naturally occurring alkaloid quinine.¹ Although initially classified as antimalarials, the positive immunomodulatory and metabolic effects of quinine, CQ and HCQ, are well-recognised.^{2 3} They are used to treat a range of rheumatological, dermatological and connective tissue disorders, including systemic lupus erythematosus (SLE), rheumatoid arthritis (RA) and Sjögren’s disease.4,6 As prescribing patterns have evolved, HCQ has now largely superseded CQ and quinine, owing to its lower risk of toxicity.⁷ Current guidelines recommend HCQ as the first-line treatment for mild or palindromic RA⁸ and all stages of SLE and Sjögren’s disease.^{4 6}

One of the most recognised complications of HCQ use is retinopathy, which is characterised by photoreceptor loss with or without retinal pigment epithelial (RPE) loss in the early stages, beginning in the parafoveal region and progressively affecting the other regions of the retina, eventually resulting in a characteristic ‘bulls-eye maculopathy’ and potentially widespread retinal atrophy.⁹ Sight loss follows a similar pattern with predominantly paracentral, then peripheral sight loss reflecting the loss of outer retina and RPE. The exact mechanism of HCQ-induced retinopathy is unknown, but a wide range of antimicrobials and rheumatic drugs, including all 4-AQs, are known to accumulate in melanin-containing tissues,¹⁰ and amino quinolone-mediated disruption of RPE autophagy and lysosomal functions has been proposed as a mechanism of retinopathy, resulting in lipofuscin accumulation and subsequent outer retinal atrophy.¹¹ However, some animal model and human studies suggest that direct neuroretinal injury (to ganglion cell and consequent nerve fibre layers) may occur first and precede retinal atrophy and RPE dysfunction.^{12 13} Once structural and functional retinal complications of HCQ have occurred, changes are generally irreversible and, in more advanced stages where atrophy has begun, can continue to progress despite medication cessation.¹⁴ Importantly, however, stopping HCQ before the onset of significant structural photoreceptor and RPE injury can halt progression, preventing sight loss.¹⁵

Cumulatively, there are an estimated 320 000 patients receiving HCQ treatment in the UK.¹⁶ A population-based, observational study using the health improvement network estimated that, between 2007 and 2016, over 2000 individuals per year were started on HCQ for RA or SLE.¹⁷ The prevalence of HCQ retinopathy was initially thought to be very low, and early attempts at UK screening programmes in the 1990s were considered not to be cost-effective.^{18 19} Later reports suggested higher prevalence²⁰; and a study by Melles and Marmor²¹ suggested that the prevalence of retinopathy among HCQ users could be as high as 7.5% (with the primary risk factors being the duration of treatment>5 years, dose greater than 5 mg/kg per day actual body weight, estimated glomerular filtration rate of<60 ml/min/1.73 m² and concomitant tamoxifen use) prompting the Royal College of Ophthalmologists (RCOphth) to recommend screening annually from year 1 in patients with risk factors and from year 5 in those without.¹⁶ More recent point prevalence estimates in the UK, using updated guideline-recommended classification criteria, suggest that HCQ retinopathy could affect up to 6% of patients on treatment²² or around 20 000 patients in the UK.

The current RCOphth HCQ-retinopathy screening guidelines recommend multiple tests, beginning with dilated spectral-domain optical coherence tomography (SD-OCT) and fundus autofluorescence (FAF) imaging, following which patients are categorised as ‘NO’ retinopathy where both tests are normal, ‘POSSIBLE’ retinopathy where findings are indeterminate and require further visual field (VF) and multifocal electroretinography (mfERG) testing or ‘DEFINITE’ retinopathy where both tests are abnormal and consistent with HCQ retinopathy (figure 1—Flowchart for HCQ screening). The process of screening is laborious for both patients and healthcare services, requiring the use of hospital eye services (HES), non-mobile devices and specialist expertise in test interpretation. This has been reflected in associated costs, with 1 UK-based council of 9 commissioning groups estimating the supportive cost of yearly screening for 6600 patients locally at over £500 000,²³ and a US-based study assessing the yearly costs of screening~150 000 patients receiving HCQ at $40.7 million.²⁴ This has limited the setup of HCQ screening services, for example, Scotland does not undertake HCQ-retinopathy monitoring.²⁵ There exists a need for a deliverable, reliable screening test that can accurately detect HCQ retinopathy, possibly before the onset of structural retinopathy, which is acceptable to patients and can be used by non-specialists, ideally outside of HES, such as rheumatology clinics.

Figure 1. Current recommended flow diagram for screening of patients on HCQ or CQ. Reproduced from the RCOphth HCQ and CQ screening guidelines. Abbreviations: CQ, chloroquine; eGFR, estimated glomerular filtration rate; FAF, fundus autofluorescence; HCQ, hydroxychloroquine; HVF, Humphrey automated visual field; mfERG, multifocal electroretinography; SD-OCT, spectral-domain optical coherence tomography.

Electroretinography (ERG) objectively measures the retinal cellular response to light stimuli. Under dark and light conditions (dark and light adaptation), light stimuli with varying intensity, frequency and patterns are delivered to each eye separately, and the electrical response generated by the retina is detected and quantified using ocular or periocular surface electrodes. The patterns of light stimuli under differing conditions isolate specific retinal pathways and can quantify retinal dysfunction²⁶ that may not, otherwise, appear on imaging modalities that evaluate structural injury (such as SD-OCT, FAF or even MRI of the optic nerve). The retina can be tested globally using full-field ERG or segmented into and tested by region using mfERG testing. The mfERG is considered by some authors as the gold-standard test for HCQ retinopathy,1627,29 has high sensitivity and specificity2830,32 relative to other diagnostic tests and has shown potential to detect retinal injury before the onset of structural damage.2730 32,34 However, the availability of ERG testing is limited because of the highly rigorous testing methods (typically requiring over 40 min) and the limited number of devices and expertise to conduct and interpret the tests. Consequently, very few units offer ERG testing and interpretation, and these tests are generally limited to the diagnosis of rare diseases or research studies. To overcome this challenge, more portable devices have been designed, which are easier to use and incorporate faster automated testing protocols and more comfortable skin electrodes while still adhering to the testing standards and tenets as established by the International Society for Clinical Electrophysiology of Vision (ISCEV).³⁵

The main purpose of this study is to evaluate the diagnostic accuracy of two new ERG devices developed by LKC Technologies (Maryland, USA) in the detection of HCQ retinopathy—the hand-held RETEval, which undertakes full-field flash and flicker ERG testing, and the trolley/table-mounted UTAS, which undertakes mfERG testing. Both devices adhere to ISCEV testing standards and protocols, but employ the following novel changes.

• More portable design (handheld and trolley mounted, respectively).

• Skin-contact electrodes attached beneath the lower eyelid incorporating active, reference and ground electrodes in a single lead.

• Mydriasis-free testing.

• Faster testing protocols.

These devices will be compared with standard screening tests for reference, with masked independent grading by retinal specialists, to determine their diagnostic accuracy in terms of sensitivity, specificity and predictive values. This study also aims to determine the acceptability of device testing, and feasibility outcomes of recruitment rate and proportion of eligible participants recruited, to inform the likely patient acceptance of these devices in actual screening services, refine technologies and testing methods and inform the feasibility of further research.

If proven to have acceptably similar accuracy to current HCQ-retinopathy screening tests and methods, these novel ERG testing devices, which are intended to be less burdensome to patients and to have greater clinical utility to non-ERG-expert clinicians, could have a significant impact on the capacity and availability to check patients for this important sight-threatening side effect, and in more places, than is currently possible. For example, annual screening could take place in general clinics where patients on HCQ (for SLE, RA or other disorders) are routinely followed up, thereby potentially reducing burdens of onwards referral to eye screening services, which may or may not be easily available or accessible depending on local services.

Study objectives

The primary aim of the study is given as follows.

• To determine the sensitivity and specificity of both ERG devices to detect and categorise patients with DEFINITE, POSSIBLE and NO HCQ retinopathy compared with current screening tests.

The secondary aims are as follows.

• To determine the effect of undilated compared with dilated testing in the UTAS mfERG device.

• To determine what ERG waveform features from the outputs of both devices can better discriminate patients with NO HCQ retinopathy from those with POSSIBLE and DEFINITE HCQ retinopathy.

• To determine the degree in which device waveforms correlate with standard mfERG waveforms in patients receiving HCQ.

• To determine the patient acceptability of hand-held ERG and UTAS mfERG testing.

• To determine the proportion and recruitment rate of patients in each category of HCQ retinopathy who consent to join this study.

Methods

Study design

This is a prospectively recruited, four-group, consecutively sampled, reference-test grader masked, diagnostic device accuracy and feasibility study. The current study protocol (v1.1, 14 February 2024) is available in the appendix.³⁶ Outputs of two novel ERG devices (RETEval full-field flash and flicker ERG and UTAS mfERG) will be compared with standard screening tests (macular SD-OCT and FAF) with all data collected at a single visit. The four predetermined disease-state categories are as follows.

• DEFINITE HCQ retinopathy.

• POSSIBLE HCQ retinopathy.

• NO HCQ retinopathy.

• Healthy controls with no previous HCQ exposure.

Study setting

Participants will be recruited and tested from a single centre (King’s College Hospital), which has a large rheumatology service. Two additional sites will be set up as participant identification centres (PICs)—St Thomas’ Hospital, which hosts the primary South London HCQ-retinopathy screening service, and Moorfields Eye Hospital, which is one of the primary London referral units for ERG testing in suspected HCQ-retinopathy cases.

Eligibility criteria

Eligibility will be assessed by a study investigator (ophthalmologist) based on the following criteria.

Inclusion criteria

1. Age≥18 years.

2. Taking HCQ and eligible for national screening OR diagnosed with HCQ retinopathy whether or not still taking HCQ.

3. Control group.

Exclusion criteria

1. Cataract grade≥3 of any subtype.

2. Cataract surgery within 4 weeks of recruitment.

3. Significant media opacity or corneal disease, including, but not limited to, corneal oedema, corneal scarring, keratoconus, previous corneal transplants and severe keratoconjunctivitis sicca (requiring the use of topical serum, immunosuppressive or analogous therapy or procedural treatment).

4. Significant macular copathology, including, but not limited to, macular degeneration, macular scarring, cystic macular oedema (for any reason) and staphyloma.

5. Inherited retinal and/or macular dystrophies, including colour vision deficiencies.

6. Active or previous posterior uveitis or panuveitis.

7. Aphakia.

8. High refractive error>8.00 dioptres.

9. Amblyopia.

10. Diabetes.

11. Retinal angiopathies, including, but not limited to, retinal vein occlusion, retinal artery occlusion, ocular ischaemic syndrome, HIV retinopathy, sickle cell disease and radiation retinopathy.

12. Visually significant surgical retinal pathology, including epiretinal membrane, macular holes, retinal detachment and retinal tears.

13. Previous retinal laser, retinal surgery or intravitreal treatment.

14. Primary or secondary glaucoma with established neuroretinal rim thinning, uncontrolled IOP of≥24 mm of mercury or mean deviation of −6.00 dB or greater.

15. Optic atrophy.

16. Photosensitive epilepsy.

17. Ungradable HCQ-retinopathy screening images.

18. Periocular infection or rash (recruitment can be deferred until the acute disease has resolved).

19. Unable or unwilling to undertake study activities.

20. Light-adapted, undilated pupil size>6 mm diameter or<2 mm diameter.

21. Current or previous use of any medication detailed in Box 1.

Box 1. Prohibited concurrent medications.

Amiodarone

Canthaxanthin

Deferoxamine

Digoxin

Ethambutol

Interferon-α

Melatonin

Nefazodone

Sildenafil

Vigabatrin

Chloroquine

Quinine

Identification of participants

In respect to potential participants receiving HCQ or having been diagnosed with HCQ retinopathy:

Potential participants at the recruiting site (King’s College Hospital) will be identified through electronic patient record searches for the following.

• Current and past patients undergoing HCQ-retinopathy screening or having been diagnosed with HCQ retinopathy in the eye clinic lists.

• Current and past patients taking HCQ in the rheumatology clinic lists.

Potential participants at PICs will be identified by ophthalmologists directly involved in eye screening and diagnosis of HCQ retinopathy, who will explain the study to potential participants and ask for consent or encourage them to contact the study team to consider enrolment.

With respect to healthy controls:

Potential participants will primarily be staff and colleagues of King’s College Hospital. We will intentionally aim to invite individuals who live in the South London region to limit the burden of travel and inconvenience of the research study visit, and also aim to recruit those in a similar age and sex demographic as the majority of patients taking HCQ so as to have a comparative healthy control group.

Sampling and recruitment

Sampling and recruitment will be on a consecutive recruitment basis. Table 1 shows the schedule of activities in order of testing, and figure 2 shows the participant flow diagram.

Table 1. Schedule of activities.

Activity	Patient receiving HCQ (n=105)	Healthy control (n=35)
Consent	X	X
Medical, ophthalmic and drug history	X	X
Standard clinic visual acuity*	X	X
Slit-lamp examination and intraocular pressure†	X	X
Skin prep and electrode placement‡	X	X
Undilated RETEval full-field flash and flicker ERG testing	X	X
Undilated UTAS mfERG testing§	X	X
Dilation drops applied to both eyes	X	X
Dilated UTAS mfERG testing§	X	X
Patient acceptability questionnaire	X	X
SD-OCT	X	X
Autofluorescence	X	X
Safety call 1-week poststudy testing	X	X

^*Undertaken in both eyes, using normal spectacle prescription, including colour vision.

^†Undertaken in both eyes, including IOP and fundus examination to determine eligibility.

^‡See Appendix A for details.

^§Undertaken two times per eye, before and after dilation.

ERGelectroretinographyHCQhydroxychloroquinemfERGmultifocal electroretinographySD-OCTspectral-domain optical coherence tomography

Figure 2. Study participant flow diagram. Abbreviations: ERG, electroretinography; FAF, fundus autofluorescence; HCQ, hydroxychloroquine; mfERG, multifocal electroretinography; OCT, optical coherence tomography.

Test methods

Index tests

The index tests under evaluation in this study are full-field flash and flicker ERG delivered by the RETEval hand-held ERG device and multifocal ERG delivered by the UTAS mfERG device, both manufactured by LKC Technologies (Maryland, USA).

RETeval ERG

The hand-held RETeval ERG performs full-field flash and flicker testing using a modified Ganzfeld dome placed over the eye, with a soft rubber adaptor that blocks ambient light. Standard testing takes approximately 3–5 min per test for both eyes and sequentially delivers 30 flashes (retinal illuminance of 85 Td·s) at 2-Hz frequency (full-field flash), then 141–424 flashes (retinal illuminance of 85 Td·s) at 28.3-Hz frequency (‘30 Hz flicker’), both with a white background of retinal illuminance of 850 Td. Each eye is tested individually, and each segment (full-field flash and flicker) will be repeated up to three times per eye to enable averaging of responses.

The device has been validated in the detection of diabetic retinopathy,³⁷ uses active pupil-tracking software to measure pupillary area (mm²) and adjusts luminance to maintain consistent retinal illuminance. The figures for retinal illuminance given above are equivalent to those delivered by standard ISCEV photopic stimuli of 3 cd·s/m² on a 30 cd/m² background, through a dilated pupil of diameter 6 mm. This has been shown to be consistent for pupil sizes up to 6.5-mm diameter,³⁸ with aberrations of extended implicit time introduced with pupils larger than 6.5 mm or smaller than 2 mm, possibly due to the Stiles–Crawford effect.³⁹ This means that for pupils between 2 and 6.5 mm, which encompasses the majority of the population under photopic conditions,⁴⁰ the RETeval ERG can be conducted without the need for dilating drops.

For the purposes of this study, participants will undergo standard full-field flash and flicker testing for each eye, predilation only, and will have three test cycles undertaken per eye to quantify device precision and reliability. Data will be pseudoanonymised at the time of testing and extracted using LKC RFF extractor software for analysis.

UTAS mfERG

The UTAS mfERG selectively stimulates retinal areas with light using flashing hexagons projected from a trolley-mounted screen at 36 cm and a screen luminance of 400 cd/m². Electrical responses are recorded using electrodes placed on the ocular or periocular surface to generate the maps of retinal function, with increasingly detailed regional segmentation based on the number of hexagons projected (61 and 103 being the standard, with 19-hexagon protocols also described for more rapid screening). Both eyes are tested simultaneously. For the purposes of HCQ-retinopathy assessment, mfERG changes in amplitude and implicit time are greatest in the pericentral region³³ corresponding to rings 1–3 on a standard 61-hexagon test. LKC has reported that testing can be undertaken without dilating drops, although this has yet to be validated with skin electrodes or in HCQ retinopathy. For the purposes of this study, patients will be tested with both 19- and 61-hexagon tests per eye, pre and postdilation, and the sensitivity and specificity of each testing protocol, pre and postdilation, will be compared with standard screening tests. Data will be pseudoanonymised at the time of testing and extracted directly for analysis.

Skin-contact electrodes

Skin-contact electrodes have been increasingly investigated in recent years due to their ease of use, better patient comfort and potentially lower susceptibility to lid and blink reflexes. Comparisons between LKC skin-contact electrodes and traditional Dawson–Trick–Litzkow corneal-contact electrodes using the hand-held RETeval ERG have shown that, with repeat testing, skin electrodes have good reliability and no loss of signal with respect to implicit time, but generally have lower waveform amplitudes and potentially lower signal-to-noise ratios.⁴¹ However, a recent study by Hobby et al⁴² determined that a lack of consistent electrode positioning and skin preparation is a potential cause of amplitude issues in previous studies. The LKC skin-contact electrodes have design advantages relative to older iterations in which they incorporate active, reference and ground electrodes in a single strip, simplifying the device testing setup and methodology. For the purposes of this study, a skin electrode placement protocol will be used using the methods described by Hobby et al and recommendations by the manufacturer. Ocular disorders that will confound amplitudes obtained using skin electrodes will be excluded as a part of the eligibility criteria.

Device outputs

The primary device outputs are continuous and summarised below.

RETeval full-field flash

Three full-field waveforms will be obtained per eye. The following data outputs will be recorded.

• Maximum stimulus luminance (cd·s/m²).

• Minimum stimulus luminance (cd·s/m²).

• Amplitude (mV) and a- and b-wave peak times.

RETeval flicker

Three flicker waveforms will be obtained per eye. The following data outputs will be recorded.

• Flicker amplitude and peak time (ms).

UTAS mfERG 19-hexagon test

1-min pre and postdilation, and 3.8-min pre and postdilation 19-hexagon mfERG tests of right and left eyes of each participant will be conducted, which include segmented waveform trace arrays, amplitudes and implicit times.

UTAS mfERG 61-hexagon test

3.8-min pre and postdilation 61-hexagon mfERG tests of right and left eyes of each participant will be conducted, which include segmented waveform trace arrays, amplitudes and implicit times.

Trace arrays from both testing protocols (19- and 61-hexagon) will be scrutinised and mapped to SD-OCT and FAF images to help delineate the reasons for localised reductions in amplitude. Averaged ring-response amplitudes and implicit times will be calculated.

Reference standard tests

The reference standard tests to be used in this study are SD-OCT and FAF performed using the Heidelberg Spectralis SD-OCT device (Heidelberg Engineering, Heidelberg, Germany). These are standard and widely used imaging tests in diagnosing and monitoring macular diseases, including HCQ screening, and provide a structural representation of macular anatomy. OCT and FAF will be taken after study ERG testing has been completed.

The principle of OCT uses light interferometry, comparing the light reflected from the back of the eye against a reference to construct a tomographic 2-D image of the tissue in question (in this case, the macula and retina). Iterations on technology have allowed faster and more detailed image acquisition compared with original OCT methodologies, leading to the current standard of SD-OCT, which uses infrared light (800–870 nm wavelength), eye-tracking software and a very high scan rate of 20 000–40 000 scans per second (compared with original OCT rate of 400 per second) to produce high-resolution, motion-controlled images with high signal-to-noise ratio.⁴³ Independent, masked retinal specialists will assess all study-acquired SD-OCT images to determine the presence and classification of HCQ retinopathy, in combination with FAF, using the RCOphth-suggested SD-OCT signs of HCQ retinopathy as reference,¹⁶ which include the following.

• Thinning of the outer nuclear layer.

• Disruption of the photoreceptor layer.

• Disruption of the inner segment/outer segment (OS) junction (ellipsoid zone).

• Loss of space between ellipsoid zone and interdigitation zone (IZ) (photoreceptor OS layer).

• Loss of IZ.

• RPE layer loss and accumulation of debris.

• Increased choroidal reflectance secondary to RPE loss.

The principle of FAF relies on the inherent autofluorescent properties of lipofuscin, which are collections of mostly lipids and proteins accumulated in the RPE, derived from the breakdown of retinal and macular photoreceptors. When illuminated with light, lipofuscin has a peak absorption light wavelength of approximately 470 nm and peak emission light wavelength of 600 nm. These absorption and emission properties can be leveraged through imaging to qualitatively and quantitatively estimate the health of RPE and outer retina—with hyperautofluorescence and hypoautofluorescence denoting RPE and outer retinal stress and death/absence, respectively, the pattern of which can inform the potential underlying cause. The spectralis device uses a bright monochromatic light (488-nm wavelength) projected onto the retina and the reflectance is captured using a barrier filter placed across the camera (500-nm wavelength). Independent, masked retinal specialists will assess all study-acquired FAF images, in combination with SD-OCT, to determine the presence and classification of HCQ retinopathy using the RCOphth-suggested signs of HCQ retinopathy as reference,¹⁶ which include parafoveal or paracentral hyper or hypoautofluorescence with or without an annular-shaped distribution.

Rationale for choosing reference standard

Both SD-OCT and FAF are recommended as first-line screening tests for HCQ retinopathy, which is the primary rationale for choosing these as reference standards. Although SD-OCT has been shown to have higher sensitivity than FAF,²⁸ their combined use likely improves the overall diagnostic accuracy of each test in isolation, and as they are widely available and can be undertaken on the same devices, it makes practical sense to combine both, as the guidelines recommend. There are two other second-line tests used in the diagnosis of HCQ retinopathy, which will not be used as a part of the reference standards for the primary outcome of this study—VF testing (mostly using the 10-2 automated Humphrey Field Analyser protocol) and mfERG. VF testing is the only test recommended in the diagnosis or screening of HCQ retinopathy that is subjective in nature that is, relies on the performance of a patient to accurately and reliably signal when they can see a light projected in the peripheral or central VF of the tested eye. Although useful when done to a high standard, this test requires a great deal of concentration from patients to be reliable and practically is much more difficult to deliver on a regular basis, as such it is recommended as a secondary confirmatory diagnostic tool. The standard mfERG has drawbacks as outlined in the introduction and rationale section above, primarily surrounding the burden of testing to patients, the limited availability of testing and high expertise needed to interpret findings, meaning it is only practically used in cases of diagnostic uncertainty. It is possible that patients diagnosed with HCQ retinopathy will have had one or both additional tests in the past at the time of diagnosis; however, guidelines recommend that clinicians can stop HCQ based on SD-OCT and FAF alone, so it cannot be guaranteed that participants on this study will have had historical testing to use as reference. For these pragmatic reasons, these two tests will not be used as reference standards for the purposes of the primary outcome analysis of this study.

If new cases of possible or definite HCQ retinopathy are detected as a part of recruitment for this study, we will arrange for follow-up in standard-of-care retinal monitoring clinics, and we will collect any additional test results (including standard mfERG results) for analysis and comparison to the study index tests as a secondary outcome. The participant and their prescribing clinician will also be informed of the result, so an informed decision can be reached with respect to ongoing exposure to HCQ.

Definition of and rationale for test positivity cut-offs or result categories of the index tests, distinguishing prespecified from exploratory

It is unknown what the exact, most useful clinical test positivity cut-off will be for each device and each category of HCQ retinopathy, determined by standard reference tests. ERG results, in general, lack widely available reference database ranges (other than in-built reference ranges usually acquired as a part of device certification), and the few publicly available databases and in-built ranges may not be representative of the population of interest in this study. For example, they will have been undertaken in patients with no prior exposure to HCQ, who will likely not have the diseases in-question necessitating HCQ treatment (such as SLE or RA) and may be of different ages and racial makeup than the participant groups in this study.

Full-field flash and flicker ERG have not been widely reported in the diagnosis or screening of HCQ retinopathy except in older case series describing advanced disease, and although mfERG is more widely reported in the diagnosis and screening of HCQ retinopathy, much of the interpretation hinges on qualitative interpretation of mfERG trace arrays to identify ‘patterns’ of loss consistent with HCQ retinopathy. Recent work has reported on more quantitative methods to try to standardise this practice, particularly the application of applying ring-ratio analysis to summate and average trace arrays in five rings extending out from the fovea. However, there still is not a clear consensus regarding what cut-offs should be used to define abnormal, particularly when skin electrodes are used and eyes are tested without dilating eyedrops.

As such, this study will not prespecify index test cut-offs of abnormality for each category of HCQ retinopathy but will instead report on the most clinically useful cut-offs to yield the highest sensitivities and specificities compared with reference standard tests.

Definition of and rationale for test positivity cut-offs or result categories of the reference standard, distinguishing prespecified from exploratory

Reference standard test cut-offs will use clinician-determined categorisation, on the basis of multimodal interpretation, in-line with current RCOphth guideline recommendations.¹⁶ Result categories will be as follows.

• DEFINITE HCQ retinopathy (both OCT and FAF abnormal, or confirmed with further testing).

• POSSIBLE HCQ retinopathy (either OCT or FAF abnormal, without further testing).

• NO HCQ retinopathy (OCT and FAF normal).

Whether clinical information and reference standard results were available to the performers/readers of the index test

Performers of the index test will be masked to any previous HCQ screening grading results when the testing is conducted.

In the case of participants with known definite HCQ retinopathy, to standardise the test and minimise potential bias, testing will be done by an independent, masked member of the study team following protocols for lead placement, manufacturer’s recommendations for test conduct using the RETEval and UTAS and standard ISCEV protocols in respect to evaluating ERG trace quality for accepting or rejecting a trace.

Whether clinical information and index test results were available to the assessors of the reference standard

Assessors of the reference standard (independent, masked retinal specialists) will be masked to the results of the index test, but will have access to the participant’s clinical information (eg, total dose and duration of HCQ exposure and underlying diagnosis). Healthy control participants with no history of HCQ exposure will have their imaging reviewed by the study principal investigator for the confirmation of eligibility but will not undergo unnecessary masked retinal specialist assessment.

Testing protocol

Participants will be given an ethics-approved participant information sheet and will have a minimum of 24 hours (ideally a week) to consider whether to take part in the study. The study will be explained to participants by the principal investigator or a delegated subinvestigator, and informed consent will be signed prior to study testing.

Study tests will be conducted in the following order.

• Past medical, ocular, drug history, including HCQ duration and dose, and assessment of known risk factors for HCQ retinopathy.

• Visual acuity assessment, including colour vision (Ishihara testing).

• Slit-lamp examination, including pupil size measurement, intraocular pressure and biomicroscopy examination of the posterior segment. Cataracts will be graded using the lens opacities classification system III.⁴⁴

• Once eligibility has been confirmed, further testing will be conducted in the following sequence.

  • Skin prep and electrode placement.

  • Undilated RETEval full-field flash and flicker ERG testing.

  • Undilated UTAS mfERG testing.

  • Dilation drops applied to both eyes.

  • Dilated UTAS mfERG testing

  • SD-OCT and FAF.

• Patient acceptability questionnaire.

• A safety call will be conducted 1 week following study testing to assess for any subsequent adverse events or serious adverse events.

No refractive correction should be used during RETeval testing. The participant’s usual refractive correction (eg, glasses) should be used predilation for mfERG testing. Postdilation mfERG testing should be undertaken with correction of refractive error by inputting the participant’s usual refractive correction into a trial frame and adding a+2.75 dioptre lens to account for the testing distance of 36 cm and loss of accommodation (ensuring that the lens or trial frame rim does not block the stimulus).

All study tests are intended to be completed in a single visit. For pragmatic reasons, where patients are coming in as a part of standard HCQ-retinopathy screening, the study visit will be conducted alongside this, and images taken as a part of standard screening. Otherwise, a separate research clinic visit will be arranged for study testing.

The total duration of study testing is not likely to exceed 90 min per participant.

Recruitment Timelines

The key logistical challenge for this study will be the recruitment of participants with POSSIBLE and DEFINITE HCQ retinopathy, as they have an estimated prevalence of approximately 6.3% and 1.6%, respectively, among all individuals receiving HCQ.²²

The eye clinic at King’s College Hospital has had approximately 100–130 patients undergoing ad hoc screening for HCQ retinopathy in the 15 months prior to January 2024 and has a large rheumatology service with patients on HCQ. St Thomas’ Hospital HCQ screening clinic and Moorfields Eye Hospital ERG clinic will be set up as PICs. Using St Thomas’ Hospital HCQ screening clinic as a reference for timeline estimation, this clinic sees 40–60 patients per week for HCQ screening. Assuming 70% agree to take part and a potential screen fail rate of approximately 10–20%, we estimate the following proportions of eligible patients and timelines to recruit 35 patients in each category of HCQ retinopathy.

• NO HCQ retinopathy=90 potentially eligible patients per month who agree to study recruitment and pass screening. If screen 3 per week=3–6 months to recruit 35 patients.

• POSSIBLE HCQ retinopathy=6 patients per month who agree to study recruitment and pass screening. If screen 2 per week=6–9 months to recruit 35 patients.

• DEFINITE HCQ retinopathy=1–2 patients per month who agree to study recruitment and pass screening. If screen 1 per week=18–24 months.

• CONTROL=4 participants per month, 6–9 months to recruit 35 participants.

By the end of month 10, therefore we predict having recruited.

• A minimum of 35 patients with NO HCQ retinopathy.

• A minimum of 30 patients with POSSIBLE HCQ retinopathy.

• A minimum of 30 healthy control participants.

• A minimum of 15–18 patients with DEFINITE HCQ retinopathy.

To improve the likelihood of achieving the target number of patients with definite HCQ retinopathy, who will be the hardest to recruit, we will intentionally invite patients with known HCQ retinopathy to consider participating in this study. These participants will have study index tests undertaken by a member of the study team masked to the previous screening grading results.

Patient and public involvement

A patient involvement group was convened during study protocol development stage to review activities and methodologies, comprising four patients who had recently undergone HCQ-retinopathy screening at King’s College Hospital. The group reviewed and gave their approval for the patient-facing documents and also advised the team on study practices. One specific outcome of the advisory group involvement was to ensure that the reports of masked screening results would be communicated to participants and acted on in the event of any abnormalities being detected.

The involvement group will continue to be periodically consulted throughout the course of the study in the event any issues arise, such as with recruitment or complaints, and towards the end of the study as the report and publications are being prepared to comment on findings and review plans for dissemination and publication.

Analysis

Baseline patient-level and eye-level demographic data will be presented for each group, including age, sex, ethnic group, eye colour, refractive error, underlying diagnosis (reason for taking HCQ), current dose, estimated cumulative dose and duration of HCQ use and the presence of categorical retinopathy risk factors (such as renal failure).

Methods for estimating or comparing measures of diagnostic accuracy

With respect to the index tests, continuous repeat-test outputs of the full-field flash and flicker ERGs will be averaged per eye, respectively, and compared for analysis. mfERG response density topographic maps will be constructed, and ring-response averages will be computed as the averages of rings 1–4 compared with 5 (1–4:5). Intereye correlation will be accounted for using standardised methods for small samples.⁴⁵ Analysis will begin with a simple, two-sided means comparison test to test the hypotheses that (1) there is a statistically significant difference between HCQ users with NO retinopathy and standard controls, (2) there is a statistically significant difference between HCQ users with DEFINITE retinopathy and healthy control participants and (3) there is a statistically significant difference between those HCQ users with NO retinopathy and those with DEFINITE retinopathy.

There are multiple possible test positivity cut-offs for each specific device output. Receiver operating characteristic (ROC) curves will be generated for each device output, for each category of retinopathy, to demonstrate various test positivity cut-offs. Summative area-under-ROC-curve analyses to show the overall performance of each device output, for each category of retinopathy, will be compared using DeLong’s test to determine the most accurate average device output. Additional data summations, such as total receiver operating characteristic curves and Gini coefficients, may be generated to present a comprehensive overview of device accuracy.

Correlation of reference standard test results between graders will be assessed using Cohen’s kappa.

How indeterminate index test or reference standard results will be handled

With respect to the index tests, as stated, there are multiple possible test positivity cut-offs for each specific index test output, so an intentional part of the analysis will be reporting the proportion of indeterminate test results for the most clinically useful cut-off.

In respect to reference standard tests, two masked retinal specialists will grade the imaging results and, in the event of disagreement, a final decision will be decided by a third, masked retinal specialist. As the decision of retinopathy presence includes the interim category of ‘POSSIBLE’ retinopathy, which acknowledges uncertainty, this will form a separate category for indeterminate grading results. It is highly unlikely that a grading result will return a result of ‘NO’, ‘POSSIBLE’ and ‘DEFINITE’ retinopathy for the same eye; however, if this does occur, we will ask for qualifying reasons from each masked grader.

How missing data on the index tests and reference standard will be handled

All study tests will be conducted in a single visit, at a single site, and checked prior to participants leaving, so we anticipate a low likelihood of missing data. However, it is possible that data may be missing by omission (eg, a repeat ERG test not being conducted and being missed on data checking prior to participant leaving) for practical reasons (eg, a participant does not withdraw consent but decides that they cannot stay to complete, or feasibly undergo all study tests at that visit) or for other reasons (eg, a participant withdrawing consent while the visit is being conducted).

In the former two situations, the study team will make all efforts to rebook the participant for a separate day to complete all assessments. If a suitable appointment cannot be made within six calendar months, all study assessments will be repeated.

In the latter situation, we will retain all data collected up to the point of withdrawal of consent. Any data collected from participants who withdraw consent prior to completion of study testing will be fully anonymised at the point of withdrawal.

Any analyses of variability in diagnostic accuracy, distinguishing prespecified from exploratory

As the sample size of this study is small and it is intended as a feasibility study, all analyses of variability to model the effect of important covariates (including age, sex, ethnicity, eye colour and refractive error) on the ROC performance will be exploratory.

Intended sample size and how it was determined

These specific types of skin electrodes, used in conjunction with these devices, have not been investigated in the screening of HCQ retinopathy, so data informing possible device sensitivity are lacking. Sample size calculation for this study, therefore, is a balance between setting a sufficiently high target device sensitivity and precision to be useful in screening practice while acknowledging the low prevalence of definite HCQ retinopathy affecting recruitment rate and timelines.

Targeting a conservative device sensitivity of 80%, CI of 90% and precision of 12% gives a required sample size of at least 28 cases of definite HCQ retinopathy. To account for a conservatively high 10–20% screen fail rate (eg, participants who are unexpectedly found to have other retinal diseases or whose pupils are too small to be detected by the automated pupil-tracking software on the RETeval and cannot, therefore, complete RETeval testing), this gives a target sample size of 35 cases for each category of HCQ retinopathy.⁴⁶ An additional 35 healthy age- and sex-matched controls with no history of HCQ use will establish a healthy reference dataset using skin-contact electrodes for comparative analysis.

Implications for practice

The purpose of this study is to evaluate the preliminary diagnostic accuracy of two novel ERG devices in the detection and classification of HCQ retinopathy and the feasibility of larger studies of these devices in HCQ retinopathy. As such, this study has both utility outcomes (the accuracy of each device, and any specific features of device performance that better discriminate abnormalities) and feasibility outcomes (the testing protocols, recruitment rates and timelines and acceptability of testing) that will inform the need for, and feasibility of, future studies and application in current National Health Service (NHS) screening services.

If found to have low accuracy, low participant acceptability or high levels of technical and feasibility challenge, this may encourage further development of this technology and testing protocols to improve its accuracy and ease of use before it can be considered for wider adoption in screening programmes or other NHS services.

If found to have acceptable levels of accuracy, participant acceptability and technical and feasibility challenge, this will set the evidence base for a larger study to confirm these findings, and/or ultimately lead to wider adoption of this technology in screening for this important side effect in non-eye specialist settings, such as general and rheumatology clinics and optometrists. Furthermore, as these devices are intended to ease the delivery and availability of ERG testing, which is itself a highly useful test in ophthalmic practice that is primarily limited by the ease, capacity and availability of testing, this may open the door to wider use of these devices to detect and monitor other rare or common diseases that affect sight. Future studies can also formally assess cost-effectiveness, informed by findings of the current study.

Registration number

This study is registered on ClinicalTrials.gov (NCT06035887).

Study protocol

The current ethics-approved study protocol version is 1.1 dated 14 February 2024 (available in the online supplemental appendix).

Data availability statement

No data are available.