Development of Novel Patient-Reported Outcome (PRO) and Observer-Reported Outcome (ObsRO) Instruments in Retinitis Pigmentosa (RP) and Leber Congenital Amaurosis (LCA): ViSIO-PRO and ViSIO-ObsRO

Abstract

Introduction

Retinitis pigmentosa (RP) and Leber congenital amaurosis (LCA) are rare inherited retinal degenerative disorders resulting in visual impairments and impacts on patients’ vision-dependent activities of daily living (ADL), mobility and distal health-related quality of life (HRQoL). This study aimed to conduct qualitative research to understand the patient experience of RP/LCA across genotypes and inform development of patient- and observer-reported outcome (PRO/ObsRO) instruments in RP/LCA.

Methods

Research activities included a qualitative literature review and review of existing visual function PRO instruments in RLBP1 RP, and concept elicitation (CE) and cognitive debriefing (CD) interviews of existing PRO instruments with patients with RLBP1 RP, expert clinicians, and payers. In wider RP/LCA, a social media listening (SML) study and a qualitative literature review was conducted, while psychometric evaluation of a PRO instrument in LCA was performed. Input from expert clinicians was sought at key stages.

Results

Findings from the qualitative literature reviews identified a range of visual function symptoms which had significant impacts on patients’ vision-related ADL and distal HRQoL. Patient interviews identified additional visual function symptoms and impacts not previously reported in published literature. These sources informed development and refinement of a conceptual model displaying the patient experience of RP/LCA. Review of existing visual function PRO instruments, and CD interviews evaluating their content validity, confirmed that no existing instrument provides a comprehensive assessment of all concepts relevant to patients with RP/LCA. This highlighted the need for development of the Visual Symptom and Impact Outcomes PRO and ObsRO instruments to adequately assess the patient experience of RP/LCA.

Conclusions

Results informed and supported development of the instruments to assess visual functioning symptoms and vision-dependent ADL, mobility and distal HRQoL in RP/LCA, in accordance with regulatory standards. Next steps to further support use in RP/LCA clinical trials/practice includes content and psychometric validation of the instruments in this population.

Supplementary Information

The online version contains supplementary material available at 10.1007/s40123-023-00724-x.

Key Summary Points

Why carry out this study?

Retinitis pigmentosa (RP) and Leber congenital amaurosis (LCA) are rare inherited retinal degenerative disorders resulting in visual impairments and impacts on patients’ vision-dependent activities of daily living (ADL), mobility and distal health-related quality of life (HRQoL). However, there is a paucity of qualitative research exploring the patient experience of RP/LCA and currently no disease-specific clinical outcome assessments for use in RP/LCA.

Visual function symptoms and impacts on vision-dependent activities are important to capture and best assessed via patient- or observer-reported outcome (PRO/ObsRO) instruments reported directly by the patient or a caregiver. Qualitative research is critical to identify concepts of interest relevant to the target population to inform development of fit-for-purpose instruments in the evaluation of treatment benefit.

The purpose of this study was to better understand the patient experience of RP/LCA and initiate development of PRO and ObsRO instruments for use in RP/LCA populations through the conduct of various qualitative research activities.

What was learned from the study?

This study provides a valuable contribution to the literature by obtaining insight into the patient experience of RP/LCA and evaluation of existing PRO/ObsRO instruments in RP/LCA, both of which were lacking in the literature but are important to understand in these rare diseases.

The study confirmed that there were no suitable disease-specific clinical outcome assessments for use in RP/LCA, and the findings supported the development of the novel Visual Symptom and Impact Outcomes PRO (ViSIO-PRO) and ObsRO (ViSIO-ObsRO) instruments in RP/LCA. Further research will explore the content and psychometric validation of the instruments in an RP/LCA population.

Introduction

Retinitis pigmentosa (RP) and Leber congenital amaurosis (LCA) are a group of rare inherited retinal diseases (IRDs), characterised by the progressive loss of rod and cone photoreceptors [1, 2]. This typically leads to the loss of night vision in adolescence, peripheral vision in young adulthood and central vision in later life [3]. The visual impairments experienced as part of RP/LCA are also often accompanied by difficulties with dark/light adaptation, colour vision and vision in bright lighting. These visual function symptoms can impact significantly on RP/LCA patients’ vision-dependent activities of daily living (ADL), mobility and health-related quality of life (HRQoL) [4–7].

Autosomal dominant, autosomal recessive or X-linked forms of RP/LCA are associated with various gene mutations such as those seen in RHO, USH2A, RPGR and RP2 genes, as well as rarer mutations seen in RPE65 and RLBP1 genes [8]. Although a range of RP/LCA genotypes have been identified and documented within the literature, much less is known about the visual impairments associated with these different RP/LCA genotypes, particularly regarding the severity and progression of the disease, and how these affect patients’ vision-dependent ADL, mobility and HRQoL [9].

There are numerous clinical measures of visual acuity and visual field, which typically form primary and secondary endpoints in clinical trials for ophthalmological conditions such as RP/LCA. These include visual acuity (VA) and visual field (VF) tests, full-field light sensitivity threshold (FST) testing, and pupillometry and contrast sensitivity testing. However, in ophthalmology it is also important to assess the impacts of visual impairments on functioning, as this can demonstrate improvements following treatment that are not always apparent on visual function tests, and can provide a more comprehensive understanding of a disease to facilitate accurate assessment of impacts as a proxy for disease severity. Such measures are best assessed directly from the patient perspective using patient-reported outcome (PRO) instruments or, for younger patients who may not be able to reliably self-report, via an observer-informant using an observer-reported outcome (ObsRO) instrument [10]. PRO and ObsRO instruments can provide insight and demonstrate improvements on concepts which cannot be measured objectively through biological or functional assessments in clinical trials or clinical practice [11].

Current regulatory guidance including the Food and Drug Administration (FDA) PRO guidance [12], patient-focused drug development (PFDD) series [13], European Medicines Agency reflection paper [14] and ISPOR Task Force papers [10, 15, 16], specify that PRO and ObsRO instruments intended for use in clinical trials to support treatment benefit and labelling claims should be informed by direct input from patients and caregivers, respectively. It is imperative that qualitative research be conducted to identify concepts of interest relevant to the target population and support the content validity of clinical outcome assessments (COAs) by demonstrating understanding and comprehension as well as adequate assessment of concepts of interest prior to use in clinical trials.

Given the paucity of qualitative research documenting the patient experience of RP/LCA, and to support the inclusion of PRO and ObsRO instruments in future RP/LCA clinical trials, qualitative research activities were conducted, including a qualitative literature review to understand the patient experience of RP/LCA across genotypes and a review of existing instruments developed in ophthalmological conditions to evaluate their suitability for use in RP/LCA. This ultimately led to the development of the novel Visual Symptom and Impact Outcomes PRO (ViSIO-PRO) and ObsRO (ViSIO-ObsRO) instruments for use in non-syndromic RP/LCA, in accordance with regulatory guidance [10, 12–16]. The methods and results of the qualitative research which contributed to the development of the ViSIO-PRO and ViSIO-ObsRO are summarised here.

Methods

The ViSIO-PRO and ViSIO-ObsRO instruments were developed in four key stages (Fig. 1), composed of a series of qualitative research activities in RLBP1 RP and wider RP and LCA. The qualitative data obtained from stages 1 and 2 were used to develop and refine a conceptual model for RP/LCA (i.e. a visual framework of the symptoms and HRQoL impacts associated with RP/LCA) and input from expert clinicians was sought at each key stage throughout the process.

Fig. 1.

Overview of study activities to inform item generation and instrument development

Stage 1: Qualitative Research in RLBP1 RP

Qualitative Literature Review and Instrument Review in RLBP1 RP

A qualitative literature review was conducted to identify the global prevalence and humanistic and economic burden of RLBP1 RP. Keyword searches of electronic databases EBM Reviews, EMBASE and Medline were conducted in March 2015 via the OVID platform (Table S1 in Supplementary Materials). Given that RLBP1 RP is a rare disease, no search limitations were applied relating to time of publication, but articles were limited to the English language, and animal studies were excluded. To identify existing COA instruments in RLBP1 RP, searches were conducted on ClinicalTrials.gov, and clinical trials that included a patient population with RP were included for analysis.

Concept Elicitation and Cognitive Debriefing Interviews of Existing Instruments with Patients with RLBP1 RP

Combined qualitative concept elicitation (CE) and cognitive debriefing (CD) interviews were conducted with patients with RLBP1 RP to gain an understanding of the patient experience and to assess the appropriateness of existing PRO instruments [National Eye Institute Visual Functioning Questionnaire (NEI VFQ-25), Low Luminance Questionnaire (LLQ) and Visual Activities Questionnaire (VAQ)] to assess vision-dependent ADL in RP and other similar ophthalmologic conditions. Telephone interviews were also conducted with expert clinicians with experience of treating patients with RLBP1 RP to explore the clinical relevance of concepts included in the instruments to RLBP1 RP and to gain insight into clinical priorities in terms of treatment and disease management.

Ethical approval was obtained in Canada (reference no. 2016.224) and Sweden (reference no. 2016/357–31). Documentation of informed consent was obtained prior to any research activities being conducted. The study was performed in accordance with the Declaration of Helsinki of 1964 and its later amendments [17]. Further details on the study methodology are reported elsewhere [5].

Stage 2: Qualitative Research in Wider RP/LCA

Social Media Listening (SML) study

Social media sources including Twitter, news outlets, blogs and forums were searched from December 2016 to June 2018 to identify key visual function symptoms and functional vision impacts of RP/LCA; key stakeholders in RP/LCA management; how stakeholder interactions shape RP/LCA market dynamics; gaps in care of patients with RP/LCA; and what emerging therapies are being discussed. The identified social media platforms were reviewed for content applicable to RP/LCA and discussions among audience members were analysed relating to the topics of interest.

Qualitative Literature Review

A targeted review of published literature was conducted to better understand the patient experience of patients with RP/LCA more broadly, beyond the RLBP1 RP genotype. Published peer-reviewed articles were identified via searches of keywords and Medical Subject Heading (MeSH) terms using disease-specific (i.e. relating to RP/LCA) and qualitative research search terms combined using Boolean logic commands in Embase, MEDLINE and PsycINFO between September and November 2018 (Table S2 in Supplementary Materials). Searches were limited to articles in the English language and those published within the last 20 years (1998–2018). Supplementary searches of Google Scholar and key conference proceedings in ophthalmology were also performed. Specific inclusion and exclusion criteria were not employed as it was anticipated that available relevant literature would be minimal. Articles relating to the patient experience of RP/LCA were reviewed to identify the relevant concepts to assess in this population, as well as information regarding the study aim(s), sample, methodology and results.

Stage 3: Psychometric Evaluation of the mVFQ-25

Psychometric evaluation of a modified version of the Visual Function Questionnaire (mVFQ-25) was conducted with individuals with RPE65-mediated inherited retinal degenerations (IRD) LCA using data from a Phase III trial of voretigene neparvovec pooled across treatment groups (ClinicalTrials.gov identifier: NCT00999609). These analyses were conducted to gain insights into the relative performance of different mVFQ-25 items to help inform the development of the novel COAs in an RP/LCA population. The trial included 31 patients (mean age 15.1 years, range 4–44 years), including children aged 3–11 years, adolescents 12–17 years and adults 18 years or older. A total of 18 patients were female (58%) and 13 were male (42%). The mVFQ-25 includes content that is substantially different from the original VFQ-25, due to revisions implemented to ensure the instrument was relevant and appropriate for the trial population (patients with biallelic mutations of the RPE65 gene). The mVFQ-25 consists of 25 items assessing ADL affected by patients’ vision impairment. Each item has a one-month recall period and a 0–10 numerical response scale with anchors of ‘always’ and ‘never’; higher scores indicate better functional vision. The mVFQ-25 was completed by patients and/or caregivers. Core follow-up visits were baseline, 30 days, 90 days, 180 days and 1-year follow-up. The mVFQ-25 was administered alongside other clinical and functional assessments at the same timepoints. The Multi-Luminance Mobility Test (MLMT) is a performance-related outcome (PerfO) which assesses functional vision in different lighting conditions. MLMT change scores were calculated as the difference between scores on the lowest light level passed at follow-up compared with baseline. Change scores of ≥  +1 indicated improvement, 0 stability and ≤  −1 worsening. Additional assessments included VA, VF and FST. Patients were grouped into the following severity groups on the basis of VF scores; mild (89.0–100%), moderate (72.0–88.0%), severe (45.0–71.0%) and very severe (0–44.0%) [18].

Psychometric analysis performed, using SAS version 9.4 or higher, included assessment of item properties, dimensionality, test–retest reliability, construct validity (convergent/divergent validity, known-groups analysis) and responsiveness (Table 1). The study protocol and individual institutional informed consent documents associated with the Phase III trial of voretigene neparvovec were reviewed and approved by the relevant committees.

Table 1.

Overview of psychometric analyses

Analysis		Description
Item properties	Quality of completion	Missing data at the item level at baseline and 1-year follow-up
	Item response distributions and floor/ceiling effects	Examined for each item at baseline and 1-year follow-up to identify any skewed distributions
Reliability	Internal consistency	Cronbach’s alpha was calculated at baseline and 1 year
	Test–retest reliability (TRT)	Evaluated by examining the stability of scores between days 30–90 and days 90–180 in stable patients
		An ICC coefficient of ≥ 0.70 was considered evidence of good TRT [19]
Construct validity	Convergent validity	Evaluated by examining correlations between mVFQ-25 scores and other measures at baseline and 1-year follow-up
		Items assessing related concepts were expected to correlate at ≥ 0.50 (MLMT; bilateral change score and assigned first eye, FST, and VF). Measures assessing unrelated concepts (VA) were expected to correlate at < 0.30 [19]
	Known-groups analysis	mVFQ-25 scores were compared between groups defined in terms of VF severity category (mild and moderate versus severe) and MLMT scores [19]
Ability to detect change	Responsiveness	mVFQ-25 scores were compared among groups defined as ‘improved’ or ‘stable/worsened’ on the basis of changes in MLMT scores [19]

Stage 4: Development of the ViSIO-PRO and ViSIO-ObsRO Instruments

The qualitative data obtained in stages 1 and 2 informed the development and refinement of a conceptual model for RP/LCA, which is essential when developing a new instrument [20]. The data from all three stages of research, alongside input from expert clinicians, informed the development of draft items, instructions, and response options, selection of an appropriate recall period, and hypothesized conceptual domain structures for the two versions of the ViSIO instruments: the ViSIO-PRO for adult and adolescent patients with RP/LCA (aged ≥ 12 years) and the ViSIO-ObsRO for completion by caregivers of paediatric patients with RP/LCA (aged 3–11 years).

Results

Stage 1: Qualitative Research in RLBP1 RP

Qualitative Literature and Instrument Review in RLBP1 RP

A total of 10,191 abstracts were returned from the search of bibliographic databases for the combined qualitative literature and instrument review. Following removal of duplicates and abstracts that were not relevant or did not have relevant research objectives, a total of 111 full text articles were extracted. Upon detailed review of the full text articles, 66 articles were excluded, resulting in 45 relevant full text articles included in the review (list of publications used in review are provided in Table S3 in Supplementary Materials). Findings from the qualitative literature review provided evidence that vision-related functioning and HRQoL is significantly affected in patients with RP, and even more so for patients with the RLBP1 mutation. Specifically, patients with RLBP1 RP may lose as much as 45% of highest achievable vision-dependent functioning, which is a greater loss than that experienced by general patients with RP. Improvement in general vision, distance vision, near vision, or peripheral vision were most likely to have the greatest impact on vision-dependent functioning and HRQoL, while rates of depression were reported to be significantly high, which may impact both subjective and objective measures of visual function.

Most of the clinical trials for RP included tests of VA and VF as outcome measures. The search identified three existing visual function PRO instruments that have been used in RP populations: the NEI VFQ-25 [21, 22], the LLQ [23] and four selected items from the VAQ [24]. No existing ObsRO instruments were identified. The NEI VFQ-25 was the most commonly used PRO instrument in RP clinical trials, with limited evidence for use of the VAQ and LLQ. All three PROs assessed peripheral vision, and LLQ subscales of driving and emotional distress corresponded to VFQ-25 subscales of mental health and social functioning. However, none of the instruments assessed all concepts relevant and important to the vision-related functioning of patients with RP.

Concept Elicitation and Cognitive Debriefing Interviews of Existing Instruments with Patients with RLBP1 RP

Many visual function symptoms (e.g. night blindness, difficulty seeing in bright lighting) and proximal impacts on ADL (e.g. difficulties reading) and physical functioning (e.g. difficulties with mobility) were discussed by patients with RLBP1 RP. The visual function symptoms and impacts on vision-dependent ADL reported by the expert clinicians were broadly consistent with patient reports.

The CD portion of the interviews found that the NEI VFQ-25, LLQ and VAQ instruments did not provide a comprehensive assessment of visual function and vision-dependent ADL in RLBP1 RP. In addition to gaps in conceptual coverage, the instruments contained items that lacked relevance and included outdated examples such as sewing or using hand tools for work or hobbies. Some items were identified as being difficult for participants to interpret due to being insufficiently specific in terms of lighting conditions or familiarity of environments. Further detail regarding the study results has been reported elsewhere [5].

Stage 2: Qualitative Research in Wider RP/LCA

Social Media Listening Study

The review of social media posts including Twitter, news outlets, blogs and forums found that key visual functioning symptoms discussed by patients with RP/LCA and/or caregivers in social media posts (n = 209) included loss of night vision, tunnel vision/reduced visual field, and problems with colour vision. Challenges associated with those symptoms included difficulty reading, recognizing faces, looking at objects, and vision in low light. Impacts of RP/LCA on adult patients’ HRQoL (n = 185) included stress, depression, unemployment, dependence for mobility and inability to drive. Impacts on children with RP/LCA included difficulty reading a whiteboard, inability to play video games and caregivers’ inability to understand their child’s frustration.

Qualitative Literature Review

Searches returned 601 abstracts, of which 8 full-text publications and 2 abstracts were identified as relevant and reviewed in full (list of publications are provided in Table S4 in Supplementary Materials). Key concepts were identified relating to a wide range of visual function symptoms (e.g. night blindness and reduced/loss of peripheral vision), impacts on vision-dependent ADL (e.g. difficulties driving, reading, doing household chores and with self-care activities), mobility (e.g. navigating) and distal HRQoL (e.g. impacts on emotional wellbeing, social functioning, work, and school). Coping strategies and the use of visual aids to help patients manage their condition on a day-to-day basis were also reported. Environmental factors such as bright or dim lighting and unfamiliarity of environment were reported as having negative impacts on functional vision, and considered important factors when collecting patient-reported or caregiver-reported data (Table 2).

Table 2.

Summary of concepts reported in qualitative literature review

Domain	Concepts identified
Visual function symptoms (n = 10)	Night blindness (n = 6), restricted/loss of peripheral vision (n = 6), reduced light/dark adaption (n = 3), difficulty seeing contrast (n = 3), photopsia (n = 2), reduced vision in bright light (n = 2), restricted/loss of central vision (n = 2), colour blindness (n = 1), blurred vision (n = 1)
Impacts on vision-dependent daily activities (n = 10)	Mobility (n = 5), falling and tripping (n = 5), sports/physical activity (n = 4), walking into objects (n = 3), navigation (n = 2), hand–eye coordination (n = 1), balance issues (n = 1) Driving (n = 7), reading (n = 5), viewing digital screens (n = 3), travel/transport (n = 3), shopping (n = 2), household chores (n = 1), misplacing things (n = 1), self-care (n = 1), difficulties engaging in hobbies (listening to music, knitting, gardening, dancing and singing; n = 1 each)
Distal HRQoL (n = 10)	Emotional wellbeing (n = 9), social functioning (n = 8), independence (n = 6), interpersonal and family relationships (n = 6), on work and school (n = 6), financial impact (n = 3)
Coping strategies (n = 8)	Emotional coping [n = 7; including acceptance (n = 5), optimism/positivity (n = 5), humour/laughter (n = 2) and appreciating life (n = 1)], learning to do things in different ways (n = 1), planning ahead (n = 1), listening to music (n = 1), watching TV (n = 1), doing housework (n = 1), taking more time to complete activities (n = 1), meditation (n = 2), spirituality (n = 2), prayer (n = 2), yoga (n = 1), counselling (n = 1), engaging in new forms of physical activity that are more accessible to those with visual impairment (n = 1), family support (n = 3) and social support (n = 2)
Visual aids (n = 6)	White cane (n = 4), guide dog (n = 3), low vision assistive devices (n = 3) and technology at work (n = 1)
Environmental factors (n = 7)	Bright or dim lighting (n = 6), unfamiliarity of environment (n = 4), busy/confined environments (n = 3) and bright or overcast weather conditions (n = 2)

Development and Refinement of a Conceptual Model

The conceptual model displays the key concepts associated with RP/LCA, including visual function symptoms, moderating environmental factors, impacts on vision-dependent ADL, mobility, and distal HRQoL and coping strategies/visual aids used by patients to manage their disease (Fig. 2). Impacts were grouped according to their proximity to the condition and categorized as either impacts on patients’ ADL and mobility or other aspects of HRQoL.

Fig. 2.

Conceptual model of RP/LCA

Stage 3: Psychometric Evaluation of the mVFQ-25

The psychometric analyses of the mVFQ-25 data were used to identify items that performed well across most analyses, some analyses or those that performed poorly (Table 3). Overall, 15 items performed well across most psychometric tests (e.g. strong item properties, good convergent validity, were responsive to change over time), while 10 items performed poorly on several psychometric tests (e.g. high ceiling effects, poor test–retest reliability). The concepts assessed by items that performed well in the psychometric evaluation were used to inform the development of the ViSIO-PRO and ViSIO-ObsRO instruments. Previous qualitative work identified that no existing PROs were suitable for use in RP/LCA, including the modifications to the mVFQ-25. Details of the psychometric analyses conducted on the mVFQ-25 total score can be found in Viriato et al. (2019) [25].

Table 3.

Summary of psychometric performance of mVFQ-25 items

Psychometric performance	Modified VFQ-25 (mVFQ-25) item
Items performed well across the majority of the psychometric tests carried out: demonstrating strong item properties, convergent validity (moderate-to-strong correlations at 1-year follow-up with MLMT, FST and visual field scores), good item-level known-groups analysis (distinguished between patients who differed in their visual field and MLMT scores) and were responsive to changes over time from baseline to 1 year	• Item 4 (walk in unfamiliar outdoor places without help at dusk); • Item 19 (bump into objects when they are in a new position); • Item 21 (run into things in unfamiliar environments); • Item 25 (overshoot objects or run into things if they have been moved from their usual spot)
Items performed well for most of the psychometric analyses conducted (e.g. test–retest reliability, convergent validity) but were not able to detect changes in scores over time	• Item 2 (locate doorknobs and handles without first passing your hand over them); • Item 3 (find things that are moved from their normal spots quickly); • Item 7 (read normal print books or see details in pictures without magnifiers or computer/television screens); • Item 8 (see a movie in a theatre or a planetarium show); • Item 9 (find a plate, fork and spoon in a restaurant that is dimly lit); • Item 10 (read labels on cans/food products/medicine bottles); • Item 11 (read street signs or signage in buildings) Item 14 (trouble identifying simple shapes); • Item 16 (put things down in specific places so that you can find them by feel/touch); • Item 20 (need assistance to walk confidently in new places); • Item 24 (hesitate before using stairs or getting onto an escalator or going through a revolving door)
Items performed poorly on several of the psychometric analyses, including item response distributions (ceiling effects), unsupported convergent validity, poor test–retest reliability, and nonresponsive to change	• Item 1 (read lighted dials or lights on electronic equipment); • Item 5 (recognize people first by vision alone); • Item 6 (find a new bus stop); • Item 12 (difficulty judging whether someone is male or female using only vision); • Item 13 (trouble determining the expression on unfamiliar faces using only vision); • Item 15 (difficulty seeing which direction hands on a clock (non-digital) are facing); • Item 17 (make mistakes when dressing in good light); • Item 18 (make mistakes when using the bathroom or bathing); • Item 22 (get lost or disoriented in familiar environments); • Item 23 (activities that make you uncomfortable or nervous because of the way your vision is)

Stage 4: Development of the ViSIO-PRO and ViSIO-ObsRO Instruments

Informed by the concordance of findings across stages 1–3 and expert clinician input, it was decided that to sufficiently assess visual function and vision-dependent ADL, mobility and HRQoL, two instruments should be developed for patient (ViSIO-PRO) and caregiver (ViSIO-ObsRO) completion. Development of two instruments was deemed suitable for the target population: a PRO for patients aged ≥ 12 years and an ObsRO for parents and/or caregivers of children aged 3–11 years. Separate versions were chosen in line with ISPOR Task Force [10] recommendations and FDA PRO guidance [12], which prefer self-report for individuals aged 12–18 years, while acknowledging that some paediatric populations are unable to report their own health status and recommend observational measures rather than proxy measures in such cases. For the ObsRO, the chosen age range (3–11 years) also reflected current therapies available, whereby patients as young as 3 years in the USA are eligible for ocular gene therapy (Luxturna) [26]. As some patients’ visual impairments may mean they are unable to self-complete the questionnaire, a self-administered and interviewer-administered version of the PRO was developed.

The instruments were designed to assess severity or frequency of concepts over a 7-day period. This recall period was considered short enough to avoid recall bias but long enough to allow items to be sensitive to changes over time and for patients to have the opportunity to do the daily activities assessed (e.g. household chores). The ViSIO-PRO utilises a 5- or 7-point verbal descriptor response scale. The Likert-type response options in the VFQ-25, LLQ and VAQ assessing difficulties and/or frequency were understood by most in the CD interviews (stage 1). The ViSIO-ObsRO instrument uses fewer response option categories (4- or 5-point scales) as it was expected to be difficult to answer on a more granular scale on the basis of only observations.

Items were developed to specify lighting conditions and familiarity of environment. Findings from the CD interviews (stage 1) indicated that patients found it difficult to respond to items if the lighting condition or familiarity of environment was not specified, as this would affect patient ability to perform activities, resulting in different interpretations of items. The ObsRO instrument includes fewer items specifying lighting/familiarity conditions so that they are more likely to be observed during the 7-day recall period. Key lighting conditions were selected on the basis of the previous qualitative research in RLBP1 RP (stage 1).

The developed 44-item ViSIO-PRO (v1.0) and 26-item ViSIO-ObsRO (v1.0) instruments are designed to assess impacts on vision-dependent ADL, mobility and distal HRQoL by measuring the level of difficulty experienced by patients with RP/LCA (aged ≥ 12 years) or children with RP/LCA (aged 3–11 years), respectively, when in specific situations or performing a variety of ADL that significantly rely on visual function. An overview of the concepts reported within each of the different sources used to inform item development is provided in Fig. 3.

Fig. 3.

Summary of concepts reported across sources used for item development

Although the mVFQ-25 included an item assessing ability to identify shapes, shape recognition did not directly map on to any of the identified key concepts relating to RP/LCA, and was therefore not included in either the ViSIO-PRO or ViSIO-ObsRO instruments. Similarly, driving was a concept which lacked relevance for most patients in the interviews with patients with RLBP1 RP, as their condition was too severe to do this activity. As such, driving was not included in v1.0 of the ViSIO-PRO. Additionally, a number of concepts were not included in the ViSIO-ObsRO instrument as it was considered unlikely that they could be reliably observed and reported on (i.e. night blindness, vision in bright lighting, contrast sensitivity, loss of peripheral vision, recognising people, embarrassment, worry), or they were inappropriate, as children may rely on the parent/caregiver to help with certain activities (i.e. self-care activities).

Discussion

To understand the patient experience of RP/LCA and support the inclusion of PRO and ObsRO instruments in future RP/LCA clinical trials, the present study summarizes the various research activities undertaken to develop the ViSIO-PRO and ViSIO-ObsRO instruments for use in RP or LCA populations. Research activities included: a qualitative literature review and review of pre-existing visual function PRO instruments in RLBP1 RP; qualitative CE and CD interviews of pre-existing instruments with patients with RLBP1 RP, expert clinicians, and expert payers; a SML study; a qualitative literature review in wider RP/LCA; and psychometric evaluation of the mVFQ-25 in RPE65-mediated IRD (LCA). This ultimately led to the development of the draft ViSIO-PRO and ViSIO-ObsRO (v1.0) instruments in accordance with regulatory guidance including the US FDA PRO guidance for industry [12], PFDD series [13], the EMA HRQoL reflection paper [14] and the ISPOR Task Force papers [10, 15, 16].

Findings from the qualitative literature reviews in RLBP1 RP and wider RP/LCA provided insights as to the range of visual function symptoms that patients experience and the significant impact that these have on patients’ vision-related ADL and distal HRQoL. In-depth qualitative interviews with patients with RLBP1 RP identified additional visual function symptoms (e.g. impaired depth perception) and impacts on vision-related functioning (e.g. cooking) that had not been previously reported in the published literature. These sources informed the development and refinement of a conceptual model; a visual representation of the patient experience and key features of RP and LCA.

The instrument review, as well as cognitive debriefing of the pre-existing instruments identified (the NEI VFQ-25, VAQ and LLQ), confirmed that there were no instruments suitable to comprehensively assess the concepts of relevance to patients with RP/LCA, even if administered in combination. Inconsistencies in item and response option interpretation and limited relevance of item concepts further highlighted the need for the development of PRO and ObsRO instruments to adequately assess the patient experience of RP and LCA.

Data generated from the research activities outlined here informed the development of the 44-item ViSIO-PRO (v1.0) and 26-item ViSIO-ObsRO (v1.0) instruments, designed to assess vision-dependent ADL, mobility and HRQoL by measuring the level of difficulty experienced by patients with RP/LCA (aged ≥ 12 years) or children with RP/LCA (aged 3–11 years), respectively, when in specific situations or performing a variety of ADL that significantly rely on visual function. Each item is assessed over a 7-day period and uses a severity or frequency 5–7 point verbal descriptor response scale. The ViSIO-PRO is a self-report instrument and developed for self- or interviewer-administration. The ObsRO is based on informant reporting (parent/caregiver of a child aged 3–11 years with RP/LCA), and so caregivers will self-complete the instrument. As the ObsRO aims to assess observable content (i.e. behaviour) as much as possible rather than subjective aspects of the child’s experience (a proxy measure) to align with regulatory and best practice recommendations, [10, 27] the ObsRO includes less items compared with the PRO.

In parallel to the development of the ViSIO-PRO and ViSIO-ObsRO instruments, the Michigan Retinal Degeneration Questionnaire (MRDQ), a PRO instrument designed for use in IRD, was developed [28]. Although the MRDQ has been developed for, and undergone content and psychometric validation in an IRD population, it is not validated for use in RP and LCA populations specifically. The MRDQ also focuses on capturing visual functional symptoms only. As demonstrated through the in-depth qualitative research conducted with patients with RP/LCA and their caregivers, a number of vision-dependent activities are reported to be significantly impacted within this population (e.g. ADL, mobility), and are therefore important to capture within a PRO or ObsRO to comprehensively assess the patient experience of RP/LCA.

The current study has a number of key strengths. The ViSIO-PRO and ViSIO-ObsRO account for the limitations of previous PRO instruments that have been used (but not developed for use) in RP/LCA, including the assessment of concepts in different lighting conditions and familiarities of environment. Furthermore, various sources of data were used to inform item generation and development (e.g. clinician input, literature review, patient interviews, and psychometric data from a similar instrument), which in the context of a rare disease is important in generating adequate evidence of the patient experience and supporting content validity within the population of interest.

However, the study does have the limitation that the qualitative interviews were conducted primarily with patients with the RLBP1 RP genotype. Despite this, findings from the qualitative literature review in wider RP indicate that similar concepts may be experienced across RP/LCA genotypes, supporting the development of an instrument for use in broader RP/LCA. Additional qualitative research will explore the patient experience in a larger sample of patients with RP/LCA and caregivers of child patients with a variety of genotypes.

Conclusions

The results from the qualitative research supported the development of the ViSIO-PRO and ViSIO-ObsRO instruments to assess visual functioning symptoms and vision-dependent ADL, mobility and HRQoL in RP/LCA, in accordance with regulatory requirements. Qualitative research is going to assess the content validity of the instruments for use in this population and evaluate their measurement properties. Following content and psychometric validation, the final versions of the ViSIO-PRO and ViSIO-ObsRO instruments could be used in future RP/LCA clinical trials to inform trial endpoints and support product label claims, as well as in clinical practice or larger research studies to track disease severity.

Supplementary Information

Below is the link to the electronic supplementary material.