The Michigan Vision-related Anxiety Questionnaire: A Psychosocial Outcomes Measure for Inherited Retinal Degenerations

Abstract

Purpose:

Construct and validate a patient-reported outcome measure for screening and monitoring vision-related anxiety in patients with inherited retinal degenerations.

Design:

Item-response theory and graded response modeling to quantitatively validate questionnaire items generated from qualitative interviews and patient feedback.

Methods:

Patients at the Kellogg Eye Center (University of Michigan) with a clinical diagnosis of an inherited retinal degeneration (n=128) participated in an interviewer-administered questionnaire. The questionnaire consisted of 166 items, 26 of which pertained to concepts of “worry” and “anxiety”. The subset of vision-related anxiety questions was analyzed by a graded response model using Cai’s Metropolis-Hastings Robbins-Monro algorithm R (version 3.6.3) mirt package. Item reduction was performed on the basis of item fit, item information, and item discriminability. To assess test-retest variability, 25 participants completed the questionnaire a second time 4-16 days later.

Results:

The final questionnaire consisted of 14-items divided into two unidimensional domains: rod-function and cone-function anxiety. The questionnaire exhibited convergent validity with the Patient Health Questionnaire (PHQ-4) for symptoms of depression and anxiety. This vision-related anxiety questionnaire has high marginal reliability (0.81, 0.83) and exhibits minimal test-retest variability (ρ=0.81 (0.64, 0.91); 0.83 (0.68, 0.92)).

Conclusions:

The Michigan Vision-related Anxiety Questionnaire (MVAQ) is a psychometrically validated 14-item patient-reported outcome measure to be used as a psychosocial screening and monitoring tool for patients with inherited retinal degenerations. It can be utilized in therapeutic clinical trials for measuring the benefit of an investigational therapy on a patient’s vision related anxiety.

INTRODUCTION

Inherited retinal diseases (IRDs) are a diverse set of conditions caused by mutations in over 270 different genes identified to-date¹. These conditions present with a wide range of clinical phenotypes and unique symptomatology depending on the particular retinal physiological pathways affected. These phenotypic patterns include: rod-cone, cone-rod, cone, and macular dysfunction.

IRDs can often lead to severe vision impairment which may substantially impact psychosocial aspects of a patient’s quality-of-life. Current literature suggests that visually impaired populations suffer higher rates of depression, and are more likely to seek mental healthcare^2,3. Furthermore, as visual function declines, patients may experience worsening symptoms of depression and anxiety^4-6.

Changes in mental health are noteworthy outcome measures that have been minimally considered in IRD treatment. It is important for clinicians to understand vision-related anxiety to better guide treatment. Obtaining patient reported outcomes (PROs) by means of valid and reliable questionnaires are useful in measuring and monitoring changes in disease state based on a patient's perspective and experience. While some existing PRO measures incorporate mood-related questions^7-9, there is currently no sufficiently validated and reliable tool to measure psychosocial changes related to IRD conditions and their affected physiological pathways. Given the mixed phenotypes of IRDs, it is best to understand which aspects of visual dysfunction are causing patients to experience challenges in daily life and emotional well-being. There is an unmet need for a vision-related anxiety PRO instrument for patients with IRDs that can be used as; 1) a screening tool for symptoms of mental health decline, and 2) a monitoring tool for therapeutic interventions and low-vision rehabilitation.

This article describes in detail the results of a psychometric validation of the Michigan Vision-Related Anxiety Questionnaire (MVAQ), a PRO instrument created in accordance with U.S. Food and Drug Administration guidelines¹⁰ and capable of detecting manifestations in psychosocial symptoms pertaining to an IRD condition. The application of this tool is an opportunity to capture a previously unmeasured health outcome in a context that is meaningful to a patient’s daily life.

METHODS

Approval from the University of Michigan Institutional Review Board (HUM00115127) was obtained prior to the study and the research was performed in accordance with the Declaration of Helsinki. Participants were all English-speakers, at least 18 years of age, and able to provide informed consent. Participants were recruited during routine clinical visits from December 2016-March 2020 at the Kellogg Eye Center (University of Michigan) Retinal Dystrophy Clinic, and had a clinical diagnosis of an IRD that was confirmed by a fellowship-trained specialist (KTJ, ATF). Participants with other ocular conditions affecting visual function were excluded. Electronic health records were reviewed for pertinent clinical data such as visual acuity, Goldmann visual field testing, full field electroretinography (ffERG), spectral domain optical coherence tomography (SD-OCT), fundus auto fluorescence, clinical examination, pedigree analysis and ophthalmic medical history. Participants were not excluded on the basis of prescription medications, including anti-depressants and/or anxiolytics.

Phase 1: Item generation

PRO content items were generated through a rigorous qualitative analysis of content information obtained from in-depth patient interviews designed by a multidisciplinary group of IRD care-providers. A preliminary questionnaire was developed and iterative revisions of the questions and responses were based on patient feedback, as previously described¹¹. Of the 166 drafted items, 26 items were designated as pertaining to “worry” or “anxiety”.

Phase 2: Administration

Participants were consented in-person at the Kellogg Eye Center. Each interview with the 166 originally drafted items ranged from 35-50 min and was verbally administered by a research assistant. A subset of participants completed the questionnaire a second time 4-16 days later to assess test-retest variability in order to determine measurement error for longitudinal studies.

Phase 3: Psychometrics Analysis and Item reduction

Graded Response Model

Item Response Theory analysis for the MVAQ was performed using a methodology similar to the development of the Michigan Retinal Degeneration Questionnaire (MRDQ)¹². Unidimensional domains were identified by factor analysis and then fit to a Graded Response Model using Cai’s Metropolis-Hastings Robbins-Monro algorithm¹³ in R mirt package¹⁴ (version 3.6.3). Patient-level vision anxiety is represented by θ, which describes the latent trait measured by a singular domain. Within the model, θ scores are centered at zero where a higher person-score represents greater anxiety. Each item within the model was evaluated for inclusion in a domain by analysis of Item Probability Traces and Item Information Curves. Overall domain performance and ability to discriminate person-level (θ) anxiety were assessed using Test Information Curves, standard error functions, and marginal reliabilities. Model fit was evaluated by Standardized Root Mean Square Residual, Root Mean Square Error of Approximation, Comparative Fit Index, Tucker-Lewis Index, and Cai and Monro’s C2¹⁵. The graded response models were compared to Method of Successive Dichotomizations models with likelihood ratio statistics¹⁶. After item-reduction, Differential Item Functioning analysis was conducted for sex, age, visual acuity, IRD phenotype, Patient Health Questionnaire (PHQ-4) score, and use of anti-depressant or anxiolytic medications. Likelihood ratio tests and Bonferroni-corrected p-values were used to examine differential item functioning.

Domain & Trait Associations and Reliability

Participant characteristics including corrected Snellen visual acuity¹⁷, age, sex, and IRD phenotype were evaluated for association with domain scores using bivariate linear models. Reliability was assessed via Pearson correlations of test and retest scores and mean and standard deviation of domain change scores between repeated test administrations (Bland-Altman analysis).

Patient Health Questionnaire-4 (PHQ-4) and Self-Rated Health

After completing the preliminary questionnaire, participants were asked items from the validated PHQ-4^18,19 to assess symptoms of depression and anxiety. The four-item PHQ short form was selected to minimize burden of subject fatigue. The PHQ-4 questionnaire gives a total score on a 0-12 point scale as well as two 0-6 point subscales: depression and anxiety. Additionally, participants were asked two questions to self-rate their general health and quality of vision, each on a 5-point Likert scale. Responses were compared to MVAQ using one-way ANOVA and linear regression.

RESULTS

Phase 1: Item generation

Fifty-five adult patients with IRDs were interviewed to generate content items targeting four conceptual domains pertaining to visual function¹¹. Repeated patient feedback in the early interviews revealed a need to address themes of worry and anxiety, which prompted the generation of items targeting vision-related anxiety.

Phase 2: Administration

One hundred and twenty-eight patients with IRDs were asked items pertaining to vision-related anxiety in functional daily tasks. Participant characteristics are shown in Table 1. Genetic testing results with a conclusive pathogenic variant were available for 77 patients. The questionnaire was administered a second time for test-retest variation in 25 patients.

Table 1:

Participant Characteristics and Demographics

Total sample size, n	128
Female, n (%)	65 (50.8)
Age (yr), Median (Range)	49 (18-88)
Corrected visual acuity, Median (Range)
Better Eye	20/42 (20/16 – NLP)
Worse Eye	20/60 (20/18 – NLP)
Conclusive Genetic
Test Result, n (%)	77 (60.2)
IRD phenotype, n (%)
Rod-cone	69 (53.9)
Cone/cone-rod	30 (23.4)
Macular	29 (22.7)
Race/Ethnicity, n (%)
White, non-Hispanic	107 (83.6)
Black/African-American	10 (7.8)
Asian	3 (2.3)
Hispanic	5 (3.9)
Unknown	3 (2.3)

IRD=Inherited retinal dystrophies; NLP=No light perception

Phase 3: Psychometrics Analysis and Item reduction

Factor analysis revealed that the 26 items designated as “worry/anxiety” loaded on two unidimensional vision-related anxiety domains: 1) rod function related anxiety, and 2) cone function related anxiety. Six items were eliminated due to poor factor loading on either domain. In the rod domain, three items were removed due to low information and poor item probability trace performance. In the cone domain, two items were removed due to poor conceptual fit and one item was removed for poor item information curve and item probability trace performance. No items were removed due to differential item functioning. Supplementary Table 1 shows the removed items.

Domain Score Associations and Reliability

Domain scores for rod function and cone function related anxiety were significantly correlated with visual acuity in the better eye (p=0.0128, p=0.0006), in the worse eye (p=0.0166, p=0.0019), and IRD phenotype (p= 0.0001, p=0.0198). For the two visual acuity measures, the associations were characterized by more anxiety with poorer visual acuity. Given the heterogeneity of IRD phenotypes, either rods or cones can be predominantly affected, in which some IRD conditions may correlate more strongly with either MVAQ domain. Patients with rod-cone dystrophies had more rod-predominant anxiety than patients with macular dystrophies (p<0.001, Figure 1). Patients with cone/cone-rod dystrophies had more cone-predominant anxiety than patients with rod-cone dystrophies (p=0.005, Figure 1). Age and sex showed no significant correlation with either domain score (Table 2). Twenty-four (18.8%) participants self-reported taking anti-depressant and/or anxiolytic medications. When comparing repeat administrations of the MVAQ, participants exhibited minimal change in MVAQ domain scores between repeated administrations. Test-retest reliability and Pearson’s correlations are shown in Table 2.

Figure 1: Domain Score and Phenotype Associations.

Associations between Domain Scores (θ) and IRD Phenotype compared using Kruskal-Wallis test, followed by pairwise comparisons were undertaken using Mann-Whitney test with Bonferroni corrections.

Table 2:

Reliability, Trait Associations, and Model Fit Statistics for 128 participants

	Rod-function anxiety	Cone-function anxiety
No. of Questions	6	8
Marginal Reliability	0.81	0.83
Test-Retest, (95% CI)
(n=25)
ρ Correlation	0.81 (0.64, 0.91)	0.83 (0.68, 0.92)
Mean Change	−0.14 (−0.37, 0.09)	−0.05 (−0.24, 0.14)
SD ME	0.43 (0.31, 0.55)	0.36 (0.22, 0.50)
Trait Associations, R2 (p-value)
VA Better Eye	4.9 (0.013)	8.9 (<0.001)
VA Worse Eye	4.5 (0.017)	7.4 (0.002)
Date of MVAQ	0.1 (0.716)	0.3 (0.555)
Age	0.0 (0.845)	0.4 (0.470)
Sex	2.7 (0.068)	1.4 (0.184)
IRD Phenotype	15.4 (<0.001)	7.6 (0.020)
Model Fit Statistics
SRMSR	0.09	0.14
RMSEA	0.4	0.18
CFI	0.85	0.88
TLI	0.75	0.83
M2	182	32.6
df	9	20
p-value	<0.001	<0.001

Marginal reliability estimated from original 128 participants. Three test-retest statistics (Pearson correlation, mean difference, and standard deviation of measurement error) and their 95% confidence intervals were computed from 25 pairs of tests taken approximately two weeks apart.

Associations between Domain Scores (θ) and Participant Characteristics measured by adjusted R² of linear model and by p-value of the F-test of no association. Corrected logMAR visual acuity (VA) taken closest to questionnaire administration.

Model fit statistics provided include the Standardized Root Mean Square Residual (SRMSR), root mean square error of approximation (RMSEA), Comparative Fit Index (CFI), Tucker-Lewis Index (TLI), and the hybrid C2 of Cai and Monro (2014) with its degrees of freedom (df) and p-value.

Michigan Vision-related Anxiety Questionnaire (MVAQ)

After item-reduction, the complete MVAQ is a 14-item instrument with two domains, rod (6 items) and cone (8 items) function-related anxiety. The final MVAQ items are presented in Table 3 along with graded response model parameter estimates. Figure 2 contains, for each domain, a person-item map, which describes the distribution of participant scores and item difficulty. See supplementary materials for domain information and standard error (Supplementary Figure 1) and eliminated items (Supplementary Figure 1).

Table 3:

Graded Response Model parameter estimates for 14 questions of MVRAQ.

Item	Question	α	β1	β2	β3
	Rod-function anxiety
Q01	Worry bumping into people/objects during the night	3.64	−0.15	0.48	0.97
Q02	Worry bumping into people/objects in poorly lit areas	3.74	−0.19	0.47	1.02
Q03	Worry about seeing during the night	9.12	−0.17	0.26	0.78
Q04	Worry about seeing in poorly lit areas	10.34	−0.23	0.24	0.67
Q05	Worry about uneven ground during the night	7.48	−0.32	0.14	0.51
Q06	Worry about uneven ground in poorly lit areas	7.28	−0.34	0.15	0.52
	Cone-function anxiety
Q01	Worry when reading up close	1.73	0.30	1.21	1.69
Q02	Worry when reading at a distance	1.52	−0.65	0.70	1.69
Q03	Worry when distinguishing colors	3.50	0.39	1.15	1.78
Q04	Worry when seeing against similar backdrops	3.90	0.06	0.97	1.53
Q05	Worry about recognizing faces	2.44	0.32	0.94	1.50
Q06	Worry when going to unfamiliar places during the day	1.94	−0.24	0.66	1.19
Q07	Worry about bright fluorescent lights	1.91	0.42	1.26	1.81
Q08	Worry about going out on bright sunny days	1.52	0.81	1.71	2.40

Figure 2: Person-item Maps for Rod and Cone Function Anxiety.

PHQ-4 and Self-Rated Health

A statistically significant relationship was observed between domain anxiety and PHQ-4 screening score (Table 4). A positive linear association was observed between PHQ-4 total score and both rod-function (p=0.0013) and cone-function (p=0.0035) anxiety. This trend remains when analyzing rod and cone function relative to the PHQ-2 depression (p<0.001, p=0.0054) and GAD-2 anxiety (p=0.0233, p=0.0113) subscales of the PHQ-4. Furthermore, an association was observed between rod and cone function domain anxiety and patient self-reported feelings about heath (p=0.012, p=0.0154) and quality of vision (p<0.001, p<0.001).

Table 4:

Associations between Domain Scores (θ), PHQ-4, and self-rated health

	Rod-function anxiety	Cone-function anxiety
Feelings about health	7.0 (0.012)	6.5 (0.015)
Feelings about vision	14.7 (<0.001)	17.6 (<0.001)
PHQ Categorized Total Score	15.9 (<0.001)	4.6 (0.031)
PHQ4 Total Score	18.9 (<0.001)	14.6 (0.002)
Depression subscale	11.7 (0.002)	7.2 (0.020)
Anxiety subscale	10.3 (0.003)	5.7 (0.032)

Measured by adjusted R² of linear model and by p-value of the F-test of no association.

DISCUSSION

In the era of emerging IRD therapeutics, the development of standardized, validated, and reliable outcome measures is critically important²⁰. Previously, patients with IRDs had minimal treatment options, but recent scientific progress has led to numerous therapeutics under development, including novel gene therapies^20-22. Furthermore, as critical evaluation of potential therapeutics are required, investigators and clinicians must consider how treatment efficacy can be measured in a way that is meaningful to patients’ daily lives²³.

The MVAQ will add a new dimension by which providers can follow a previously unmeasured potential signal of treatment efficacy. While those who care for patients with IRDs recognize the psychosocial impact of IRDs, little attention has been paid to sound measurement of this impact. There is good evidence that mental health challenges disproportionately affect patients with visual impairment. Patients face greater stress, report more symptoms of depression, and increased risk of suicide^24-28. The literature has already demonstrated that psychosocial health can affect patient performance on visual function testing such as visual field testing^29-31. Depression and poor mental/emotional health have repeatedly been correlated with worse visual function, and in fact may be predictive of poorer physical health in an IRD population^4,5,32-36. Furthermore, self-rated health has been more strongly associated with quality of life than traditional clinical tests^2,37, suggesting that a patient’s perception of their quality of vision may be more impactful on quality of life. Previous literature has focused more on the effects of depression; while the specific role of anxiety in visually-impaired populations is less well understood³⁸. Clinicians and investigators should consider these patient perspectives when designing and evaluating treatment options.

Interventions that improve psychosocial outcomes are recognized and have been documented in treatments for chronic pain³⁹ and coronary artery disease⁴⁰. In a meta-analysis of psychosocial interventions for low-vision, a small effect was observed in reducing depression, while no significant effect was observed in other psychosocial symptoms⁴¹. In studies of visual impairment in older populations, interventions such as “self-management” techniques have shown efficacy in managing depressive symptoms^42-44. While improved vision through low-vision rehabilitation may relieve some psychosocial symptoms, it is not clear if psychosocial changes in response to mental health interventions or low-vision rehabilitation can be reliably measured⁴⁴. Existing studies suggest that certain patient-level factors such as positive coping strategies and social support are associated with better psychosocial health while other factors such as age and negative life experiences are associated with risk of “vision-related distress” ^27,36,45-49. In addition to these sociodemographic factors, understanding the underlying vision pathways that influence vision-related anxiety can lead to further targeted psychosocial interventions.

The National Eye Institute Visual Function Questionnaire-25⁹ (NEI VFQ-25), Vision Quality of Life Index (VisQoL)^7,50, Impact of Vision Impairment (IVI)^51,52 and EuroQol EQ-5D (EQ-5D)^53,54 are widely used and validated PRO measures, that have been employed to monitor and evaluate patients^55-60. While some have attempted to address different facets of psychosocial well-being, they were not developed for the application to an IRD population. As such, these existing PRO measures may not be sensitive to detect a change in low-vision IRD populations undergoing therapeutic or rehabilitative interventions^23,44. Furthermore, the NEI VFQ-25 and VisQoL include emotional well-being questions within the overall questionnaire, but do not address the specific anxiety constructs identified in our IRD population. While the IVI has an independent emotional well-being domain and has been modified to become a sample-independent PRO measure⁶¹, the IVI domain addresses emotions of daily life and does not inform the underlying retinal pathologic pathway leading to the emotional distress.

PRO item banks developed by Prem-Senthil and colleagues are well constructed via both qualitative and quantitative techniques, and targeted to the hereditary retinal disease population^62-64. However, to our knowledge, these Hereditary Retinal Disease Item Banks do not specify the visual/retinal pathophysiological pathway relating to responses in emotional well-being. Understanding and measuring anxiety of patients in cone-related and rod-related anxiety axes enables better targeting of interventions and measurement of efficacy of the interventions. While vision-related anxiety might be predominant in either the rod or cone axes, all IRD patients should be evaluated for both rod and cone-related anxiety.

The MVAQ’s development followed U.S. Food and Drug Administration recommendations, its patient-generated content, and its strong psychometric properties, are all encouraging features that contribute to its applicability as an IRD psychosocial outcome instrument. Despite the recent popularity of Rasch methodology in item response theory analysis, a graded response model was determined to be a more appropriate item response theory model given its flexibility to incorporate items without the need for uniform item discimination^65-68. When a polytomous Rasch model was compared to a graded response model, the more restrictive Method of Successive Dichotomization model was rejected for each domain. The graded response model is a recognized method by the PROMIS initiative⁶⁹ and has already been employed in ophthalmology PRO measures^65,67,70.

An important new feature of the MVAQ is its ability to relate psychosocial reports to a retinal physiologic pathway. Furthermore, the MVAQ disability scores exhibit correlations with VA, phenotype, and self-reported feelings. Other PRO measures ask about a patient’s feelings or concerns, but their responses cannot be linked to the pathologic defect underlying their visual impairment. In contrast, the MVAQ allows a clinician to map the concerns of the patient into an actionable area for improvement. As opposed to knowing that a patient experiences “fear” or “worry”, with the MVAQ a clinician can identify whether a patient’s rod dysfunction is the predominant cause of their anxiety, and recommend accommodations to their living setting and vision-related activities to address these specific anxiety-provoking situations. These findings serve to alert providers to other sources of anxiety that may have previously been overlooked.

The Patient Health Questionnaire (PHQ-4) is an established metric for assessing symptoms of depression and anxiety which has been validated and in-use in low vision populations ^19,48,71-76 The correlation of MVAQ with the PHQ-4 demonstrate the MVAQ’s convergent validity and suggest that the latent traits measured in the MVAQ are related to symptoms of clinical depression and generalized anxiety. As already documented, patients with low vision and IRDs are at increased risk for poor mental health, quality of life, and suicide^2,4,25,26. IRD specialists are often a patient’s consistent point of contact with the healthcare system; yet in a survey of eye care providers, “absence of standard procedures”, “limited knowledge”, and “lack of training” were the most commonly reported barriers to addressing depression in patients with visual impairment⁷⁷.

There were still several limitations in the validation of the MVAQ. The study was performed at an academic institution and in a largely white, non-Hispanic population. This survey should be validated in other more diverse populations with distinct cultures and languages in order to establish its generalizability. The current PRO is limited to the adult population and future efforts should be directed towards developing a similar tool for the pediatric population. Even with 14 items and five response choices interrogating anxiety, there was a bimodal distribution of rod and cone vision-related anxiety measured by theta scores. As would be expected, patients with rod-cone dystrophy had more rod-related anxiety and patients with cone/cone-rod dystrophy had more cone-related anxiety.

Proposed comprehensive care models have emphasized a multidisciplinary approach to include genetic counseling and psychosocial resources^78-80. With additional training for eye care providers, commonplace psychosocial screenings during ophthalmic visits can be an opportunity for mental health outreach and a means to connect patients with resources for care and support⁸¹. Furthermore, MVAQ allows for a new dimension of patient-partnered care in which psychosocial concerns are better understood by IRD providers. This provides a tool to bridge the understanding and provide a more holistic form of patient care. To solve the unmet need in IRD care and research, the MVAQ is available as a screening tool, and with further longitudinal evaluation, the MVAQ can be used as a monitoring tool.

CONCLUSION

The current study demonstrates psychometric validation of items pertaining to vision related anxiety in patients with IRDs and collectively form the MVAQ. The MVAQ measures vision-related anxiety due to dysfunction in either a predominantly cone pathway or a rod pathway. It can be used as a screening tool to direct the attention of a clinician to counseling and psychosocial interventions, as well as to measure baseline and follow-up vision related anxiety for targeted low vision rehabilitation and psychotherapy. Finally, the MVAQ may also be incorporated in therapeutic clinical trials to understand the benefit of a novel therapeutic intervention for reducing a patient’s vision-related anxiety.

Supplementary Material

Appendix A: Supplementary Data

Supplemental Figure 1: Domain Test Information and Standard Error

Solid line represents overall domain information curve. Dotted line represents domain standard error.

Suppl Table 1

Supplemental Table 1: Original items not included in final MVRAQ and reasons for exclusion

IPT=Item probability trace; IIC=Item information curve; CD=Concept divergence; FL=Factor loading

HIGHLIGHTS.

• Patient-reported outcomes measure specific to inherited retinal degenerations

• Development of the Michigan Vision-related Anxiety Questionnaire (MVAQ)

• Screening and monitoring tool for psychosocial health in the context of visual function