Comparison of Physician-Predicted to Measured Low Vision Outcomes

Abstract

Purpose

To compare low vision rehabilitation (LVR) physicians’ predictions of the probability of success of LVR to patients’ self-reported outcomes after provision of usual outpatient LVR services; and to determine if patients’ traits influence physician ratings.

Methods

The Activity Inventory (AI), a self-report visual function questionnaire, was administered pre and post-LVR to 316 low vision patients served by 28 LVR centers that participated in a collaborative observational study. The physical component of the Short Form-36, Geriatric Depression Scale, and Telephone Interview for Cognitive Status were also administered pre-LVR to measure physical capability, depression and cognitive status. Following patient evaluation, 38 LVR physicians estimated the probability of outcome success (POS), using their own criteria. The POS ratings and change in functional ability were used to assess the effects of patients’ baseline traits on predicted outcomes.

Results

A regression analysis with a hierarchical random effects model showed no relationship between LVR physician POS estimates and AI-based outcomes. In another analysis, Kappa statistics were calculated to determine the probability of agreement between POS and AI-based outcomes for different outcome criteria. Across all comparisons, none of the kappa values were significantly different from 0, which indicates the rate of agreement is equivalent to chance. In an exploratory analysis, hierarchical mixed effects regression models show that POS ratings are associated with information about the patient’s cognitive functioning and the combination of visual acuity and functional ability, as opposed to visual acuity or functional ability alone.

Conclusions

Physicians’ predictions of LVR outcomes appear to be influenced by knowledge of patients’ cognitive functioning and the combination of visual acuity and functional ability - information physicians acquire from the patient’s history and examination. However, physicians’ predictions do not agree with observed changes in functional ability from the patient’s perspective; they are no better than chance.

Visual impairment can cause significant interference with one’s ability to function independently or perform activities of daily living. According to the American Optometric Association guidelines¹ of caring for patients with vision impairment, one of the major goals of a low vision evaluation is to “identify patients with visual impairment(s) who might benefit from low vision care and rehabilitation.” The same goal is expressed by the ophthalmology profession, as summarized by Markowitz² and as outlined in the American Academy of Ophthalmology’s Preferred Practice Pattern Guidelines for Vision Rehabilitation.³ The goal of Low Vision Rehabilitation (LVR) is to optimize the remaining vision of patients suffering from vision loss and to enhance independence and functioning. Thorough case history and evaluation often are employed to generate an understanding of the patient’s functional difficulties and estimate the patient’s rehabilitation potential.

Like other areas of medical decision-making, the development of an optimal plan for LVR requires the clinician to evaluate the patient and choose interventions that have the highest likelihood of producing the desired outcome. Low vision rehabilitation, like other skilled health care services, requires the clinician to have the knowledge and experience to estimate potential outcomes of different interventions conditioned on the traits of the patient. The aim of the patient evaluation is to provide the clinician with the information needed to create a rehabilitation plan, which includes choices of vision assistive equipment in addition to choices types of adaptations, accommodations, and visual skills instruction. To choose the best intervention for a given patient, the clinician must be able to predict the likelihood of success of different alternatives for that patient.

Estimating the effectiveness of intervention requires knowledge of outcomes, either from personal experience or from evidence in the literature. Although many low vision rehabilitation outcome studies have been performed, a recent systematic review of studies of the effectiveness of low vision services⁴ concluded there is wide variability in the magnitude of the effects of service, the choice of outcome measures, and the quality of research. Prior to 2000, LVR outcome studies did not agree on the choice of outcome measure,⁵ with a variety of observations being used, such as asking patients how satisfied they were with low-vision intervention^6,7 or asking them about their use of low-vision aids.^8,9 Since 2000, most outcome studies have used patient-reported outcome measures obtained with visual function questionnaires^10,11 and/or quality of life questionnaires. ^12–15 Binns et al. summarized the LVR outcome literature and reported that approximately 90% of LVR outcome studies used patient-reported visual function as the primary outcome measure.⁴ Other outcome measures that have been used to assess the effectiveness of LVR include the frequency of use of low vision aids by patients,^8,16,17 gained independence⁷ and measures of accuracy and speed of performing surrogate daily living tasks.^18–21

Although several studies have shown that low vision rehabilitation can be successful in improving functional ability,^22–32 and other aspects of quality of life,^12,33 little is known about the information clinicians use to predict the outcomes of intervention, or if those predictions, i.e. prognoses, are accurate. Prognostication is important for all types of health care management. The physician uses known facts and prior knowledge to predict according to present indications or signs. Having expectations for an intervention is important in developing a plan of care, i.e., to know which methods will produce the greatest likelihood of a successful outcome for an individual patient. Few studies have evaluated prognostication of patient outcomes and most have been performed in intensive care units, where predictions are related to patient survival. In these few studies, physician predictions were not accurate.^34,35 Patients and clinicians often differ in terms of identifying the problem, prioritizing examination information and classifying an outcome as “successful”. Regarding eye care, patients’ definitions of LVR success are often centered on visually-guided, task-specific, functional measures such as reading, driving or cooking. Conversely, in most cases physicians treating the patient’s disease formulate definitions of success based on anatomical, structural or quantitative measurements, such as reducing intraocular pressure, retinal edema or neovascularization. Because the definitions of success in these cases differ between physicians and their patients, it is questionable whether both parties would rate an outcome the same way. However, in the case of low vision physicians, one would assume that alignment with the patient’s perspective in outcome expectations would be more likely because the aim of low vision specialists is to improve patients’ function.

During a low vision examination, the case history is a major part of the evaluation. The discussion of activities of daily living helps the clinician identify areas of functional difficulty and importance. The rehabilitation plan is set to achieve the activity-specific goals of improving visual function. Estimation of the patient’s rehabilitation potential influences the clinician’s decisions in creating the plan of care. The plan of care includes the clinicians’ judgment about which goals the patient will be able to achieve, which in large part, relies on patient-reported functional difficulties. A discussion over functional domains (e.g., reading, mobility, visual information processing and visual motor skills) helps the clinician understand the deficits encountered with the patient’s daily activities due to the vision impairment. The clinician elaborates on problems with activities of daily living the patient identifies as being difficult due to his or her vision loss. Based on the discussion, the clinician begins to develop a plan of care to know the direction to emphasize during rehabilitation.

Predictions commonly are incorporated in all aspects of examination. Prior experience or evidence-based practices drive actions. The ability to predict outcomes influences patient care and drives decisions in terms of device recommendations, rehabilitation training and treatment goals. To date, very little has been reported on outcome predictions and prognostication in eye care. The aim of this study is to elicit physicians’ predictions of the outcomes of LVR for individual patients, which presumably incorporate their estimates of the patient’s rehabilitation potential, and 1) compare them to actual post-LVR outcome measures based on patient self-reports and (2) compare their predictions to patient traits that could determine rehabilitation potential.

METHODS

Subjects

As part of a larger collaborative observational outcome study of usual low vision care, the present study was conducted on 316 new low vision patients across 28 outpatient LVR centers in the United States, by 38 optometrists and ophthalmologists who specialize in low vision rehabilitation. All participating clinical centers provide low vision services according to the AOA guidelines.^1,36 There were no visual acuity, visual field or diagnosis eligibility requirements for patient participation because the study aim was to evaluate typical patients presenting for low vision rehabilitation services. Subjects were recruited by the participating centers from their appointment lists before being seen in the low vision clinic and oral informed consent was obtained by telephone. The study protocol was approved by the Johns Hopkins Human Subjects Research Institutional Review Board (IRB) and, when required, by the IRBs of the participating centers. The 316 subjects enrolled in this study represented a subset of a larger group of 764 subjects who participated in the parent study. The study reported here started midway through the parent study. The distributions of visual impairments, disorder diagnoses, co-morbidities, and other baseline traits among study patients were the same as those for the larger group of patients (described in detail in reference 37). Telephone interviews to obtain baseline data were conducted before patients were evaluated by the low vision rehabilitation center. There was no intervention protocol – the participating centers provided their usual clinical low vision services.

Procedures

The primary aim of this study is to compare low vision physicians’ estimates of the likelihood of patients having a successful LVR outcome to the actual LVR outcomes from the patient’s perspective. The secondary aim is to determine which patient traits are likely to influence low vision physicians’ estimates of patients’ rehabilitation potential, expressed as an estimate of the likelihood of a successful outcome. To achieve these aims, we measured LVR outcomes using an adaptive patient self-report instrument, obtained likelihood estimates of a successful outcome from low vision physicians, and obtained measures of patient’s visual acuity, psychological state, physical functioning ability, cognitive function, and baseline visual functioning ability.

LVR Outcome Measure that Represents the Patient’s Perspective

The problem with most previously used patient-reported outcome measures is that they do not accommodate real world constraints on clinicians who must consider the differing needs, limitations, and preferences of individual patients. An exception, but not previously used as a LVR outcome measure, is Goal Attainment Scaling (GAS), which is scored by estimating how far individual patients are from their rehabilitation goals.³⁸ Specific individualized goals are created for each patient, which means the outcome measure must be adaptive; this approach is very different from the traditional methods that employ fixed-item questionnaires. Although the goal-specific nature of GAS is consistent with creating an individual plan for LVR, it has not been generalized and therefore makes comparisons of outcomes between patients very difficult to interpret.¹⁶ Additionally, the plan of intervention usually is created by the doctor or therapist in collaboration with the patient, but may be limited by what is thought to be achievable and not exclusively by what the patient considers to be important.

When considering interventions for the plan of care, it is important to take into account the patient’s perspective, focusing on activities that reportedly are important and difficult to the patient. Although GAS tries to bring measurement into agreement with the individual rehabilitation plan approach, GAS produces a therapist-reported outcome, not an outcome reported by the patient. Thus, at best GAS is a measure of LVR efficacy under the controlled conditions of the clinical encounter, not a measure of LVR effectiveness in improving the patient’s daily life.

Like GAS, the Activity Inventory (AI) is an adaptive outcome measure, customized to assess function relative only to those activities that are important to the patient and difficult to perform, and therefore in need of rehabilitation. But unlike GAS, outcome measurements with the AI are made from the patient’s perspective. The Activity Inventory (AI), a well validated self-report instrument,^39,40 provides quantitative estimates of functional ability in reading, mobility, visual motor, and visual information processing domains and an estimate of a more global visual ability variable that summarizes the low vision patient’s self-perceived overall ability to perform daily activities considered important to daily living by the patient. The AI item-bank consists of 50 general activity goals and 460 specific cognitive and motor tasks nested under the goals. For example, a goal of “cook daily meals” includes tasks such as “read recipes”, “measure ingredients”, and “read the stove or oven dial”. The AI is administered by asking the subject to rate the importance of independently performing each goal activity. If the subject reports that a goal is not important, the interviewer moves to the next goal. Otherwise, the subject is asked to rate the difficulty of performing the goal activity without the assistance of another person. If the subject reports that the goal is not difficult, the interviewer moves on to the next goal. If the subject responds with any other difficulty rating, he or she is then asked to rate the difficulty of each task under that goal, or report that the task is not applicable to them.

The AI was administered at pre-rehabilitation baseline by telephone interview³⁹ to the 316 new low vision patients enrolled in the study. The AI report included visual ability scores on a logit scale at the goal level and for each functional domain. Additionally, the report provided descriptive detail on the rated difficulty of each goal and task that the patient determined was important and at least slightly difficult to perform. Results of the patient’s responses to the AI at baseline in report form were then provided to the low vision clinician prior to her or his evaluation of the patient; however, there was no way of knowing if clinicians used the information provided.

Measures of Patient Health States

In addition to the AI, a series of general health state questionnaires were administered to the patient by telephone interview prior to his or her initial clinic visit. These questionnaires included an intake survey that elicited details about the patient’s ocular, medical, physical, psychological and social history;³⁷ the physical function component of the MOS Short Form 36 (SF-36);⁴¹ the 15-item version of the Geriatric Depression Scale (GDS);⁴² and the Telephone Interview for Cognitive Status (TICS).⁴³ The results of the intake survey, GDS and TICS were reported to the low vision clinician prior to the patient’s initial visit.

Patient Evaluation and Low Vision Physicians’ Estimates of the Likelihood of a Successful Outcome

Patients underwent a low vision evaluation as regularly done by the low vision specialist. Following their initial patient evaluation, the 38 participating optometrists and ophthalmologists who specialize in LVR estimated the probability of outcome success (POS) for each of their enrolled patients using a 0 to 100% visual analog scale.^44,45 No criteria or definitions of success were given to the physician.

All clinicians rating patients in this study were experienced LV specialists with years of rehabilitation experience, and were included in the Low Vision Research Network (LOVRNET) Study Group authorship. All were aware they were participating in an observational outcome study of usual care, using the patient-reported AI as the outcome measure, and no other variable.

Rather than providing specific instructions regarding the criteria or definition of success, it was expected that each patient would likely have a different plan of treatment, with the assumption that the clinician’s definition of success would be unique to the individual and based on the plan of treatment and the patient’s rehabilitation potential.

Clinician ratings of estimates of success were submitted to the data coordinating center online via secure web server through the Low Vision Research Network website within 48 hours following the initial evaluation of the patient. Data were collected prior to additional LVR. Patients then underwent usual care. The AI was re-administered by telephone interview 6–9 months later, after completion of LVR services. These two sets of AI measures formed the data set used for this study. Clinicians’ assessments of each individual patient were compared to the patient’s self-reported outcomes measured with the AI.

Analysis

For the purpose of this study, a Rasch model was employed to estimate two interval-scaled measures from the AI difficulty ratings: overall visual ability from the patient’s difficulty ratings of AI goals and reading ability from the patient’s difficulty ratings of AI reading tasks. Results from the AI included only responses to those goals and tasks that were reported by the patient as being at least “slightly difficult” at baseline. Goals and tasks to which the patient responded “not difficult” at baseline were excluded from the estimation of both the baseline and the post-intervention AI person measures because they would not be targeted by the patient’s rehabilitation plan and LVR changes the item, not the person (i.e., intervention-specific differential item functioning,) consequently unaddressed items artifactually reduce the magnitude of the effect of intervention.⁴⁶

The AI person measures for each patient at baseline and at post-intervention follow-up were estimated using Winsteps (v 3.65)⁴⁷ with the Andrich rating scale model⁴⁸ while the AI goal and task item measures and response category thresholds were anchored to low vision population values estimated from the baseline responses of more than 3200 low vision patients. Baseline and post-rehabilitation person measures were estimated for visual ability from difficulty ratings of AI goals and for reading function from difficulty ratings of AI reading tasks. Outcomes of low vision rehabilitation were calculated by subtracting the baseline person measure from the post-intervention person measure and expressed as a change score. Change scores greater than zero indicate overall improvement after treatment in the patient’s self-reported visual or reading ability to perform the activities that would be targeted by low vision rehabilitation. Change scores less than zero indicate worsening of the patient’s visual or reading ability after treatment. A change score of zero indicates neither improvement nor worsening after treatment.

Raw analog scale scores for the low vision physician’s POS were compared to the patients’ goal change scores for visual ability (estimated from AI goal difficulty ratings) and reading function (estimated from AI reading task difficulty ratings). Analyses of agreement between clinicians’ predictions of the likelihood of successful outcomes and AI change scores were performed in two ways. First a hierarchical random effects model (Systat 10) with POS as the dependent variable and visual ability or reading change score as the independent variable was employed with patients grouped by clinician (clinicians were not grouped by clinical center in the hierarchy because most clinical centers had only one clinician). Second, each clinician/patient pair was treated as an independent observation of a POS/reading or visual ability change score pair. Each observation was then dichotomously scored as “successful” or “unsuccessful” based on a criterion value of POS and scored as “successful” or “unsuccessful” based on a criterion value of the visual ability or reading change score. A matrix was created to compare different combinations of criteria for dichotomizing POS ratings and AI change scores. A concordance analysis was performed for all combinations of criteria ranging from 50% to 90% in increments of 10% for POS ratings and from 0.2 logit to 1.2 logit in increments of 0.2 logit for AI change scores. The kappa statistic was used to test the concordance between clinician predictions of outcomes and AI change scores for each combination of criteria. The kappa statistic incorporates the assumption that agreement between observations will sometimes occur by chance. Thus, a kappa of 1 indicates perfect agreement, whereas a kappa of 0 indicates agreement equivalent to chance. Bonferroni adjustment was used to correct p values for an alpha level of 0.05 with multiple comparisons.

Interval-scaled measures of physical functioning were estimated from Rasch analysis of patient ratings of the 10 physical functioning items in the SF-36. Similarly, interval-scaled measures of depressed mood were estimated from Rasch analysis of patient responses to the 15 items in the GDS. Cognitive functioning was estimated using TICS raw scores. Other patient health state measures included logMAR visual acuity, visual ability and reading function at pre-LVR baseline. The effects of the various health state measures on clinicians’ POS ratings were evaluated individually with hierarchical mixed effect models (i.e., patients grouped by clinicians) and together with a multivariable regression model.

RESULTS

Figure 1 is a histogram that illustrates the distribution of clinicians’ POS ratings across all patients. The POS scores ranged from 0 to 100% likelihood of success. The trend suggests modes in the distributions at 30%, 50%, 80% to 90%, and 100%, which implies that clinicians divided the continuous rating scale into four or five response categories. Clinicians tended to rate the likelihood of success high for most patients, indicated by a median score of 75%.

Figure 1.

The distribution of clinicians’ POS ratings across all patients.

Figure 2 is a histogram of the average POS score on the visual analog scale for each physician across all the patients evaluated by that physician. Overall, the physicians had a tendency to rate high with a median average POS score across clinicians of 75%.

Figure 2.

The average POS score for each physician across all the patients evaluated by that physician.

Figure 3 illustrates histograms for each of the measured patient health states: reading ability and visual ability (goals) at baseline, logMAR, TICS, GDS and SF-36. Figure 3a shows reading ability at baseline, measured by the AI in logits; the higher the value, the greater the patient’s ability. Figure 3b shows overall visual ability at baseline, measured by the AI in logits. Higher scores indicate greater visual ability. Figure 3c shows the distribution of visual acuity at baseline in units of logMAR. Higher values of logMAR indicate worse visual acuity. Figure 3d shows the distribution of patient cognition, measured by the TICS. Higher scores indicate higher cognitive ability. Traditionally, scores >31 indicate normal cognition, 27 to 31 indicate mild cognitive impairment, and scores ≤27 indicate dementia.⁴³ Figure 3e shows the distribution of patient-rated depression, measured by the GDS. Higher values correspond to greater levels. GDS raw scores have standard cut-offs. A raw score in the range of 0 to 4, translated by Rasch analysis to values less than −0.96 logit, indicates no depression. A raw score of 5 to 10, translated to the range of 0.96 to 1.5 logits Rasch score, indicates mild depression. Raw scores ≥11, translated to >1.5 logits Rasch score, indicate severe depression.⁴² Figure 3f shows the distribution of physical ability as measured by the physical component of the SF-36. Higher values indicate greater physical ability. Interpretation of Rasch scores for physical functioning capability measured with the SF-36 is best done by comparing the measure to specific activities the person would be able to perform. For example, people with a score of 3.2 logits or greater would be able to perform “vigorous activities such as running, lifting heavy objects, or participating in strenuous sports”; people with a score >1.35, but <3.2 logits, would report having no problems “walking more than a mile” but would not be able to engage in vigorous activities; people with a score >.25, but <1.35 logits would report no problems with “bending, kneeling or stooping” but would not be able to walk more than a mile; people with a score in the range of −1.5 to 0.25 logits would report being able to “walk one block” but would not be able to bend, kneel, or stoop; people with a score in the range of −3.5 to −1.5 logits would report having no problems “bathing and dressing yourself” but would not be able to walk one block; and patients with a score <−3.5 logits would not be able to bathe and dress themselves.⁴¹

Figure 3.

Histograms for each of the measured patient traits: physical functioning (SF-36), depressed mood (GDS), cognitive functioning (TICS), visual acuity (logMAR), baseline visual ability (goals) and baseline reading function (reading).

Figure 4 illustrates the distributions of AI change scores. Figure 4a shows the results for overall visual ability (Goals) and Figure 4b shows the results for reading function. The visual ability change scores ranged from −1.96 to 4.1 logits. The reading function change scores ranged from −2.9 to 4.59 logits. For both measures, AI change scores greater than zero indicate improvement in self-reported ability and AI change scores less than zero indicate worsening of self-reported ability. The mean visual ability change score is 0.75 and the standard deviation is 0.86 logit (mean change score is significantly different from zero, p<0.0001). The mean reading function change score is 0.48, and the standard deviation is 1.00 logit (p<0.0001).

Figure 4.

Histograms illustrating the distributions of AI change scores for visual ability (Goals) and reading function (Reading).

The average standard error of the estimate across patients is 0.8 logit on the visual ability change score and 0.54 logit on the reading change score. A clinically meaningful change score for each patient is a change that has less than 5% chance of being the result of measurement error (>1.96 SE). Using this criterion, 23% of patients had a clinically meaningful improvement in visual ability (0.7% of patients had a clinically meaningful decrease in visual ability) and 27% of patients had a clinically meaningful improvement in reading function (5% had a clinically meaningful decrease in reading function).

Figure 5a is a scatter plot of the clinician’s POS rating vs. the visual ability change score for all patients. The Pearson correlation is −0.076, which is not significantly different from 0 (p=0.094). Figure 5b is a similar scatter plot of the clinician’s POS rating vs. the reading function change score for all patients. The Pearson correlation is −0.05, which is not significantly different from 0 (p=0.201). Figure 5c is a spaghetti plot of regression lines fit to POS vs. visual ability change score across patients for each clinician. The regression coefficient from the linear hierarchical random effects model is −0.012, which is not significantly different from 0 (p=0.423). The constant for the model is 0.764, which is significantly different from 0 (p<.0001). Similarly, figure 5d is a spaghetti plot of regression lines fit to POS vs. reading function change score across patients for each clinician. The regression coefficient from the linear hierarchical random effects model is −0.003, which is not significantly different from 0 (p=0.823). The constant for the model is 0.749, which is significantly different from 0 (p<.0001).

Figure 5.

(A) scatter plot of the clinician’s POS rating vs. the visual ability change score for each individual patient. (B) illustrates a scatter plot of the clinician’s POS rating vs. the reading function change score for all patients. (C) is a spaghetti plot of regression lines fit to POS vs. visual ability change score across patients for each clinician. (D) is a spaghetti plot of regression lines fit to POS vs reading function change score across patients for each clinician.

The results of concordance testing for POS with variable criterion values for success (ranging from 50 to 90%) versus visual ability (Goals) and reading function (Reading) AI change score criteria for a successful outcome (ranging from 0.2 logit to 1.2 logit) are displayed in Table 1. Each cell of the matrix contains a kappa value for that combination of outcome criteria. Positive values of kappa mean that the frequency of agreement between observations is better than expected by chance; negative values of kappa mean that that the frequency of agreement between observations is less than would be expected by chance; and a kappa value of zero means that the frequency of agreement is equal to that expected by chance alone. Across all comparisons, kappa ranges from −0.13 to 0.088 for visual ability change scores (AI Goals) and −0.09 to 0.09 for reading function change scores (AI reading). Correcting for multiple comparisons, none of the kappa values are significantly different from zero (for a criterion alpha of 0.05).

Table 1.

The results of concordance testing for POS with variable criterion values for success ranging from 50 or greater to 90% or greater (rows) versus visual ability (Goals) and reading function (Reading) AI change score criteria for a successful outcome ranging from 0.2 logit or greater to 1.2 logit or greater (columns). Each cell of the matrix contains a kappa value. None of the combinations of criteria for POS scores or AI change scores was significantly different from zero.

Kappa
				Goals AI
		0.2	0.4	0.6	0.8	1.0	1.2
POS	50%	0.064724	−0.07991	−0.11384	−0.9309	−0.0674	−0.07707
	60%	0.002761	−0.10565	−0.1187	−0.11403	−0.08908	−0.11206
	70%	−0.00237	−0.05491	−0.08239	−0.07078	−0.03837	−0.0801
	80%	0.011474	0.000909	−0.01126	−0.00865	0.036382	−0.02422
	90%	−0.01821	0.000361	0.007021	−0.04241	−0.02119	−0.08099
				Reading AI
		0.2	0.4	0.6	0.8	1.0	1.2
POS	50%	−0.04452	−0.0796	−0.06422	−0.03391	−0.01767	−0.00542
	60%	−0.0618	−0.08606	−0.07407	−0.04312	−0.01733	−0.01121
	70%	−0.03493	−0.06933	−0.07836	−0.04833	−0.00092	−0.00248
	80%	0.048898	0.013536	0.0272	0.006893	0.001394	0.001583
	90%	0.032501	−0.02013	−0.03693	−0.06365	−0.01757	−0.02215

Table 2 is a correlation matrix for all combinations of POS and the patient trait variables. The Pearson correlations range from 0.02 for GDS vs log MAR to 0.77 for baseline visual ability vs baseline reading.

Table 2.

A Pearson a correlation matrix for all combinations of POS and the patient trait variables.

	POS	SF-36 PF	GDS	TICS	Vis Ability	Reading
POS
SF-36 PF	0.16
GDS	−0.16	−0.37
TICS	0.25	0.30	−0.20
Vis Ability	0.21	0.31	−0.34	0.27
Reading	0.18	0.18	−0.24	0.25	0.77
LogMAR	−0.17	0.04	0.02	−0.14	−0.27	−0.44

Table 3 illustrates the results for a series of multiple variable hierarchical mixed effects regression models to explore the effects of individual patient traits or interactions of traits on clinician ratings of POS. For each regression model, POS is the dependent variable and patients are at the first level of the hierarchy and clinicians are at the second level. All the models assume a fixed coefficient across clinicians but a random constant, which represents expected differences between clinicians in response bias.

Table 3.

Illustrates the results for a series of multiple variable hierarchical mixed effects regression models to explore the effects of individual patient traits or interactions of traits on clinician ratings of POS. For each regression model, POS is the dependent variable and patients are at the first level of the hierarchy and clinicians are at the second level. All the models assume a fixed coefficient across clinicians but a random constant, which represents expected differences between clinicians in response bias.

Regression model A	Estimate	p-value
constant	0.554	0
TICS	0.006	0.003
GDS	−0.007	0.337
LogMAR	−0.078	0.003
SF-36	0.008	0.161
Vis Ability	0.034	0.142
Reading	−0.001	0.928
Regression model B	Estimate	p-value
constant	0.77	0
Reading	−0.001	0.934
Vis Ability	0.06	0.01
Reading × Goals	0.022	0.03
Regression model C	Estimate	p-value
constant	0.783	0
Reading	−0.007	0.681
LogMAR	−0.046	0.087
Reading × LogMAR	0.053	0.002
Regression model D	Estimate	p-value
constant	0.791	0
Vis Ability	0.002	0.943
LogMAR	−0.037	0.152
Vis Ability × LogMAR	0.091	0.001
Regression model E	Estimate	p-value
constant	0.763	0
Vis Ability × LogMAR	0.071	0.018
Reading × LogMAR	0.01	0.617
Vis Ability × Reading	−0.007	0.527
Regression model F	Estimate	p-value
constant	0.496	0
TICS	0.007	0
Vis Ability × LogMAR	0.099	0
LogMAR × TICS	−0.001	0.396
Goals × TICS	0	0.433
Regression model G	Estimate	p-value
constant	0.51	0
TICS	0.007	0.001
SF-36	0.013	0.678
Vis Ability × LogMAR	0.084	0
TICS × SF-36	0	0.8
Regression model H	Estimate	p-value
constant	0.452	0
TICS	0.008	0.005
GDS	−0.031	0.49
Vis Ability × LogMAR	0.082	0
TICS × GDS	0.001	0.618

The first model, shown in Table 3a has scores from the TICS, GDS, and SF-36, along with log MAR and baseline visual ability and baseline reading function measures serving as independent variables; no interactions between the independent variables are included in this model. Besides the constant, only the TICS score and log MAR coefficients are significantly different from zero. However, in Table 2 we can see that log MAR moderately correlates with baseline reading function (r = −0.44) and weakly correlates with baseline visual ability (r = −0.27), while baseline reading function strongly correlates with baseline visual ability (r = 0.77). To explore the predictive power of these variables, the next model displayed in Table 3b includes only baseline reading function, baseline visual ability, and the interaction between these two variables. With this combination of variables, only baseline visual ability and the interaction proved to be significant predictors of POS ratings. In the third model, shown in Table 3c, we substituted log MAR for baseline visual ability and observed that only the interaction of log MAR and baseline reading function is a significant predictor. The fourth model, shown in Table 3d, paralleled the third model but used baseline visual ability, log MAR, and their interaction as variables. The results were the same – only the interaction is significant. These models indicate that baseline visual ability, baseline reading function, and log MAR are not significant independent predictors, but the interactions of these variables are significant predictors.

The fifth model, displayed in Table 3e, included only the interactions between log MAR and baseline visual ability and between log MAR and baseline reading ability. In this case only the interaction between log MAR and baseline visual ability is significant. Therefore, the sixth model, displayed in Table 3f, which evaluates the predictive power of the TICS score, includes only the interactions between log MAR and baseline visual ability, between log MAR and the TICS score, and between baseline visual ability and the TICS score. Only the independent effect of the TICS score and the interaction between log MAR and baseline visual ability are significant.

The seventh and eighth models evaluate the independent effects of the SF36-physical functioning score and the GDS score on the POS and their interactions with the TICS score, shown respectively in Tables 3g and 3h. Again only the independent effects of the TICS score and the interaction between log MAR and baseline visual ability are significant.

DISCUSSION

Accurate outcome prediction in LVR is an important issue because the clinicians’ perceived outcome drives decisions regarding patient care. In LVR, the patient evaluation is focused on the patient’s ability to perform task-specific and goal-oriented activities. Based on this principle, it is reasonable to assume that the patient and low vision physician would have a high rate of agreement in terms of outcome success. Unfortunately, this study demonstrates the opposite findings, showing no agreement between low vision clinician predicted outcome success and actual outcomes based on patient self-report.

The aim of the study is to assess usual care provided by LV specialists in the U.S. The most common type of low vision service in the U.S. is as described in the AOA guidelines.¹ These services tend to be clinic-based, with the exception of comprehensive rehabilitation like those provided in the Veterans Affairs Blind Rehabilitation Centers⁴⁹ or those provided by private and state vision rehabilitation agencies.⁵⁰ The clinician uses the initial evaluation to create a plan of care with optimal intervention based on the clinician’s decisions about the patients. But, the results of this study show that clinician estimates of patients’ likelihood of success are poorly correlated with the patient-reported outcomes. Thus, if the clinician is using information gathered during the initial evaluation to create the plan of care, the information gathered, although reasonable for estimating rehabilitation potential, does not appear to be helping the clinician predict the outcomes. This result does not mean the clinician is misinterpreting her or his observations, but does suggest that the information being collected is not sufficient to predict patient-reported outcomes accurately.

Clinicians tend to rate the POS on the high end of the scale, with a median score of 75%. Only 12% of patients rated were given a score <50%. This distribution of ratings suggests that either clinicians are generally optimistic or they had reason to assume high rehabilitation potential for most of their low vision patients. Even though most ratings were high, the scores were distributed along the upper end of the scale, which indicates the clinicians noted differences in rehabilitation potential between the patients rated. The disagreement between clinician ratings and measures of LVR outcomes suggests that low vision clinicians lack feedback from the patient and most likely are not aware of their patients’ outcomes.

One problem is that low vision patients commonly do not return to the low vision center following the initial visit for follow up as recommended. The lack of independent transportation (only 30% drive)³⁷ coupled with high reliance on others for activities of daily living support⁵¹ make compliance with follow up appointments difficult. Additionally, because of a lack of familiarity with and education on low vision services, patients seeking outpatient care may anticipate restorative rather than rehabilitative approaches, which could affect acceptance of treatment and the provision of feedback. This type of miscommunication could occur at the start of the evaluation. Physicians obtain critical information during the case history, as reported by the patient. Unfortunately, the patient-supplied information may over-emphasize chief complaints and not adequately cover the full range of problems the patient has performing activities important to daily living. The difficulties with non-discussed activities would not be included in the rehabilitation plan, or taken into account when assessing the patient’s rehabilitation potential. Because to a large extent LVR is activity-specific, if daily activities important to the patient and difficult to perform are not addressed in the rehabilitation plan, improvement in the patient’s ability to perform those activities probably will not occur.

Results of the exploratory models support the hypothesis that many physicians’ predictions of rehabilitation outcomes are driven by information about the patient’s cognitive functioning and the combination of visual acuity and the patient’s overall functional ability, as opposed to visual acuity or functional ability alone. Although these are rational choices for estimating the patient’s rehabilitation potential, we have to conclude that this information is insufficient for making accurate predictions of functional outcomes important to the patient. Unfortunately there is very little information available in the literature on the relationship between patient traits and LVR outcomes that can help inform the clinician’s decision-making. This study suggests the need for developing a system with which LVR clinicians can obtain unbiased outcome measures on their own patients and see how those outcomes are influenced by patient traits, which could inform their clinical decisions and improve the quality and effectiveness of LVR.