Comments about outcome measures for clinical trials of interventions for post-stroke patients with hemianopia

Dear Editor

We write to comment on the recent paper by Rowe and colleagues¹ reporting the results of a pilot randomized clinical trial of the effectiveness of treatments for post-stroke patients with hemianopia. Although the study followed generally accepted criteria for the design of clinical trials, there was a major problem with the selection of the primary outcome measure, and, as a result, with the use of that primary outcome measure as the basis of the sample size calculation for a future clinical trial. There were also problems with the choices of some of the secondary outcome measures. The authors reported in their earlier protocol paper,² that one of the aims of the pilot clinical trial was to consider “the appropriateness of the outcome measurements” (page 8). We feel the results reported in the recent pilot trial results paper,¹ and our comments here, require that there be a fundamental re-consideration of these outcome measures before they are implemented in a future full-scale clinical trial.

Participants were randomized to one of three arms: 1) peripheral prism glasses, 2) visual search training, or 3) standard care. Peripheral prism glasses are designed as a mobility aid to assist with detection of blind side obstacles.³ When the prisms are worn, they provide expansion of the field of view measurable with standard perimetry,^3–5 which has been reported by patients to be helpful for obstacle avoidance when walking.^{3, 4, 6, 7} When not worn, the prisms are not expected to provide any help or effect. Visual search training is expected to increase visual exploration (scanning) toward the blind hemifield,^8–10 which may in turn improve detection of objects on the blind side.^{8, 11}

Outcome measures should be selected to be relevant to the expected effects of the interventions (in this case, improved blind side detection performance). However, in the Rowe et al. pilot study, the primary outcome measure was the relative change in visual field area from baseline to the 26-week follow up. Visual field measurements were performed without the prism glasses^{1, 2} so the visual field expansion effects of the prism glasses were never considered. When performing traditional visual field measurements, scanning is prohibited, so visual search training would not be expected to have any effects on visual field measurements. As the Rowe et al. study did not include any other objective measures of blind side detection performance, either with prisms or with scanning, neither the effects of the prisms nor the effects of the visual search training on blind side detection could be directly measured. The authors did not provide any rationale, either in the protocol paper² or the pilot results paper,¹ for why change in visual field area was selected as the primary outcome measure.

Rehabilitation for individuals with visual impairment (whether of visual acuity or visual field) is not expected to change the impairment itself, but is rather to reduce the effect of that impairment on the ability to perform activities of daily living. Earlier studies have examined separately the effect of peripheral prisms, and the effect of visual search/explorative saccade/compensatory scanning training, in stable hemianopia. Some of those studies used performance-based measures of effectiveness relevant to activities of daily living, such as avoidance of obstacles in a mobility course,^{8, 11} responses to potential hazards during on-road driving,¹² and responses to pedestrian hazards in a driving simulator.¹³ All these measures were found to be responsive to the intervention.

In the Rowe et al pilot study, participants were recruited between 2 and 26 weeks post stroke. Prior research suggests that there may be spontaneous recovery of the visual field up to 3 months (12 weeks) or even 6 months (24 weeks) following the stroke.¹⁴ Therefore it is quite possible that some participants might have experienced some recovery of the visual field and it was appropriate to check visual fields at the end of the trial as a control, but not as a primary outcome. In the discussion of the pilot results paper,¹ the authors report “minimal non-significant increase in visual field across all three arms of 5, 8 and 3.5%” and then go on to state “the insignificant change in visual field was expected given the deliberate recruitment of stable hemianopes to the trial. Other trials recruiting stable hemianopes also report no significant change to extent of visual field loss”. Yet the visual field measure was described as the “primary efficacy outcome” (results paper,¹ section 2.10), which implies that the interventions were expected to result in some recovery of the visual field (i.e. an increase in area from baseline). If the authors expected no significant change in the visual field, then it is hard to understand why a change in visual field area was included as the primary measure of efficacy.

There are many reasons to suggest that visual field area should not be considered as an efficacy outcome measure for the interventions in the future clinical trial proposed by Rowe and colleagues. There are neither theoretical reasons nor empirical evidence to support the idea that either visual search training or peripheral prisms would result in visual field recovery. In fact, prior studies have reported no effects of search training^{8, 9, 11} or peripheral prisms⁴ on visual field size (when measured without prisms). Indeed, the Rowe et al. paper¹ even cited a recent Cochrane systematic review¹⁵ of interventions for post-stroke field loss stating that the review “concluded that, generally, interventions for homonymous hemianopia do not result in improvement of visual field”. That review paper addressed prism glasses, visual search/scanning training and even treatments that are intended to increase the visual field.

One of the aims of the Rowe et al. pilot clinical trial¹ was to use the visual field data as a basis for a sample size calculation for a future clinical trial. The authors computed sample sizes for various minimally clinically important changes in visual field area (200, 400 and 600 degrees²). However, no rationale was provided as to why an increase in field area of any of these amounts would be clinically important, and the calculations seem to have been a meaningless exercise in sample size estimation when the underlying expectation was no change in visual field area. For context, the 40 Δ prisms used in the study would have provided field expansion areas of about 400 degrees² in both the upper and lower visual fields for a total of 800 degrees² when in use.

In the conclusions of the pilot results paper,¹ the authors suggested that an alternative primary outcome measure might be needed for a future clinical trial. “Given that visual fields did not change significantly and require patients to attend follow-up appointments at hospital eye clinics (a potential deterrent to trial participation) an appropriate alternative primary outcome measure may be a vision-related quality-of-life questionnaire such as the VFQ25.” The National Eye Institute Visual Function Questionnaire-25 (VFQ-25) was one of the secondary outcome measures in the study, along with a number of other questionnaires addressing visual function, mobility, and activities of daily living.¹ The selection of questionnaires was justified by “All measures have been used extensively in previous stroke research and are sensitive to change, valid and reliable” (page 6, protocol paper²). These statements may be true for some of the questionnaires in some instances, but no supporting evidence was cited. More importantly, the authors did not specifically address whether there was any evidence that the questionnaires were sensitive (relevant) to the expected effects of the interventions used in their pilot study. Indeed, one questions whether the VFQ-25 would be a good outcome measure because it includes very few questions that are specific to the expected effects of visual search training or the use of peripheral prism glasses. Similarly, the Rivermead Mobility Index has many items which relate to physical limitations on mobility (e.g., moving from lying to sitting; running 10 m in 4 s) which are unlikely to be responsive to either a visual or an oculomotor rehabilitation strategy, making it not valid for use in this context.

Performance on the Radner reading test was also included as a secondary outcome measure.^{1, 2} However it had been previously reported that reading speed was unaffected by compensatory scanning training.^8–10 In a cross-over study in which specific oculomotor training to rehabilitate reading was provided, in addition to compensatory scanning training, it was found that the training was specific to the intended modality, and did not transfer to the other task.¹⁶ Peripheral prism glasses were also never intended to help with reading and patients are usually told not to wear them for prolonged reading.^{3, 4, 6} Therefore, the reading measure was irrelevant to the expected effects of either therapy, and no rationale was given for why it was used as an outcome measure.

Systematic reviews^{15, 17} have highlighted the need for randomized clinical trials of interventions for patients with homonymous visual field loss, and we would fully support this contention. The pilot trial by Rowe and colleagues was undoubtedly well intentioned in this respect. However, the primary outcome measure (change in visual field area) could never have provided any useful information about the efficacy of the two interventions and, therefore, is also inappropriate as the basis for a sample size calculation for a future clinical trial. The choice of some of the secondary outcome measures for the pilot study was equally unjustifiable and they would not make reasonable alternate primary outcome measures for the future trial. It does not matter how rigorous the design of a clinical trial, if the outcome measures and the basis of the sample size calculation are not appropriate, then the results will be meaningless. Development of good performance-based outcome measures that are relevant to the effects of the interventions, relevant to activities of daily living of patients with hemianopia¹⁸ and suitable for implementation in clinic-based randomized controlled trials, is an ongoing challenge for the field of hemianopia rehabilitation research.