Reply to Helgeson et al.: Remote 6-minute-Walk Testing in Patients with Pulmonary Hypertension: Further Validation Needed?

From the Authors:

We read with interest the letter by Helgeson and colleagues and appreciate the opportunity to continue this important and timely discussion on the value of remote 6-minute-walk testing (6MWT) in pulmonary hypertension (PH). We agree that the risk versus benefit of in-clinic unmasked walks is unfavorable given concerns about communicable respiratory viruses (including severe acute respiratory syndrome coronavirus 2 [SARS-CoV-2]) in our vulnerable patient population. We acknowledge that our study (1) requires additional validation in more settings and in more patients and applaud the recent work by Helgeson and colleagues (2) to this end. For the sake of discussion, we raise the following points.

While it is certainly possible that masking does not impact 6-minute-walk distance (6MWD), we note that in studies where conditions like masking are compared retrospectively, it is not possible to control for an order effect. As the order (masked versus unmasked) is not randomly counterbalanced, correlation cannot be distinguished from causation. In our prospective study, the order of 6MWTs was performed at random and we completed all walks over a six-week period, which is approximately half the time of the in-clinic walks reported by Helgeson and colleagues. It is possible that a temporal effect confounded the results of both studies. As masking cannot be blinded, perceived exertion during a masked 6MWT may be subject to reporting and recall bias. Swiatek and colleagues did randomize facemask order in their design with healthy participants and found no evidence of a difference in the distance walked but indeed an increase in dyspnea with use of a mask (3). Although masking order was counterbalanced in our study, the systematic bias we observed whereby masked participants achieved shorter 6MWD than when unmasked could be caused by cardiopulmonary limitations due to PH, psychology, or both. The act of wearing a facemask may cause patients to perceive a restriction of air, which may negatively influence exercise capacity; likewise, wearing a facemask may lead to self-regulation and shorter 6MWD. Surprisingly, in our study Borg Scales for breathlessness and fatigue were concordant in masked versus unmasked walks and not significantly different (Figure 1). As pointed out by Helgeson and colleagues, an observer effect is also possible—patients may perform better (or worse) in front of clinical staff or trusted companions. Studies manipulating observers and real versus sham facemasks (facemasks with all layers removed but the paper top) could be designed to address these limitations. Of course, to demonstrate evidence of “no difference” with any intervention, a study must be (a priori) designed and powered to test for either noninferiority or equivalence. Alternatively, concordance can be evaluated, as we have done here.

Figure 1.

Concordance between masked versus unmasked Borg Scales. (A) Bland-Altman plot with solid black line as reference line. Solid red line is the observed difference between masked and unmasked walks in the Borg score for breathlessness, dashed red line is ±2 SD, and dashed green line is ±3 SD. (B) Bland-Altman plot with solid black line as reference line. Solid red line is the observed difference between masked and unmasked walks in the Borg score for fatigue, dashed red line is ±2 SD, and dashed green line is ±3 SD.

Second, the distinction between absolute versus relative differences in walk distance should be considered. For example, take a subject who walked 375 m in-clinic (at the lower end of the distribution in our study) but achieved 400 m remotely—a “moderate” absolute difference of 25 m and a relative increase of 7%—and compare this to a second subject who walked 600 m in-clinic (the upper end of the distribution) but 650 m remotely—a “large” discrepancy of 50 m but only an 8% relative increase. The discrepancy of the latter participant is twice as large as that of the former and surpasses the minimally important difference, though the relative differences are nearly the same (7% vs. 8%). It is for this reason that comparing absolute distances between participants on different ends of a spectrum should be done with caution. To further address this issue, we natural-log transformed distance from our study and re-ran the Bland-Altman analysis. As shown in Figure 2, the relative discrepancies are consistent with the previously published analysis of absolute discrepancies (1). In other words, our results are consistent for both relative and absolute differences in 6MWD.

Figure 2.

Natural-log transformed values of in-clinic versus remote and masked versus unmasked 6-minute-walk distance, respectively. Note that participants at the extreme may be artificially concordant because of a ceiling effect. (A) Bland-Altman plot with solid black line as reference line. Solid red line is the observed difference between in-clinic and remote walks, dashed red line is ±2 SD, and dashed green line is ±3 SD. (B) Bland-Altman plot with solid black line as reference line. Solid red line is the observed difference between masked and unmasked walks, dashed red line is ±2 SD, and dashed green line is ±3 SD.

Finally, we acknowledge that the differences between in-clinic and remote tests are “noisier” compared with facemask versus no facemask. These results are not surprising as in-clinic is a controlled setting and likely familiar to all participants as established patients at our centers, while remote was not. Conversely, there was less variation in conditions and likely less noise in the mask analysis. As the fourth walk in our study was optional (and the walk used to compare masking), we may have selected for PH participants who were willing to do the extra walk, healthier, and more prone to a ceiling effect. It is our hope that larger, prospective studies can address these gaps in studies to-date on this topic as we integrate mobile health into the care of patients with PH.