Accuracy of Patient Recall of Hand and Elbow Disability on the QuickDASH Questionnaire Over a Two-Year Period

Abstract

Background:

Patient self-reporting questionnaires such as the QuickDASH, a shortened version of the Disabilities of the Arm, Shoulder and Hand (DASH) outcome measure, are critical to current orthopaedic outcomes research. The use of these questionnaires could introduce recall bias in retrospective, case-control, and cross-sectional studies if no preoperative data has been collected prior to study inception. The purpose of this study was to quantify recall accuracy on the QuickDASH questionnaire as a function of the duration of the recall interval.

Methods:

This cross-sectional study enrolled 140 patients with nontraumatic hand and elbow diseases. Patients were stratified into groups of thirty-five based on the time since their initial office visit (three months, six months, twelve months, or twenty-four months). All patients had completed the QuickDASH as part of a standard intake form at the time of the initial office visit (actual baseline score). Patients were contacted by phone and asked to recall their upper extremity disability from the time of the initial office visit with use of the QuickDASH questionnaire. Patients also completed the QuickDASH to rate their current disability. Actual and recalled QuickDASH scores for each group were statistically compared. Kruskal-Wallis analysis was used to determine any differences in recall accuracy between the groups. Pearson correlation coefficients quantified relations between recall accuracy and patient age and current function (absolute QuickDASH scores).

Results:

Mean differences between recalled QuickDASH scores and actual scores were all less than the QuickDASH minimal clinically important difference (MCID) of 13 points at different time points: three months (–7.1, p < 0.01), six months (0.8, p = 0.79), twelve months (–2.3, p = 0.43), and twenty-four months (–2.8, p = 0.26). There were no significant differences in recall accuracy across the four groups (p = 0.77). Recalled QuickDASH scores were highly correlated with actual baseline values (r_p ≥ 0.74). Recall accuracy was neither correlated with patient age nor current QuickDASH scores (r_p ≤ 0.04).

Conclusions:

Patients with a nontraumatic hand or elbow diagnosis are able to recall prior level of function accurately for up to two years with the QuickDASH questionnaire. Although data collected prospectively remain optimal, our data suggest that research conducted with use of recalled QuickDASH scores produces reliable assessment of disability from common upper extremity diagnoses with acceptable recall bias.

Level of Evidence:

Diagnostic Level III. See Instructions for Authors for a complete description of levels of evidence.

Orthopaedic outcomes research is increasingly emphasizing the use of standardized patient-rated questionnaires. In prospective study designs, investigators collect such measures of disability prospectively before the intervention and also after the intervention to quantify treatment effect. However, case series, case-control studies, retrospective cohorts, and even cross-sectional studies may rely on patients’ ability to recall preintervention impairment in order to determine an intervention’s effectiveness^1,2.

Despite the convenience of using recall patient-rated questionnaires, this reliance on patient memory introduces the possibility of recall bias. Recall bias is a well-known form of systematic error. In orthopaedic surgery, this may bias results secondary to differential recall of a prior health state that is either skewed by the current health state or by the severity of the pretreatment condition. Alternatively, random error related to the duration of time elapsed since the health state of interest may be introduced during recall. Few studies have measured agreement between patients’ recalled and actual preintervention function on validated orthopaedic patient-rated outcome measures^2-4. Furthermore, these studies have not evaluated recall bias as a function of time. The existing studies that have attempted to measure recall accuracy at a single time point after intervention demonstrate varying levels of agreement between what the patient recorded at the time of initial assessment and what the patient remembers^2,3,5-7. There is also limited evidence on factors that may contribute to the reliability of patient recall⁶. Without this knowledge, the validity of clinical research based upon post hoc patient recall of their preintervention state of function remains uncertain.

The QuickDASH questionnaire, a shortened version of the Disabilities of the Arm, Shoulder and Hand (DASH) outcome measure, is a validated, standardized questionnaire that is a responsive and reliable assessment of patients’ upper extremity disability^8-12. The eleven-question set is a subset of the DASH, one of the most commonly utilized patient-rated outcome measures in recent hand surgical series¹³. This study was designed to determine if patients could accurately recall prior self-rated function (QuickDASH scores) between three months and twenty-four months after their initial office visits (IOVs). We hypothesized that patient recall accuracy would decrease with increasing time since the IOV. Our null hypothesis was that there would be no effect of recall duration (time from the IOV) on the patient’s accuracy in recalling their preintervention functional state. Second, we sought to test the null hypothesis that patient factors would not affect patient recall accuracy. We expected that recall bias would exist, with recall error being positively correlated with current health such that patients with better current function would underestimate past impairment.

Materials and Methods

We obtained institutional review board approval prior to conducting this cross-sectional study. The study was designed to directly compare current QuickDASH scores and recalled baseline QuickDASH scores with actual baseline scores obtained at the IOVs. Current and recalled baseline QuickDASH completion was performed explicitly for research purposes and followed verbal consent by all participants. Actual baseline QuickDASH scores obtained during new patient visits as part of standard clinical practice for the senior authors (M.I.B. and R.P.C, two fellowship-trained hand surgeons) were accessed from the electronic medical record. Four separate time points after the IOV were studied. These included three months (ninety days ± fourteen days), six months (180 days ± fourteen days), twelve months (365 days ± fourteen days), and twenty-four months (730 days ± fourteen days). Patients in each cohort were contacted by phone and asked to complete the QuickDASH questionnaire over the phone, including their recollection of both their disability at the time of the IOV and their present disability. At the time of phone contact, the two interviewers (J.G.S. and D.A.L.) were unaware of the actual baseline QuickDASH score. The recalled baseline scores were compared with each patient’s actual QuickDASH score that had been completed at the time of the IOV. This retrospective callback design was used because it most accurately reproduced actual retrospective studies that have used recalled information.

Inclusion criteria required participants to be over eighteen years of age, to have had a hand or elbow diagnosis established at the IOV, to have properly completed the QuickDASH questionnaire at the time of the IOV, and to have provided verbal consent to research participation.

Any patients unable to provide verbal consent for study participation and all patients without proficiency in the English language were excluded from the study. Patients were also excluded if they had completed a second QuickDASH questionnaire following the IOV, and if the reason for the office visit was secondary to acute trauma or infection. Trauma and infection were exclusion criteria because patients with these acute changes in health often present to our practice and express difficulty completing the QuickDASH secondary to recent injury, after which they have not attempted many of the queried tasks. This is supported by prior studies documenting difficulty in accurate recall of function upon initial presentation following treatment for traumatic injury or acute infection^14,15.

Electronic health records of new patient visits to the senior authors’ hand clinics were searched from July 2010 to May 2012. Eligible patients were determined for each cohort with the criteria outlined above. With use of a standardized phone script, eligible patients were contacted and asked first to complete the QuickDASH questionnaire over the phone based on their recalled level of disability at the time of the IOV. In order to assess patients’ current functional state, patients then completed the QuickDASH questionnaire based on their current disability. This order was chosen to ensure that our main outcome (the recalled QuickDASH score) was not affected by possible systematic bias caused by “learning” of the questions. The original office visit QuickDASH questionnaire as well as patient age, presenting diagnosis, and intervention were recorded from the electronic health record to determine if these factors affected patient recall accuracy. In cases of multiple interventions, the most invasive treatment was documented.

Statistical Analysis

Sample size estimates, with use of an alpha of 0.05 and power of 90% to detect a conservatively estimated minimal clinically important difference (MCID) of 13 points and a standard deviation of 23 for the QuickDASH, yielded group sizes of thirty-five^8,16-18. Potential participants were called and excluded if the current date placed them out of any time interval ± fourteen days. When the cohort size of thirty-five was reached for a given time point, all remaining patients in the group were excluded.

Descriptive statistics were produced to describe the enrolled cohort of patients. The three separate QuickDASH scores (actual baseline, recalled baseline, and current scores) were calculated from the recorded raw data with use of the standard QuickDASH scoring algorithm to provide a score from 0 to 100 (0 = no disability, 100 = maximum disability). Two-sided paired t tests and the Wilcoxon signed-rank test for nonparametric data were used to evaluate the difference between mean recalled and actual QuickDASH scores for each time point. Since the variances differed between groups, a Kruskal-Wallis test (nonparametric analysis of variance) was used to compare the calculated difference between actual and recalled QuickDASH scores between the groups.

Pearson correlation coefficients were calculated to evaluate overall concordance between actual and recalled QuickDASH scores at each time point. After confirming that no statistical differences existed across the time points, pooled data from all of the time points were used for similar correlation analysis to quantify the effects of patient age, disability at the time of the IOV (actual baseline QuickDASH score), current disability (current QuickDASH score), and actual change in disability (whether a patient improved or worsened measured by actual baseline/current QuickDASH score) with accuracy of patient recall. We believe that r_p values of <0.35 demonstrated a weak correlation, 0.35 to 0.7 demonstrated a moderate correlation, and >0.7 demonstrated a strong correlation¹⁹.

To determine if patient diagnosis affected recall accuracy, we again used a Kruskal-Wallis test. We grouped distinctive diagnoses into five different categories to increase our power to detect a significant difference between the groups (patients with tendonitis, nerve compression disease, arthritis, cysts or masses, and multiple or other diagnoses). The Mann-Whitney U test was used to determine the difference in recall accuracy between intervention types (surgical versus nonoperative management and procedural intervention versus noninvasive treatment). The above analyses were also conducted with pooled data.

Study data were collected and managed with use of Research Electronic Data Capture (REDCap) tools hosted in the biostatistics division of our university. REDCap is a secure, web-based application designed to support data capture for research studies²⁰.

Sources of Funding

This publication was supported by the Washington University Institute of Clinical and Translational Sciences grant UL1 TR000448, subaward TL1 TR000449, from the National Center for Advancing Translational Sciences. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. This work was also supported by a grant from the Doris Duke Charitable Foundation to Washington University. Each award supports research time for the authors (J.G.S. and D.A.L.) as opposed to directly funding this investigation.

Results

A total of sixty-one, sixty-eight, fifty-three, and fifty-nine patients were identified as eligible patients after chart review in the three-month, six-month, twelve-month, and twenty-four-month groups, respectively. Potential participants were called on the telephone until thirty-five patients were enrolled in each group. Of the 146 patients contacted, only six refused to participate (two at three months, three at twelve months, and one at twenty-four months). In the final 140 participants, the ages ranged from twenty-three to eighty-six years of age. Table I shows the demographics, actual baseline levels of function (QuickDASH scores from the IOVs), current QuickDASH scores, diagnoses, and treatments at the four different follow-up time points.

TABLE I.

Descriptive Statistics for Each Patient Group^*

	Time After Initial Evaluation
Variable	3 Months	6 Months	12 Months	24 Months
Men (%)	10 (28.6%)	16 (45.7%)	12 (34.3%)	15 (42.9%)
Mean age at the time of follow-up (±SD)	55.0 (±11.5)	55.9 (±13.1)	54.9 (±15.7)	54.7 (±14.1)
Actual baseline QuickDASH (±SD)	36.8 (±23.0)	35.5 (±24.1)	43.2 (±24.3)	36.0 (±22.5)
Current QuickDASH (±SD)	22.3 (±24.8)	22.9 (±24.9)	26.1(±26.3)	15.6 (±21.6)
Diagnosis†
Arthritis	3	6	8	4
Trigger finger	11	6	9	6
de Quervain syndrome	3	3	4	3
Dupuytren contracture	3	1	1	3
Carpal and cubital tunnel syndrome	9	12	13	16
Cyst/mass	5	7	2	3
Other	2	2	1	2
Treatment
Surgery	3	13	6	7
Steroid injection	13	8	16	10
Braces/medication	12	9	10	14
No treatment	4	3	3	2
Other‡	3	2	0	2

^*SD = standard deviation.

^†Seven patients had multiple diagnoses.

^‡There were three needle aponeurotomies, two aspirations, one sclerotherapy, and one unknown treatment.

Figure 1 presents patients’ mean actual and recalled baseline QuickDASH scores at each time interval. A significant difference between actual and recalled baseline QuickDASH scores only occurred at three months. At all other time points, the actual and recalled scores were statistically equivalent (Table II). The mean difference noted at three months, despite being significant, was less than the predetermined MCID of 13 on the QuickDASH. There was no significant difference in recall accuracy between the four time groups (p = 0.77), with no evidence of decreasing recall accuracy over time.

Fig. 1.

Comparison of the mean of actual baseline QuickDASH scores and recalled baseline QuickDASH scores for each time point. The three-month time point was the only time point to show a significant difference between actual baseline and recalled baseline QuickDASH scores (p= 0.001).

TABLE II.

Intragroup Comparison of Patient Recall Accuracy (Actual/Recalled QuickDASH Scores)^*

	Time After Initial Evaluation
	3 Months	6 Months	12 Months	24 Months
Actual QuickDASH (±SD)	36.8 (±23.0)	35.5 (±24.1)	43.2 (±24.3)	36.0 (±22.5)
Recalled QuickDASH (±SD)	43.9 (±24.6)	36.4 (±24.5)	45.5 (±21.8)	38.8 (±24.4)
Mean difference (95% CI)	−7.1 (−11.0 to −3.2)	0.8 (−5.2 to 3.6)	−2.3 (−8.1 to 3.5)	−2.8 (−7.2 to 1.6)
P value	0.001	0.79*	0.43	0.26†

^*SD = standard deviation and CI = confidence interval.

^†Related samples from the Wilcoxon signed-rank test.

The correlations between recalled and actual baseline QuickDASH scores in each cohort were significant (Figs. 2, 3, 4, and 5). All relationships were strongly correlated: r_p = 0.89 at three months, r_p = 0.86 at six months, r_p = 0.74 at twelve months, and r_p = 0.85 at twenty-four months.

Fig. 2.

Scatter plot of actual QuickDASH score obtained at the IOV compared with recalled baseline QuickDASH scores obtained three months after the IOV with a best-fit line.

Fig. 3.

Scatter plot of actual QuickDASH score obtained at the IOV compared with recalled baseline QuickDASH scores obtained six months after the IOV with a best-fit line.

Fig. 4.

Scatter plot of actual QuickDASH score obtained at the IOV compared with recalled baseline QuickDASH scores obtained twelve months after the IOV with a best-fit line.

Fig. 5.

Scatter plot of actual QuickDASH score obtained at the IOV compared with recalled baseline QuickDASH scores obtained twenty-four months after the IOV with a best-fit line.

There was no observable association between patients’ age, change in disability from IOV to time of follow-up, baseline disability, or current disability and recall accuracy (measured by the actual/recalled baseline QuickDASH score). Table III presents the Pearson correlation coefficients of each of these patient factors. Figure 6 graphically presents a representative poor correlation between the absolute change in recall score and patient age in a scatter plot (r_p = 0.037).

TABLE III.

Correlation of Patient Factors with Recall Error (Actual/Recalled QuickDASH Score)^*

Patient Factors	Pearson Correlation
Change in disability (QuickDASH actual/current score)	−0.077
Actual baseline disability (QuickDASH actual score)	0.062
Age	0.037
Current disability (QuickDASH current score)	−0.011

^*All p values of >0.05.

Fig. 6.

Scatter plot of age at time of phone follow-up compared with recall accuracy measured by the absolute difference between recalled baseline and actual baseline QuickDASH scores with a best-fit line. QD = QuickDASH score.

There was no significant difference in recall accuracy between any of the groups of diagnoses (p = 0.31) or when comparing recall accuracy between patients who underwent surgical versus nonsurgical management (p = 0.10). When procedure-based management (e.g., surgery, corticosteroid injection, or needle aponeurotomy) was compared with nonprocedure-based management (e.g., brace, medication, or no treatment), the difference in recall remained statistically insignificant (p = 0.69).

Discussion

Our data indicate that patients can reliably recall their previous level of function two years after the IOV with use of the patient-rated QuickDASH questionnaire. Each time point showed a strong linear correlation between recalled and actual QuickDASH scores. The three-month time point was the only time point to show a significant difference between actual baseline and recalled baseline QuickDASH scores (Table II). This isolated statistical difference, however, was not clinically significant since the mean recall error (±95% confidence interval) was less than the estimated MCID for the QuickDASH.

Two studies have evaluated patients’ ability to recall prior function with the Western Ontario and McMaster Universities Osteoarthritis Index (WOMAC) scale (a validated patient-rated questionnaire). One was conducted six weeks after hip arthroplasty, and the other was conducted three months after total knee arthroplasty^2,7. Patient recall following total knee arthroplasty demonstrated moderate agreement with the actual prearthroplasty functional state, while patient recall following total hip replacement showed excellent agreement with baseline scores. We have expanded on these prior studies by incorporating evaluations at multiple time points (up to twenty-four months) following initial survey completion. Our data support the conclusion that when utilizing a standardized outcome measure, patient recall is highly reliable. However, the lower accuracy of recall after total knee arthroplasty reported previously suggests that recall accuracy may vary by the anatomic location of relevant disease and treatment.

Patient recall following lumbar fusion in patients with chronic low back pain has been evaluated over a large postoperative time interval of two months to five years⁵. Patients’ recalled pain and function showed only moderate agreement with preintervention status, with a tendency to overestimate preoperative pain. It is possible that the longer duration of potential follow-up in the lumbar spine study decreased patients’ recall accuracy; however, the study’s use of nonvalidated patient-rated questions is more likely the reason for only moderate recall accuracy. This highlights the importance of using validated questionnaires with strong psychometric properties when collecting recall data. According to our data, cross-sectional, retrospective, and case-control trials with use of the QuickDASH questionnaire can accurately assess patient function retrospectively for up to two years. This finding allows for greater flexibility in future research design, and it may save both time and money since a recalled retrospective study with use of the QuickDASH score could potentially be performed in lieu of a study with a prospective design.

Previous studies have also shown that patient factors, such as intensity of current pain and relief from an intervention, are important in patient recall accuracy^6,21. Our data do not support these conclusions. It is possible that persisting extreme levels of pain may impact recall accuracy, but patients with mild remaining impairment, as in our series, are not overly influenced by their current health state when recalling prior function.

Several limitations should be noted. The QuickDASH questionnaire has not been validated formally for use over the telephone. However, with a time-sensitive design and because e-mail addresses were unavailable for most patients, we did not deem it feasible to contact patients by mail or bring them into the office for completion of the QuickDASH questionnaire. Lastly, “learning” of the QuickDASH questions may have caused a systematic bias when patients were asked to complete the QuickDASH questionnaire a second time to document current disability. This may have affected the current QuickDASH scores. We were able to avoid this “learning” in our primary outcome measure of recalled QuickDASH scores since the recalled QuickDASH questionnaire was always administered prior to querying about the current impairment level.

We cannot generalize our results to patients treated for traumatic injury, acute infection, or diseases outside of the upper extremity. Despite this, we were able to use prospectively collected data over a broad range of hand and elbow diseases and treatment types without excluding patients for other ailments or health problems. Furthermore, there was no difference in recall accuracy between the varying diagnoses or treatments. Additionally, all participants in the study were insured (i.e., Medicaid, Medicare, workers’ compensation, or private insurance) and were selected from a specialty clinic (hand surgery practice). Our findings may not be generalizable to uninsured populations, but we have no evidence that socioeconomic status affects patient recall. Although we did not find recall accuracy deterioration with advancing age, all patients enrolled demonstrated cognitive abilities sufficient to verbally comprehend the purpose of this study and consent to participation. As a result, the findings of our study are not presumed reliable if patients with cognitive impairment are being studied.

We believe that designing our study similar to a case-control or a retrospective case series (likely to use recalled data) improved its external validity. Patients were called at a specific time after the IOV and were asked to recall prior level of function and current level of function without prior consent to study participation, which would have given them the expectation that they would be contacted in the future to recall their QuickDASH answers. All of the patients were treated in a standardized manner; the use of only two data collectors minimized any potential bias that the interviewers might have introduced.

We failed to reject the null hypothesis: our data demonstrate that patients can accurately recall the level of disability reported at their IOV for up to two years. Prospectively collected data will always provide the strongest levels of clinical evidence; however, recalled data may reliably estimate or even match the performance of prospective data. In particular, our results show that researchers who use recalled patient function via the QuickDASH questionnaire can assume that a population of interest should be able to accurately recall their prior level of function for up to two years after an intervention. Because certain individuals in our study did demonstrate inaccurate recall of their prior state of function, we recommend caution if relying on recalled functional scoring in small groups of patients because of the possibility of undue influence by any single outlier.

Supplementary Material

Supporting Data

Disclosure of Potential Conflicts of Interest