Concurrent validity and precision of the thumb disability examination (TDX) in first carpometacarpal osteoarthritis

Abstract

Study Design:

A descriptive psychometric study of precision and concurrent validity of the Thumb Disability Examination (TDX).

Introduction:

Thumb carpometacarpal osteoarthritis (CMC OA) is a painful joint condition impacting the functionality of the hand. Therapists use patient-reported outcome measures to evaluate change in disability and symptomology in response to interventions. The TDX is the only condition-specific outcome measure for persons with thumb CMC OA. Its responsiveness, test-retest reliability and concurrent validity with the DASH are published, yet it’s precision and concurrent validity with a hand-region-specific tool has not yet been established.

Purpose of the Study:

We aimed to determine the precision and concurrent validity of the TDX with a region-specific outcome measure in people with thumb CMC OA.

Methods:

Sixteen individuals with a medical diagnosis of CMC OA or a positive pressure-shear test completed the TDX across two visits and the Brief Michigan Hand Questionnaire (bMHQ) at the initial visit. The second visit was 7 to 21 days after the first. Self-administration of the TDX and bMHQ were observed by a licensed occupational therapist.

Results:

Across total and subscale scores of the TDX, standard error of measurement (SEM) values are used to indicate the precision of tool and demonstrate how confident a user can be that change in score exceeds the error inherent to the tool. Minimal detectable change percentage (MDC%) values for the TDX are acceptable (<30%). The TDX demonstrated high concurrent validity with the bMHQ (r_s = −0.733; P = .001).

Discussion:

Precision of the TDX is acceptable and the concurrent validity of the TDX with a commonly used region-specific scale is high. The study was limited by a small, demographically homogeneous sample due to difficulty in recruitment.

Conclusions:

The TDX is a precise and valid outcome measure for individuals having a clinical diagnosis or indications of having thumb CMC OA.

Introduction

Osteoarthritis (OA), particularly of the hands, hips, and knees, is the most common form of arthritis in the United States.¹ Thumb carpometacarpal osteoarthritis (CMC OA) is a highly prevalent condition that can cause pain, impair function, and impact quality of life.² Its development is linked to ligamentous laxity, joint hypermobility^3–6 and forceful pinching.⁶ Thumb pain, often associated with thumb CMC OA, has a high potential to cause dysfunction in many areas of life including activities of daily living and work⁷.

Therapists use standardized patient-reported outcomes to assess change in disability and symptomology^8,9 Existing questionnaires, such as the Disability of the Arm, Shoulder, and Hand (DASH) questionnaire, have been used as an outcome measure for thumb CMC OA¹⁰; however, it has been noted that using a general upper extremity or a non-specific hand questionnaire may yield results that are not responsive to changes in specific regions of the hand, such as the thumb CMC joint.^11,12 Knowing the DASH is not selectively responsive to changes in the thumb CMC joint, Noback et al.¹³ created the Thumb Disability Examination (TDX), a disease-specific questionnaire for persons with thumb CMC OA. The TDX is a patient-reported outcome measure that contains twenty items which measure pain, thumb impairment, and activity limitations from the patient’s perspective. The TDX has high internal consistency, excellent test-retest reliability, low concurrent validity with the PROMIS Upper Extremity (UE) CAT¹⁴, moderate-to-low concurrent validity the Disabilities of the Arm, Shoulder, and Hand (DASH)¹⁵ questionnaire and comparable content validity with the Australian/Canadian Osteoarthritis Hand Index (AUSCAN) and the Patient-Rated Wrist/Hand Evaluation.^13,16 While there is some emerging evidence on the psychometrics of the TDX, its “precision,” or measurement error, has not yet been explored. Additionally, while the TDX’s concurrent validity with two ‘global’ upper limb disability scales has been established, its concurrent validity with well-established hand region-specific scales has not. The Michigan Hand Questionnaire (MHQ) and its brief version (bMHQ) are both hand-region specific scales and have been used in numerous studies on hand arthritis^17–20,7 but their concurrent validity with the TDX has not yet been explored.

Purpose of the Study

There is currently no “gold standard” patient-reported outcome used to measure impairment and activity limitation in patients with thumb CMC OA. The TDX is unique in that it specifically targets activity limitations and impairments common to those with basal joint OA.¹³ However, the TDX is a recently developed measure and its precision, or how well a measure can estimate the patient’s score or change in their score, has yet to be established. Furthermore, the concurrent validity of the TDX with other hand region-specific scales has not yet been studied. Therefore, the purpose of this study is to describe a clinical measurement through 1) confirming the concurrent validity between the TDX and the bMHQ in persons with thumb CMC OA and 2) exploring the precision of the TDX when conducting baseline and follow-up assessments.

Methods

Study Design

This paper describes a descriptive psychometric study of precision and concurrent validity of the Thumb Disability Exam (TDX).

Instruments

As described by the developers of the TDX in Noback et al (2017), the TDX is a 20 question, self-administered questionnaire that evaluates function, satisfaction, and pain in persons with suspected CMC OA.¹³ It includes questions related to function of one-handed (affected hand) and two-handed activities, thumb pain, and satisfaction of the affected thumb within the past week. All questions are rated on a scale from one to five with one being most function, no pain, and greatest satisfaction. The TDX can be scored as a whole as well as by subsection on a scale of zero to 100 with zero indicating the least amount of disability.¹³

The bMHQ is a 12 item, hand region-specific questionnaire that evaluates overall hand function, activities of daily living performance, pain, work performance, aesthetics, and patient satisfaction with hand function. All questions are rated on a scale from one to five. Eight of the twelve questions are reversed and re-coded. The items are totaled and normalized to get a score of zero to 100, with zero indicating the worst overall functioning and satisfaction.²¹ The bMHQ has been shown to retain the psychometric properties of the MHQ and remains appropriate for the measurement of hand disability.¹⁰ Unlike the MHQ, the bMHQ does not distinguish between hands.¹⁸ The bMHQ was selected as the comparator for this study given its relative ease of administration and reduced respondent burden when compared to the 37-item MHQ. Specifically, the time for bMHQ administration more closely resembled that of the TDX (i.e., 2 minutes rather than the 10–15 required for the MHQ).^22,21

Participants

Participants were respondents to recruitment letters, flyers, and social media posts. The inclusion criteria for this study included adults ages 18 and older who have a diagnosis of thumb CMC joint OA from a physician or a positive pressure shear test of the thumb CMC joint, when administered by a certified hand therapist.²³ To complete the test, the examiner applied pressure over the volar beak of the thumb metacarpal base and created a shearing force across the CMC joint by rocking the MC across the trapezium in a medial-lateral direction with the thumb CMC resting in midposition. The test was considered positive when the participant reported pain in the CMC region.

Exclusion criteria for this study included individuals who had received steroid injection treatments within the past 3 months and had other comorbid hand conditions, women who self-reported being pregnant, individuals who have had CMC joint replacement, individuals who had median or radial nerve issues, individuals who were not fluent in English, and individuals who were unable to consent to research participation or follow the standardized commands. Eligibility for the study was assessed by a telephone screening call prior to the first session. Written informed consent was obtained from all participants prior to their participation in the study. Approval to conduct human subjects research was granted by a Midwest University’s institutional review board (approval no. 1204M12262).

Procedure

Intake information included relevant diagnoses, non-occupational therapy co-interventions (e.g., medication and injections), age, gender, comorbidities, and radiographic staging of arthritis (if known). The TDX and bMHQ were administered to a group of 16 individuals during the initial visit. After 7 to 21 days of the initial visit, these participants were then asked to return and complete only the TDX questionnaire a second time. This duration was chosen given that a 1x/week for 1 month frequency/duration has been described to be most common hand therapy practice pattern in this population.²⁴ This period of time was considered long enough to avoid potential recall bias, but short enough to avoid progression of arthritis.¹³ A licensed occupational therapist administered the TDX and the bMHQ at the same time of day (morning vs afternoon) so as to reduce the confounding influence of time of day on arthritis symptomology.

Data Analysis

All data was analyzed using SPSS version 25 software (Armonk, New York). For each participant, total scores for the bMHQ and first and second administrations of the TDX were calculated according to the standardized instructions.^13,7 Subscale scores for the first and second administrations of the TDX were also calculated as per the standardized instructions.¹³ In order to categorize the sample, the Bureau of Labor Statistics was consulted; codes from this website were utilized to group participants into similar job categories.²⁵ Additionally, participants were categorized as metropolitan or non-metropolitan based on their zip codes.²⁶ Demographic and descriptive data of the exams (means, counts, frequencies, and standard deviations) were reported. Statistical analyses were done to determine concurrent validity and precision as follows:

Concurrent Validity: Concurrent validity between the total scores of the bMHQ and first administration of the TDX was determined through a Spearman rho test (r_s).²⁷ The Spearman rho was used because it is a nonparametric measurement, which is appropriate for this sample size of less than 30. The strength of the concurrent validity, as per the r_s statistic, would be defined as very high (≥ 0.9), high (0.70–0.89), moderate (0.5–0.69), low (0.2–0.49), or negligible (< 0.2). A P value of less than.05 was considered statistically significant.^28,29

Precision of the TDX: As a prerequisite to determining precision of the TDX, test-retest reliability was established. Test-retest reliability was measured via use of the intraclass correlation coefficient (ICC). Higher ICC values indicate greater reliability, with an ICC of 1 indicating perfect agreement and 0 indicating only random agreement. An ICC of less than 0.4 was defined as poor, 0.40 to 0.59 as fair, 0.60 to 0.74 as good, and 0.75 and greater as excellent reliability.³⁰ From the test-retest reliability findings, precision was determined. The benefit to understanding the precision of a scale such as the TDX when it is used to monitor progress over time is that it offers the therapist perspective on what change must occur for it to be considered real or not attributable to measurement error. Two measures of precision were used in this study. The standard error of the measurement (SEM) and minimum detectable change (MDC) were used to analyze the precision of TDX within a single rater. The SEM, estimated as the pooled standard deviation of the assessments multiplied by the square root of (1− ICC), (i.e., SEM = SD*√(1-ICC)), is an estimate of within-subject variability, and reflects the amount of instrument measurement error.³¹ The MDC, another measure of precision, is calculated by multiplying the SEM by the z-score for the 95% confidence interval and the square root of 2 (i.e., MDC₉₅ = 1.96*(SEM*√2)). The MDC represents the smallest amount of meaningful change in scores that can be detected by a statistical test and can be interpreted clinically.³² The ICC, SEM, and MDC were reported as they are considered standards for reporting precision.³³ Additionally, the MDC was then divided by the range of TDX values to calculate the ‘MDC%’. This new value illustrates the measurement error relative to the range of values one might expect for a particular measure and allows for comparisons across scales. An MDC% of less than 30% was defined as acceptable and less than 10% as excellent.^34,35

Sample Size: The minimum sample size required to assess the concurrent validity was determined to be 14 participants.³⁶ This estimate was based on a two-tailed α ≤ 0.05, statistical power greater than 80%, and an r_s threshold of 0.70 based on the concurrent validity findings of Noback et al.¹³ This r_s threshold was set according to our assumptions that the association between the bMHQ, a hand-region specific scale, would exceed the association between DASH, an upper quadrant-specific scale, and TDX scores ( r = 0.63) in this population.

Results

Demographics

In total, there were 16 participants. Seven participants were respondents to recruitment letters, and nine participants were recruited via flyers and social media posts. Those responding to recruitment letters were selectively recruited through a translational science institute whereby letters were sent to those with a medical record of an ICD-9 or ICD-10 diagnosis of thumb CMC OA. Those recruited by flyers and social media self-reported a physician diagnosis of thumb CMC OA and were later screened positive by an occupational therapist who is a certified hand therapist on the research team for CMC OA via a pressure-shear test.²³ There were four males and twelve females with an average age of 64.6 years. Most participants were white with a graduate level education. Fifty percent of participants had pain in both of their thumbs, 31.3% had pain in the right thumb, and 18.8% in the left, and 81.3% of participants were right-hand dominant. Further descriptive statistics can be found in Table 1.

Table 1.

Participant demographics

Category	All Subjects
Age (y)	64.81 + /− 8.66
Gender, n (%)
Male	4 (25)
Female	12 (75)
Racial Background, n (%)
White	15 (93.8)
Multiracial	1 (6.3)
Education, n (%)
Associates	2 (12.5)
Bachelors	5 (31.3)
Graduate	9 (56.3)
Painful Thumb, n (%)
Right	5 (31.3)
Left	3 (18.8)
Both	8 (50)
Dominant Hand, n (%)
Right	13 (81.3)
Left	3 (18.8)
Area of Residency, n (%)
7-county metro	14 (87.5)
Non-metro	2 (12.5)

Note: n = Number of participants; y = Years; % = Percentage of sample. Metro = Urban and Suburban regions, Non-Metro = Rural regions.

Descriptive Data

The mean scores of all participants were calculated for the bMHQ and the first and second administrations of the TDX. For the bMHQ, an increased score indicates less disability, whereas for the TDX, an increase in the total score indicates more impairment. As seen in Figure 1 and Table 2, there is an inverse relationship between the mean score of the bMHQ and the two mean scores of the TDX (trials 1 and 2). Again, this is because a higher score on the TDX indicates poor thumb function, high pain, and poor satisfaction, whereas a higher score on the bMHQ indicates higher thumb functioning, lower pain, and higher thumb satisfaction. The mean score of the bMHQ for all participants was 72.01 (SD = 11.95). Mean TDX total and subscale scores can all be found in Table 2.

Fig. 1.

Concurrent validity of the TDC and bMHQ.

Table 2.

Intra-rater reliability of the TDX: Descriptive Data, Intraclass Correlation Coefficients (ICC), Standard Error of Measurement (SEM), and Minimal Detectable Change (MDC₉₅) for comparing two trials of TDX

	Time 1Mean (SD)	Time 2Mean (SD)	ICC (CI-95)	SEM	MDC95 (MDC%)
Total Score	31.95(14.13)	30.16 (13.72)	.81 (.55–.93)	6.02	16.69 (29.02)
Subscale I: Function	21.38 (7.01)	20.75 (6.66)	.82 (.56–.93)	2.9	8.04 (29.78)
Subscale II: Pain	11.31 (3.09)	11.06 (3.15)	.77 (.45–.91)	1.51	4.19 (32.20)
Subscale III: Satisfaction	12.88 (3.26)	12.31 (3.59)	.87 (.68–.95)	1.23	3.42 (28.46)

Note: ICC = Intraclass Correlation Coefficient; SEM = Standard Error of Measurement = SD * √(1-ICC); MDC = 1.96 * (SEM * √2); MDC% = MDC/range of all observations per measure

Concurrent Validity

Our findings indicate high and significant negative associations (r_s = −0.733; P =.001) between the first administration TDX Total and bMHQ Total scores, as shown in Figure 1. It is important to note that the flipped scales (i.e., lower TDX scores indicating less disability and higher scores on the bMHQ indicating the same) of the two assessments resulted in a negative correlation coefficient. Individuals who had a high score on the bMHQ were likely to have a lower score on the TDX, resulting in a negative correlation coefficient.

Precision

From participant’s baseline and follow up TDX assessments, the ICC, SEM, MDC, and MDC% were calculated for the total and three subscale scores, shown in Table 2. The Satisfaction subscale had the highest ICC, but the SEM was higher for the total score and the MDC. When the MDC values were normalized to the MDC% the Satisfaction Subscale had the lowest value and the Pain Subscale had the highest.

Discussion

Development of new patient-reported outcomes are becoming more essential as therapists seek to evaluate patient improvement from baseline measurements and to compare patient performance to population norms. While many upper extremity outcome measures exist, the TDX measures outcomes specifically important to individuals with CMC OA of the thumb (including functional performance, pain, and satisfaction).

The Spearman’s rho coefficient of −.73 indicates high concurrent validity between the TDX and bMHQ.²⁷ This is encouraging because, although the MHQ helped to inform the generation of five of the “activity” items in the TDX, there are only two activity items in the bMHQ which resemble items in the TDX.¹³ These findings are also encouraging given that the TDX has previously been found to have low to moderate concurrent validity with the DASH and PROMIS¹³ and is even more promising considering the relatively small sample ( n = 16). This relatively higher concurrent validity is likely attributable to the similar qualities of the TDX and bMHQ, asking the subjects about function and symptoms specific hand or the thumb ^7,13 whereas the DASH and PROMIS examine broad upper-limb functions and symptomology. However, the validity of TDX subscales could not be tested as there is no validated method of calculating subscores for the bMHQ.

Although not the primary focus of this study, the ICC measures for the total score for all subsections indicated excellent test-retest reliability between administrations of the TDX (ICC >.75).³⁰ However, it is important to consider the 95% confidence interval generated for each ICC produced. The small sample, a wide range of scores between TDX trials, and variance present in this data set indicates that the true ICC value could fall to values that are below the “excellent” category.³⁰ Still, these findings align with the excellent test-retest reliability outcomes previously reported by Noback et al.¹³

The SEM values for the total score and the subsections of the TDX indicate the precision of the TDX measurements from initial to follow-up assessment. The reported outcomes seen within Table 2 illustrate the smallest detectable change in this instrument between assessments that is not attributable to error, as measured in the units of the TDX. The MDC₉₅ values represent the smallest amount of detectable change not attributable to error that represents clinically significant change at 95% confidence (i.e., distribution-based Minimal Clinically Important Difference).^32,33 These measures are more conservative, but are more likely to represent true changes in the function, pain, and satisfaction of a patient’s thumb. According to our findings, a hand therapist or surgeon can be 90% (SEM) or 95% (MDC) confident that a change in the TDX total score is not attributable to error if it exceeds ~6 or ~17 points respectively. The same reasoning can be applied when interpreting changes in the TDX subscales. Additionally, according to our analyses of the MDC%, only the “Pain” Subscale had an MDC% that was greater than 30%, exceeding the threshold of what is considered an acceptable measurement error relative to the range of the measures.³⁵ Until this study can be expanded to a larger sample, the use of this subscale to measure change in response to treatment should be done with caution.

When considering our findings, it is important to note that MDC% is usually calculated for samples greater than 30 participants.³⁴ Recruiting more participants in future studies will likely result in lowered SEM, MDC, and MDC% values. Still, the measures of MDC and MDC% produced for this study indicate that this test has metrics that produce similar results between tests that can be used to determine statistical or clinical changes between scores over time.

At present, there is increasing institutional pressure for hand therapists and other providers, such as occupational therapists, physical therapists, physicians, and hand surgeons, to prove that their services are making a positive difference in their patient’s ability and, in return, qualifying their services to be covered under insurance.³⁷ Thus, hand therapists need to use valid and reliable tools to measure the outcomes of patients. Given the high volumes of persons with thumb CMC pain receiving hand therapy services,²⁴ hand therapists should select a patient-reported outcome which is population-specific and has strong psychometrics.

While other hand-OA scales exist, they are limited in their application to persons with thumb CMC OA. The Functional Index of Hand Osteoarthritis (FIHOA) only enquires about daily impact and does not assess symptomology. Additionally, there are no published details on its precision or psychometrics in a thumb CMC OA-specific group, and its test-retest reliability of the English version is only established at one hour,³⁸ which limits the application to clinical settings. Lastly, its gender role-specific assessment of daily activity performance would appear to be somewhat short-sighted in present times.

Although the AUSCAN deals with symptoms and daily activity performance, has strong test-retest reliability at 1 week, and has strong content validity in persons with hand OA,^16,20,39 it requires licensing and has no published psychometrics specific to this sub-population. The TDX, however, does not require licensing, was developed with specifically those with thumb CMC OA in mind, and has strong emerging evidence to support its use with those with thumb CMC OA in clinical settings.

Limitations

Several limitations apply to this study. The largest limitation of this study was the small sample size. Although adequately powered according to the sample size estimation, and while the study yielded strong concurrent validity, a larger sample size would have enhanced our study’s rigor. In addition, per the study criteria, individuals didn’t need a confirmed medical record diagnosis of CMC OA to participate. For those individuals without a confirmed diagnosis of CMC OA, a pressure shear test to the CMC joint was conducted. While this test has excellent sensitivity and specificity,²³ this does not clinically diagnose participants with CMC OA. These results cannot be generalized to other populations and to situations where more than one therapist is involved in using the TDX over time. Further testing is needed to determine its inter-rater reliability.

Recommendations for future study

Replication of this study with a larger sample size is recommended. Additional research on the TDX should focus on its relative responsiveness when compared against other hand arthritis scales such as the AUSCAN and the FIHOA as well as hand region specific scales such as the PRWHE and MHQ/bMHQ. Additionally, establishing its Minimal Clinically Important Difference through anchor-based methods is necessary. Finally, the TDX questionnaire should be tested in more diverse populations. Our sample included those with thumb CMC OA who were primarily white and over the age of 65. By extending research to a more diverse population, the TDX can be more confidently used with those with other conditions of the thumb and differing backgrounds.

Conclusion

We recommend therapists consider using the TDX outcome measure when assessing thumb functionality, pain, and satisfaction in persons with thumb CMC OA. We present additional evidence to justify the clinical use of this tool and be confident that the changes measured within therapy are not due to error.