Reliability of hand diagrams for the epidemiologic case definition of carpal tunnel syndrome

Abstract

Introduction:

The purpose of this study was to evaluate the inter-rater reliability of hand diagrams, which are commonly used in research case definitions of carpal tunnel syndrome (CTS). To evaluate the potential of non-random misclassification of cases, we also studied predictors of rater disagreement as a function of personal and work factors, and of hand symptoms not classic for CTS.

Methods:

Participants in a longitudinal study investigating the development of CTS completed repeated self-administered questionnaires. Three experienced clinicians, blind to subjects’ work or personal history, independently rated all hand diagrams on an ordinal scale from 0 to 3. Disagreements between ratings were resolved by consensus. Reliability was measured by the weighted kappa statistic. Logistic regression models evaluated predictors of disagreement.

Results:

Three hundred and thirty-three subjects completed 494 hand diagrams. Eighty-five percent were completed by self-administered questionnaire and 15% by telephone interview. Weighted kappa values representing agreement among the three raters, were 0.83 (95% CI: 0.78, 0.87) for right hand diagrams and 0.88 (95% CI: 0.83, 0.91) for left hand diagrams. Ratings from hand diagrams obtained by telephone interview produced better agreement. Agreement among raters was not affected by subjects’ personal or work factors. Disagreement among raters was associated with the presence of hand/wrist symptoms other than classic CTS symptoms.

Conclusions:

Overall, high levels of agreement were attained by independent raters of hand diagrams. Personal factors did not affect agreement among raters, but presence of non-CTS symptoms seemed to affect results and should be considered in studies focused on diverse populations with heterogeneity of upper extremity symptoms.

Introduction

Carpal tunnel syndrome (CTS) is one of the most common diagnoses of the upper extremity. The prevalence of this disorder has been estimated between one and five percent in the general population [1–3] with higher estimated rates of 10% or more reported among workers in some industries [4–6]. The direct costs for treatment and indirect costs for lost work time and permanent functional disability make this syndrome costly for patients as well as employers [7, 8].

Carpal tunnel syndrome is clinically diagnosed based on a specific pattern of symptoms with observable clinical signs sometimes noted in the later stages. The typical symptom pattern is paraesthesia in the median nerve distribution, often described as numbness, tingling, burning or pain in the first three digits (thumb, index, and long) of the hand [9, 10]. The symptoms are usually intermittent in the early stages, often occurring nocturnally. Variations of this classic pattern include the presence of symptoms during active hand use or location of symptoms in a larger area of the hand than the distal sensory distribution of the median nerve. In more advanced stages of the disorder, symptoms may include the motor component of the median nerve, thus causing weakness, incoordination, and visible muscle atrophy. The pathophysiologic mechanism is not well understood, although compression of the median nerve in the carpal canal is a leading theory [11, 12].

The number of cases identified depends upon the case definition used to make the diagnosis. Rempel and colleagues [13] described consensus criteria recommended for use in population based epidemiologic studies. The case definition recommended by this consensus panel includes positive electrodiagnostic findings as well as characteristic symptoms in the median nerve distribution. These criteria are supported by other authors [14–16]. Inclusion of only one of these variables (electrodiagnostic results or median nerve symptoms) increases the number of cases substantially [17, 18] but increases misclassification of cases. Varying the electrodiagnostic cut points for an abnormal classification will also alter the number of cases. Physical examination findings, though commonly employed, have shown poor validity and reliability in epidemiological settings [5, 17–21].

The hand diagram is a frequently used instrument for assessing symptoms in population based epidemiologic studies. It was originally designed by Katz [22, 23] with simplifying modifications made by Franzblau [24]. Diagrams are completed by the subject indicating the location of symptoms, and are then scored by a rater on a four point ordinal scale expressing the likelihood of CTS (unlikely, possible, probable, or classic). The self-administered drawings rated by consensus have been described as a valid method for classifying pathology with sensitivity of 80% and specificity of 90% in a referral patient-based population [23]. Work-based population studies that screen workers who were not seeking medical attention showed lower sensitivity (0.19–0.40) with high specificity (0.83–0.95), using NCS results for case classification [24]. Similarly, general population based studies have shown a broad range of sensitivity and specificity values [5, 25–27]. Despite varying validity, reliability has shown consistently high results with kappa and intraclass correlation coefficients of 0.89 to 0.93 [24, 25].

Given the common use of the hand diagram, it is important to evaluate the potential for misclassification. Numerous studies have found associations between personal risk factors and physical work exposures, and carpal tunnel syndrome. It is unknown whether these same personal or work factors may influence the scoring of hand diagrams that are used in case definitions of carpal tunnel syndrome. In addition, rating hand diagrams containing upper extremity symptoms other than the classic symptoms of carpal tunnel syndrome- numbness, tingling, burning, and pain- may cause disagreement among raters. Subjects with hand conditions other than CTS may be unable to clearly differentiate symptoms of stiffness, soreness, or aching from numbness when completing the hand diagram, or may have pain in the median nerve distribution from a different condition. This could potentially lead to differential or non-differential misclassification of hand diagrams used in CTS case definitions.

To gain more information about the usefulness of the hand diagram in outcome assessment for epidemiologic studies of CTS, this study evaluated the inter-rater reliability of hand diagram scores for CTS in a diverse population from a broad range of industries. We also evaluated the predictors of disagreement in models containing personal and work factors, and in models containing hand symptoms in addition to classic CTS symptoms.

Methods

All data are from the Predictors of Carpal Tunnel Syndrome (PrediCTS) Study, an ongoing prospective study of newly hired workers in different industries that was initiated in July 2004. The purpose of the overall study is to investigate personal and work-related risk factors associated with the development of carpal tunnel syndrome. This study was approved by the Institutional Review Boards of all participating institutions, and all subjects provided written informed consent.

Study participants were recruited from eight employers and three apprenticeship programs in the St. Louis, Missouri area. Newly hired workers were invited to participate either at their new hire orientation, post-offer health screening, or at entry-level classes in the apprenticeship programs. Subjects were required to be at least 18 years old, English speaking, and working a minimum of 30 hours per week in a new job or with a job change to regular benefit status. Exclusion criteria included current pregnancy, prior diagnosis of CTS or other peripheral neuropathy, or contraindications to nerve conduction testing.

Participants underwent a one-hour baseline testing protocol that included nerve conduction tests, a structured physical exam of the arms and hands, and a self-administered questionnaire. The questionnaire assessed demographics, past work history, medical history, work exposures at the previous job, and upper extremity symptoms of the neck/shoulder, elbow/forearm and hand/wrist. The questionnaire incorporated items from previous research on upper extremity disorders, including items previously shown to have good to excellent test-retest reliability [19, 24, 28–32]. Follow-up questionnaires with similar questions to the baseline questionnaire were completed at approximately 6 months, 18 months, and 36 months after baseline testing. These follow-up questionnaires were either mailed to subjects or, when applicable, distributed and collected at apprenticeship training classes or the worksite. To increase the response rate, subjects were mailed a second questionnaire if they did not return a completed questionnaire. Subjects who failed to return a follow-up questionnaire were called by a study team member as a reminder, and were offered the chance to complete the survey by telephone. We pursued subjects with unreturned follow-up questionnaires up to six months after the due date.

Hand diagrams were completed as part of the baseline and follow-up questionnaires for subjects who reported hand or wrist symptoms lasting more than seven days or occurring three or more times in the previous year (or previous six months for the sixth-month follow-up). Subjects who described the hand symptoms as tingling, numbness, burning or pain in one or both hands were asked to complete a hand diagram and shade in the location of symptoms on the volar and dorsal aspects of a diagram of the right and left hands (see Figure 1). Subjects who exclusively reported symptoms of soreness, aching, cramping, tightness, and stiffness of the hands and wrists were instructed to not complete the hand diagram. Hand diagrams were completed by eligible subjects at baseline and at 6-, 18-, and 36-month follow-up.

Figure 1.

Hand diagram schemata from self-administered questionnaire

In order to increase the response rate, some questionnaires were completed by telephone interview. This format was not previously described in the prior hand diagram protocols [24, 33]. To complete the hand diagram by interview, we developed a series of questions that systematically reviewed the presence, quality, and location of symptoms on the hand. Subjects were asked to describe the type of symptoms from a menu (burning/pain, tightness/stiffness, soreness/cramping/aching, and numbness/tingling) by general area (wrist, hand, and finger) of both the right and left hands (Appendix A). For each symptomatic hand, interviewers used a branched series of questions to determine the specific location of symptoms: which fingers if any were affected, which phalanges were affected; volar and dorsal location of symptoms, and symptoms extending into the palm or the dorsum of the hand. After completing the interview, the interviewer shaded the corresponding locations on the hand diagram and reviewed the symptom distribution with the subject. The time for completing the interview depended upon the variability and complexity of the distribution of symptoms; the estimated range for completion time was two to twenty minutes.

Three expert raters including two occupational medicine physicians and one occupational therapist independently scored each hand diagram following the scoring criteria described by Franzblau et al. [24]. All raters had prior research and clinical experience addressing upper extremity problems. The scoring criteria were unlikely (0), possible (1), probable (2) or classic CTS (3) (Table I). The raters were masked to subjects’ personal, work, and medical information except for the shadings drawn on the hand diagrams and a table listing the nature and general location of symptoms (Figure 1). On all hand diagrams where there was not complete agreement between the three independent ratings, the raters discussed the diagram to reach a final consensus rating. Several additions and clarifications to the scoring criteria were made as the study progressed to address the most frequently encountered ambiguities in the application of the scoring criteria. These modifications are shown in italics in Table I.

Table I.

Scoring Criteria for Hand Diagrams

Rating	Description of area shaded on the hand*
Classic (3)	Tingling, numbness, burning or pain in at least 2 of the digits (thumb, index and long). Symptoms in palm and dorsum of hand excluded; small finger symptoms, wrist pain or radiation proximal to the wrist allowed. For index and long digits, must include shading between the distal tip and the proximal finger crease volarly, and include >1/2 of the middle phalanx &/or some of the distal phalanx. For thumb, must include shading in the distal phalanx volarly. Digit may include shading dorsally from fingernail to the distal MP mark on the hand diagram. If joint of digit (including MP) is the only area shaded and less than half of two adjacent phalanges, this may be considered arthritic complaints.
Probable (2)	Same shading as for classic but allowed the shading to extend into the palm volarly unless it was confined to the ulnar aspect of the palm.
Possible (1)	Tingling, numbness, burning, or pain in at least one of the digits (thumb, index and long). May include the dorsum of the hand
Unlikely (0)	No shading of the primary digits or shading restricted to the dorsum of the digits only.

^*Modifications to rules in italics

Analysis

For evaluating interrater reliability, hand diagrams completed by subjects at one or more points in time were included in the analysis; from the perspective of the raters each hand diagram was an independent event coded anonymously. Right and left hand data were analyzed as two separate datasets. The primary analysis used weighted kappa statistics to assess agreement among raters [34].

In addition, chi-square tests and logistic regression analyses were performed to examine potential predictors of disagreement among raters. For these analyses, we compared cases where all three raters agreed to those without complete agreement. Because we examined person-level characteristics, each subject contributed only a single hand diagram to these analyses; for subjects who completed a hand diagram at more than one study point, we used the diagram from the earliest time point in the study. The first series of chi-square tests and the logistic regression analyses examined personal characteristics as predictors of disagreement including age, sex, job category, race, the presence of other upper extremity symptoms in the elbow/forearm or neck/shoulder, and diseases including diabetes and arthritis. The statistical significance for these analyses was evaluated with a p-value < 0.05 as these tests were related to previously known personal risk factors.

The second series of chi-square tests and the logistic regression analyses examined presence of hand or wrist symptoms in addition to the characteristic symptoms of numbness/tingling or burning/pain that were required to trigger completion of a hand diagram. There were a total of 12 symptom variables created from three body parts (wrist, hand, and finger) and four groups of symptoms (burning/pain, tightness/stiffness, soreness/aching/cramping, and numbness/tingling). We ran several models predicting disagreement among raters adjusting for the presence of one or more than one of the 12 symptom variables. We ran approximately 100 individual tests to determine the relationship between the presence of symptoms and disagreement between raters. Using the 12 different symptom variable groups, we ran chi-square tests evaluating each individual symptom variable to the outcome of disagreement among raters. We also ran logistic regression analyses using individual symptoms and combinations of the symptom variables as independent predictors in the models. The symptom variables were entered as separate variables, multiple symptom variables from a single body part, and multiple symptom variables within multiple body parts. In all, we ran about 50 tests for each hand. These analyses were intended to determine whether the presence of hand/wrist symptoms not characteristic for CTS produced greater disagreement among raters. As this was an exploratory analysis, we used a Bonferroni adjustment for our observed significance level dividing the original alpha level by the number of tests conducted, resulting in a significance level of p ≤ 0.0005 required for these comparisons. Though described as a conservative method, the Bonferroni adjustment has been suggested by Perneger as an “acceptable [method] when searching for significant associations without pre-established hypotheses” [35, page 1237]. We used all self-administered questionnaires with complete symptom data for these analyses. We included multiple questionnaires completed by the same individual because we did not adjust for personal characteristics in these models. Analyses were conducted using the statistical software package R [36].

Results

Of the 1108 subjects enrolled in the PrediCTS study, 333 subjects identified tingling, numbness, burning or pain symptoms in at least one hand and completed a set of hand diagrams for both the right and left hands on at least one questionnaire. Hand diagrams were completed by self-administered questionnaire or by telephone interview at four different time points: 141 (29%) at baseline, 179 (36%) at six months, 156 (32%) at 18 months, and 18 (4%) at 36 months for a completion of 494 total questionnaires. Self-administered questionnaires account for 419 (85%) of the sets of hand diagrams with 75 (15%) completed by telephone interviews. The majority of the subjects (n = 217) completed bilateral hand diagrams at only one point in time (65%) and 116 subjects (35%) completed more than one set of hand diagrams.

Table II shows the demographic characteristics of the subjects who completed at least one hand diagram by self-administered questionnaire versus those who completed all surveys by telephone interview. Subjects were predominantly right handed and male, with a mean age of 31 and 32 years (SD 10). The subjects worked in a variety of job categories (construction: carpenters, floorlayers, sheetmetal workers; office/technical: computer and laboratory workers; service: housekeepers and food service workers). Five percent or less of the subjects reported a past medical diagnosis of diabetes or arthritis. A large portion of the subjects (28%–41%) reported additional symptoms in the elbow/forearm or neck/shoulder locations of the upper extremity.

Table II.

General personal characteristics and occupational job categories for 333 subjects who completed hand diagrams

Characteristics	Self-administered n= 288		Telephone interview n= 45
	n	(%)	n	(%)	p-value*
Age in years mean (SD)†	31	(10)	32	(10)	0.50**
Gender
Males	200	(70%)	24	(53%)	0.03
Females	88	(30%)	21	(47%)
Race
Caucasians	206	(72%)	17	(38%)	<0.01
Others‡	82	(28%)	28	(62%)
Job Category
Construction§	158	(55%)	14	(31%)	<0.01
Office/technical∥	78	(27%)	13	(30%)
Service¶	52	(18%)	18	(40%)
Medical diagnoses
diabetes	9	(3%)	0	0	0.23
arthritis	13	(5%)	1	(2%)	0.48
Handedness
Right handed	257	(89%)	37	(82%)	0.09
Left handed	20	(7%)	7	(16%)
Ambidextrous	6	(2%)	0	0
Missing	5	(2%)	1	(2%)
Other location of upper extremity symptoms
Elbow/forearm	80	(28%)	13	(29%)	0.88
Neck/shoulder	119	(41%)	15	(33%)	0.31

^†Standard Deviation,

^‡Others – African Americans, Asians, Native Americans, others,

^§carpenters, floorlayers, sheetmetal workers,

^∥computer, laboratory workers,

^¶housekeepers, food service workers

^*chisquare test,

^**t-test

Testing for group differences by demographic characteristics using chi-square and t-tests showed that there were a greater proportion of females, service workers, and non-Caucasians that completed the hand diagrams by telephone interview rather than by written questionnaire.

Agreement

Reliability analyses were run separately for the hand diagrams completed by self-administered questionnaire and those completed by telephone interview. Three surveys were removed from the reliability analysis for the self-administered group and one from the telephone interview group due to missing data points for some of the three independent ratings.

Of 416 self-administered questionnaires used for the analyses, hand diagram ratings were analyzed separately for the right and left hands. Figure 2 shows a plot of percent agreement by category for each rater separately compared to the consensus results. As shown in Table III, agreement was generally higher for the left hand compared to the right hand although there were a low proportion of abnormal hand diagrams for the left hand. The highest agreement was found for the ‘unlikely’ category (0), with very high agreement found for both the ‘possible’ (1) and ‘classic’ (3) categories. The lowest agreement was shown for the ‘probable’ (2) category. A small percentage of hand diagrams received unique ratings from all three raters (2% right hand, 2% left hand). Ratings of self-administered hand diagrams produced weighted kappa scores of 0.83 (95% CI: 0.78, 0.87) for right hand diagrams and 0.88 (95% CI: 0.83, 0.91) for left hand diagrams with similar results found for intraclass correlation coefficient (ICC) agreement and consistency analyses.

Figure 2.

Comparison of hand diagram consensus scores among three raters to the individual raters’ scores of unlikely (0), possible (1), probable (2) and classic (3).

Table III.

Proportion of ratings by coding scale and percent agreement by all raters for the left and right hand diagram completed by self-administered questionnaires

	Left Hand n=416		Right Hand n=416
Consensus ratings	% of completed questionnaires	Complete agreement by three raters (%)	% of completed questionnaires	Complete agreement by three raters (%)
Unlikely (0)	67.8	94	50.0	86
Possible (1)	14.4	73	24.5	69
Probable (2)	9.1	47	13.7	53
Classic (3)	8.7	72	11.8	69
All diagrams	100	85	100	75

Telephone interview hand diagram ratings produced higher agreement among raters (n = 74). Independent ratings showed weighted kappa scores of 0.96 (95% CI: 0.93, 0.99) for right hand diagrams and 0.93 (95% CI: 0.86, 0.98) for left hand diagrams.

Personal Factors as Predictors of Disagreement

In order to determine whether subjects’ personal factors contributed to systematic misclassification of our diagnostic outcome, we ran logistic regression analyses to predict disagreement among the raters. This analysis was restricted to the first hand diagram completed by self-administered questionnaire for each subject (n=288). The outcome for this analysis was complete agreement among raters versus at least one rater with a different score. Agreement among raters was not predicted by the subjects’ age, sex, job category, race, the presence of other neck/shoulder or elbow/forearm symptoms, or other diseases including diabetes and arthritis.

Additional Upper Extremity Symptoms as Predictors of Disagreement

In order to determine whether the presence of additional hand or wrist symptoms, not classic for CTS, contributed to disagreement on the classification of hand diagrams, we ran logistic regression analyses to predict disagreement among the raters. Of the 416 self-administered questionnaires used in the analysis, the prevalence of symptoms of burning/pain, tightness/stiffness, soreness/cramping/aching, and numbness/tingling was 39%, 42%, 49%, and 74% respectively for the right hand/wrist and 20%, 27%, 32%, and 44% respectively for the left hand/wrist. The number of completed hand diagrams that reported only one symptom was 118 (28%) for the right hand and 85 (20%) for the left hand. Multiple symptoms were reported for 257 (62%) right hand diagrams and 166 (40%) left hand diagrams. The remaining hand diagrams reported no symptoms but were completed because the opposite hand had symptoms.

Chi-square tests evaluated whether there was greater disagreement than expected in the presence of individual symptoms. Using p ≤ 0.0005 as a conservative cut point for statistical significance, the results showed that several of the 12 individual hand/wrist symptom variables were associated with greater disagreement among raters. Analyses were conducted separately for the right and the left hand with results shown in Table IV. Correlations between individual symptom variables produced a large number of moderate to strong association values between symptoms. We ran several logistic regression analyses to predict agreement among raters in the presence of symptoms not characteristic for CTS. Each prediction model showed that the presence of one or more additional hand/wrist symptoms predicted disagreement among raters, with a p ≤ 0.0005.

Table IV.

Chi-square tests to evaluate the association with disagreement of raters for individual symptom variables run separately for the right and left hand (n = 416)

	Left Hand	Right Hand
	p-value	p-value
Wrist
burning/pain	0.1504	0.8931
tightness/stiffness	0.0293	0.8915
soreness/cramping/aching	0.0006	0.2845
numbness/tingling	0.0011	0.7104
Hand
burning/pain	0.0017	0.0200
tightness/stiffness	0.0616	0.0556
soreness/cramping/aching	0.0001*	0.0007
numbness/tingling	<0.0001*	<0.0001*
Finger
burning/ pain	0.0005*	0.0037
tightness/ stiffness	0.0025	0.0153
soreness/cramping/aching	0.0019	0.0357
numbness/tingling	0.0001*	0.0003*

^*p≤ 0.0005

Discussion

Overall, we found high levels of agreement among three experienced raters of hand diagrams in a cohort of workers newly employed in several industries. Small differences among raters existed, and consensus rating allowed resolution of differences among raters. Most subjects with hand symptoms did not have a symptom pattern suggestive for CTS. Agreement among raters was not affected by subjects’ differences in demographic characteristics and job category. However, the presence of non-CTS symptoms was associated with disagreement among raters.

The hand diagram was developed as a self-administered drawing on a schemata of a hand to represent CTS symptoms. Subjects transferred their perceived symptoms to the drawing or picture. The diagram was not intended to illustrate the severity of hand symptoms but to show the location and quality of the symptoms in the hand. The original publication describing the hand diagram showed detailed drawings with shadings lying clearly within the median or ulnar nerve distributions [23]. In our experience, it is more common to receive self-completed diagrams that show much less clarity and adherence to anatomic boundaries. Despite clear instructions to shade the area of the hand where the subject has experienced numbness, tingling, burning or pain, the subject may circle parts of the hand diagram or use a careless shading method resulting in many stray lines. Judgment and interpolation are required by raters. Many past studies that have used this tool have not described the protocol for how the diagrams were rated, nor the consistency of agreement among raters [16, 26, 37]. In order to better understand the validity and reliability of case definitions derived from hand diagrams, researchers should report these methods and results.

The prevalence of abnormal findings on hand diagrams will depend upon the population under study. Our population of newly hired workers was slightly higher for combined classic/probable ratings (right hand 25.5%) than in previous studies of general and active worker population that showed rates of 11 to 18% for combined classic/probable results [3, 26]. These classic and probable rates are generally higher in clinic based studies given that patients are seeking treatment for a symptomatic hand related disorder [22, 37].

When the reliability of hand diagrams has been reported, the intraclass correlation coefficient or kappa values have been very high, from 0.89 to 0.93 [24, 25]. Our study also showed similar findings with weighted kappas of 0.83 (right hand) and 0.88 (left hand). These results indicate that hand diagrams can be a reliable tool for use in population-based epidemiologic studies of CTS.

Evaluations of the validity of hand diagrams in determining a diagnosis of carpal tunnel syndrome have shown good results for clinic based studies with somewhat variable values for population based studies. Referral clinic based studies using physician diagnosis, nerve conduction results, or a combination of the two have generally shown high sensitivity (76–80%) and specificity (79–90%) [21, 23, 38]. Patients seeking medical attention may bias physician ratings of a hand diagram, increasing the sensitivities found in past studies [17, 22]. Work and general population studies have shown less ability to predict nerve conduction abnormalities from hand diagram results with a wide range of sensitivity (0.19–0.90) and specificity (0.39–0.95) [5, 26, 27].

A commonly used research case definition for carpal tunnel syndrome requires the presence of symptoms in the median nerve distribution. [13, 14, 15]. Collection of symptom information may use personal interviews or self-administered questionnaires including hand diagrams. Our study explored potential systematic misclassification of hand diagram ratings based on personal factors. Many studies have shown associations between carpal tunnel syndrome and personal factors including age, body mass index, gender and medical history of diabetes [39, 40]. Our results found no associations between disagreement in hand diagram ratings and all examined personal risk factors for CTS including age, sex, race, diabetes, arthritis, job category, and the presence of symptoms in the neck/shoulder and elbow/forearm. These results suggest that systematic misclassification of cases is unlikely to account for associations observed between CTS and these personal risk factors.

The presence of additional hand and wrist symptoms other than CTS symptoms was associated with greater disagreement among raters. These additional symptoms included soreness, cramping, aching, stiffness, and tightness. This is an important consideration in population studies, where subjects are not seeking treatment but may be experiencing a variety of hand symptoms related to the nature of their work activities. These subjects may be less aware of the type and location of hand symptoms compared to patients seeking medical treatment for suspected carpal tunnel syndrome, leading to less precise symptom reporting on hand diagrams and subsequent misclassification. In addition, these symptoms may be more transient, or affected by recent work tasks. Szabo and colleagues [21] reported decreased predictive ability to classify positive CTS cases accurately (positive predictive value) in subjects with a physician diagnosed condition other than CTS, indicating that the presence of hand problems other than CTS decreases the discriminative value of the hand diagram. Additional caution should be used when evaluating populations that have a high prevalence of hand and wrist symptoms from other causes. The disagreement seen among our raters suggests that CTS could be under- or over-ascertained in populations with a higher prevalence of other hand and wrist disorders.

Shading of the hand diagram was originally intended to be completed by the subject. However, in order to increase our questionnaire response rate in this longitudinal study, we offered subjects telephone interviews if they did not respond to repeated mailed questionnaires. In order to complete the hand diagram by phone, we crafted a detailed script to obtain the necessary information about the type and location of symptoms for each hand. The agreement between ratings from these interview completed hand diagrams were slightly higher than for the self-administered hand diagrams. It is unknown how well these telephone diagrams would compare to self-administered hand diagrams but the high rater agreement indicates this method of data collection is promising. One possible explanation for the observed higher agreement might be greater clarity of the drawings completed by the interviewer. Past studies have shown that telephone respondents are different from mailed respondents [41, 42]. Improving response rates by including results from subjects less willing to return self-administered questionnaires gives greater confidence in the internal and external validity of study results. For studies that include the use of hand diagrams and may resort to telephone interviews to increase response rates, further investigation should be considered to evaluate the agreement of the results between phone interviews and mailed questionnaires.

These results show the hand diagram is a useful method for identifying individuals with symptoms suggestive of CTS. Our study had several potential limitations. First, we used only three raters. All raters were experienced in the scoring of hand diagrams and spent time working together early in the study to develop a similar understanding of the hand diagram coding definitions. Studies that use less experienced raters who do not have the opportunity to work together on consensus ratings may have lower levels of agreement among raters. We also had a small number of subjects with hand diagrams coded as ‘classic.’ This is an expected ‘classic’ CTS rate in a population based study in which subjects are not seeking medical treatment, despite potential hand symptoms. As previous researchers have shown, comparison of reliability results from population based studies may be lower than that seen in clinic-based studies.

Use of the telephone interview hand diagrams was a novel aspect of our study that produced promising results. Telephone interviews increase the likelihood of capturing information from hard to reach subjects, particularly for our longitudinal study with repeated assessment of symptoms. Further evaluation of hand diagrams derived from telephone interviews compared to self-administered surveys is warranted.

Conclusions

The hand diagram tool produced highly reliable results in a diverse working population from a broad group of industries. Given the associations between personal factors, job classification, and work-related musculoskeletal disorders that have been shown in past literature, it is reassuring to know these same factors do not impact the reliability of hand diagram results. We found that other hand symptoms appeared to affect raters’ agreement on classifying hand diagrams. Hand symptoms unrelated to carpal tunnel syndrome are common in manual working populations. Population based studies focused on these work groups should consider the effects that hand/wrist symptoms not characteristic of CTS may have on CTS case definitions. Overall, the hand diagram is a simple tool that produces reliable results even in diverse populations. Completion of hand diagrams via telephone interview appears to produce results that are similar to self-administration.