Validity of Nordic-style questionnaires in the surveillance of upper-limb work-related musculoskeletal disorders

Abstract

Objectives

The study aimed to compare results of the standardized Nordic-style questionnaire to those of a clinical examination in two large surveys on upper-limb work-related musculoskeletal disorders (UWMSD).

Methods

The “Repetitive task” survey based on 1757 workers in 1993–1994 and 598 workers in 1996–1997 aimed at studying UWMSD risk factors in a population exposed to repetitive work. The “Pays de la Loire” survey, based on 2685 workers in 2002–2003, was part of a population-wide surveillance system. In both surveys, each worker completed a Nordic-style questionnaire and underwent a standardized clinical examination. Presence of at least one UWMSD was compared, with evaluation of sensitivity, specificity, and kappa value, considering clinical examination as the reference method. In the second survey, a score based on a numeric scale for severity of symptoms at the time of examination was evaluated in the same way (plus ROC curves).

Results

Agreement between questionnaire and examination was different in the two surveys: from kappa=0.22 [0.19–0.23] in the “Pays de la Loire” survey to 0.77 [0.74–0.80] in 1993–1994 in the “Repetitive task” survey. Sensitivity was excellent in all situations (from 82.3% to 100%). Specificity was variable, from 51.1% in the “Pays de la Loire” survey to 82.4% for score ≥ 2 based on the severity of symptoms in the survey.

Conclusion

Nordic-style questionnaires exploring symptoms in the past year can be considered as useful tools for surveillance of UWMSD, especially if they include numerical scales on symptom severity. Physical examination remains essential for a medical or clinical diagnosis assessment.

Introduction

One of the dimensions of Upper Limb Work-related Musculoskeletal Disorders (UWMSD) prevention is implementation of surveillance systems. These systems are based on various data sources, such as population-based or company-wide systems. Development and use of these systems imply that appropriate tools for evaluation of health status of the worker are available (1;2).

Some authors have proposed a multi-level model for surveillance of UWMSD and their risk factors, such as a first level using questionnaires and checklists, for a rapid assessment, and a second level including clinical examinations and in-depth job analysis by trained health care providers (1;3;4). Various questionnaires developed for musculoskeletal disorders, can be used to assess functional status (5–12). However, they are better adapted to a clinical than a surveillance context. The Standardized Nordic questionnaire, published in 1987 (13) is the most frequently used symptom questionnaire, initially designed for all musculoskeletal disorders, mainly for low back pain. The published version includes specific sections for the lower back, neck, and shoulder regions. Similar sets of questions can be added for different parts of upper limb (elbow, wrist, hands). Here, we use the term “Nordic-style” for questionnaires using a similar format to the original Nordic questionnaire, focusing on upper limb disorders.

The repeatability (or reliability) of the original Nordic questionnaire and other structured symptom questionnaires has been studied but there is scant information about the validity of this type of questionnaire (13–17).

Such questionnaires have been used in several surveillance studies in France, prompting a preliminary validity study comparing the answers to the questionnaire to a physical examination. This study was performed in the setting of an in-plant surveillance program in a shoe factory with a small sample of subjects, and concluded that Nordic-style questionnaires seemed to be useful in workplace surveillance programs (18;19)

In order to confirm the conclusion of this small study, we analysed data from two large French surveys on UWMSD with slightly different designs: the “Repetitive task” and the “Pays de la Loire” surveys. In both, we compared a Nordic-style questionnaire with a clinical examination, considered as the reference method.

Methods

Nordic-style questionnaire

The Nordic questionnaire was developed in the framework of a project supported by the Nordic Council of Ministers (13). It consists of structured, forced, multiple choice questions and can be used as a self-administered questionnaire or as an interview. The original version consists of several parts: a general questionnaire, and three specific parts focusing on the lower back, shoulders, and neck. The questionnaire was designed to answer the following question: “Do musculoskeletal troubles occur in a given population, and if so, in what parts of the body are they localised?” With this consideration in mind, a questionnaire was constructed in which the human body (viewed from the back) is divided into nine anatomical regions. The question “At any time during last 12 months/7 days have you had trouble (ache, pain, discomfort) in the lower back (shoulders, neck, etc.)?” is asked for each anatomical area in turn. Specific questions then concentrate on each anatomic region. These anatomic areas were, in the original version, the lower back, the shoulders, and the neck. However, similar questionnaires can be developed with a focus on upper limb anatomic areas, as was done in several studies in France. In the “Repetitive task” survey, a rather detailed questionnaire was used, with 121 question about upper-limb (20–23). In the “Pays de la Loire” survey, there were fewer questions (40 items), and the severity of symptoms at the time of the examination was also assessed using a numerical, rather than a dichotomous, measure (24).

Design

The objectives of the two surveys were different: descriptive and etiologic objective in the “Repetitive task” survey, and surveillance of UWMSD at a regional level in the “Pays de la Loire” survey. The study design and the population have been described earlier (20–24). In the presentation below, we describe more precisely the diagnostic criteria. Table 1 summarizes major points concerning the questionnaires and the clinical examination.

Table 1.

The questionnaire and the clinical examination criteria in “Repetitive task” and “Pays de la Loire” surveys.

-	-	“Repetitive task” survey		“Pays de la Loire” survey
Type of criteria		“Complete”*	“Restricted” *	-
Years		1993–1994 1996–1997	1993–1994 1996–1997	2002–2003
ULWMSD questionnaire	Place	The worker alone in the occupational health service	The worker alone in the occupational health service	The worker alone in the occupational health service
	Number of pages	10	10	3
	Place in the questionnaire	At the end	At the end	At the beginning
	Symptoms	Pain and discomfort	Pain and discomfort	Ache, pain, discomfort
	Recall period	Six months	Six months	Twelve months
	Region	Neck/shoulder to fingers	Neck/shoulder to fingers	Shoulder to fingers
	Drawing	Yes	Yes	Yes
	Criteria for positive case	At least one symptom in the last 6 months	At least one symptom in the last 6 months	At least one symptom in the last 12 months/or last week
Clinical examination	Criteria	Standardized criteria specific of the study	Standardized criteria specific of the study	SALTSA criteria
	Number of clinicians	39 in 1993–1994 18 in 1996–1997	39 in 1993–1994 18 in 1996–1997	80
	Number of disorders	33 in 1993–1994 35 in 1996–1997	6 6	6
	Uncommon disorders	Yes	No	No
	Specific tests	Systematic specific tests	Systematic specific tests	Specific test if symptoms at the interview by the practitioner
	Blinding of examiners (towards symptoms)	No	No	No
	Criteria for positive case	Positive if: -proved diagnosis at the examination -proved diagnosis before to the examination -suspected diagnosis	Positive if: -proved diagnosis at the examination -proved diagnosis before to the examination -suspected diagnosis	Positive if one specific test was positive

^*“complete”, analyses considering all the disorders in the clinical examination; “restricted”, analyses restricted to six disorders.

The “Repetitive task” survey

Study population

In 1993–1994, 1757 workers completed a self-administered questionnaire about their work conditions and UWMSD. The mean age was 38 years, and the percentage of women was 76%. All the workers underwent a standardized clinical examination by the occupational health physician responsible for medical surveillance in their company.

The workers were selected according to occupational criteria. They were required to be exposed to repetitive work in one of five activity sectors: (i) assembly-line manufacture of small electrical appliances, motor vehicle accessories, or ski accessories, packaging excluded; (ii) clothing or shoe industry, packaging excluded; (iii) food industry, packaging excluded (mainly the meat industry); (iv) packaging (primarily in the food industry); (v) supermarkets (cashiers). A final control group (vi) was made up of workers from the same industries who were not exposed to repetitive work. All subgroups included both women and men except for the supermarket cashiers who were exclusively women.

Eighteen of the 39 occupational health physicians who participated in the 1993–1994 evaluations were able to repeat the study 3 years later. The 700 workers in 18 different firms whom they had examined in 1993–1994 were thus the target population of the longitudinal study. In all, 598 (85.4%) workers (mean age 41 years, 70% of women) completed a self-administered questionnaire and were re-examined by the same physician three years after the baseline examination.

Questionnaire

Each worker completed a 10 page Nordic-style questionnaire on ULWMSD symptoms occurring in the previous six months. Assessments took place in the occupational health clinic both in 1993–1994 and in 1996–1997. After one page of general questions, the questionnaire included three pages for each anatomical region (shoulder, arm, elbow, forearm, wrist and hand) regarding pain, treatment and the consequences of the disorder. This questionnaire followed 13 pages of questions, mainly about exposure, in 1993–1994 and 9 pages in 1996–1997.

Clinical examination

Each worker underwent a standardized clinical examination, performed by the occupational physician immediately after the worker had completed the questionnaire. A list of criteria for the diagnoses recorded in the questionnaire was prepared for the clinical examination. These guidelines covered 33 diagnoses in 1993–1994 and 35 in 1996–1997 (the slight changes between the two lists were limited to shoulder tendonitis). One or two regional meetings with the occupational physicians took place before the baseline survey. Presentation of the guideline and training for the standardised physical examination was included in these meetings. The presentation of the guideline was included again in the regional meetings organised before the second survey.

The following three classifications of UWMSD were possible from the clinical examination: (i) diagnosis proved during the medical examination, (ii) diagnosis “proved” before the medical examination (for example, previous diagnosis by a specialist for a problem present in the last six month), and (iii) suspected diagnosis (not all the criteria were met in the medical examination or the diagnosis was based on the description of symptoms in the last six month but no longer present at the examination). Our definition of UWMSD determined by the clinical examination included “proved”, suspected and prior diagnoses.

Experimental network of epidemiological surveillance of UWMSD in the Pays de la Loire (“Pays de la Loire” survey)

The data for this investigation were collected as part of a surveillance study of work-related upper limb musculoskeletal disorders launched by France’s National Institute for Health Surveillance and set up in the Pays de la Loire region (Loire Valley district, West-Central France, population 3 220 000) (24).

Study population

All occupational physicians who practice in the Pays de la Loire region (n=460) were solicited to participate in the survey. A total of 80 agreed to participate. Each participating physician followed a standardized random selection procedure for inclusion of a sample of workers (30 for those working full-time, 15 for those working part-time). The demographics and occupational distribution of the sample was similar to that of the salaried workforce in the Pays de la Loire region and characteristic of France. In two years (2002 and 2003), 2685 workers were included, with 42% of women, and a mean age at 38 years.

Questionnaire

Participating workers were invited to complete a 3-pages Nordic-style questionnaire on UWMSD in the previous 12 months. Neck pain was not considered in the study. Each worker completed a numeric scale (0 to 10) assessing the intensity of pain in each anatomical region at the time of the examination. A global numerical scale score (“NS”) was made by summing the region scores (minimum 0; maximum 40, neck region not considered). The Nordic-style questionnaire was followed by 12-pages with questions on work exposures, as well as general and medical items.

Clinical examination

The occupational physicians were trained by the study investigators to perform a standardized physical examination, based on an international protocol for the evaluation of work-related upper-limb musculoskeletal disorders (Saltsa) (25). The training was similar to that of the “Repetitive task” survey. They began the examination by asking the worker about upper-limb pain in the preceding 12 months. Participants with pain in the shoulder/arm, elbow, hand/wrist, or fingers underwent a standardized localized clinical examination. The physical examination allowed the detection of six disorders: rotator cuff syndrome, lateral epicondylitis, ulnar nerve entrapment at the elbow (cubital tunnel syndrome), extensor/flexor tendonitis/tenosynovitis, de Quervain’s disease and carpal tunnel syndrome. The physical examination was considered as positive if any of the 6 principal upper limb disorders was present.

Analyses

The questionnaire was considered as positive if the worker indicated at least one symptom in any region for the recall period. The clinical examination was considered as positive in the “Pays de la Loire” survey if any of the specific clinical examination tests was positive. In the “Repetitive task” survey, the presence of UWMSD according to the clinical examination was defined to include cases proved in the examination, proved before the examination, or suspected. In order to increase comparability, we performed additional analyses restricted to the six disorders in the “Pays de la Loire” survey (“restricted analysis” in table 1).

The validity of the questionnaire was studied by calculating Cohen’s Kappa, sensitivity, specificity and predictive values, considering the clinical examination as the reference. In the “Pays de la Loire” study, we conducted additional analyses using a more restrictive definition of a “positive questionnaire”: presence of symptoms at the time of the examination (intensity one or more according to the NS score). Since several thresholds for the numerical score (NS) were available for the definition of a “positive questionnaire”, a Receiver Operation Characteristic (ROC) curve was drawn to choose the optimal (best sensitivity and specificity) threshold for the NS (26).

SAS v8.2 and SPSS v11.01 were used for analyses (27;28).

Results

The questionnaires’ validity, considering the clinical examination as the reference, is presented in table 2. The proportion of positive questionnaires was higher than the proportion of positive clinical examinations in both surveys. In the “Repetitive task” survey, the proportion of positive clinical examinations was high (47 to 71%). Kappa coefficients were fair to good in the “Repetitive task” survey. In the “Pays de la Loire” survey, the value of the kappa coefficients differed according to the definition of a “positive questionnaire”. If the definition was extensive (at least one symptom in the 12 months), the agreement between the Nordic questionnaire and the clinical examination was low. If the questionnaire case definition took into account the presence of symptoms at the time of the examination (“NS” score >0), the agreement with the clinical examination was higher.

Table 2.

Comparisons of Nordic-style questionnaire in “Repetitive task” and “Pays de la Loire” surveys, considering the clinical examination as the reference.

		“Repetitive task” survey				“Pays de la Loire” survey
		1993–1994 “Complete”*	1996–1997 “Complete”*	1993–1994 “Restricted”*	1996–1997 “Restricted”*	Nordic Questionnaire**	Nordic Questionnaire*** NS score >0	Nordic Questionnaire*** NS score 2
Total number of subjects		1757	598	1757	598	2685	2685	2685
Frequencies, positive and negative cases	Positive questionnaire, n (%)	1198 (68.2%)	387 (64.7%)	1198 (68.2%)	387 (64.7%)	1490 (55.5%)	731 (27.2%)	696 (25.9%)
	Positive clinical examination, n (%)	1155 (65.7%)	423 (70.7%)	818 (46.6%)	306 (51.2%)	345 (12.8%)	345 (12.8%)	345 (12.8%)
	True negative cases	490	136	531	170	1195	1897	1928
	False negative cases	69	75	28	41	0	57	61
	False positive cases	112	39	408	122	1145	443	412
	True positive cases	1086	348	790	265	345	288	284
Performance	Sensitivity (%)	94.0	82.3	96.6	86.6	100	83.5	82.3
	Specificity (%)	81.4	77.7	56.6	58.2	51.1	81.1	82.4
	Negative predictive value	87.7	64.5	95.0	80.6	100	97.1	96.9
	Positive predictive value	90.7	89.9	65.9	68.5	23.2	39.4	40.8
	Kappa	0.77	0.57	0.52	0.45	0.22	0.44	0.45
	Lower limit	0.74	0.40	0.48	0.38	0.19	0.40	0.41
	Upper limit	0.80	0.64	0.55	0.52	0.23	0.48	0.49

^*“complete”, analyses considering all the disorders in the clinical examination; “restricted”, analyses restricted to six disorders.

^**Standard Nordic style questionnaire, including questions about last week and/or and last year

^***Nordic style questionnaire with numerical scale (NS) score based on severity of symptoms at the time of the clinical examination

The negative predictive value was good in both surveys (64.5% to 100%), with a very good sensitivity (82.3% to 100%). The positive predictive value was high in the “Repetitive task” survey, and lower in the “Pays de la Loire” survey. The specificity was also lower in the “Pays de la Loire” survey, especially if the definition for “positive questionnaire” was an extensive definition.

If the list of symptoms in the “Repetitive task” survey was restricted to those in the “Pays de la Loire” survey, sensitivity and specificity were similar in the two studies, but the predictive values were different (table 2).

The ROC curve based on the “NS” score in the “Pays de la Loire” survey showed that specificity was at 82.4% with sensitivity at 82.3% for a more restrictive case definition, with a score at least 2 (fig. 1). The area under curve was calculated at 0.85 [95%CI 0.82–0.87].

Figure 1.

Receiver Operation Characteristic (ROC) curve between the level of the Numerical Scale score (NS score) and the clinical examination in the “Pays de la Loire” survey.

(Ss = sensitivity; Sp= Specificity).

Discussion

This study illustrates the use of Nordic-style questionnaires in two surveys with different designs and populations (table 1). In the “Repetitive task” survey the population was highly exposed to repetitiveness, with high prevalence of UWMSD (40% to 70% in the clinical examination depending on the criteria). In the “Pays de la Loire” survey, the population was representative of the working population, with a lower prevalence of UWMSD (13% in the clinical examination). Despite of the differences between the two surveys, sensitivity and specificity were similar if the analyses were restricted to the six diagnosis of the “Pays de la Loire” survey. Positive predictive values remained different, which was expected since positive predictive value depends on the prevalence of the outcome.

The administration of the questionnaire, especially the place and the context of the questionnaire could have an effect on the reported prevalence of symptoms. In a study conducted among bus drivers, the prevalence rates of UWMSD differed between survey formats, including a general health questionnaire with a Nordic style questionnaire completed before a periodic health examination and an health questionnaire completed after the periodic health examination (29). The authors concluded that prevalence increased if the subjects focused their interest on the musculoskeletal system. In some contexts, the workers might also tend to under-report symptoms if they are afraid of possible consequences for their jobs. In order to avoid information bias, the workers must be confident about data protection.

In this study, we considered the clinical examination as the method of reference. Clinical examination in the two surveys included information on present and past symptoms and physical findings, as recommended and usually done in regular medical activity. However, a questionnaire is more formal and describes a rather long period for history of symptoms whereas a standardised physical examination generally describes only the present situation. The choice of clinical examination as the reference method could be argued (30). In the “Pays de la Loire” survey, only major disorders were investigated and participants suffering from disorders without a clear diagnosis were not considered as cases. This could have decreased the clinical examination’s positive predictive value. However, symptoms represent also an important dimension. Nordlund et al. conclude that the level of questionnaire based self-reported musculoskeletal symptoms predicts future health problems, in a follow-up study comparing 129 clinically examined UWMSD cases and 655 controls (31). Feuerstein et al describe a tool for predicting clinical outcomes in patients with UWMSD, which takes into account pain severity (32). Baron et al. concluded that the reliability and validity of symptom data were acceptable for the purposes of workplace ergonomics programs (16).

The authors of the Nordic questionnaire indicate that the validity of the questionnaire, studied in a small sample, was good (13). Olhsson et al. reported the sensitivity and specificity of a screening questionnaire for neck and upper limb complains in a sample of 165 women (33). The results for sensitivity were similar to our findings (92% for shoulder, 66%–79% for others regions). The specificity was similar to that found in the “Repetitive task” survey and in the “Pays de la Loire” survey with NS score >0. Palmer et al. assessed the validity of a Nordic-style questionnaire in a sample of 105 hospital outpatients (17). Regional pain reports proved to be very sensitive in relation to specific UWMSD, in particular for cervical spondylosis (sensitivity 90% for neck pain in the last year), adhesive capsulitis (sensitivity 100% for shoulder pain in the last year), lateral epicondylitis (sensitivity 90% for elbow pain in the last year) and Raynaud’s phenomenon (sensitivity 78% for hand/wrist pain in the last year). However, with the exception of reported finger blanching in patients with Raynaud’s phenomenon, specificity was low (range 33%–38%). Silverstein et al., in a study comparing data source for UWMSD surveillance, found that the sensitivity of a symptoms questionnaire compared to physical examination plus interview was relatively high (78% to 88%), but the specificity (21%–38%) and positive predictive value (31–50%) were low (34).

In a preliminary study we conducted using data collected among workers of a shoe factory, the sensitivity of the questionnaire ranged from 65% to 70%, with a high negative predictive value (18). In this study, the possibility of bias in the prevalence rate had been discussed, with high prevalence of UWMSD according to the clinical examination (36% to 44%) and with only 46% to 58% of positive questionnaire (18). This might be due to underreporting of symptoms where workers were not confident about protection and use of these data. Overestimation of sensitivity may have occurred in the “Pays de la Loire” survey in which sensitivity was 100%. The reason probably was that physical examination was performed only if the interview of the worker indicated the presence of symptoms at the time of the examination. If the interview by the occupational physician and the questionnaire give similar information, then sensitivity is expected to be high. However, the agreement between symptoms reported by the Nordic-style questionnaire and by the physician-lead interview was poor [kappa= 0.27 (95%CI: 0.25–0.29], with 39% of workers misclassified. We found strongest disagreement if the questionnaire was positive and the interview negative (38.9%, n=1045). This difference could be partly explained by the period of symptoms which was longer in questionnaire than in interview.

In our study, the values for specificity range from 51% to 82%. Rather low values may be due to the fact that the time period of the questionnaire and the physical examination differ, between past year or past six months for symptoms in the questionnaire, and present time for the physical examination, at least in the “Pays de la Loire” survey. Some workers could have been sick and have recovered during this period, which would artificially increase the number of false positive cases and would decrease the questionnaire’s specificity. In the “Repetitive task” survey, the physical examination included among “suspected diagnosis” symptoms no longer present at the examination (or “prior diagnosis” when made by a specialist). The specificity was thus very high in this study (>75%).

In the “Pays de la Loire” survey, specificity increased from 51% to 81% if the answers to the numerical scale of intensity of pain for each region at the time of the consultation were taken into account. The lower specificity in the “Pays de la Loire” survey could be explained by the limited number of diagnoses in the physical examination, which is consistent with the results obtained for the “Repetitive task” survey with a restricted list of diagnosis.

In conclusion, Nordic-style questionnaires exploring symptoms in the past year can be considered as useful tools for the surveillance of UWMSD, especially if they include numerical scales on symptom severity. The physical examination remains essential to establish a medical or clinical diagnosis. For other purposes, questionnaires remain useful tools, giving information on functional, psychological and psychosocial dimensions of musculoskeletal disorders.