The Reliability of Maitland's Irritability Judgments in Patients with Low Back Pain

Abstract

Maitland's construct of musculoskeletal pain irritability is widely used by physical therapists but has not been defined to the extent that its measurement properties can be tested. The purpose of this study was to examine the inter-rater reliability of physical therapists' irritability judgments during application to patients with low back pain (LBP). Eighty patients with LBP received two consecutive examinations at their initial clinic visit by two physical therapists. Patients reported pain location and intensity prior to each evaluation. Therapists judged subjects' LBP as irritable or non-irritable. Inter-rater agreement of physical therapist irritability judgments was moderate (kappa = 0.44, prevalence-adjusted kappa = 0.50). This level of reliability of therapists' LBP irritability judgments may be improved upon by development of an operational definition of pain irritability. Further research is needed to identify measures appropriate for inclusion in an operational definition of pain irritability and to assess the value of making pain irritability judgments in evidence-based physical therapy practice.

Musculoskeletal pain irritability is a clinical characteristic developed by Maitland and used by physical therapists to modify the vigor of examination and treatment techniques^1–4. Physical therapists report that they frequently use irritability judgments in the clinical decision-making process^3,5,6. Although widely used, low back pain (LBP) irritability has not been defined to the extent that its measurement properties can clearly be tested, and no published investigations exploring the measurement characteristics of LBP irritability were found by these authors.

When evaluating a patient's musculoskeletal problem, Maitland considers the patient's pain severity and irritability, as well as the nature of the disorder². The interaction of these characteristics is used to guide the therapist's examination and treatment decisions. Pain is considered severe if the intensity is sufficient to interrupt a patient's activity. Pain severity is used as a consideration in determining pain irritability. The nature of a disorder includes considerations such as stability of the problem (worsening, improving, stabilized), stage of healing, known pathologic processes, and/or fear-avoidance behaviors.

Pain irritability is assessed by judging: 1) the vigor of activity required to provoke a patient's symptoms, 2) the severity of those symptoms, and 3) the time it takes for the symptoms to subside once aggravated (i.e., pain persistence)^1–3. Physical therapists who judge a patient's LBP as irritable should decrease the vigor of the physical examination and exercises initially prescribed to the patient². The value of regulating the vigor of examination and initial treatment is to avoid exacerbating the patient's symptoms while maximizing the patient's mobility. Some mobility exercises are designed to decrease low back and/or lower extremity pain, such as those given to patients with McKenzie's derangement syndrome⁷. Since the goal of these types of exercises is to reduce or control easily exacerbated LBP, they are appropriate for prescription to patients with irritable LBP. Physical and mechanical agents (e.g., temperature and/or electrical modalities, traction) might also be provided to patients with irritable LBP with the goal of decreasing pain and improving mobility. The benefit of judging LBP irritability over a course of treatment has not been demonstrated to improve patient outcomes. However, maximizing the mobility of a patient with LBP has been shown to improve outcomes^8–16. Therapists use LBP irritability status as a tool to gauge the patient's “mobility starting point.”

Maitland judges a patient to have irritable LBP when the pain is easily aggravated, severe, and persistent for a prolonged period following a cessation of the aggravating activities^1,2. A primary hypothesis of the extent of LBP irritability is made by the therapist following the medical history and is modified based on findings in the examination.

Maitland's instructions on how to make irritability judgments are not stated in a clearly defined operational definition but rather as vignettes of patients' clinical presentations. Maitland provides an example of making an irritability judgment using the low vigor activity of ironing clothes². Suppose that ironing clothes for half an hour elicits LBP to the degree that the patient must stop the activity. If the pain subsides within a half an hour, allowing the patient to iron for another half hour, Maitland would judge the patient's disorder to have “minor irritability.” However, if the patient could not iron again until sleeping through the night, Maitland would consider the disorder to be “irritable.” An example given by Koury et al of an irritable shoulder condition illustrates consideration of the same variables¹. They described a patient who flexes his or her shoulder once or twice resulting in increase in symptoms that persist for 30 minutes as having irritable pain; alternately, if the symptoms dissipate immediately on return from flexion, the pain is considered non-irritable.

Because a measurement construct of irritability has not been operationally defined, we contend that when therapists assess LBP irritability, judgments based on these assessments are likely to contain a substantial amount of variability. This variability will diminish the reliability of therapists' judgments of irritability and reduce their value for making clinical decisions. Since pain irritability

is a concept that is widely used by physical therapists as a major consideration in examination planning and treatment decisions, an exploration of the measurement properties of irritability is warranted. The purpose of this study was to examine the inter-rater reliability of Maitland's construct of irritability in patients with LBP.

Methods

Raters

Thirty physical therapists with experience in the evaluation and treatment of patients with LBP participated in this investigation; 27 were employees of the participating clinics and 3 were recruited from outside the clinic. The clinic-employed therapists had been practicing on average 6.1 years (range: new graduate to 21 years) and received on average 98 hours of advanced training in the evaluation and treatment of patients with LBP in the form of short- and long-term continuing education courses. The three non-employee physical therapists had been practicing an average of 24 years (range 22 to 25 years), and each had received more than 500 hours of advanced training in the evaluation and treatment of patients with LBP, also in the form of short- and long-term continuing education courses. None of the participating therapists were certified as an Orthopedic Clinical Specialist by the American Physical Therapy Association, were fellows in the American Academy of Orthopedic Manual Physical Therapy, or had received an Australian Masters or Graduate Diploma in Manipulative Physiotherapy at the time of the study.

Pre-Test Training

Participating physical therapists all attended an orientation prior to beginning data collection that covered the purposes and procedures of this investigation. Maitland's definition of LBP irritability was reviewed by identifying the characteristics that guide classification: time and vigor of activity required to aggravate pain, severity of pain, and persistence of pain after aggravating activities are stopped. All therapists were told that irritable LBP is easily aggravated, of moderate to severe intensity, and persistent after the aggravating symptoms are stopped (though no specific time guideline for pain persistence was provided). Non-irritable LBP was described as not easily aggravated, usually not severe, and not persisting after the aggravating activity is stopped. Therapists were told that the treatment for patients with irritable LBP might include manual techniques, pain-relieving agents, and/or exercises designed to decrease or avoid aggravation of symptoms. Treatment for non-irritable LBP might include flexibility, strengthening, and/or conditioning exercises with minimal concern for aggravating pain complaints. Specific guidelines for deciding irritability status based on the interaction between ease of aggravation, pain severity, and pain persistence were not provided to therapists because none are known to exist. Treatment prescriptions were left to the therapists' judgment based on their evaluations. No assessment of therapists' concept of irritability was administered prior to data collection.

Patient Sample

Data for this investigation were gathered from 80 subjects consecutively recruited from two physical therapy clinics: one in Sacramento, CA, and one in Davis, CA. Patients were admitted to the study if they were: a) at least 18 years of age, b) able to speak and read the English language fluently (no interpreters were available), and c) available to be contacted by phone for a one-year period (for research purposes not covered in this report). Patients were excluded from the study if they a) had undergone back or neck surgery, b) were diagnosed with a vertebral or hip fracture, c) had plans for surgery in a hospital within six months of recruitment, d) were unable to walk one city block for a reason other than LBP, e) were pregnant, or f) reported progressive lower limb weakness or recent incontinence. Demographic, anthropometric, and clinical characteristics of the sample are described in Table 1.

TABLE 1.

Subjects' characteristics.

	Clinic #1 n=53	Clinic #2 n=27	Total n=80
% Male/female	43%/57%	33%/67%	40%/60%
Age (SD∗)	43.6 (14.4)	49.1 (12.6)	45.4 (14.0)
Body Mass Index (SD)	26.4 (4.9)	27.4 (4.7)	26.7 (4.8)
Years Education (SD)	16.6 (2.7)	15.2 (3.1)	16.1 (2.9)
Race	n (%)	n (%)	n (%)
White	44 (83%)	18 (67%)	62 (77%)
Hispanic†	1 (2%)	2 (7%)	3 (4%)
Other	8 (15%)	7 (26%)	15 (19%)
Distal Symptoms Present	68%	58%	65%
% with Chronic LBP‡	34%	41%	35%
Average Initial RMDQ⊠ Score	10.6 (5.7)	11.1 (6.5)	10.7 (6.0)
Initial Pain Intensity Score§	3.7 (2.3)	4.4 (2.3)	4.0 (2.3)
Insurance Types
HMO/PPO	35 (66%)	18 (67%)	53 (66%)
Medicare/Medicaid	5 (9%)	7 (26%)	12 (15%)
Worker's Compensation	5 (9%)		5 (6%)
Other	8 (15%)	2 (8%)	6 (8%)

^∗SD = Standard Deviation.

^†Hispanic is an ethnic group, not a race.

^‡Chronic LBP defned as greater than 6 months duration.

^§Pain Intensity Score = Rating of current pain intensity level on a scale of 0 to 10.

^⊠RMDQ = Roland-Morris LBP Disability Questionnaire (0 to 24 range, 24=maximum disability).

Experimental Protocol

This study employed a test-retest design to determine the inter-rater reliability of physical therapists' LBP irritability judgments. Prior to data collection, the procedures and subject informed consent forms were approved by the California State University, Sacramento Committee for the Protection of Human Subjects, the Sutter Health Central Area Institutional Review Committee, and the University of California, Davis Institutional Review Board.

When a patient first contacted the clinic, he or she was recruited to participate in the study. If tentative agreement was obtained, eligibility was determined and, if eligible, demographic and anthropometric information was obtained. Eligible patients signed an informed consent on the day of their initial examination.

Subjects then underwent two examinations for LBP; the second occurred immediately following the first. Prior to each LBP examination, subjects completed a pain drawing and a numeric pain rating scale that asked subjects to rate their current LBP intensity. The numeric pain rating scale ranged from 0 to 10 with 0 being no pain and 10 being extremely severe pain¹⁷.

Subjects were first evaluated by a physical therapist who was not an employee of the physical therapy clinic. A non-employee therapist was used to minimize impact on the clinics' scheduling procedures. The first evaluating physical therapist did not provide the subject with any treatment. After the first physical therapist completed her evaluation (all were women), she recorded her irritability judgment as either irritable or non-irritable.

The subject completed the second pain drawing and again recorded his or her current pain intensity on the 0 to 10 scale so that any changes in the subject's status between the first and second evaluations could be noted. A change in pain intensity of more than two points between the first and second evaluation was considered meaningful based on the reported minimal detectable change of a similar numeric pain rating scale¹⁸. A patient's pain drawing changing to include or exclude at least one joint was considered a meaningful difference in pain location.

Next, the second evaluating physical therapist, an employee of the clinic, evaluated the subject. Following the evaluation, this therapist provided the subject with treatments consistent with the standard of practice at the participating clinic. The therapist then recorded his or her irritability judgment. Each evaluating physical therapist was blinded to the judgment of the other.

Analysis

To assess the reliability of the physical therapists' judgment of LBP irritability, an unweighted kappa statistic was used to assess their level of agreement beyond chance. A kappa statistic adjusted for prevalence effects and rater disagreement bias (PABAK) was also reported^19–21. The percentage of agreement of therapists' judgments was reported as well. The McNemar's test²² was used to determine if the proportion of irritable LBP judgments was significantly different for employee and non-employee physical therapists. A difference found in the proportion of subjects judged to have irritable LBP by employee and non-employee therapists would suggest that the therapists' level of training or the order in which the evaluations were performed were factors in the number of disagreements.

Version 8.1 of SAS (SAS Institute Inc, Cary, NC) was used to compute descriptive statistics. A macro created on Microsoft Excel 2002 was used to compute the kappa statistics. The McNemar's test was computed manually.

Results

Of the 80 repeated examinations, 53 were conducted at one clinic and 27 at another. Table 2 describes the agreement of LBP irritability status judgments between two physical therapists evaluating the same subjects. The therapists agreed upon subjects' LBP irritability status 71% of the time. An unweighted kappa statistic of 0.39 (95% CI = .19 to .60) was calculated based on the irritability judgment agreement between the two therapists. The PABAK statistic was 0.43 (95% CI = .23 to .62).

TABLE 2.

Low back pain irritability judgment agreement between two raters (n=80).

		Rater 2 (Non-Employee Physical Therapist)
		Irritable	Non-irritable
Rater 1 (Employee Physical Terapist)	Irritable	19	9
	Non-irritable	14	38

The percentage agreement of irritability judgments for the three non-employee physical therapists with employee therapists were 69%, 73%, and 83%. The percentage agreement at each of the two clinics was 70% and 72%. The McNemar's test revealed no significant difference in the proportion of irritable LBP judgments between employee physical therapists and non-employee physical therapists (p>.05).

A total of 48 of the 80 subjects reported pain location and pain intensity measures that did not change between the repeated examinations. For this subsample of 48 subjects, therapists agreed on the LBP irritability status of subjects 77% of the time, and a kappa statistic of 0.44 (95% CI = .17 to .70) was computed (Table 3). The PABAK statistic was 0.50 (95% CI = .26 to .74).

TABLE 3.

Low back pain irritability judgment agreement between two raters (n=48) excluding subjects with a change in pain location or lbP intensity level.

		Rater 2 (Non-Employee Physical Therapist)
		Irritable	Non-irritable
Rater 1 (Employee Physical Terapist)	Irritable	10	6
	Non-irritable	6	26

Discussion

The reliability of the participating physical therapists' current method of judging LBP irritability was found to be fair to moderate. A kappa statistic of 0.39 computed when considering all subjects (n= 80) reflects fair reliability of the physical therapists' judgment of subjects' LBP irritability²³. A kappa statistic of 0.44 when considering only subjects who did not experience changes in pain intensity or pain location between evaluations (n=48) reflects a moderate reliability of the physical therapists' judgment of subjects' LBP irritability²³. When these values are adjusted for prevalence effects and differences in proportions of rater disagreements, the kappa statistic is adjusted upward (PABAKs of 0.43 and 0.50, respectively). Thus, the reliability of the therapists' irritability judgments was found to be solidly in the moderate range. The confidence intervals for the kappa values were large due to limited sample sizes, and they encompassed the fair to moderate ranges of reliability. A more accurate kappa statistic estimate derived from a larger sample would likely remain in the fair to moderate range.

No differences were found in the reliability of the therapists' irritability judgments when subjects with acute LBP (onset < 6 months) and subjects with chronic LBP were considered separately. The number of different physical therapists who participated in the study precludes analysis of patterns of agreement based on therapist pairing.

The modest level of reliability found in this study is not surprising given the lack of quantifiable guidelines in Maitland's definition. A number of factors may have contributed to the level of reliability observed. One factor may be that the physical therapists were asked to make a dichotomous irritability judgment based on a definition that considers three or more variables. Irritability judgments are considered by some therapists as multi-level and ordinal (non-irritable, mildly irritable, and very irritable). Providing therapists with more levels of LBP irritability to choose from might improve the reliability of irritability judgments, though from a statistical viewpoint, as the number of categories to choose from increases, the extent of agreement generally decreases²⁴.

It is possible that the estimates of the reliability statistics are modest due to a training bias introduced in the study design. There was a difference in the amount of advanced training between the employee and non-employee physical therapists, though the employee therapists, averaging over six years of experience treating patients with LBP, were not novices. The McNemar's analysis found both groups of therapists rated approximately the same proportion of subjects to have irritable LBP, suggesting that the error observed was random rather than due to a training effect. It is possible the reliability of irritability judgments would be higher if all therapists involved in a study had advanced training in Maitland's treatment philosophy, but the ability to generalize the findings from such a design to practicing physical therapists would be diminished.

An information bias also needs to be considered as having a possible effect on the level of reliability found in this study. The non-employee physical therapists made an irritability judgment after only an initial evaluation while the employee physical therapists made the judgment knowing the subject they were examining had already received a physical therapy examination. The employee therapist also provided the subject with the initial treatment prior to completing the questionnaire on the subject's irritability status. Thus, the employee physical therapist had more information on which to base an irritability judgment than the non-employee physical therapist. While it would have been preferable to blind the employee therapist to the fact that the subject had already received one physical therapy examination and for the employee therapist to complete the irritability judgment form prior to providing treatment, logistically it was not practical.

To address this issue, we performed the reliability analysis only on those subjects who did not have changes in pain intensity or pain location between the first and second evaluation (i.e., therapists likely saw the same clinical presentation). For these “stable” subjects, the reliability of therapists' judgments was still in the “moderate” range. A McNemar's analysis of this subgroup also found that both groups of therapists rated the same proportion of subjects to have irritable LBP, suggesting that the error observed was again random rather than due to an information bias.

Of the three clinical factors (ease of aggravation, severity of symptoms, and pain persistence), two are not clearly defined, which can lead to variability in the therapists' judgments. When a therapist judges the ease of LBP aggravation, both the vigor and duration of the aggravating activity are considered². The vigor of the activity reflects the physical demands of the activity being performed (e.g., sitting versus lifting). The duration of the activity reflects how long an individual can perform the aggravating activity before he or she cannot continue². The subjectivity of the interaction between these two variables needs to be addressed for consistent irritability judgments to be made.

The severity of pain symptoms appears to be adequately defined. Maitland states that “it is useful to describe the symptoms of a patient as ‘severe’ if the activity that causes the pain needs to be interrupted because of the intensity of the pain”².

Judgments of LBP persistence may vary considerably depending on patients' willingness to aggravate symptoms. If LBP has affected the patient's function to the degree that the patient has limited his or her activities, the patient may not experience periods of persistent pain because the patient has altered his or her activities to avoid extended periods of LBP. Medication may also be used by the patient to minimize persistent pain levels, further clouding pain persistence reporting. Thus, the therapist may find it difficult to consistently assess the extent of LBP persistence with some patients.

Reliability alone does not define the value of a clinical measure in physical therapy practice. The descriptive values assigned to reliability coefficients are arbitrary and should be considered in the context of the validity of the clinical measure²⁵. The ability of the clinical measure to predict the effectiveness of a treatment intervention is the most important reflection of the measure's value. Low reliability may impair the predictive validity of a clinical measure so the reliability should be evaluated to determine if the reliability and subsequent predictive validity of the clinical measure can be improved upon. Some clinical measures have reasonable predictive validity even with modest reliability values²⁵. Some clinic measures with modest reliability have been combined into clusters of measures to improve their reliability and predictive validity^26–28. Therefore, it is important not to simply discard a frequently used clinical measure because it has found to have low reliability. It is important to recognize if the reliability can be improved upon and to consider reliability in relation to the predictive validity of the measure.

Developing a standardized operational definition to determine a clinical construct's “score” can improve the reliability of the construct, thus improving its potential predictive validity. There is enough vagueness in the current definition of pain irritability that further investigation toward the development of an operation definition is warranted.

Conclusion

The reliability of LBP irritability judgments as currently made by physical therapists was found to be moderate. Developing an operational definition for judging irritability may improve its reliability. Further research is needed to identify measures appropriate for rating patients' pain irritability and assessing the value of making irritability judgments in evidence-based physical therapy practice.