Randomized Controlled Trial of Education and Feedback for Implementation of Guidelines for Acute Low Back Pain

Abstract

OBJECTIVE

The effect of clinical guidelines on resource utilization for complex conditions with substantial barriers to clinician behavior change has not been well studied. We report the impact of a multifaceted guideline implementation intervention on primary care clinician utilization of radiologic and specialty services for the care of acute low back pain.

DESIGN

Physician groups were randomized to receive guideline education and individual feedback, supporting patient education materials, both, or neither. The impact on guideline adherence and resource utilization was evaluated during the 12-month period before and after implementation.

PARTICIPANTS

Fourteen physician groups with 120 primary care physician and associate practitioners from 2 group model HMO practices.

INTERVENTIONS

Guideline implementation utilized an education/audit/feedback model with local peer opinion leaders. The patient education component included written and videotaped materials on the care of low back pain.

MAIN RESULTS

The clinician intervention was associated with an absolute increase in guideline-consistent behavior of 5.4% in the intervention group versus a decline of 2.7% in the control group (P = .04). The patient education intervention produced no significant change in guideline-consistent behavior, but was poorly adopted. Patient characteristics including duration of pain, prior history of low back pain, and number of visits during the illness episode were strong predictors of service utilization and guideline-consistent behavior.

CONCLUSIONS

Implementation of an education and feedback-supported acute low back pain care guideline for primary care clinicians was associated with an increase in guideline-consistent behavior. Patient education materials did not enhance guideline effectiveness. Implementation barriers could limit the utility of this approach in usual care setttings.

In an effort to improve outcomes, diminish practice variation, and/or reduce cost, many groups have produced evidence-based best-practice recommendations or clinical practice guidelines. Studies of guideline implementation have demonstrated mixed findings of effectiveness.¹ Simple passive dissemination of information has rarely been effective in changing physician behavior.^2–12 Methods that have been shown to be effective in specific settings include peer-opinion leader models,^13,14 audit and feedback processes,^15–17 educational interventions,¹⁸ small group consensus processes,¹⁹ more intensive “academic detailing” models,^20,21 prospective “reminder” systems,²² and computer-based implementation.²³ Many of the interventions have been investigated in nonprimary care settings or for conditions of low complexity (drug prescribing, preventive health interventions). Few studies have attempted to change primary care physician behavior related to complex, undifferentiated conditions.

Acute low back pain is a common condition with high direct and indirect medical costs.^24–28 Recent studies and published guidelines have supported a conservative approach to the medical management of this condition.^29–38 Nevertheless, a pilot study in primary care practices revealed substantial levels of potentially unnecessary resource utilization for episodes of low back pain.³⁹ One other published study of guideline implementation in a managed care setting targeted utilization of imaging studies for patients with low back pain.⁴⁰ In this study, a peer-developed guideline was implemented by brief group education followed by passive dissemination and performance feedback. No changes in radiographic utilization rates were noted following guideline implementation in the intervention versus control group, though the “guideline consistency” of utilization events was not assessed.

We designed the present study to investigate the effect of physician education and individual performance feedback with or without patient educational materials on adherence to a clinical practice guideline for the care of acute low back pain. The physician implementation strategy combined use of local opinion leaders, small group educational sessions, and individual performance feedback utilizing principles of academic detailing. These methods had demonstrated efficacy in other situations and a multifactorial approach was considered necessary for changing clinician behavior in the face of significant barriers. We hypothesized that the availability of patient education materials specifically supportive of the guideline would facilitate clinician behavior change for this condition.

METHODS

Clinician Randomization

The study was conducted at 14 group practice sites affiliated with two not-for-profit group model HMOs in metropolitan Washington, DC. The study subjects were 120 internists, family physicians, and associate practitioners (nurse practitioners and physician assistants) caring for 160,000 HMO members assigned to these sites. Fourteen of these clinicians did not accrue eligible patients in both the baseline and intervention periods and were excluded from analysis (effective clinician sample size = 106). Clinician practices were stratified by affiliation (academic vs nonacademic) and then, using sealed envelopes, randomized by an investigator (DV) to 4 groups in a 2 × 2 factorial design: (1) a physician education and feedback intervention supporting the guidelines; (2) patient educational materials (written and video) consistent with the guideline; (3) both of these interventions; (4) no intervention.

Data Collection

The baseline year extended from July 1, 1993 to June 30, 1994 and the study year from August 1, 1994 to July 31, 1995. During these time periods, patients with diagnosis codes related to back pain or spinal disorders were identified. Three months after the index visit, trained chart abstractors, blinded to group assignment, reviewed the patient's clinical record for study eligibility. Patients were eligible for study inclusion if they met all three of the following criteria: (1) presence of low back pain; (2) duration of current symptoms less than 6 weeks; and (3) no episodes of pain reported or office visits for low back pain in the preceding year. The reason for the latter criteria was to enable comparable, but mutually exclusive, baseline and intervention year patient samples as well as to exclude patients with chronic low back pain. Chart and electronic record review was utilized to gather information on subject history and physical findings, radiographic services, specialty or physical therapy referrals, additional office or telephone consultations, and treatment provided during the 3 months following the index visit.

Intervention

A clinical practice guideline was developed through a process of literature review and consultation with national and local experts from relevant domains (General Internal Medicine, Neurosurgery, Rheumatology, Physical Medicine/Rehabilitation). The guideline suggested optimal strategies for the initial evaluation, testing, and treatment of acute low back. Supporting patient education materials, including a pamphlet and 10-minute educational videotape, were prepared by a group including physicians, a health services researcher with expertise in consumer issues in medicine, and an expert in the development of patient education materials. These materials lent support to the guideline-based back pain evaluation and management strategy.

Prior to the start of the study year, clinicians assigned to receive guideline implementation completed a standardized 90-minute educational session, which included an introduction to the guideline, a description of its development, and a series of interactive educational vignettes designed to highlight application of the guideline to various types of patients. These educational sessions were delivered by recognized clinical leaders at each of the respective institutions. Ninety percent of the assigned clinicians attended the education sessions. All clinicians received a copy of the guideline.

Following the educational session, each clinician was given an audit report summarizing their performance vis-à-vis the guideline in the care of patients with acute low back pain during the baseline year. Over and under-utilization of clinical services were highlighted and the rationale for each classification was explained (see below). Nonattendees received a copy of the guideline, their individualized audit report, and a follow-up phone call from one of the study investigators.

All clinicians in the guideline implementation group also received an individual follow-up visit from one of the study investigators 6 months into the study year. At this meeting the guideline was reviewed, questions or concerns were addressed, and another audit report covering low back pain encounters for the first 6 months of the study year was reviewed.

Clinical sites assigned to receive patient education materials received copies of the videotape and pamphlet along with a TV/VCR during a visit by one of the study investigators. Both the pamphlet and the video conveyed general information about acute low back pain and translated the guideline recommendations into lay terms. All clinicians were encouraged to review the pamphlet and videotape personally. Clinicians at patient education sites received two additional written reminders to use the materials during the first 3 months of the study year.

Outcome Evaluation

The clinician's decision to provide each of four clinical services (plain x-rays, CT or MRI, subspecialty referral, physical therapy referral) was assessed for consistency with the guideline for each patient encounter during the baseline and study years. Plain lumbar radiographs were consistent with the guideline when obtained at the first visit if any of the following conditions were present: trauma, age > 50 years, fever, history of malignancy, neurological deficits on physical examination, or if obtained greater than 6 weeks after the index visit if no improvement in the clinical condition.

Computed tomographic scans or magnetic resonance imaging scans of the lumbar spine were consistent with the guideline at any time in the presence of cauda equina syndrome or progressive neurological deficits and if obtained greater than 4 weeks from the index visit in patients with neurological deficits or sciatica that failed to improve.

Subspecialty referral (Neurosurgery/Orthopedics/Rheumatology/Spine Center) was consistent with the guideline at any time in the presence of cauda equina syndrome or progressive neurological deficits or if obtained greater than 4 weeks from the index visit in patients with neurological deficits or sciatica that failed to improve.

Physical therapy referrals were consistent with the guideline if obtained greater than 6 weeks from the first visit if symptoms had not improved.

To confirm the reliability of the chart audit assessment of guideline-consistent behavior, two clinician investigators independently reviewed 100 patient records (oversampled for utilization events), blinded to the prior audit findings. Though the κ between investigators and audit was only 0.55, a review of classification disagreements between the algorithm and the clinicians revealed that almost all were the result of a misinterpretation of one criterion (dating the start of the episode from reported symptom onset rather than index visit date) by the investigators. The audit classified these events appropriately. Residual misclassification occurred in less than 2% of events and was the result of errors in chart abstraction or data entry.

Patient Survey

Following determination of study eligibility, patients were surveyed by telephone at least 3 months after the index low back pain encounter. The survey gathered information regarding beliefs about the care of low back pain, satisfaction with care, and measures of clinical outcome using previously validated instruments.^41,42

Clinician Survey

At the conclusion of the study year, we surveyed all clinician subjects accruing at least 8 back pain patient encounters in both baseline and intervention periods (N = 53). We assessed perceptions of the guideline and patient education materials for the relevant subgroups as well as contamination across intervention groups.

Statistical Analyses

Utilization of services (plain x-ray, MR/CT, specialty referral, physical therapy) consistent with the guideline was assessed for each encounter. Each clinician's guideline-consistent utilization was averaged across all encounters for both the baseline and intervention periods. The 106 clinicians accrued a mean of 19 patients in each period. The outcome measure thus reflected the proportion of encounters in which services were utilized consistent with the guideline. The principal study outcome was the summary measure reflecting guideline-consistent utilization of all services. The clinician level model utilized analysis of variance weighted by the reciprocal of the error variance for each physician estimate (reflecting the number of patients accrued). Separate models limited to clinicians accruing at least 8 patients in each period yielded similar results. A nested model (physician within site) was utilized because of the randomization at the practice site level with the baseline level of guideline-consistent utilization included as a covariate. In addition to the summary analysis, the effect of the intervention on each of the four individual services was assessed in separate models. To identify whether the intervention affected total utilization as opposed to only guideline-consistent utilization, these analyses were repeated using the raw utilizaton data as the dependent variable.

A two-tailed P value of .05 was the criterion for statistical significance of the principal summary outcome measure (i.e., overall guideline-consistent utilization of services). Power analysis indicated that there was > 80% probability of detecting a 10% difference between groups with respect to aggregate utilization rates based on the actual sample size. In light of multiple comparisons, an intervention effect on secondary outcome measures (e.g., utilization of individual services) was considered statistically significant at P = .01. Analyses were conducted using the SAS System release 8.02 (SAS Institute, Inc., Cary, NC).

Descriptive models were constructed to evaluate the effect of patient characteristics on the utilization of services. The dichotomous-dependent variable was utilization of each of the 4 services (with a fifth model evaluating utilization of any of these services). Logistic regression was used to model the effect of patient age, gender, prior history of low back pain, duration of current symptoms, presence of reported sciatica, neurological findings on exam, number of visits in this episode of care, and history of cancer while controlling for intervention group and time period.

RESULTS

Table 1 describes patient and physician characteristics of the four study groups. Since analyses of the impact of the patient education intervention on the utilization of clinical services revealed no effect, the four intervention groups were collapsed into two (clinician intervention vs no clinician intervention) for analysis and reporting purposes. Analyses performed with and without the inclusion of the 21 nurse practitioners and physician assistants produced similar results (aggregate results are reported).

Table 1.

Patient and Clinician Characteristics by Study Group

	Control Group	Patient Intervention Only	Clinician Intervention Only	Patient/Clinician Intervention
Study patients
N (baseline year)	590	416	481	533
N (study year)	544	284	588	630
Gender, % female	54	55	58	54
Age, mean	45.5	43.0	45.4	45.3
Prior low back pain, %	42	37	33	32
Sciatica, %	33	32	29	31
Neuro exam signs, %	8	7	4	6
Study clinicians
N (physicians)	20	24	20	21
N (NP or PA)	6	6	4	5
Years practice, mean	8.7	9.3	10.5	11.0
Gender, % female	64	42	47	55

Though randomization appeared successful in achieving fairly similar groups (Table 1), subsequent analysis of utilization data (Table 2) suggested important differences between them. The intervention group had substantially higher utilization of radiologic and specialty services during the baseline period. Similar baseline differences were found for utilization of services inconsistent with the guideline. These differences remained, though were diminished, after adjustment for patient characteristics (duration of symptoms, history of low back pain, and number of visits during episode) that were strongly associated with utilization.

Table 2.

Utilization of Clinical Services (% of Patients Based on Episode of Care)

	Physician Education and Feedback				Physician Control Group
	Baseline Year		Intervention Year		Baseline Year		Intervention Year
LS spine x-ray	31*	14.5†	19*	8.1†	21*	8.2†	18*	8.6†
PT referral	12*	10.0†	10*	9.2†	13*	10.9†	13*	12.0†
LS CT or MRI	7.6*	5.7†	5.6*	3.5†	5.6*	3.5†	7.1*	5.4†
Specialty referral	12.0*	9.5†	8.6*	7.1†	5.9*	4.0†	7.1*	5.6†
≥1 of above	41*	29.9†	31*	21.2†	33*	21.5†	32*	24.3†

lS, lumbosacral; PT, phqsical therapy; CT, computed tomography; MRI, magnetic resonance imaging.

^*Total utilization of clinical services (% of patients based on episode of care).

^†Utilization not consistent with guideline (% of patients based on episode of care).

The primary analysis of intervention effectiveness revealed a significant increase in guideline-consistent behavior among physicians assigned to receive the education and feedback intervention. At the physician level, guideline-consistent behavior increased by 5.4% in the intervention group versus a 2.7% decline in the control group (P = .046). This was paralleled by an overall decline in raw utilization of services (without respect to guideline consistency) of 8.5% in the intervention group versus 0.6% in the control group (P = .042). Models controlling for patient-level covariates, including prior history of back pain, duration of symptoms, and number of visits, did not diminish the effect size. Though there were trends toward improved guideline-consistent utilization of some individual services, none of these changes was statistically significant.

The patient education intervention produced no effect. Providers receiving the patient education materials demonstrated no change in overall guideline-consistent actions from baseline to intervention periods, whereas those without access for their patients to the study videotape and pamphlet showed a 2.8% increase in guideline-consistent utilization (P = .71 for the difference).

Patient characteristics were strong predictors of the utilization of all services during an episode of low back pain (Table 3). Controlling for the study period and study group assignment, the odds of receiving any service (plain x-ray, CT/MRI, physical therapy and/or specialty referral) was independently associated with a history of cancer, age > 50 years, prior history of low back pain (though without clinical services in the past year), duration of pain > 3 weeks prior to the current index visit, sciatica, neurological findings on exam, and the number of visits for back pain during this episode of care. The receipt of each individual service was also predicted by number of visits, longer reported duration of acute pain, documented history of prior low back pain, presence of sciatica (except physical therapy), and documented neurological findings. Only plain films were predicted by age and history of cancer. Modeling guideline-consistent utilization as a function of the same independent variables yielded similar results.

Table 3.

Independent Predictors of Utilization of Clinical Services

	LS Spine XR		PT Referral		CT/MRI		Specialty Reference		Any Service
	OR	P Value	OR	P Value	OR	P Value	OR	P Value	OR	P Value
Hx of cancer	1.9	.001	NS*		NS		NS		1.7	.006
Gender	NS		NS		NS		NS		NS
Age > 50 years	1.5	<.0001	NS		NS		NS		1.3	.0005
Hx low back pain	1.2	.03	1.5	.0004	1.3	.04	1.4	.003	1.6	<.0001
Pain > 3 weeks	1.5	.0003	1.9	<.0001	1.8	.0006	1.8	.0002	1.9	<.0001
Sciatica	1.3	.002	NS		2.5	<.0001	2.1	<.0001	1.5	<.0001
Neuro findings	1.7	.0008	1.7	<.007	3.6	<.0001	3.3	<.0001	2.4	<.0001
# visits in episode	1.9	<.0001	2.8	<.0001	2.9	<.0001	1.9	<.0001	3.0	<.0001

ls, lumbosacral; pt, phqsical therapy; Hx, history of; xr, x-ray; CT, computed tomography; mri, magnetic resonance imaging.

^*NS = not statistically significant (P > .05).

The number of visits for back pain was the strongest predictor of utilization in the model. Though only 32% of intervention group patients had more than one visit, 60% of guideline-inconsistent utilization occurred in this subgroup. To elucidate cause versus effect, we examined the likelihood of services being ordered on the first visit versus subsequent visits in the intervention group (Table 4). The odds of ordering were higher at subsequent visits for all services, dramatically so for referrals and advanced imaging.

Table 4.

Ordering of Services at Initial Versus Subsequent Back Pain Visit

	LS Spine XR	PT Referral	CT/MRI	Specialty Ref.	Any Service
Initial visit, %	14.1	4.4	2.4	2.6	20.1
Subsequent visit, %	22.3	23.0	11.2	17.1	49.1
Odds ratio	1.7*	6.4*	5.2*	7.8*	3.8*

ls, lumbosacral; pt, phqsical therapy; ct, computed tomography; mri, magnetic resonance imaging.

^*P < .0001.

Although 80% of surveyed physicians in the patient education groups reported awareness of the educational materials, only 33% reported that they occasionally or usually distributed the pamphlet and only 13% noted occasionally using the videotape. The study patient survey (response 44%) showed no substantial difference in reported receipt of patient education materials (59% vs 54%) between the patient education intervention and control arms. Only 3.2% of study patients versus 2.2% of control patients reported viewing a low back pain videotape.

Surveyed patients frequently held beliefs about the care of back pain that conflicted with the guideline recommendations. For instance, 79% believed that low back pain should be treated by a specialist and 78% believed that x-rays should be ordered. The patient education intervention did not affect these beliefs.

Among clinicians in the guideline intervention group attending the original educational sessions, 100% reported that they were slightly or somewhat more knowledgeable about back pain care and more confident in their approach to such patients. All such respondents reported that they agreed with “many” or “almost all” aspects of the guideline and reported that the guideline at least “slightly” altered their care of patients with low back pain. However, 75% of the respondents reported that unique circumstances related to individual patients were “very important” reasons for their deviation from the guideline.

Among clinicians not in the guideline intervention groups, 47% reported hearing of the guideline. However, only 21% reported knowledge of the guideline and 16% reported that knowledge had slightly changed their practice behavior.

CONCLUSION

An intervention based on accepted strategies of physician education, practice audit with performance feedback, and peer opinion leader use produced a modest but significant increase in physician actions consistent with a clinical guideline for the care of acute low back pain. The addition of a patient education intervention did not produce significant changes in the outcome measures of interest, likely a result of failures in implementation and adoption.

There are several potential explanations for our findings. While the randomization process did produce similar groups from a demographic perspective, subsequent utilization data obtained in the baseline year revealed greater procedure and consultant utilization in the intervention group. It is possible that the improvement in clinician utilization behaviors could at least partly be explained by regression to the mean during the intervention year. The fact that the disparity in baseline utilization rates between the groups diminished after adjustment for visit frequency and other clinical covariates suggests a more severe case mix in the intervention group. However, the intervention effect size did not diminish after adjustment for patient clinical covariates.

Since baseline utilization was lower and guideline-consistent behavior higher than expected when compared with other primary care studies of low back pain,^9,43,44 as well as preliminary audit data from the study site,³⁹ there was less room for improvement than had been anticipated. We believe that the high control and baseline rates of guideline-consistent behavior are at least in part explained by two factors. First, it may have reflected increased general attention to back pain care in the literature, exemplified by the release of the AHCPR back pain guideline (with similar recommendations to the intervention group's guideline) in the midst of the intervention year.³⁸ It also may indicate greater emphasis on utilization issues by the participating managed care organizations—though there were no specific initiatives in this regard, both organizations studied were under substantial financial duress during this period. Though contamination of control clinician groups by the guideline intervention could have contributed to their low utilization, the physician survey suggests that knowledge of the specific study guideline was not widespread. Furthermore, control group utilization remained stable between baseline and intervention periods.

One concern about clinical guidelines for acute low back pain is that they might prompt increased utilization of services, rather than the generally intended opposite effect.^44,45 Contrary to this theoretical concern, we observed no increase in service utilization in our study (Table 2). Study clinicians, practicing in a managed care environment, were likely to be more cognizant of utilization concerns and therefore less prone to request services fulfilling “soft” guideline criteria (such as age > 50 for plain films). Additionally, the managed care patient population tended to be relatively young and healthy and thereby less likely to fulfill some of these same criteria. Guideline-driven increased service utilization remains a legitimate concern and may depend on the clinical setting as well as the guideline criteria and language.

Preexisting patient and physician beliefs may have reduced the study guideline's effect. Physicians harbor substantial concerns about the impact of clinical practice guidelines in general on medical practice,⁴⁶ and many hold beliefs about the value of diagnostic and therapeutic interventions for back pain that conflict with guidelines for its care.^47–49 Particularly when confronted with patient expectations for services, it may be harder to convince physicians to do less for patients rather than more,⁵⁰ and research clearly demonstrates that patient expectations play a significant role in the utilization of services for low back pain.^51–55 Most of the patients surveyed in our study had strong beliefs about the need for testing and referral that conflicted directly with the guideline recommendations. Consequently, we felt that a patient education intervention to address this barrier to guideline-consistent actions would complement the physician intervention.

The patient education intervention was designed to support the guideline's evaluation and management strategy by addressing frequent misconceptions about low back pain care. Though we structured the low-intensity pamphlet and videotape-based approach so that it could be practical for wide dissemination, implementation and adoption problems limited any potential effect. Greater attention to integration of the educational materials into the practice structure via use of nurses, office staff, or automated triggers may have led to greater use and impact.^56,57 However, though more intensive patient education interventions for low back pain have been successful in altering patient beliefs and satisfaction, they have shown little impact on measures of functional outcome or resource utilization.^8,58–60 Patient intervention efforts may need to add cognitive and behavioral approaches to the more traditional educational ones in order to improve, albeit modestly, low back pain outcomes.⁶¹

In addition to patient beliefs, individual factors related to the illness episode may be important to the success of guideline implementation strategies. In this study, the frequency of encounters during the back pain episode was strongly associated with utilization of clinical services that was not consistent with the guideline. This finding is concordant with prior studies that have suggested that the chronicity and severity of symptoms may compel physicians to do more in the absence of other firm indications for additional services.^{4,52,53,62–64} Our results suggest that guideline implementation efforts in this area might benefit from strategies directed specifically at these issues, especially that of return visits for persistent symptoms. Guideline adherence could have been facilitated by offering physicians intermediate strategies for patients suffering with more severe or prolonged pain, but without worrisome neurological features. In retrospect, the recommendation of a strictly conservative approach during the first 4 to 6 weeks of care may have left the clinicians with little defense against the onslaught of patient symptoms and expectations. If the guideline had supported earlier use of physical therapy, for instance, there might have been lower use of other specialty services and advanced imaging in the absence of clear indication. However, such an approach would likely have resulted in more costly overall care without significant improvement in clinical outcomes.⁶⁵ Alternative “stepped-care” approaches have been suggested but have not been empirically validated.⁶⁶ Though separate evidence-based systems of care for low back pain appeared to utilize services efficiently in one study,⁶⁷ their generalizability is questionable.

In this study, clinicians exhibited greater guideline-consistent clinical practice following a multifaceted, moderate-intensity active intervention. Another trial⁴⁰ utilizing predominantly passive dissemination of guidelines and performance feedback in a similar setting failed to show an effect, suggesting that the threshold of intervention intensity necessary to affect clinician behavior in areas with substantial barriers to change may lie between these two studies. Given the difficulties inherent in translating research into practice, it is reasonable to question whether the interventions we describe are generalizable to other health care settings and conditions. Even a well-organized group might encounter difficulties targeting multiple illnesses for similar active guideline-based interventions. To test and apply implementation strategies such as we studied to other usual care settings might prove to be prohibitively costly or impractical. Alternative strategies to alter systems of primary care practice and reduce dependence on direct clinician interventions may have promise.^68–70 As electronic medical record technology matures and is adopted, it will also provide a strong platform for guideline dissemination and implementation strategies.

In summary, this study demonstrates that an intervention based on accepted principles of group education, performance audit and feedback, and use of opinion leaders can result in clinician behavior change in an area with substantial barriers to improvement. It is clear, however, that the challenges to generalizing and extending such success are considerable.