Adherence to and uptake of clinical practice guidelines: lessons learned from a clinical practice guideline on chemotherapy concomitant with radiotherapy in head-and-neck cancer

Abstract

Background

Clinical practice guidelines (cpgs) are systematically developed statements designed to assist practitioners and patients in making decisions about appropriate heath care interventions. Clinical practice guidelines are expensive and time-consuming to create. A cpg on concurrent chemotherapy with radiation therapy (ccrt) was developed in Ontario at a time when treatment approaches for head-and-neck cancer were changing significantly.

Methods

An assessment of treatments and outcomes based on electronic and chart data obtained from a population-based study of 571 patients with oropharynx cancer treated in Ontario (2003–2004) was combined with a review of relevant knowledge transfer (publications and presentations at major meetings) to understand variation in adherence to a cpg.

Results

In 9 Ontario cancer treatment centres, ccrt was used for 55% of all patients with oropharyngeal cancer; however, at the centres individually, that proportion ranged from 82% to 39%. Furthermore, there was no agreement on the chemotherapy regimen: 2–4 years later (a period during which newer regimens were emerging), only 4 of 9 centres were following the guideline for most patients. When outcomes of treated patients were compared for centres with “higher” and “lower” use of ccrt, no difference in survival was observed (p = 0.64).

Conclusions

At a time of treatment evolution, the new guideline was controversial, and there are many reasons for the mixed adherence. An estimation of adherence should be included during both development and review of guidelines.

1. INTRODUCTION

Clinical practice guidelines (cpgs) are systematically developed statements designed to assist practitioners and patients in making decisions about appropriate heath care interventions for specific clinical circumstances¹. Evidence demonstrates that cpgs can influence the processes and outcomes of care^2–4.

To inform their recommendations, high-quality cpgs use evidence that is typically selected, appraised, and synthesized using systematic review methods⁵. In trying to craft recommendations for important clinical questions, the cpg process can be challenged when the quality of the evidence is poor or its quantity is scant³. Such was the case with a cpg produced by the Head and Neck Cancer Disease Site Group of the Program in Evidence-Based Care, the cancer guidelines program of the provincial cancer agency Cancer Care Ontario (cco). One cpg published in 2000 addressed the question of the effectiveness of the addition of chemotherapy to radiotherapy for head-and-neck cancer^6,7 based on a systematic review of the evidence between 1970 and 2000.

The cpg recommended that concomitant chemo-therapy with conventional radiotherapy (ccrt) be the treatment of choice for patients with locally advanced squamous cell carcinoma, noted that the ideal agent had yet to be identified, and proposed two regimens as reasonable options outside the setting of a clinical trial. However, from the outset, considerable concern was raised concerning the lack of quality data^8,9 about the incidence of major late toxicity^10–13, with a modest-at-best survival advantage of 8%¹⁴ against a backdrop of years of unsuccessful trials investigating the use of chemotherapeutic agents in patients with cancers of the head and neck. Despite those limitations, 60% of practitioners responding during the external review step of the guideline development process¹⁵ indicated that the guideline should be approved, and 75% indicated that they were likely or very likely to use it⁷.

The objective of the present study was to document adherence to the cpg recommendations by the cancer centres across Ontario that approved it and to examine changes in the evidence that were occurring at the time. The manuscript is divided into sections about the clinical story and the educational events near the time of the guideline. The first part uses a population-based study of patients from 2003–2004 to examine patients, treatments, and outcomes. The second part reviews the frequency and content of ccrt presentations at major conferences and in publications between 1997 and 2003. With respect to the clinical study, the years 2003 and 2004 were chosen specifically because uptake was known to vary, thus creating natural experiments that would allow for an assessment of outcomes despite selection bias^16–18.

2. THE GUIDELINE

The cpg⁷ was published in 2000 based on survival evidence from 20 randomized comparisons (1970 into 2000) of patients with locally advanced stage iii or iv squamous cell head-and-neck cancer identified in literature searches (medline, cancerlit, Healthstar, and Cochrane Library) and concluded that survival improved with ccrt compared with conventional radiotherapy alone. These were some of the key recommendations:

• Concomitant chemotherapy with conventional fractionated radiotherapy should be the treatment of choice for patients with advanced squamous cell head-and-neck cancer,

• At this time, the data are insufficient to recommend the use of concomitant chemotherapy with altered fractionation schedules.

• The choice of concomitant therapy should take into account the toxicity produced by various regimens and the convenience of treatment administration. An examination of individual trial results and toxicity profiles with the use of concomitant cisplatin-based treatment suggests that, given the circumstances, reasonable options outside a clinical trial include either single-agent daily cisplatin or carboplatin with conventional radiotherapy, or alternating split-course radiotherapy with cisplatin plus infusional 5-fluorouracil.

3. METHODS AND RESULTS

3.1. Part 1: The Clinical Study

The methods and results of a population-based study of all patients with squamous cell cancer of the oropharynx treated in Ontario during 2003–2004 have previously been published¹⁹. The study used information from the population-based Ontario Cancer Registry located at the Division of Cancer Care and Epidemiology (formerly the Radiation Oncology Research Unit) of the Queen’s Cancer Research Institute. All potential patients were identified based on International Classification of Diseases codes with a date of diagnosis between January 1, 2003, and December 31, 2004. The outpatient charts, with treatment records, were reviewed by experienced abstractors at the Queen’s Cancer Research Institute for data on patient characteristics, extent of disease, investigations, treatments, and outcomes. After exclusions, 571 cases of oropharyngeal carcinoma were retained.

Patient variables included the Adult Comorbidity Evaluation²⁰ and staging according to the TNM classification (6th edition)²¹. Patient status with respect to the human papilloma virus (hpv) was not available. We defined four initial treatment options of radiotherapy (rt), chemoradiotherapy (ccrt), surgery, or palliation, including treatment of residual disease (because that would be assumed to be part of the initially planned treatment). The surgery cohort included patients who received planned postoperative rt. The palliation cohort was identified based on a statement of intent in the chart for palliative or no treatment, regardless of the actual treatment administered. We recorded the cancer treatment centre at which the initial treatment was performed, and all centres were anonymized for analysis. Using the percentage of patients receiving ccrt at each centre, we compared outcomes at cancer treatment centres with similar protocols and stratified the centres based on a cut-point of 50% into those with “higher” and “lower” use of ccrt. Survival outcomes for the two groups were compared by the Kaplan–Meier method and Cox proportional hazards regression, controlling for age, sex, comorbidity, sub-site, T stage, and N stage.

Table i describes the patient population. Treatment selection was evident in the data: overall, the patients receiving ccrt were younger and healthier, with more advanced disease and a higher rate of base-of-tongue cancers. In comparing patients managed at the 9 treatment centres, we observed no statistically significant difference in age (p = 0.87), comorbidities (p = 0.39), sub-site (p = 0.65), N stage (p = 0.19), or TNM group (p = 0.22).

TABLE I.

Patient and tumour characteristics^a

Characteristic	Value
	(n)	(%)
Sex
Men	434	76.01
Women	137	23.99
Age
<50 Years	113	19.79
50–59 Years	180	31.52
60–69 Years	139	24.34
70–79 Years	116	20.32
≥80 Years	23	4.03
ace-27 score
0	224	39.23
1	180	31.52
2	106	18.56
3	61	10.68
Cancer site
Base of tongue	197	34.50
Faucial arch	345	60.42
Other	29	5.08
Histologic grade
i/iib	269	47.1
iii/ivc	164	28.7
Missing	138	24.2
Primary tumour
T1	130	22.77
T2	205	35.90
T3	100	17.51
T4	136	23.82
Lymph nodes
N0	130	22.77
N1	97	16.99
N2a	49	8.58
N2b	158	27.67
N2c	114	19.96
N3	23	4.03
Metastasis
M0/Mx	555	97.20
M1	16	2.80
TNM stage
i/ii	82	14.36
iii	101	17.69
iv	388	67.95

^aAdapted from Hall et al.¹⁹ with permission from Wiley–Blackwell.

^bWell or moderately differentiated.

^cPoorly differentiated or undifferentiated.

ace-27 = Adult Comorbidity Evaluation.

Table ii presents the initial treatments. More than 85% of the patients were treated with either rt or ccrt, but there was little agreement or consensus about treatment among the 9 centres (p < 0.0001). Although approximately 10% of patients at all centres received palliative treatment, the rate of ccrt varied from 78.26% to 30.14% (p < 0.0001). When patients receiving palliation and surgery were excluded, variation in the use of ccrt ranged from 82% to 39% (Figure 1).

TABLE II.

Initial treatments delivered at the nine centres^a

Treatment	Value
	(n)	(%)
Radiotherapy	235	41.2
Chemoradiotherapy	256	44.8
Surgery	14	2.5
Palliative or no treatment	66	11.5
TOTAL	571

^aAdapted from Hall et al.¹⁹ with permission from Wiley–Blackwell.

FIGURE 1.

Patients treated with radiotherapy (rt) or concurrent chemotherapy and radiotherapy (ccrt), by treatment centre (anonymized). Excludes palliative and primary surgery cohorts. Adapted from Hall et al.¹⁹ with permission from Wiley–Blackwell.

Table iii presents the six chemotherapy regimens used in the study population. In the centre comparison presented in Figure 2, the regimens are grouped into 4 classes of therapy (low-dose daily cisplatin or carboplatin, mid-dose weekly, high-dose every 3 weeks, and any combination with 5-fluorouracil). In 5 centres, one regimen (that aligned with the cpg) was used almost exclusively. In 2 centres, two primary regimens (one of which did not align with the cpg) were used. In 1 centre, all four chemotherapy options were tried during the 2 years of interest.

TABLE III.

Chemotherapy regimens used at the nine centres^a

Drug	Regimen	Value
		(n)	(%)
Cisplatin	Daily	82	32.0
	Weekly	24	9.4
	Every 3 weeks	110	42.9
Carboplatin	Daily	2	0.7
	Weekly	20	7.8
5fu combinations		18	7.0
TOTAL		256

^aAdapted from Hall et al.¹⁹ with permission from Wiley–Blackwell.

5fu = 5-fluorouracil.

FIGURE 2.

Chemotherapy regimens used, by treatment centre (anonymized). 5fu = 5-fluorouracil; q3 = every three.

In comparing the outcomes of all patients and of those treated at the centres with “higher” and “lower” use of ccrt, we observed no difference in 3-year overall survival (66.8% vs. 66%), 5-year overall survival (58.8% vs. 57.5%), 3-year disease-specific survival (72.3% vs. 72.5%), and 5-year disease-specific survival (68.9% vs. 67.7%) (Figure 3). The hazard ratio in the Cox model was 1.028 (95% confidence interval: 0.775 to 1.363; p = 0.85).

FIGURE 3.

Disease-specific survival of patients treated at centres with higher and lower use of concurrent chemotherapy and radiotherapy.

3.2. Part 2: Knowledge Transfer

The cpg process and the guideline itself provided information about randomized trials and meta-analyses about ccrt regimens up to 2000 for oncologists in Ontario, but other sources of information such as journal articles and presentations at major meetings were available to clinicians. A medline search spanning 1950 to the present combined the key word “head and neck neoplasms” with “combined modality therapy” (mesh) and each of the following phrases as text words: “concomitant or combined,” “radiotherapy,” “chemotherapy,” “radiochemotherapy,” and “chemoradiotherapy.” The reference lists of relevant studies were used to identify additional publications. To address the emergence of high-dose regimens, the foregoing terms were then combined with the search term “high dose.” We tabulated the number of publications describing only high-dose concomitant ccrt schedules between January 1997 and December 2003, because those publications might have influenced oncologists in Ontario.

Presentations at major meetings are another source of information for clinicians, especially if the presentations are given by opinion leaders. The American Society of Clinical Oncology Web site (http://www.asco.org) and the American Society for Radiation Oncology Web site (http://www.oncolink.org) were searched for relevant conference proceedings from meetings during 1997–2003 of head-and-neck poster or oral presentations describing high- and low-dose ccrt administered for head-and-neck cancer. Unfortunately, programs and abstracts for the annual meetings of the Canadian Association of Medical Oncologists and the Canadian Association of Radiation Oncology during the same period were not available online. For the present review, only podium presentations of high-dose ccrt regimens for head-and-neck cancer were included. However, we classified posters (American Society of Clinical Oncology head-and-neck and central nervous system group, 1998–1999; and head-and-neck cancer group, 2000–2003) as high-dose, low- or medium-dose, or any ccrt schedule.

Table iv shows the numbers of manuscripts, presentations, and posters that were located during the search. Over time, the number of published manuscripts about high-dose regimens remained constant. The number of posters on ccrt presented each year increased, and the number of posters on high-dose and low- or mid-dose regimens held steady. However, between 2000 and 2002, 14 podium presentations about high-dose regimens were delivered; few such presentations were delivered before 2000, and one was delivered in 2003.

TABLE IV.

Publications, podium presentations, and posters about chemoradiotherapy, high-dose chemoradiotherapy, and lower-dose chemotherapy for head-and-neck cancer, 1997 to 2003

Forum	Topic	Year
		1997	1998	1999	2000	2001	2002	2003
Publication	High-dose chemoradiotherapy	17	8	8	10	9	9	8
Podium presentationa	High-dose chemoradiotherapy (astro)	na	na	0/44	0/33	4/63	3/54	0/32
	High-dose chemoradiotherapy (asco)	na	0/23	0/31	2/32	3/67	2/56	1/58
Poster	Chemoradiotherapy (asco)	21/101	25/120	40/92	39/85	44/100	34/74	42/101
	High-dose chemoradiotherapy (asco)	12/101	13/120	14/92	13/85	18/100	19/74b	18/101c
	Low- or mid-dose chemoradiotherapy (asco)	3/101	4/120	15/92	13/85	13/100	7/74d	13/101e

^aDenominator indicates the total number of key podium presentations at the meeting.

^bIncludes 10 selected oral presentations.

^cIncludes 13 selected oral presentations.

^dIncludes 3 selected oral presentations.

^eIncludes 10 selected oral presentations.

astro = American Society for Radiation Oncology; asco = American Society of Clinical Oncology; na = information not available online.

4. DISCUSSION

Given the resources and time required to follow the cpg development cycle and to create a cpg¹⁵, adherence ought to be high. Typically, adherence in Ontario to the cgps created by cco is high²². However, the present study demonstrates that 2–4 years after approval of a guideline, adherence was mixed and much lower. The 9 Ontario cancer treatment centres used ccrt for 55% of oropharynx cancer patients overall (range: 30%–80%), and although not all patients would have been eligible for ccrt, we observed little difference in patient characteristics between centres. Furthermore, no agreement on the drug regimen was evident: 4 centres were following the guideline for most patients, 2 centres were following the guideline for some patients, and 3 centres were using regimens not supported by the guideline.

Cabana et al.²³ cited 7 reasons why physicians do not follow guidelines, including lack of awareness, lack of familiarity, lack of agreement with the evidence, lack of outcome expectancy, lack of self-efficacy, inertia of previous practice, and external barriers. The reason for the varied uptake of ccrt in Ontario might have been lack of agreement with the evidence (despite the practitioner feedback process), external barriers, or the inertia of previous practice. However, why were 3 centres exclusively using a high-dose regimen that was not recommended in the cpg? In Table iv, the data show continued interest in ccrt whether high- or low-dose, but the number of podium presentations on the new high-dose regimens at major meetings was increasing. It could be that some Ontario centres were influenced by those presentations more than by the cpg (noting that there had not been—and never has been—a clinical trial comparing the high- and low-dose regimens).

The development of the 5-6a cpg carefully followed the steps of the guideline development cycle¹⁵ used in 2000 and almost certainly would have satisfied all the criteria of the newer agree tools for creating or assessing guidelines (http://www.agreetrust.org/)²⁴. Why, then, did adherence vary?

First, for a new guideline to be accepted by groups and organizations, the process ought to address the culture and receptivity of the physicians and organizations the guideline is intended to influence. The new cgp, together with the momentum of the ccrt revolution of that time, was led by medical oncologists at a time when the head-and-neck teams, site groups, tumour boards, and management care conferences were traditionally staffed by radiation oncologists and surgeons. The role of medical oncology in head-and-neck cancer was limited to palliative care or research within approved clinical trials—because, aside from a U.S. Veterans Affairs trial²⁵, chemotherapy had not been showing much progress or promise. Although the medical oncology community was no doubt enthusiastic, acceptance by the radiation oncologists and head-and-neck surgeons almost certainly varied depending on institutional practices.

At the institution level, the political landscape can also be a factor in adherence to any new guideline, and interestingly, in 2000, cco had approved divestment of the provision of cancer care services at Ontario cancer treatment centres to the hospitals hosting the regional centres, a process that was completed in 2002. That change coincided with production of the guideline, and it is possible that the head-and-neck site groups at the host hospitals did not feel as responsible to cco-created cpgs once the line of responsibility had changed.

A second potential reason for varied adherence is acceptance of the opinions of the guideline development committee. The committee members ought to be representative of the potential target physicians and familiar with the process. The committee at that time (2000) consisted of volunteer members from some specialties at each cancer treatment centre whose opinions might or might not have reflected the opinions of other members of the local head-and-neck treatment team. The chair at the time was very knowledgeable about the cpg development cycle, but many members of the Head and Neck Cancer Disease Site Group were experiencing their first encounter with the complex and prolonged evidence-based guideline process. The newness of the process almost certainly slowed the process in a clinical environment that was continuing to evolve.

A third reason for varied adherence is the need to address major concerns or potential concerns. In 2000, ccrt was a very controversial treatment, and very real concern was being expressed by most head-and-neck teams about the validity of the data on late toxicity^8,9, especially permanent dysphagia^10–13. Based on a review of the clinical trials, one author⁹ had even stated that “no data were collected on the critical items of swallowing and airway function. These gaps, inconsistencies and variations in reporting practices indicate that published toxicity reports are not only frequently lacking key information, but also suggest that they likely contain significant underreporting, bias and errors.” It is not surprising that the enthusiasm of physicians and centres varied. Addressing concerns might also require the inclusion of competing evidence and, in this case, evidence about other non-chemotherapy options. Published trials had reported that, compared with conventional rt, hyperfractionated rt without chemotherapy was associated with better outcomes²⁶, and some rt departments in Ontario were particularly interested in that treatment option. The cpg meta-analysis had included two studies on hyperfractionation, but did not directly compare hyperfractionated regimens with ccrt regimens; and although the statement “the data for concomitant chemoradiotherapy are at least as good as the data for accelerated fractionation the body of evidence is higher” appears in the discussion, it does not appear in the recommendations. The use of hyperfractionated rt with an increased total dose was subsequently shown to provide an effect (8% difference with a 22% reduction in the hazard of death) identical to that achieved with ccrt^27,28.

A fourth potential reason for varying adherence is that a guideline might be an inappropriate approach for controversial topics with an evidence base that is rapidly expanding and evolving. By the time this particular cpg was published, the recommendations were clearly considered historical by some centres. A looser or more fluid statement—with some combination of evidence updates, ongoing review, or reassessment of the recommendations—would perhaps have provided more and better information to the head-and-neck oncologists of Ontario.

A final reason for varied uptake is the lack of a mechanism for feedback (Smith and Hillner²⁹). Given the controversy concerning the guideline, a built-in mechanism for direct feedback to or an audit on uptake after 1–2 years might have proved useful for the Head and Neck Cancer Disease Site Group, the Program in Evidence-Based Care, the guideline, and physicians.

To summarize, the observed variation in adherence was likely attributable to receptivity, the process, new evidence with new treatments, a lack of feedback, and concern over incomplete data.

Figure 3 further complicates the story: Our results suggest that the addition of chemotherapy to rt had a minimal effect on survival for oropharyngeal cancer patients in 2003–2004. The complete analysis has already been reported¹⁹. In our study, the univariable and multivariable analyses might be confounded by hpv status (although the effect of hpv status was not known at the time of the guideline); however, it is unlikely that the rate of hpv-positive cases varied in the populations of the “higher-use” and “lower-use” centres in our study. Our findings have two possible explanations: either the evidence about ccrt for oropharyngeal cancer was not, outside the biases of the clinical trials, generalizable to the real world of community practice in the 9 Ontario cancer treatment centres³⁰, or the evidentiary base concerning the addition of chemotherapy to rt for head-and-neck cancer was confounded by the evolving causative role of hpv. Clinical trials and population-based studies are under way to answer that fundamental question.

A review of the extensive literature on compliance with or adherence to cpgs is beyond the scope of this report; review articles by Smith and Hillner²⁹, Francke et al.³¹, Grimshaw and various colleagues^2,3,32, and Tan³³ are recommended. Three previous studies on compliance with cpgs specifically in head-and-neck oncology (including thyroid cancer) have been published.

Van Agthoven et al.³⁴ compared practice based on chart reviews before and after a guideline on the diagnosis, treatment, and follow-up of patients with laryngeal cancer in the Netherlands. Those authors reported that “in general, the guideline recommendations were properly complied with” and that compliance was related to the quality of the evidence. They noted that a plan for direct feedback on performance to clinicians and some form of accountability as proposed by Smith and Hillner²⁹ might have improved their results and, subsequently, the quality of the guideline. The cco 5-6a guideline would have been improved by some form of built-in feedback, especially given that treatments were evolving.

Another relevant adherence study was reported by Lewis et al.³⁵, based on a chart review of 107 new patients referred with recurrent or residual head-and-neck cancer to the MD Anderson Cancer Center. Using the U.S. National Comprehensive Cancer Network head-and-neck guidelines as a comparator, those authors found that 58.7% of the patients had received inadequate surgery; 10.9%, inadequate adjuvant therapy; and 4.4%, inadequate rt.

Panigrahi et al.³⁶ compared prior clinical practice with a guideline on the treatment of medullary thyroid cancer that had undergone many iterations over time. Based on data for 2033 patients from the U.S. Surveillance, Epidemiology and End Results database, those authors found that the extent of surgery was inappropriate in 41% of cases. They identified differences by geographic region, year of treatment, tumour size, age, and sex as contributing factors. Our study found regional differences in adherence, and the guideline might not have improved survival outcomes.

5. CONCLUSIONS

By 3 years after publication, the recommendation of the cco cpg on the use of ccrt over rt alone for patients with advanced squamous cell cancers of the head-and-neck was adopted for 55% of patients with oropharyngeal cancer across Ontario; however, the uptake by centre varied from 82% to 39%, possibly because of concerns about evidence quality, medical culture, delay in process, and evolving treatment options not addressed in the guideline. The drug regimen recommendation was modified by many centres (possibly on the basis of presentations by opinion leaders) in the period immediately after publication of the cpg, despite no comparative evidence being available. Clinical practice guidelines ought to include a more precise estimate of adherence, performance ought to be frequently monitored when the guideline is controversial, and a more fluid guideline format might be more appropriate in situations in which the evidence is evolving rapidly.

7. CONFLICT OF INTEREST DISCLOSURES

We have read and understood Current Oncology’s policy on disclosing conflicts of interest, and we declare that we have none.