Effect of periodic letters on evidence-based drug therapy on prescribing behaviour: a randomized trial

Abstract

Background

The effect of regular and expected printed educational materials on physician prescribing behaviour has not been studied. We sought to measure the impact of a series of evidence-based drug therapy letters mailed to physicians in British Columbia on prescribing to newly treated patients.

Methods

A paired, cluster randomized community design was used. The study population included 499 physicians from 24 local health areas in British Columbia. Local health areas were paired by number of physicians, and 1 of each pair was randomly selected and its physicians assigned to an intervention group or a control group. The intervention was 12 issues of an evidence-based series called Therapeutics Letter. Physicians in the control group (n = 241) received the letters 3–8 months after physicians in the intervention group (n = 258). The impact on prescribing to newly treated patients (defined as patients who had not previously made a claim for any medication from the class of drugs profiled in the letter) was analyzed using the drug claims database of BC Pharmacare, a publicly funded drug benefits program that covered all seniors and people receiving social assistance.

Results

The probability of prescribing a drug recommended in the Therapeutics Letter rather than another drug in the same class increased by 30% in the 3 months after the mailing of the letter relative to the preceding 3 months, adjusted for any before–after changes in the control group (relative risk 1.30; 95% confidence interval 1.13–1.52). No letter achieved statistical significance on its own. However, 11 of the 12 letters produced prescribing changes in the predicted direction such that the overall result was significant when their effect was combined.

Interpretation

The combined effect of an ongoing series of printed letters distributed from a credible and trusted source can have a clinically significant effect on prescribing to newly treated patients.

Printed letters are regarded as a relatively ineffective method of changing behaviour.^1,2 A systematic review in The Cochrane Library concluded that printed educational materials for health care professionals had negligible impact and were of uncertain clinical significance.³ These conclusions were not based on a quantitative meta-analysis, however, in part because of the poor quality of analysis and reporting of results and in part because the different outcomes could not be easily combined. Although changes in prescribing behaviour as a result of printed materials have tended to be small, printed materials have the potential to be a cost-effective method of education.^4,5 There is also a paucity of studies in Canada on initiatives to change prescribing behaviour.⁶

Between 1994 and 1997 the Therapeutics Initiative of the University of British Columbia conducted a randomized controlled trial with its Therapeutics Letter (www.ti.ubc.ca), which is distributed to most practising physicians in British Columbia. We aimed to quantify the publication's success in producing changes in prescribing behaviour. In 1998 we reported that 2 letters on the treatment of primary hypertension did not produce a statistically significant change in prescribing.⁷ Here we report findings from a combined randomized trial of the impact of 12 of the first 20 issues of the Therapeutics Letter, which were distributed between October 1994 and December 1997. This combined analysis was undertaken to measure the series' overall impact on prescribing to newly treated patients and to provide evidence of the effectiveness of these interventions in a Canadian setting.

Methods

Impact on prescribing behaviour was measured using the drug claims database of BC Pharmacare, a publicly funded drug benefit program. During the observation period, BC Pharmacare covered all people 65 years of age and older as well as people receiving social assistance. The database was updated weekly and contains a record of all prescriptions reimbursed in full or in part by the Crown, with the exception of drugs dispensed in acute care hospitals. Physician characteristics were compared using fee-for-service payment records and physician demographic information from the BC Medical Services Plan. Physician demographic data included sex, year of graduation and specialty. Payment records were used to determine patient demographics and estimate practice sizes. Arrival dates for people who immigrated to British Columbia and registered for the Medical Services Plan during the observation period were obtained from the Registration and Premium Billing Branch.

This research was approved by the University of British Columbia Behavioural Research Ethics Committee. Each copy of Therapeutics Letter stated that effects on prescribing would be evaluated and if physicians did not wish to have their prescribing records included in the analysis they could ask to be excluded. No requests to be excluded were received.

The intervention and control groups were created by grouping an approximate 10% sample of prescribing physicians in 24 local health areas in a paired, cluster randomized design into 12 pairs based on the number of physicians in each area. Local health areas are small geographic regions designated as health analysis regions because of the availability of accurate demographic data. One local health area in each pair was randomly selected and assigned (blindly by M.M. using the RAND function in Excel) to be in the control group. The possible influence of clustering on standard errors was explored using generalized estimating equations (GEEs), which adjusted for clustering at the levels of local health area, physician and letter. The GEEs were marginal regression models and used logistic link functions, binomial error distributions and unstructured correlation matrices. We also undertook an approach that required very few assumptions, a permutation analysis examining all 2 to the power of 12 randomizations (4096 permutations).

Therapeutics Letter is a concise and colourful 2- to 4-page bulletin with an easy-to-read question-and-answer format that is sent to over 6000 physicians in British Columbia. For this study the letters were sent to physicians in the intervention group (n = 258) at the same time as to physicians who were not part of the study. Physicians in the control group (n = 241) received the letters 3–8 months later. Letters #7 and #8 on antihypertensive therapy were a special case.⁷ They formed a 2-part series and were mailed 10 weeks apart. They were treated as a single intervention; the control group received Letter #7 8 weeks after the intervention group received Letter #8. This meant that the post-intervention follow-up period for these letters was 8 weeks. For the remaining letters the post-intervention follow-up period was 3 months. For all letters the pre-intervention observation period was 3 months.

With one exception, the source population comprised all residents of British Columbia aged 66 years of age or older between 1993 and 1998 who were living at home or in a continuing care institution and who had been eligible for Pharmacare coverage for at least 1 year (i.e., since their 65th birthday, when universal coverage for seniors began). From this source population, and for each drug class analyzed, a denominator of people was drawn who had made no claim in the preceding year for any drug in that drug class. This was the population at risk for a first prescription. The exception was the use of a younger source population to measure the impact of a letter on the management of asthma. This population comprised all people 60 years of age or younger who were eligible for Pharmacare coverage because they were receiving social income assistance. This younger group of people was chosen because it was less likely to be confounded by patients with chronic obstructive pulmonary disease.^8,9

A letter had to satisfy 2 conditions to be included in the combined analysis. First, the letter had to provide a clear message that could be predicted to result in a change in prescribing behaviour. Second, any predicted change in prescribing had to be of a nature that was measurable within the limitations of the databases available for the analysis. These criteria meant that letters focusing on topics such as dose titration, drugs available without prescription or new drugs not reimbursed by Pharmacare were excluded from the analysis.

Two of us (J.M.W. and C.R.D.), blinded to the prescribing data (except for our previously published results for the letters on the treatment of hypertension), independently reviewed the first 20 letters in the series to decide which ones met the inclusion criteria. The 8 letters that were excluded from the analysis and the reasons why they were excluded are shown in Table 1. One of us (J.M.W.) also selected from the included letters the key drug for which prescription numbers were most likely to change as a result of exposure in the letter. This drug was referred to as the analysis drug. For example, the letter describing menopausal hormone therapy stated, “There are no long-term RCTs evaluating hormone therapy. Until randomized trials are completed, long-term therapy decisions have to be made in the face of uncertainty.” The analysis drug was conjugated estrogens, and the prediction was that prescriptions for them would decrease. In letters where an increase in prescriptions of 1 drug would lead to a decrease in prescriptions of another (e.g., a decrease in prescriptions for ranitidine coinciding with an increase in prescriptions for cimetidine), only 1 prediction was allowed. In one letter 2 drug combinations — metronidazole–tetracycline and metronidazole–amoxicillin — were each treated as 1 drug, the prescriptions for which were predicted to increase.

Table 1

The primary outcome was incidence of newly treated patients, instead of total patients, because it was a more sensitive measure of physician opportunity and willingness to change prescribing habits. A patient's treatment status was determined by that patient's previous drug use.⁷ For each drug studied, patients were classified as being newly treated if no drugs for that condition had been dispensed to them for at least the previous 365 days.

Changes in prescribing were measured for each letter by counting the number of newly treated patients who received that letter's analysis drug before versus after the intervention. Half of the analysis drugs were predicted to show an increase in prescriptions to newly treated patients, and half were predicted to show a decrease. For the drugs predicted to show a decrease, we switched the temporal sequence of the comparison periods so that predicted decreases would not cancel predicted increases in a combined analysis.

The probability of being prescribed an analysis drug after the intervention was defined as the number of newly treated patients (a) divided by the potential patient population (P). Impact was measured by relative risk (RR), dividing the probability in the intervention group (a_i/P_i) by the probability in the control group (a_c/P_c). In a very large trial with randomization by physician, the 2 groups would have almost equal populations (P_i = P_c) and baseline risks (b_i/P_i=b_c/P_c), where b is the number of newly treated patients in the interval before the intervention. Then, RR = a_i/a_c. Because our trial was moderate in size and randomization was by region, some imbalance at baseline was expected. We adjusted for baseline risk by dividing the after-intervention risk (a/P) by the before-intervention risk (b/P). Assuming that the populations were the same before and after (online Appendix 1, available www.cmaj.ca/cgi/content/full/171/9/1057/DC1), or that any seasonal changes were proportionate in the intervention and control groups, the Ps cancel out and the adjusted relative risk is (a_i/b_i) / (a_c/b_c), with a null expected value of 1. For example, if preference for the analysis drug quadrupled in the intervention group and doubled in the control group, we would divide the intervention group relative risk of 4 by the control group relative risk of 2, and get an adjusted relative risk of 2, meaning the letter really only doubled the preference for the analysis drug. We used an intention-to-treat analysis with no further adjustments for physician mobility, vacation periods or other variations in patient contacts, assuming these were similar in both groups because of randomization. For the combined analysis, newly treated patients prescribed the analysis drugs were summed across all letters. We defined this weighted relative risk measure as the change in physician preference for the analysis drugs.⁷

Results

Characteristics of the intervention and control physicians in 1991 are displayed in Table 2. The physicians and their patient populations were well balanced for these characteristics. The predicted and actual prescribing changes for newly treated patients are shown in Table 3. A significant change was observed in the proportion of newly treated patients receiving the analysis drugs as first-line therapy. The preference for the analysis drugs was 1.3 times more in the predicted direction in the intervention group of physicians than in the control group (95% confidence interval [CI] 1.13–1.52).

Table 2

Table 3

All of the trends in prescribing to newly treated patients went in the direction predicted except in the case of the letter on finasteride for the treatment of benign prostatic hypertrophy. However, no single letter produced a result that was statistically significant on its own. The combined result was robust to various sensitivity analyses (online Appendix 2, available www.cmaj.ca/cgi/content/full/171/9/1057/DC1). When we remove the 2 letters on antihypertensive therapy, because these were included in a previous analysis, the effect remained significant (RR 1.26; 95% CI 1.11–1.42). Permutation analysis showed that the effect of the intervention was unlikely due to chance (p < 0.001) and was consistent with our GEE model.

Interpretation

The results of this randomized controlled trial demonstrate a significant change in prescribing to newly treated patients when the impact of a series of 12 letters was subjected to a combined analysis. The results in Table 3 reveal that each letter's impact did not achieve statistical significance when considered on its own. The lack of significant impact of single letters is consistent with published evidence.^{1,2,3,4,30,31,32} However, 11 of the 12 letters produced a change in prescribing behaviour in the predicted direction among the intervention group as compared with the control group, and, when combined, the overall impact of the 12 letters on prescribing behaviour was highly statistically significant.

The impact appeared to be greater for the 6 letters in which the predicted change was an increase in prescribing than for the 6 letters in which the predicted change was a decrease (Table 3). It may be easier to persuade physicians to prescribe than to persuade them to stop prescribing. This observation should be tested in a future trial.

At least 2 factors may have contributed to the observed significant impact on prescribing. First, counting only newly treated patients increased the sensitivity of demonstrating a change in prescribing.⁷ Second, combining the effects of a series of letters created a larger sample than would have been available for any 1 letter.

There are a number of limitations to our study. We included only patients who were 66 years and older or receiving social assistance, and the proportion of these patients with private insurance was unknown. One may presume that randomization balanced these factors in the intervention and control groups, but there are no data to confirm this. The design of the study was such that follow-up periods longer than 3 months could not be evaluated because the control group became exposed to the letter at that time. Therefore, the sustainability of the impact on prescribing beyond the first few months is unknown. Furthermore, it is possible that a small number (1%–2%) of recent immigrants who were actually continuing patients were misclassified as new patients. In addition, patients captured in the analysis of the letter on the treatment of asthma may also have been misclassified owing to the transient nature of patients on the social assistance plan. It is unlikely that such misclassifications would invalidate the results, since it is more difficult to change prescriptions for continuing patients than it is to change them for new patients, and because randomization would probably ensure that the intervention and control groups were equally affected.

GEEs were estimated to explore the possibility that clustering at the level of local health area, physician or letter produced confidence intervals that were too narrow. The confidence intervals produced using these models were virtually the same, and therefore the unadjusted data are reported. However, researchers wishing to apply our design must be mindful of its susceptibility to clustering, especially when effect estimates are modest.

We conclude that printed letters distributed as an ongoing series from a credible and trusted source can have a clinically significant impact on prescribing to newly treated patients. Further work needs to be done to determine the components of the message and the characteristics of the physicians that lead to changes in prescribing.