Impact of a provincial asthma guidelines continuing medical education project: The Ontario Asthma Plan of Action’s Provider Education in Asthma Care Project

Abstract

BACKGROUND:

The Ontario Ministry of Health and Long-Term Care funded the Ontario Lung Association to develop and implement a continuing medical education program to promote implementation of the Canadian asthma guidelines in primary care.

OBJECTIVES:

To determine baseline knowledge, preferred learning format, satisfaction with the program and reported impact on practice patterns.

METHODS:

A 3 h workshop was developed that combined didactic presentations and small group case discussions. Outcome measures included a workshop evaluation, baseline assessment of asthma management knowledge and three-month postreflective evaluations.

RESULTS:

One hundred thirty-seven workshops were delivered to 2783 primary care providers (1313 physicians, 1470 allied health) between September 2002 and March 2005. Of the 2133 participants, 1007 physicians and 1126 allied health professionals submitted workshop evaluations. Most (98%) of the attendees indicated they would recommend the workshop to a colleague. The majority preferred the combination of didactic lecture plus interactive case discussions. A subset of physicians provided consent to use these data for research (n=298 pediatric and 288 adult needs assessments; n=349 postreflective evaluations). Important needs identified included appropriate medication for chronic asthma and development of written action plans. On the postreflective evaluations, 88.7% remained very satisfied, 95.5% reported increased confidence, 91.9% reported an influence on practice and 67.2% reported using a written action plan.

CONCLUSIONS:

This continuing medical education program addresses identified needs of primary care providers. Participants reported improvements in asthma care, including prescribing practices, use of spirometry and written action plans. Similar programs should be considered as part of multifaceted asthma guidelines dissemination and implementation initiatives in other provinces and nationally.

Successful translation of clinical practice guidelines into practice is contingent upon effective dissemination and implementation strategies. Despite the development, publication and regular revision of Canadian asthma management guidelines (1–8) over the past 15 years, their uptake has been suboptimal (9). In the late 1990s, only 38% of family and general practitioners surveyed were knowledgeable about Canadian asthma guidelines (10). More recently, the Ontario Asthma Regional Variation Study documented large variations in emergency department (ED) visit rates and hospitalizations for asthma (11), as well as suboptimal adherence to asthma guidelines in both primary (12) and acute care settings (13). Furthermore, in a 2001 survey, the majority of Canadians with asthma had suboptimal asthma control according to Canadian Asthma Consensus Guidelines (CACG) criteria (14).

The main challenge of guideline implementation appears to be the translation of new knowledge into changes in attitudes and practices of health care providers. Traditional methods of guidelines dissemination, such as peer-reviewed publications, mailed print materials and didactic continuing medical education (CME) programs, are often of limited success (15). Interactive sessions led by local opinion leaders and multifaceted or sequenced CME activities can lead to changes in practice and patient outcomes. Interventions that enable and reinforce behaviour change such as academic detailing, audit and feedback, and reminder systems that reinforce behaviour change are most effective but are typically labour intensive and costly. For example, a paper stamp checklist tool was recently shown to increase asthma management knowledge and reduce ED visits, particularly when combined with a CME event and incentive (16). A recent Canadian Thoracic Society-sponsored workshop identified the need for a multifaceted evidence-based national asthma (and chronic obstructive pulmonary disease [COPD]) guideline implementation strategy (9). Funding for a national strategy and its rigorous evaluation were deemed high priorities, and physicians and allied health care professionals (AH) were identified as the key ‘targets’ of such an initiative.

In 2002, the Ontario Ministry of Health and Long-Term Care (MoHLTC) invested $4 million in annual funding for a provincial Asthma Plan of Action (APA) to support the implementation of asthma guidelines in Ontario (17). The goal of the APA is to reduce asthma morbidity, mortality and costs by focusing on three areas: prevention and health promotion, management and research including surveillance. As part of the APA, The Ontario Thoracic Society of the Ontario Lung Association (OLA) was funded to develop a CME program on pediatric and adult asthma and deliver it to approximately 3000 health care providers over a three-year period. An evaluation was conducted to determine baseline asthma knowledge of primary asthma care providers, and to determine and compare the preferred learning format, satisfaction with the program and reported impact on participants’ practice patterns.

METHODS

Asthma CME development and format

The CME program was developed based upon review of the literature, with input from a steering committee consisting of pediatric and adult respirologists, a family physician, an OLA Asthma Program Manager and certified asthma educators. The content was based upon the CACG and updates (1–8). The program combined expert content, process and facilitation in a 3 h multidisciplinary evening workshop format.

Workshops were comprised of two 30 min didactic presentations by specialists on adult and pediatric asthma, each followed by 1 h of interactive small-group case discussions led by trained primary care facilitators. Standardized slide kits were developed based upon the CACG, and were updated in October 2004 to reflect the new CACG and current literature. The case discussions covered asthma diagnosis in children and adults including objective assessment of airflow rates and examples of spirometry, differential diagnoses, asthma control and severity, environmental control/triggers, pharmacological management based upon severity and development of a written action plan. A facilitator’s manual was developed for the case discussions. Specialist presenters were recruited largely from OTS membership, while primary care practitioners experienced in group facilitation were sought locally. Where present, existing asthma clinics and education centre staff often participated and were able to summarize local resources. The CME program received accreditation by the College of Family Physicians of Ontario for three Maintenance of Proficiency (Mainpro^®) ‘C’ credits.

Participants also received a package including a copy of the 2003 CACG (5) and, once published, the 2005 update (8), CACG flow sheet and action plans available on the Canadian Asthma Guidelines Web site as downloads (www.asthmaguide-lines.com/downloads.html), CACG pocket card (which contains the guideline continuum diagram, control parameters and inhaled corticosteroids dose equivalency table), OLA Asthma Action educational materials (18) for patients including the Asthma Action Handbook and Asthma Action fact sheets, Asthma in Children Handbook and two children’s books (Call me Brave Boy [19] and Asthma Active [20]).

Asthma CME delivery

Approximately 55 workshops per year were planned in target Ontario communities, identified by the MoHLTC, between September 2002 and March 2005. Primary care physicians and AH received mailed invitations to attend the asthma continuing education workshop. Mailing lists were obtained through a professional targeted marketing company. Workshops were also directly marketed through various health care facilities. There was no registration fee. The workshop included dinner and a coffee break.

Evaluation

There were three sources of data for evaluation: baseline survey of asthma knowledge and skills, workshop evaluation and three-month postreflective evaluation. A baseline survey of asthma knowledge and skills was mailed to all registrants and could be returned to the OLA up until the evening of the CME, before the program began (Appendix). It assessed baseline knowledge of asthma control parameters, differential diagnoses in adults and children, nonpharmacological and pharmacological management, and action plan use. All participants were encouraged to submit a workshop evaluation at the conclusion of the evening. Participants were asked to rate on a scale of 1 (strongly disagree) to 5 (strongly agree) whether the didactic presentations of adult and pediatric asthma, and the subsequent case discussions, were relevant to family medicine, met stated and personal objectives, were presented clearly and whether the facilitator was helpful. The postreflective evaluation was mailed three months later to physicians who completed the baseline survey and assessed satisfaction with the workshop, confidence in asthma management, helpful learning points, impact on clinical practice, barriers to practice change and patient perception of change. Physician registrants were eligible to receive Maintenance of Proficiency ‘C’ CME credits if they completed the baseline survey of asthma knowledge and skills before the CME and the three-month postreflective evaluation.

Informed consent was not required for the use of the workshop data. Consent was sought from participants beginning May 2003 to use the baseline survey and postreflective data for research. Consent was not requested in the early workshops because it had not been the intent, initially, to use these data for research purposes. Ethics approval for the study was obtained from the McMaster University Medical Centre Research Ethics Board (Hamilton, Ontario).

Statistical analysis

Data were analyzed using SPSS (Version 12.0.1, SPSS Inc, USA, 2003). Categorical data were summarized in frequency tables, while ordinal data were described as frequencies and as mean ± SD. Comparisons between groups were based on χ² tests and independent samples t tests.

RESULTS

Workshops and participants

A total of 2783 health care professionals attended 137 workshops between September 2002 and March 2005. The characteristics of participants, proportion completing workshop evaluations and proportion of physicians consenting to use of their baseline survey and postreflective evaluation for the study are outlined in Table 1.

TABLE 1.

Participants, evaluations completed, and consent information

	n (%)
Workshop participants	2783
Physicians	1313 (47.2)
Allied Health	1470 (52.8)
Workshop evaluation completed	2133 (76.6)
Of 2133 evaluations
Physicians	1007 (47.2)
Nurse practitioners	73 (3.4)
Respiratory therapists	230 (10.8)
Pharmacists	338 (15.9)
Registered nurses	250 (11.7)
Physiotherapists	111 (5.2)
Other/Not documented	124 (5.8)
Of 1313 physicians
Pediatric needs assessment completed	506 (38.5)
Consent requested	311 (23.7)
Consent provided*	298 (22.7)
Adult needs assessment completed	500 (38.1)
Consent requested	302 (23.0)
Consent provided*	288 (21.9)
Postreflective evaluation completed	547 (41.7)
Consent requested	349 (26.6)
Consent provided*	335 (25.5)

^*The large differences between respondents and those who consented were due to the fact that consent was not requested in the early workshops. Over 95% of those asked provided consent

Workshop evaluations

All responses had mean values in excess of 4.3, although the AH group consistently provided lower ratings than physicians (P<0.05). Mean responses from physicians ranged from a low of 4.6±0.7 for the adult presentation meeting their personal objectives, to a high of 4.7±0.5 for relevance to family medicine. Mean responses from AH ranged from a low of 4.3±0.8 for the applicability of case discussions to family medicine, to a high of 4.6±0.7 for clarity of presentation.

The vast majority of physicians (98.2%) and AH (98.6%) indicated that they would recommend the workshop to a colleague. Most physicians (78.5%) and AH (80.1%) preferred a combination of expert content and process/facilitation, rather than expert content alone (13.7% and 15.5%, respectively) or process/facilitation alone (8.3% and 4.4%, respectively). However, there were significant differences in preferred learning format. While 59.2% of the AH sample preferred both the lecture and case discussion be multidisciplinary, only 42.5% of the physicians preferred this format (P<0.001). Of the physicians sampled, 20.3% preferred both be separate and 16.0% preferred at least the case discussion be separate.

Strong features of the program were identified by 1664 (78.0%) participants, and weaknesses were identified by less than one-half of the participants (859 or 40.3%). Details of the strengths and weaknesses are outlined in Table 2. Many also identified the Lung Association sponsorship, MoHLTC funding and absence of pharmaceutical sponsorship bias as strong points.

TABLE 2.

Strengths and weaknesses of the program

	Physicians n=1007 n (%)	Allied health n=1126 n (%)	P*
Strong features†
Strong feature(s) provided:	824 (81.8)	840 (74.6)	<0.001
Case discussion	246 (24.4)	278 (24.7)	0.15
Presenters/experts	250 (24.8)	328 (29.1)	<0.001
Educational format	200 (19.9)	198 (17.6)	0.75
Content	332 (33.0)	337 (29.9)	0.94
Interaction with peers/experts	185 (18.4)	95 (8.4)	<0.001
Handouts/resource materials	95 (9.4)	119 (10.6)	0.11
Areas of weakness
Area(s) of weakness provided	385 (38.2)	474 (42.1)	0.39
Presenters/experts	20 (2.0)	37 (3.3)	0.13
Education format	152 (15.1)	230 (20.4)	0.002
Content	129 (12.8)	152 (13.5)	0.9
Handouts/resource material	24 (2.4)	26 (2.3)	0.7
Logistics‡	105 (10.4)	117 (10.4)	0.67
Other	27 (2.7)	55 (4.9)	0.02

^*Based on Pearson χ² tests, on the basis of those providing a response.

^†Strong features mentioned by less than 5% included written action plan, spirometry/pulmonary function test review, logistics and other;

^‡Logistics included factors such as food, room temperature and acoustics

Anticipated changes in practices were reported by 1409 participants, and are provided in Figure 1. Medication/prescribing changes were identified by a total of 570 (26.7%) participants, and included changes in inhaled corticosteroids (13.1%), prednisone (1.7%), long-acting beta₂-agonists (5.3%), leukotriene antagonists (2.3%), combination drugs (1.3%), short-acting beta₂-agonists (1.7%) and medication devices/spacers (2.0%).

Figure 1).

Anticipated changes in practice identified by physicians (n=760) and allied health (n=649). With the exception of diagnosis (P=0.66), all differences between physician and allied health responses are statistically significant (monitoring P=0.024, and P<0.001 for all others). Monitoring Monitor asthma control (symptoms and/or lung function) in follow-up; Nonpharma Nonpharmacological; PEFR Peak expiratory flow rate

Baseline survey of asthma knowledge and skills

The baseline survey was completed by over 38% of the physicians attending the workshop. Approximately one-half were asked for consent (Table 1), which was provided by over 95% of those asked. Of the 258 (88.7%) who responded to a question regarding asthma control in a sample adult case, all respondents correctly identified poor control, and 81.8% (92.2% of those who responded) identified at least one asthma control parameter, with 16.6% citing one, 22.3% citing two, 28.5% citing three and the rest citing four or more parameters. In addition, 95.5% could correctly identify at least one way to differentiate between asthma and COPD in adults. For the pediatric sample case, 98.0% listed at least one correct differential diagnosis of asthma.

A long-term management plan was completed by 99.3% of respondents for the adult case and 94.0% of respondents for the pediatric case. Nonpharmacological interventions were identified by 284 (97.6%) and 249 (82.5%) for the adult and pediatric case respectively, while pharmacological interventions were identified by 277 (95.2%) and 248 (82.1%). The pharmacological interventions were incorrect for 27 of 277 (9.7%) adult and 21 of 248 (8.5%) pediatric responses (based on CACG). An asthma action plan for the adult case was prepared by 242 (83.2%) of respondents. Of these, 154 (63.6%) used the written action plan form provided, but 46 of the 154 (29.9%) did not fill it out adequately.

Postreflective evaluation

Three months following the workshop, a postreflective evaluation was completed by 547 (41.7%) physicians. Consent was requested of 349, and provided by 335 (Table 1). Overall satisfaction with the workshop three months later was very high, with 66.0% indicating that they were very satisfied and an additional 22.7% indicating that they were extremely satisfied. In addition, 95.5% indicated that their confidence level in the management of asthma had increased.

Helpful learning points as well as the influence on their clinical practice are illustrated in Figure 2. ‘Medication/prescribing changes’ was the most commonly cited helpful learning point and influence on clinical practice. Specific medications identified include (percentages represent responses to helpful learning points, then influence on practice): inhaled corticosteroids (31.9%; 17.9%), prednisone (4.5%; 0.9%), long-acting (20.3%; 13.4%), beta₂-agonists antileukotriene (6.9%; 4.8%), combination inhaled corticosteroid/long-acting beta₂-agonist drugs (4.5%; 5.1%), short-acting beta₂-agonists (4.8%; 2.7%) and medication devices/spacers (3.0%; 1.5%).

Figure 2).

Postreflective evaluation by physicians (n=335): helpful learning points identified and influence on clinical practice. Monitoring Monitor asthma control (symptoms and/or lung function) in follow-up; Nonpharma Nonpharmacological; PEFR Peak expiratory flow rate

Notably, 67.2% indicated they had incorporated a written action plan into their practice. Those who had not done so cited reasons such as no time (8.1%), use of a verbal action plan (4.5%) and logistics (4.8%). In addition, 42.1% indicated they had encountered barriers that prevented them from incorporating positive practice strategies, such as access and time (16.1%), cost (10.4%), patient compliance (12.2%) and diagnostic tools (3.0%). However, 29.6% indicated that they had found solutions to these barriers, with solutions identified for problems associated with time (18.5%), cost (4.5%), patient compliance (6.3%) and diagnostic tools (2.1%).

Many (43.6%) indicated they believed their patients had noticed changes in their practices, including increased patient/family involvement (17.3%), more time spent on education (8.7%), prescription changes (8.4%), increased use of objective measures (4.8%) and increased patient control (10.1%). In addition, close to two-thirds (61.5%) had discussed changes to their clinical practice with their colleagues.

A large majority (83.3%) indicated that the educational materials had been helpful, particularly the action plan (39.1%), Asthma Action patient handouts (32.2%), CACG summary (21.5%) and the treatment flow chart (11.9%). However, only 24.5% indicated that they had ordered materials from the Asthma Action Program; reasons cited by others for not ordering materials included already having a current supply (26.6%), lack of time (7.8%) and lack of (perceived) need (14.3%).

DISCUSSION

This CME program represents a unique Canadian asthma guidelines implementation initiative, which was part of a multifaceted provincial asthma strategy. Novel aspects include its funding source (provincial government), agency (charitable organization), professional society endorsement, multidisciplinary target audience and scope (adult and pediatric asthma). It was delivered to almost 3000 health care providers in Ontario between 2002 and 2005, making it one of the largest asthma CME initiatives in Canada. Although physicians and AH differed in preferred learning format, the majority of both groups anticipated it would lead to changes in practice. Participants demonstrated needs for asthma knowledge and skills, in particular regarding medication plans for chronic management and use of action plans for acute management. Physicians reported improved confidence in asthma management and important changes in practice, notably medication/prescribing changes, nonpharmacological management, use of spirometry for diagnosis and/or monitoring, and incorporation of a written action plan into their practice. The Health Care Provider Continuing Education in Asthma Care Project may lead to improved asthma management and outcomes by promoting awareness, acceptance and adherence with the CACG in primary care.

Several inherent limitations in the evaluation of this CME program must be acknowledged and considered when interpreting the findings. First, the uncontrolled design precludes one from attributing improvement in outcomes to the intervention alone. Second, the self-reported anticipated and actual changes in practice may have been subject to respondent and expectation bias. Third, we do not know if the participants were representative of all Ontario health care providers (ie, selection bias) or if the subset of physicians who completed the postreflective evaluations were representative of all participants. Fourth, the impact on actual asthma control of patients was not evaluated. Consideration was given to examining hospitalization, ED visit rates and pharmacy prescriptions for asthma medication. This analysis was deferred due to concerns regarding lack of power in large communities and confounding by cointerventions such as other APA projects including the Primary Care Asthma Pilot Project, School Asthma Project and ED Asthma Care Map Project. These may be more appropriate outcome measures for the overall APA. We are exploring linking chart audit data from the Primary Care Asthma Pilot Project and plan to survey baseline action plan use in future evaluations. Despite these limitations, the overwhelmingly positive workshop and postreflective evaluations merit discussion, and may guide future asthma guideline implementation initiatives.

The combination of presentations by expert opinion leaders with case-based discussions was the educational format preferred by approximately 80% of both physicians and AH. Perhaps not surprisingly, AH favoured the multidisciplinary structure of the workshop more than physicians (approximately 60% versus 40%), but a substantial proportion of both groups would have preferred one or both components to be ‘separate’ (ie, discipline-specific). Although labour intensive and therefore costly (approximately $1.5 million for three years), our results are in keeping with published knowledge translation literature that interactive workshops combined with didactic sessions can result in moderately large changes in professional practice (15,21–24). Both groups valued the general content of the workshop and the case discussions. Physicians more commonly reported the interaction with peers as a strength, while AH valued the expert presentations slightly more than the physicians. These preferences should be considered in future programming. Lastly, the perceived lack of bias from industry is a noteworthy strength of this program.

Knowledge translation theory stipulates that effective CME interventions should be tailored to the needs of participants (23). The baseline survey was designed to assess participant’s baseline knowledge of asthma care and guidelines, reflecting actual rather than their own perceived learning needs. Before the workshop, participants were able to correctly identify poor asthma control and list between one and three of the CACG control parameters, although only one-third listed four or more parameters and 16.6% listed only one parameter. Most were able to design appropriate long-term asthma management plans; these were more accurate in adult than the pediatric cases 12% to 15 % of the time. This suggests that the nuances of pediatric asthma management and the new Canadian pediatric asthma guidelines (7) should be emphasized in future workshops. One of the main baseline knowledge/skill deficits identified related to creation of written action plans. Many respondents did not attempt the question (17%), and of those who did and used the action plan form provided (64%), 30% did not do so correctly. The interactive discussions focused on these topics and afforded the opportunity to practice skills (create an action plan for a case). In response to numerous requests, spirometry is being emphasized in future workshops.

Self-reported anticipated and actual changes in practice following the workshop must be interpreted with caution, due to the limitations in design already discussed. Nonetheless, improved confidence in asthma management is an important outcome of this study. Reported refinements in medication management included an improved understanding of appropriate dosing of inhaled corticosteroids as monotherapy in children and indications for add-on therapy in children and adults. Perhaps the most remarkable end result relates to reported action plan use. Although use of a written action plan before the workshop was not specifically recorded, 24% of physicians indicated they planned to do so after the workshop. Twenty-four per cent also cited it as a specific change they had made three months later, and 67% of physicians reported actually using a written action plan as part of their practice three months later. Respondents did not indicate whether they were preparing action plans themselves or referring clients to an asthma educator.

Due to its success, the Health Care Provider Asthma Education Project is being sustained as an OLA program with ongoing funding from Ontario’s MoHLTC. In response to actual and perceived learning needs of participants, standardized sessions on requested topics (eg, differentiating asthma from COPD) and tools to supplement the workshops (eg, CD ROMs on spirometry and devices) are being developed. Linkages are being established with other Asthma Strategy initiatives such as the Occupational Asthma Provider Education Project and the Primary Care Asthma Program. Future program evaluation should include direct assessment of its impact on asthma control and practice patterns (such as actual use of written action plans), learning needs and postreflective evaluations from AH and a cost-benefit analysis.

In summary, the OLA’s Health Care Provider Continuing Education in Asthma Care Project is an evidence-based CME program designed to support the dissemination and implementation of the CACG in Ontario, as part of Ontario’s APA. The workshops, which combined specialist presenters and peer-facilitation of case-based discussions, were extremely highly rated by both physician and AH attendees, indicating multi-disciplinary CMEs can be successful even though some do prefer discipline-specific formats. Key asthma management and skills needs of participants, including pharmacological management, particularly of pediatric asthma, utility of spirometry and development of written action plans, were addressed. The reported influence of this workshop on clinical practice suggests that it improves awareness and acceptance of as well as adherence with guidelines in primary care. Similar programs should be considered as part of multifaceted asthma guidelines dissemination and implementation initiatives in other provinces and nationally.

APPENDIX.