Workshops to disseminate the Canadian Thoracic Society guidelines for chronic obstructive pulmonary disease to health care professionals in Ontario: Impact on knowledge, perceived health care practices and participant satisfaction

Abstract

BACKGROUND:

The Canadian Thoracic Society (CTS) has developed a clinical practice guideline (CPG) regarding the management of patients with chronic obstructive pulmonary disease (COPD). Implementation of this CPG in the primary care setting requires an effective dissemination strategy.

OBJECTIVES:

To examine the change in knowledge, participant satisfaction and perceived changes in clinical practice among health care professionals working in the primary care setting following attendance at a workshop to disseminate the CTS CPG for COPD.

METHODS:

A 2.5 h workshop was conducted in three community health sites within Ontario. Each workshop comprised a didactic presentation and interactive case study discussions. Before, and one month following the workshop, a structured knowledge assessment questionnaire was administered. A structured satisfaction questionnaire and evaluative form that examined the impact of the workshop on the clinical management of COPD patients were administered immediately and three months following completion of the workshop, respectively.

RESULTS:

Sixty-nine participants attended the workshop. The mean score for the structured knowledge assessment questionnaire increased from 8.5±2.7 to 10.6±2.0 following the workshop (P=0.008). Eighty-nine per cent and 96% of participants indicated that they would recommend the workshop to a colleague and had greater confidence in their management of COPD patients, respectively. Following attendance of the workshop, 73%, 69% and 46% described increased patient education, patient monitoring and the use of objective testing in clinical practice, respectively.

CONCLUSIONS:

Workshop attendance was associated with high levels of satisfaction and important self-reported changes in clinical practice, which may reflect improved knowledge of the CTS CPG for COPD.

In 2003, the Canadian Thoracic Society (CTS) developed a clinical practice guideline (CPG) pertaining to the management of patients with chronic obstructive pulmonary disease (COPD) (1). This guideline was updated in 2007 and contains clinical recommendations derived from the best available evidence (2). Given the increased complexity of patient care, coupled with the large number of research studies available to guide clinical practice, CPGs have the potential to integrate knowledge and optimize the quality of health care for individuals with specific conditions (3,4).

Primary health care practitioners provide the majority of clinical care for patients with COPD (5). The importance of early diagnosis and initiation of appropriate management of COPD patients in the primary care setting was highlighted by the publication of a separate document by the CTS (6) that summarized their recommendations and implications of the CPG for family doctors. Specifically, early diagnosis leading to effective smoking cessation and pharmacotherapy will slow the rate of decline in lung function and improve the functional status of COPD patients, respectively (2,7). Furthermore, the accurate diagnosis of COPD, combined with appropriate pharmacological and nonpharmacological therapies, may reduce the number of exacerbations treated in primary care (8). Adoption of the CPG in the primary care setting could have a substantial impact on large numbers of patients with COPD (5).

Successful implementation of the CPG in the primary care setting requires an effective dissemination strategy. However, between 1990 and 1999, as many as 15 CPGs regarding the management of COPD were published, of which only two included a dissemination strategy (9). Approaches that include education and interaction with an influential peer are most likely to be effective in promoting guideline acceptance (10). Therefore, we hypothesized that a workshop that incorporated several teaching approaches would facilitate acceptance of the CTS CPG for COPD in primary practice. The objective of the present study was to examine the change in knowledge, participant satisfaction and the perceived changes in clinical practice among health care professionals who work in the primary care setting, following attendance at a workshop designed to disseminate the CTS CPG for COPD.

METHODS

Approval for the present study was obtained from the Research Ethics Committee at the University of Toronto, Toronto, Ontario. Participants provided written consent to use the results of the structured knowledge assessment questionnaire and the evaluation form (see Outcome measures) at the time of their workshop attendance. However, consent was not sought at this time to use data collected from the structured satisfaction questionnaire because these data were not intitially intended to be used for research. Approval was subsequently obtained from the Research Ethics Committee at the University of Toronto to use these data for research purposes in November 2008.

Workshop: Location and content

Between April and May, 2006, a single 2.5 h workshop was conducted at three community health sites within Ontario: Toronto, Sault Ste Marie and Lanark. These locations were selected by the Ministry of Health and Long-Term Care (MOHLTC) based on former experience in working with the health care professionals associated with each site.

The format of the workshop was adapted from the MOHLTC strategy for dissemination of the CTS guidelines for the management of asthma (11). Specifically, each workshop comprised a didactic 30 min Power Point presentation by a respirologist, followed by three 30 min case study discussions with interaction between participants facilitated by a family physician. The didactic presentation described the CTS guidelines (1). In particular, the presentation emphasized the importance of spirometry in the early diagnosis of COPD, patient and family education regarding the disease and the role of nonphysician health care professionals in optimizing the functional status of patients with COPD. The case discussions reviewed diagnostic tests for COPD including those that pertain to making a differential diagnosis of asthma, in addition to management approaches for patients with mild, moderate and severe COPD. Case studies included specific clinical questions and, to guide discussion, workshop facilitators were provided with answers that were endorsed by a respirologist. Both a facilitator and respirologist were available throughout the workshop to address questions posed by the participants.

Participants were provided with written materials regarding the current CTS guidelines, a COPD action plan, a COPD treatment algorithm and a listing of smoking cessation programs, as well as resources that included the addresses of appropriate Internet Web sites such as <www.copdguidelines.ca> and <www.discoverycampus.com>. These resources served as reminders of the content of the program. No follow-up workshop was scheduled. The content of the workshop was reviewed by three respirologists and one family doctor.

The workshop was reviewed and accredited by the Ontario College of Family Physicians, which allowed physicians who attended the workshop and completed all evaluative components, to earn 2.5 Maintenance of Proficiency (Mainpro C) credits of continuing medical education (CME).

Recruitment

Workshops were offered to all health care providers associated with the participating community health centres in Toronto, Sault Ste Marie and Lanark. A contact person at each target site was asked to circulate flyers and encourage all staff to attend. Flyers were also distributed to the local respiratory educators listed with the Ontario Lung Association.

Attendance was optimized by the first author sending a minimum of three reminder e-mails and making telephone contact with the designated contact person at each site, providing the workshop outside usual business hours (ie, 17:30 to 20:30) and inviting all participants to attend a dinner before the workshop.

Outcome measures

Knowledge assessment:

A structured knowledge assessment questionnaire (Appendix 1) was administered before commencing the workshop (baseline) and one month following its completion (postworkshop). The questionnaire was developed by a respirologist and respiratory therapist with expertise in the clinical management of patients with COPD. It comprised nine items, yielding a maximum possible score of 13 points. Questions pertained to the clinical features, risk factors, diagnostic tests and appropriate management for a patient with mild COPD, presented as a case study during the workshop. Higher scores indicated greater knowledge. The questionnaire was pilot-tested on four respiratory therapists and modified to optimize its clarity and face validity. The final version took approximately 10 min to 15 min to complete.

Baseline questionnaires were collected at the beginning of the workshop. The postworkshop questionnaires were mailed one month later, together with the certificate of attendance. Participants were requested to return their completed postwork-shop questionnaires to the first author either by mail or fax.

Satisfaction with the workshop:

Immediately following completion of the workshop, participants were asked to complete a structured satisfaction questionnaire. This instrument consisted of three sections, each comprising four statements that pertained to aspects of the didactic presentation, the case discussions or the format of the workshop. Participants were asked to rate their agreement with each statement using a five-point Likert scale in which a score of 1 indicated strong disagreement and 5 indicated strong agreement, yielding a maximum score of 60 points. Participants were also invited to describe the strengths and weaknesses of the workshop, and asked whether they would recommend it to a colleague.

Clinical impact of the workshop:

Three months following completion of the workshop, participants were mailed an evaluative form that sought to examine its impact on their clinical management of patients with COPD. Questions pertained to the most useful learning aspects of the workshop, level of confidence in managing COPD patients, perceived changes in clinical practice following attendance at the workshop, use of spirometry, barriers to changing the clinical management of patients with COPD and suggestions for future workshops. Participants were requested to return their completed forms to the first author either by mail or fax. Completion of this form was required to earn the CME credits.

Analyses:

Analyses were performed using SPSS version 15.0 (SPSS, USA). Data were normally distributed. The difference in the baseline and postworkshop structured knowledge assessment questionnaire scores were assessed using paired t tests. Satisfaction with the workshop was compared between participants grouped according to their profession using one-way ANOVA. The responses to the satisfaction questionnaire and evaluative form are presented as frequency histograms with all data expressed as mean ± SD. P<0.05 was considered to be statistically significant.

An earlier report (12) demonstrated an 18% increase in the total score of a structured knowledge assessment questionnaire following a targeted education intervention. Therefore, prospective power calculations based on detecting a change in the structured knowledge assessment questionnaire of 2.3 points (18% of the total score) with an anticipated SD of this change of 3.4 points using a two-tailed paired t test (80% power; alpha = 0.05) was performed. A sample size of 20 subjects was required.

RESULTS

The Toronto, Sault Ste Marie and Lanark workshops were attended by 19, 30 and 20 participants, respectively. Considering all sites together, of the 69 participants, 22 (32%) were registered nurses, 15 (22%) were physicians, 14 (20%) were nurse practitioners and 18 (26%) were from other allied health professions.

Knowledge assessment

A total of 38 (55%) and 17 (25%) participants completed the baseline knowledge assessment questionnaire, and both the baseline or postworkshop knowledge assessment questionnaires, respectively. Of those who completed both the baseline and postworkshop knowledge assessment questionnaire, three (18%) were nurse practitioners, three (18%) were registered nurses, seven (41%) were physicians and four were respiratory therapists (23%). Considering all professions together, the knowledge assessment score increased from 8.5±2.7 to 10.6±2.0 following attendance of the workshop (P=0.008).

Satisfaction with the workshop

A structured satisfaction questionnaire was collected from 55 of the participants (80%), with representation from nurse practitioners (n=12 [22%]), registered nurses (n=12 [22%]), physicians (n=12 [22%]) and other allied health professionals (n=19 [34%]). The average satisfaction score was 58.0±3.5 points. The average score for each item ranged from 4.7 (for the statement ‘the case study was applicable to my practice’) to 4.9 points (for the statement ‘the facilitator was an effective group leader’). There were no differences in satisfaction when the participants were grouped according to their profession (P=0.7). Forty-nine participants (89%) indicated that they would recommend the workshop to a colleague.

A total of 43 (78%) and 28 (51%) participants described the strengths and weaknesses of the workshop, respectively. Their responses are summarized in Figure 1.

Figure 1).

Reported strengths and weaknesses of the workshop. Percentage of participants describing the feature of the workshop as a strength (grey bars) or a weakness (black bars)

Clinical impact of the workshop

Evaluative forms were received from 26 participants (38%), with representation from nurse practitioners (n=9 [35%]), registered nurses (n=4 [15%]), physicians (n=6 [23%]) and other allied health professionals (n=7 [27%]). The most useful learning aspects of the workshop, according to 25 of the participants, are summarized in Table 1. Twenty-five participants (96%) described greater confidence in their management of patients with COPD.

TABLE 1.

Topics described as the most useful aspect(s) of the workshop. a total of 25 participants described at least one topic

Topic	Participants, n (%)
Diagnosis of chronic obstructive pulmonary disease	14 (56.0)
General content	12 (48.0)
Medications	11 (44.0)
Objective measures	11 (44.0)
Nonpharmacological management	11 (44.0)

Regarding specific changes in clinical practice following attendance at the workshop, 19 (73%) described greater patient education, 18 (69%) reported increased patient monitoring and 12 (46%) described an increased use of objective testing. The most common suggestions for future workshop topics were COPD action plans (n=15 [58%]), differential diagnosis and management of asthma versus COPD (n=11 [42%]), management of severe COPD (n=11 [42%]) and medications (n=11 [42%]). Of the 11 participants who noted barriers to implementing changes in their clinical management of COPD patients, the most commonly cited issues were patient compliance (n=5 [45%]) and access to the necessary resources (n=4 [36%]).

DISCUSSION

The present study is the first to evaluate the use of workshops for the dissemination of the CTS guidelines for COPD. This simple strategy significantly increased participants’ knowledge and was associated with high levels of satisfaction. Participants reported important changes in their clinical practice in the three months following the workshop. These results suggest that for some individuals, workshop attendance constitutes an effective method to promote acceptance of CPGs.

The score on the knowledge assessment questionnaire increased following attendance of the workshop. Although the modest sample size precluded detailed subgroup analyses of the response to each question, there was an encouraging trend for a greater proportion of participants being able to justify the use of spirometry in the diagnosis of COPD (76% versus 100%) and identify the clinical features of COPD (41% versus 71%). These findings are relevant because the infrequent use of spirometry and low awareness of symptoms by primary care practitioners have been reported as barriers to optimal care of patients with COPD (5). Despite these positive changes, it was notable that on completion of the workshop, only a minority of participants obtained a perfect score on the knowledge assessment questionnaire, suggesting that effective implementation of CPGs likely requires multimodal dissemination strategies and regular re-enforcement.

Satisfaction with the workshop was high, with the average score for all 12 items of the structured satisfaction questionnaire exceeding four out of a possible five points. The content and structure was compatible with health care professionals identifying ‘live’ CME activities centered on CPGs as important for optimizing clinical management (5). The workshop used in the present study met these preferences by describing a CPG, being accredited to allow physicians to earn CME points and providing a ‘live’ forum for education. Responses to the satisfaction questionnaire revealed that approximately one-half of the participants chose not to indicate any weakness with the workshop. Of those that did describe a weakness, issues pertaining to the content or information were reported by the largest proportion of the sample (n=12 [22%]), with the accompanying written comments indicating that participants would have welcomed more information on medications and home exercise.

Almost every participant described greater confidence in their management of patients with COPD. This is an important finding because a survey of staff at primary care practices has suggested that less than 50% of physicians experience high levels of confidence associated with managing COPD patients (5). Workshop attendance resulted in clinical practice changes, including more patient education, increased monitoring and greater use of objective testing. The large proportion of participants that offered suggestions for topics to be covered in future workshops indicates that a ‘workshop series’ may be required to meet the large range of educational needs. Such a strategy, coupled with reminder cards or e-mails, may also optimize sustained adherence with the changes in clinical practice (10).

Our findings of high levels of satisfaction with the workshop, an increased confidence with the management of COPD patients, self-reported changes in clinical practice and the indication that almost all participants would recommend the workshop to a colleague, supports the observations of Lougheed et al (11) who successfully used workshops to disseminate the CTS guidelines for the management of asthma. Based on these similarities, we speculate that the use of a workshop format that combines specialist-led didactic presentations with interactive case studies will be associated with favourable outcomes for health care professionals working with different patient populations.

The limitations of the present study are related to the incomplete response rates for all outcome measures, selection and responder bias inherent in studies of this design, the lack of a control group, and the subjective nature and short-term assessment of clinical impact, which is likely to have overestimated the actual changes in clinical practice resulting from attendance at the workshop. Future studies should consider optimizing response rates by providing postage paid preaddressed envelopes or Internet-based modalities for participants to return their responses in addition to techniques such as a Dillman approach (13), which involves sending multiple reminders to submit responses. Greater response rates would allow for subanalyses to explore differences in knowledge and impact on clinical practice between participants grouped according to their profession. Despite the absence of a control group, it is reasonable to attribute the improved knowledge questionnaire scores to attendance of the workshop. Changes in clinical practice were based on self-reports which may be skewed by bias (14). Future studies can avoid this limitation by evaluating changes in clinical practice, particularly those emphasized throughout the workshop in the short, medium and long term by obtaining objective measures of clinical practice such as via a chart review before and at various time points following dissemination strategies.

SUMMARY

Workshop attendance by health care professionals was associated with high levels of satisfaction and self-reported changes in clinical practice, which may reflect improved knowledge of the CTS CPG for COPD. This strategy should be considered in other jurisdictions to promote awareness and implementation of CPGs.

APPENDIX 1: KNOWLEDGE ASSESSMENT QUESTIONNAIRE

Patient history

Alfred, a 49-year-old factory worker, presents with shortness of breath and a morning cough which is productive of a small amount of sputum. On most days he coughs intermittently throughout the day. He thinks he has bronchitis. His wife insisted that he make this appointment because his cough has been present for many weeks without improvement, despite his using an over-the-counter cough medication. He has been with same family physician for 15 years but does not visit the office frequently. However, in the previous winter he had bronchitis twice, and both times he was treated with antibiotics. Alfred notices that he is short of breath when he walks around the factory. He has stopped playing hockey with his friends because he could not catch his breath during the game. Alfred has a 23 pack-year smoking history. He currently smokes 15 to 20 cigarettes a day. He says that his father and brother both have emphysema. His clinical examination is normal.

Questions:

• What are the possible causes of Alfred’s cough? (Give as many as you can)

• What risk factors for COPD does Alfred have?

• Would you order a chest x-ray? (Give reason for this answer)

• Would you order simple pre/post bronchodilator spirometry? (Give reason for this answer)

• Would you order a sputum culture? (Give reason for this answer)

• Name three clinical features of COPD?

• Which key issues would you discuss with the patient?

• Assuming that Alfred has COPD, what is your approach to pharmacological therapy for him?

• Outline your longer-term nonpharmacological strategy for the management of Alfred’s condition.