Measuring the impact of a live, case-based, multiformat, interactive continuing medical education program on improving clinician knowledge and competency in evidence-based COPD care

Abstract

Background:

Major clinical gaps impede the evidence-based treatment of chronic obstructive pulmonary disease (COPD) in the primary care setting. Studies are needed to measure the effectiveness of continuing medical education (CME) on improving physician competency and performance toward evidence-based COPD care.

Methods:

Between September 26, 2009 and December 12, 2009, 769 primary care physicians participated in a series of 12 regional, live, interactive, case-based, multiformat, half-day CME programs on COPD. A subgroup of randomly selected participants (n = 50) and demographically matched nonparticipants (n = 50) completed surveys that included case vignettes, a validated tool for measuring physician performance in clinical practice. Cohen’s d was used to calculate the magnitude of difference between participants and nonparticipants in the delivery of evidence-based care.

Results:

Physicians who participated in CME programs were 50% more likely to provide evidence-based COPD care than physicians who did not participate. Compared with nonparticipants, participating physicians were more likely to recognize COPD correctly in a patient presenting with dyspnea (74% versus 94%, P = 0.007), recognize that women may have a greater susceptibility than men to the toxic effects of smoking (54% versus 90%, P < 0.001), and identify the mechanisms of action of emerging therapies (33% versus 65%, P = 0.003).

Conclusion:

Physicians who participated in a half-day regional CME program on COPD diagnosis, staging, and treatment were significantly more likely than nonparticipants to deliver evidence-based COPD care. With multiformat, interactive, focused educational interventions, physicians can make diagnostic and therapeutic choices in the primary care setting that align more closely with current guidelines and clinical evidence in COPD management.

Introduction

Chronic obstructive pulmonary disease (COPD) is a major cause of mortality and morbidity worldwide. Approximately 16 million Americans are currently diagnosed with COPD, while another 14 million may be affected but remain undiagnosed.¹ Each year, COPD is responsible for more than eight million physician office and hospital outpatient visits, 1.5 million emergency department visits, and 726,000 hospitalizations.² COPD is the fourth leading cause of death in the US, and is projected to become third by the year 2020 if drastic changes in diagnosis and management are not implemented.³

Despite these striking statistics, COPD remains poorly managed in the primary care setting. One major barrier to better patient outcomes is the underdiagnosis of COPD, which delays treatment and leaves symptoms unmanaged. COPD is highly prevalent in primary care; approximately one in four adults aged 40 years or older with known risk factors for COPD, including tobacco use and self-reported symptoms of chronic bronchitis, have airway obstruction consistent with a diagnosis of COPD.¹ Yet primary care physicians will less commonly consider and pursue the diagnosis of COPD than other diagnoses with fewer symptoms, such as hypertension and diabetes.⁴ Moreover, only one in four primary care physicians adhere to guideline-recommended spirometry and medication use more than 90% of the time.⁵

According to the Global Initiative for Chronic Obstructive Lung Disease (GOLD) guidelines, COPD is both a preventable and treatable disease, and increased focus on accurate diagnosis, staging, and management may reduce the overall burden of the disease.⁶ Continuing medical education (CME) programs that address the inadequacies of COPD care may improve physician competency in providing guideline-directed COPD care. The American College of Chest Physicians has called for more research on the effectiveness of CME initiatives in improving physician competency and performance toward evidence-based care.⁷ The underlying rationale for CME is that providers who are educated about the latest standards of care will make more informed diagnostic and treatment decisions, resulting in improved patient outcomes.⁸ To date, the CME literature suggests that the most effective strategies contain several design elements, such as a rigorous and accurate assessment of need, use of active and varied learning approaches, and an evidence-based curriculum that focuses on overcoming barriers to change.⁹ Additionally, a change in physician practice is more likely with interventions that are multifaceted, interactive, and consistent with the perceived needs of the learner.¹⁰

In September 2009, we implemented a live, highly intensive, case-based educational initiative called “Improving COPD Patient Outcomes: Breaking Down the Barriers to Optimal Care”. This CME initiative consisted of a series of 12 regional half-day meetings designed to address existing knowledge and practice gaps in the management of COPD. The objective of this communication is to report the impact of this educational initiative on participants’ knowledge and competency in the guideline-based diagnosis, staging, and management of COPD.

Materials and methods

Between September 26, 2009, and December 12, 2009, 12 regional, live, interactive, case-based, multiformat, half-day CME programs were held on the topic of COPD. The program learning objectives and content were developed by a faculty panel of three leading experts in the management of COPD. Specifically, the program was designed to increase the ability of primary care providers to:

• Describe the demographics of COPD burden and explain the impact of patient comorbid conditions on COPD outcomes and management

• Obtain reliable office spirometry results through proper patient education and coaching

• Diagnose, stage, and manage a patient with COPD with a combination of management strategies, including risk reduction, pharmacologic, and nonpharmacologic interventions as per guideline recommendations

• List emerging therapies for management of COPD

• Implement patient-oriented strategies to optimize adherence and improve outcomes.

To accomplish these educational objectives, the half-day program incorporated a mix of proven adult-learning formats,⁹ including a series of short (25–30-minute) didactic lectures, with several audience response system questions designed to promote faculty-learner interaction. To enhance their patient education skills, participants watched a video about the correct use of inhaler devices. Physician learners also participated in two small-group workshops. One workshop focused on a detailed case discussion, with challenges related to diagnosis, staging, initial treatment, and management of COPD exacerbations, and follow-up care. In the second workshop, learners participated in hands-on demonstrations of handheld spirometers, and engaged in active role playing of spirometry coaching, reading, and interpretation to reinforce accurate office spirometry use.

Measurement of effectiveness

The effectiveness of the educational intervention was measured using a survey comprised of evidence-based case vignettes and questions about physician confidence in the optimal management of patients with COPD. The case vignettes were framed around several key measurement indicators, which are evidence-based statements that outline the health care performance expectations associated with the content of an educational activity. The measurement indicators identified for this program and used to develop the case vignette survey were concentrated in the areas of COPD pathophysiology, diagnosis, and management (Table 1).

Table 1.

Effectiveness of educational intervention: measurement indicators

Aspects of COPD management	Measurement indicators
Pathophysiology and natural history	In COPD, the ongoing inflammatory process leads to enlargement of the alveolar spaces, fibrosis, and destruction of the lung parenchyma; these changes increase with disease severity and persist on smoking cessation2,32,33 Cumulative exposure to noxious particles (including cigarette smoke) is the key risk factor for COPD10 Compared with men having COPD, women with COPD have greater susceptibility to toxic effects of smoking.34
Diagnosis and staging	A diagnosis of COPD should be considered in any patient who has cough, sputum production, dyspnea, and/or a history of exposure to risk factors; the diagnosis is confirmed by spirometry6,35–37 A consensus statement by the National Lung Health Education Program recommends the widespread use of office spirometry by primary care providers for patients ≥45 years old who smoke cigarettes6,5–37 A good test requires a good effort on the part of the patient and enthusiastic coaching by the technician; for spirometry to be of value, the FVC test must be performed correctly38 Staging requires knowing FEV1 and FVC and understanding how to use them to stage a patient6 Patients with FEV1 <40% have severe COPD.6
Treatment	Current pharmacotherapies do not change the natural history of COPD COPD needs to be treated early, and regular treatment with long-acting bronchodilators is an effective and convenient maintenance treatment6 For patients with severe COPD who have repeated exacerbations, glucocorticosteroids should be added to the treatment regimen6 Patients may be on a short-acting bronchodilator, a long-acting bronchodilator, a combination product, and glucocorticosteroids6 Unintentional nonadherence may reflect poor comprehension of the treatment regimen39 Nicotine dependence might function as a barrier to smoking cessation; nicotine dependence, in particular withdrawal, was related to a high number of quit attempts and to remaining a current smoker40 A number of Phase III clinical trials have assessed the potential utility of PDE4 inhibitors in the treatment of COPD.41–43

Abbreviations: COPD, chronic obstructive pulmonary disease; FEV₁, forced expiratory volume in the first second; FVC, forced vital capacity; PDE4, phosphodiesterase 4.

The case vignettes were presented (in survey format) to program participants and a demographically similar group of nonparticipants. The vignettes were designed to assess whether the diagnostic and therapeutic choices of the participants were consistent with clinical evidence presented in the content of the educational activity. The case vignettes were also used to assess whether practice choices of participants were different from practice choices of nonparticipants.

Additional survey items were included to assess barriers to the optimal management of COPD. Respondents were asked to rate the severity of five potential barriers to optimal COPD management on a scale of 1 (lowest significance) to 10 (highest significance). Respondents were also asked to rate their level of familiarity with four aspects of COPD management on a scale of 1 (least familiar) to 10 (most familiar).

All surveys were field-tested for content validity and revised based on field testing comments/suggestions prior to implementation. Finalized surveys were distributed to participants immediately following each of the 12 regional half-day programs. Survey responses were also collected from a demographically similar group of nonparticipants from the American Medical Association database, matched by degree, specialty, practice setting, and number of patients with COPD seen per week, via email.

Data analysis

Data were analyzed using the Statistical Package for Social Sciences (SPSS 17.0, SPSS Inc, Chicago, IL). Data were first arrayed using frequencies, and t-tests were then used to test for differences between the mean evidence-based responses of the participants and the nonparticipants. Differences between the two groups were considered significant if P ≤ 0.10. The educational effect size was calculated to determine the amount of difference in treatment choices between the CME participants and nonparticipants, using a method previously published by Colliver et al.¹¹ Effect size is calculated using the Cohen’s d formula, and expressed as a percentage of nonoverlap between participants and nonparticipants, or the percentage achieved by participants that was not reflected in the evidence-based responses of nonparticipants. Any positive difference between the two groups where the content is relevant to the participants’ patient populations demonstrates an important potential educational impact.

Effect size analysis was performed using physician data only. Based on total attendance, preliminary power analysis demonstrated that sample data from 50 participants and 50 nonparticipants were needed to demonstrate a significant (P ≤ 0.10) 30% difference in educational index between the physician groups.

Results

Baseline characteristics

A total of 769 health care providers participated in the educational program, including 699 physicians, 13 nurses, and 57 other practitioners. Participants reported seeing a total of at least 8459 patients with COPD per week within their clinical practices.

A total of 146 effectiveness surveys were collected from 769 participants (response rate, 19%), and 53 surveys were collected from 160 nonparticipating physicians contacted by email (response rate, 33%). A random subgroup of participants (n = 50) who completed the effectiveness survey were matched with nonparticipants (n = 50) according to clinical specialty, years in practice, and treatment setting (Table 2). Participant and nonparticipant responses to the survey questions are summarized in Tables 3–5.

Table 2.

Baseline demographics of effectiveness subgroup

Characteristic	Participants (n = 50)	Nonparticipants (n = 50)
Patients seen per week with COPD	11	15
Years in practice	28 years	24 years
Specialty
Family practice	50%	52%
Internal medicine	50%	48%
Degree
MD/DO	100%	100%
Present employment
Solo practice	45%	38%
Group practice	31%	58%
Medical school	0%	2%
Health-management organization	0%	2%
Government	25%	0%
Practice location
Urban	40.4%	30.0%
Suburban	55.3%	52.0%
Rural	4.3%	18.0%

Abbreviation: COPD, chronic obstructive pulmonary disease.

Table 3.

Case 1 survey results

Case 1: A 51-year-old woman presents for her annual visit. In reviewing her recent health, she reports mild shortness of breath when climbing a flight of stairs. Her past medical history is significant only for hypothyroidism (well controlled with thyroid replacement therapy). There is no history of early heart disease in her family. Her father died of lung cancer. The patient smoked two packs per day for 10 years but quit more than five years ago. Her husband (also a patient of yours) continues to smoke. She works as a bank teller and until this past year was moderately active (walking 30 minutes three times per week).
Case questions and responses*	Participants (n = 50)		Nonparticipants (n = 50)		Pvalue
	n	%	n	%
Based on the patient’s presentation, what is the most likely etiology for her shortness of breath?
Alveolar destruction*	46	93.9%	37	74.0%	0.007
Muscular deconditioning	2	4.1%	8	16.0%
Synovial inflammation	1	2.0%	3	6.0%
Increased left ventricular filling pressure	0	0.0%	2	4.0%
Total respondents	49	100.0%	50	100.0%
What is the most appropriate next step to diagnose her dyspnea?
Repeat thyroid-stimulating hormone test	1	2.0%	1	2.0%
Spirometry*	48	98.0%	45	90.0%	0.204
Refer for cardiac catheterization	0	0.0%	2	4.0%
Watchful waiting	0	0.0%	2	4.0%
Total respondents	49	100.0%	50	100.0%
What intervention would you recommend to improve her current condition?
Advise that she take a daily aspirin	0	0.0%	1	2.0%
Advise that she take an iron supplement	0	0.0%	0	0.0%
Advise that she get her husband to quit smoking*	46	92.0%	41	82.0%	0.137
Refer her to physical therapy	4	8.0%	8	16.0%
Total respondents	50	100.0%	50	100.0%

Notes:

^*An evidence-based answer. Not all respondents answered every question.

Table 5.

Case 3 survey results

Case 3: A 69-year-old woman presents for a regularly scheduled follow-up visit. She has type 2 diabetes, hypertension, and COPD. Spirometry results obtained two years ago showed an FEV1/FVC ratio < 0.7 and an FEV1 60% predicted. A sleep study the following year was normal. She has had three COPD exacerbations in the past two years characterized by periods of increased sputum production. Her medical regimen currently consists of metformin, lisinopril, as needed albuterol, and daily salmeterol.
Case questions and responses*	Participants (n = 50)		Nonparticipants (n = 50)		Pvalue
	n	%	n	%
What would you recommend as the next therapeutic step to improve COPD control in this patient?
Prescribe home oxygen therapy	1	2.0%	1	2.0%
Add a short-acting inhaled anticholinergic agent (ipratropium)	17	34.0%	16	32.0%
Add an inhaled glucocorticosteroid*	32	64.0%	33	66.0%	0.834
Refer for surgical evaluation	0	0.0%	0	0.0%
Total respondents	50	100.0%	50	100.0%
Compared with men, women with COPD tend to have:
Less airway hyperresponsiveness	0	0.0%	11	22.0%
Better health-related quality of life	0	0.0%	3	6.0%
Greater susceptibility to the toxic effects of smoking*	44	89.8%	27	54.0%	<0.001
Increased probability of diagnosis on initial presentation	5	10.2%	9	18.0%
Total respondents	49	100.0%	50	100.0%
Which medication classes have been shown to change the natural history of COPD?
Corticosteroid	11	35.5%	20	42.6%
Corticosteroid + bronchodilator	8	25.8%	11	23.4%
Bronchodilator	5	16.1%	9	19.1%
Antibiotic + corticosteroid	0	0.0%	1	2.1%
Phosphodiesterase 4 inhibitor	2	6.5%	0	0.0%
None*	5	16.1%	6	12.8%	0.746
Total respondents	31	100.0%	47	100.0%
For which of the following inflammatory molecular targets in COPD is there an agent in Phase III development?
PPAR-γ	8	20.0%	10	20.8%
PDE4*	26	65.0%	16	33.3%	0.003
TNF α	4	10.0%	7	14.6%
IL-8	2	5.0%	15	31.3%
Total respondents	40	100.0%	48	100.0%

Notes:

^*An evidence-based answer. Not all respondents answered every question.

Abbreviations: COPD, chronic obstructive pulmonary disease; FEV₁, forced expiratory volume in the first second; FVC, forced vital capacity; IL, interleukin; PDE4, phosphodiesterase 4; PPAR, peroxisome proliferation activated receptor; TNF, tumor necrosis factor.

Pathophysiology and natural history of COPD

When asked about the underlying pathophysiology of COPD, a greater proportion of participants than nonparticipants recognized alveolar destruction as the most likely etiology for shortness of breath in a patient presenting with dyspnea on exertion (94% versus 74%, P = 0.007, Table 3, Case 1).

Clinicians who participated in the educational intervention also showed an improved understanding of smoking as a leading risk factor for the development and progression of COPD. Participants were more likely than nonparticipants to recognize correctly that women may have a greater susceptibility than men to the toxic effects of smoking (90% versus 54%, P < 0.001, Table 5, Case 3). Participants were also more likely to recognize the importance of encouraging the spouse of a COPD patient to quit smoking (92% versus 82%, Table 3, Case 1).

Spirometry use

Familiarity with spirometry testing and interpretation was generally low among both participants and nonparticipants. Participants reported a significantly lower mean level of familiarity than nonparticipants (5.9 versus 6.9, P = 0.014). More participants than nonparticipants described themselves as not at all familiar (13% versus 4%) or somewhat familiar (62% versus 54%) with spirometry, whereas fewer reported that they were completely familiar (25% versus 44%) with the procedure (Figure 1). Participants’ answers to case study questions showed a nonsignificant trend toward a greater willingness to use office-based spirometry in the initial assessment of suspected COPD (98% versus 90%, Table 3) and in the long-term monitoring of confirmed COPD (86% versus 76%, Table 4) compared with nonparticipants.

Figure 1.

Familiarity with spirometry interpretation. Participants and nonparticipants were asked to rate their level of familiarity with spirometry interpretation on a scale of 1 (least familiar) to 10 (most familiar). Responses were analyzed by mean ranking and by proportion of responses indicating that physicians were not at all familiar (1–3), somewhat familiar (4–7), or extremely familiar (8–10) with spirometry interpretation.

Table 4.

Case 2 survey results

Case 2: A 68-year-old man who recently moved to the area presents to the urgent care clinic with an acute complaint of productive cough. He has a 20-pack year history of smoking but quit five years ago. He was started last year on an albuterol inhaler by his prior physician. On further review, he has had no fevers, sweats, weight loss, nausea, or vomiting. He reports that for the past year he typically begins each morning with productive cough but is fine the rest of the day. This pattern is somewhat worse over the past two weeks, with his cough persisting for most of the morning. On examination he is a thin man in no acute distress. He is afebrile, breathing comfortably, with decreased breath sounds but no focal wheezes or areas of consolidation on pulmonary examination. The rest of his examination is within normal limits.
Case questions and responses*	Participants (n = 50)		Nonparticipants (n = 50)		Pvalue
	n	%	n	%
What would be your next step in the management of this patient?
Prescribe a macrolide or cephalosporin antibiotic	16	32.7%	21	42.0%
Initiate a 21-day tapered oral glucocorticoid regimen	3	6.1%	4	8.0%
Initiate a long-acting beta2-agonist*	30	61.2%	25	50.0%	0.261
Admit the patient to the hospital	0	0.0%	0	0.0%
Total respondents	49	100.0%	50	100.0%
Which of the following studies would you use in his long-term management?
Chest x-ray	4	8.0%	8	16.0%
High-resolution chest CT	3	6.0%	3	6.0%
Spirometry*	43	86.0%	38	76.0%	0.202
Sleep study	0	0.0%	1	2.0%
Total respondents	50	100.0%	50	100.0%
Case 2 (Continued): The patient comes to see you two weeks later. His chest x-ray obtained at his urgent care visit was normal. His sleep study was negative. His spirometry results from last week indicated that his FEV1 was <40% predicted, FEV1/FVC <0.70. He is feeling much better and has stopped taking his inhalers.
Which of the following strategies would be most likely to improve the patient’s adherence?
Provide in clear detail the results of recent trials showing mortality trends and benefits of therapy	19	38.8%	16	32.0%
Identify the patient’s preferences about different therapeutic choices	8	16.3%	19	38.0%
Assess the patient’s comprehension of different treatment options*	14	28.6%	10	20.0%	0.320
Refer the patient to pulmonary rehabilitation	8	16.3%	5	10.0%
Total respondents	49	100.0%	50	100.0%
Based on the GOLD criteria, what severity stage is this patient?
GOLD stage 1 – mild	3	6.0%	4	8.0%
GOLD stage 2 – moderate	18	36.0%	24	48.0%
GOLD stage 3 – severe*	29	58.0%	22	44.0%	0.161
GOLD stage 4 – very severe	0	0.0%	0	0.0%
Total respondents	50	100.0%	50	100.0%

Notes:

^*An evidence-based answer. Not all respondents answered every question.

Abbreviations: CT, computed tomography; FEV₁, forced expiratory volume in the first second; FVC, forced vital capacity; GOLD, Global Initiative for Chronic Obstructive Pulmonary Diseases.

COPD staging

Participants were twice as likely as nonparticipants to report complete familiarity with the GOLD guidelines (28% versus 14%). Participants’ greater self-reported knowledge and confidence in applying the GOLD guidelines were reflected in the case vignettes, where participants were more likely than nonparticipants to determine disease severity correctly (58% versus 44%, Table 4, Case 2) according to GOLD severity criteria.

COPD management and emerging therapies

Participants and nonparticipants were equally likely to select guideline-recommended pharmacotherapy for COPD patients with repeated exacerbations (64% versus 66%, Table 4, Case 2). Participants were more likely than nonparticipants (61% versus 50%) to select a guideline-recommended strategy for maintenance therapy (Table 5, Case 3). Participants were more likely than nonparticipants to report complete familiarity with emerging therapies for COPD (21% versus 4%, Figure 1) and were more likely to identify correctly the mechanisms of action of emerging therapies (33% versus 65%, P = 0.003, Table 5).

Barriers to guideline-recommended care

When potential barriers to COPD management were evaluated, participants rated “patient lack of adherence to smoking cessation recommendations” as a more severe barrier to optimal COPD care than did nonparticipants (9.2 versus 8.6, P = 0.025, Figure 2). In addition, participants assigned a lower mean score to “lack of clarity about staging COPD severity” than nonparticipants (4.8 versus 5.6, P = 0.083, Figure 2). Participants were nearly twice as likely as nonparticipants to describe this potential barrier as not at all significant (31.9% versus 16.0%).

Figure 2.

Barriers to the management of chronic obstructive pulmonary disease. Mean barrier severity score on a scale of 1 (lowest significance) to 10 (highest significance).

Regarding the use of spirometry as a potential barrier to COPD management, participants assigned a higher mean severity score to “difficulty in obtaining spirometry results” than nonparticipants (5.4 versus 4.4, P = 0.029, Figure 2). More participants than nonparticipants described obtaining spirometry results as a very significant barrier (27% versus 12%), whereas fewer participants reported spirometry as not significant (25% versus 40%).

Educational impact of the program

Effect size analysis showed a 50% nonoverlapping difference in the evidence-based clinical choices made by participants and nonparticipants. This represents a 50% increased likelihood that clinicians who participated in the “Improving COPD Patient Outcomes: Breaking Down the Barriers to Optimal Care” program were making choices in response to case vignettes based on clinical evidence. Based on the average number of patients with COPD seen weekly by the 769 health care providers who attended the educational activity, these data indicate that at least 8459 COPD patients are 50% more likely to receive evidence-based care than those seen by health care providers who did not participate in the activity.

Discussion

Clinicians who participated in the program described here demonstrated improved knowledge and competency in a variety of areas related to the management of COPD, including etiology and risk factors, disease staging, guideline-directed patient management, and emerging treatment options. Furthermore, participating clinicians were 50% more likely than nonparticipants to provide evidence-based and guideline-driven care. Specific areas of patient care that were most affected by attending the activity included recognizing gender differences in susceptibility to the toxic effects of smoking, using spirometry in the assessment and monitoring of patients with COPD, identifying stage of COPD based on the FEV₁ value, and maintenance therapy.

Considerable research on CME has focused on the ability of specific educational interventions to improve provider knowledge, competence, and ultimately, patient outcomes. In a 2007 meta-analysis of CME interventions, the mean effect size of various interventions was 28%, with greater effect sizes observed from active learning methods than from passive educational interventions (33% versus 20%).¹² By comparison, the effect size of the “Improving COPD Patient Outcomes: Breaking Down the Barriers to Optimal Care” program was 50%, much higher than the average effect size across the spectrum of CME interventions.

Our large effect size may stem from the multiple modes of delivery. In 2009, the American College of Chest Physicians released a series of evidence-based guidelines on the development of effective CME programs.^7,9,13–15 Consistent with principles of adult learning theory, the American College of Chest Physicians emphasized the importance of using multiple instructional techniques to effect positive change on physician knowledge, practice performance, and clinical outcomes.^7,9,13–15 The program incorporated many of the educational formats recommended by the American College of Chest Physicians, including case-based learning, audience response system, lectures, hands-on demonstrations, discussion groups, and role playing.⁹ The half-day program emphasized faculty-learner interactions via discussion of practice preferences and responses to knowledge-based and competency-based audience response system questions, with ample time for open discussion. The small-group workshops provided learners with additional opportunities for interaction and faculty feedback regarding clinical decision-making in the management of COPD. With this approach, this multimodal educational activity engaged 769 physicians and potentially influenced the care of nearly 8500 patients.

Results from our study show that participation in the program improved physician knowledge of the underlying pathophysiology of COPD, as well as gender differences as a risk factor for COPD. The ability of CME activities to increase physician knowledge is well documented in the CME literature. Bruno et al found that participation in a live peer-to-peer CME activity increased pulmonologists’ knowledge about the diagnosis and management of idiopathic pulmonary fibrosis.¹⁶ Dimor et al reported immediate gains in physician knowledge, as measured by changes in response patterns before and after a 1.5-day immersion course on sleep quality.¹⁷

Response patterns to the case vignettes indicate that the physician participants increased their competency in two important areas, ie, staging COPD severity on the basis of spirometry data and managing COPD patients with pharmacologic and nonpharmacologic interventions. Within the evolving CME landscape, activities have shifted in focus in recent years from increasing provider knowledge to improving competency. Outcomes research to document gains in competency and health care delivery have followed suit. Peterson et al showed that participation in a two-day course resulted in a significant increase in self-assessed competency.¹⁸ Participants who reported substantial improvements in competency were more likely than nonparticipants to make appropriate evidence-based management decisions.¹⁸

In our study, the quality of education index of 50% in our study indicates a high likelihood that primary care providers who participated in the educational intervention will provide evidence-based and guideline-driven care for patients with COPD. Although the relationship between physician competency and clinical behavior is not well understood, studies suggest that enhancing provider competency translates into an increase in physicians’ performance in providing evidence-based and guideline-based care. In turn, improved physician performance can lead to improved patient care and patient outcomes. Cabana et al reported improved asthma symptoms and decreased health care utilization among patients whose physicians had attended two interactive seminar sessions on asthma care that included short lectures, case discussions, and video modeling of patient communication techniques.¹⁹

Ulrik et al²⁰ evaluated the effects of an intensive series of educational programs for primary care providers and their staff on the clinical management of COPD. After one year, COPD education was associated with improvements in several indicators of guideline adherence, including a significant increase in the use of spirometry (P < 0.001), documentation of smoking status (P < 0.01), counseling regarding smoking cessation, inhaler technique, physical activity, and rehabilitation (P < 0.01), and a significant shift toward guideline-directed and stage-specific pharmacologic management of COPD (P < 0.001).²⁰

Persistent clinical gaps

Our survey identified several persistent barriers to optimal COPD management that may serve as appropriate targets for additional education. Clinicians in both study groups ranked “lack of adherence to smoking cessation recommendations” as the greatest of several barriers to optimal COPD management. Participants assigned this barrier a higher score than nonparticipants, suggesting that the educational activity successfully reinforced the importance of smoking cessation as a key element of COPD management. Additional education in this area may empower clinicians to overcome this barrier to optimal COPD care successfully.

Our results are consistent with those of several others showing practice gaps related to the incorporation of spirometry into the diagnosis and management of COPD in primary care.^21–24 However, compared with nonparticipants, participating physicians considered “difficulty in obtaining spirometry results” to be a more severe barrier. Furthermore, participants described themselves as less familiar with spirometry interpretation than nonparticipants. These findings contrast with results from the Case 2 vignette (Table 4) indicating increased competency in performing spirometry and interpreting spirometric results. These data appear contradictory, but may in fact highlight the effectiveness of CME in providing a “reality check” about the practical challenges of performing and interpreting results from office spirometry tests. After experiencing hands-on demonstrations of office spirometers, role playing the task of spirometry coach, and attempting to interpret real-time spirometry results, participants were more likely to recognize and report gaps in their own competency with the correct use and interpretation of spirometry. Our findings illustrate the importance of identifying learner-reported competency gaps, such as spirometry use and interpretation, as targets for additional educational intervention.

Study limitations

In today’s CME environment, greater emphasis has been given to measuring higher level educational outcomes. The effect of physician education on patient outcomes is the ultimate endpoint, but was not measured in this assessment. While direct application of learning into practice was not assessed, the methodology in this study assumes that improvements in knowledge, competency, and performance ultimately translate into higher quality care and improved patient outcomes. Our outcomes methodology relies on comparing responses to a series of case vignettes from physicians who participated in a CME activity with those obtained from a comparable group of physicians who did not receive the same education.²⁵ Other methods for assessing the effect of physician education have been reported in the CME literature. One option involves chart abstraction. This method is limited by the potential for recording bias due to time constraints on outpatient visits and a cumbersome process of adjustment for case-mix variation.²⁶ Evaluation of deidentified claims data is another method for measuring change in physician competence. While this methodology can accurately report impact on patient care, it is highly dependent on variable insurance reporting systems, which may delay or alter data collection.²⁷ Case vignettes have gained considerable support for their value in predicting physician practice patterns. Results from recent research studies demonstrate that case vignettes, when compared with chart review and standardized patients, are a valid and comprehensive method to measure a physician’s process of care in actual clinical practice.^26,28 Furthermore, case vignettes are more cost-effective and less invasive than other means of measurement.²⁶

Assessing the participants prior to the activity, and comparing preactivity results with those obtained after the activity may also provide an objective measure of educational impact. We prefered the methodology of comparing data from participants and matched nonparticipants because pre- and post-assessments are usually associated with a high attrition rate, which would complicate the analysis. Our methodology may be limited by an imperfectly matched control group, because although it was demographically matched to the participant group, other potential key confounders, such as previous COPD training and attitudes towards COPD management, were not taken into consideration.

In our study, participants completed the case vignette surveys directly after the program, thereby capturing immediate improvements in knowledge, competency, and intention to change practice behavior. Surveying participants weeks or months following an educational activity is favored as a means for assessing long-term retention, as well as to evaluate performance change that occurred after the physicians returned to practice. Some studies have shown a decline in learning retention with follow-up testing of CME programs,^29,30 whereas others have shown a gain in retention (eg, due to the ability of learners to transfer new knowledge into practice over time).³¹ In the present study, repeat follow-up testing of participants may have shown a different level of educational impact. Nevertheless, our data project an impact on physician behavior and strongly support the benefits of CME for improving the delivery of guideline-recommended patient care. However, given that our participants were a self-selected group of primary care providers with an interest in COPD, the educational effectiveness of this program may not be generalizable to all primary care providers.

Conclusion

Participation in a half-day, highly interactive CME activity is associated with increased physician knowledge in the management of COPD, as well as increased likelihood for patients to receive evidence-based care from participating physicians. Our findings also identified opportunities for additional improvement in certain aspects of care, including spirometry use and smoking cessation.

With focused educational interventions, physicians can make diagnostic and therapeutic choices that align more closely with current guidelines and clinical evidence in COPD management. These findings strengthen the rationale for interactive CME programs to improve provider knowledge and competency, and demonstrate the ability to document the educational effectiveness of CME initiatives.