A Systematic Review of the Effects of Continuing Education Programs on Providing Clinical Community Pharmacy Services

Abstract

Objective. To summarize the effects of media methods used in continuing education (CE) programs on providing clinical community pharmacy services and the methods used to evaluate the effectiveness of these programs.

Methods. A systematic review was performed using Medline, SciELO, and Scopus databases. The timeline of the search was 1990 to 2013. Searches were conducted in English, Portuguese, and Spanish.

Results. Nineteen articles of 3990 were included. Fourteen studies used only one media method, and the live method (n=11) was the most frequent (alone or in combination). Only two studies found that the CE program was ineffective or partially effective; these studies used only the live method. Most studies used nonrobust, nonvalidated, and nonstandardized methods to measure effectiveness. The majority of studies focused on the effect of the CE program on modifying the knowledge and skills of the pharmacists. One study assessed the CE program’s benefits to patients or clients.

Conclusion. No evidence was obtained regarding which media methods are the most effective. Robust and validated methods, as well as assessment standardization, are required to clearly determine whether a particular media method is effective.

INTRODUCTION

Community pharmacists are among the most accessible health professionals. They are in a unique position to help patients manage several health conditions. In recent decades, the role of community pharmacists has shifted from providing medications to providing clinical community pharmacy services (eg, drug therapy management, residential medication management review, and pharmacy-based minor ailment treatment regiments, among others).^1-3 This shift has created a need to develop and maintain expertise and competence in new areas, including pharmacotherapy, interpersonal communication, and patient information documentation.^4,5

However, pharmacy curricula in several countries do not focus on clinical community pharmacy services. Only a few disciplines focus on developing communication skills and pharmacotherapy knowledge.^6-8 Studies report that community pharmacists in several countries have insufficient knowledge in these areas,^9-12 and community pharmacists commonly provide clinical community pharmacy services at an unsatisfactory level of quality.^13-15 Such a situation poses problems concerning appropriateness, effectiveness, safety, and adherence to patients’ drug therapy regimens.

Continuing education (CE) programs can play a vital role in expanding basic pharmacy education and enhancing therapeutic management skills, particularly in areas for which insufficient training has been received or achieved during undergraduate studies.¹⁶ Increasingly, more countries are implementing CE program for community pharmacists as obligatory lifelong learning programs in an attempt to improve clinical community pharmacy services. Nevertheless, much remains unclear about which media methods in CE programs provide the most effective educational approach to providing clinical community pharmacy services. “Media method” refers to the method by which the CE program activity is delivered. We adopted the definition of media methods proposed by the Johns Hopkins Evidence-based Practice Center (Table 1).¹⁷ To our knowledge, no published systematic review evaluates the effects of various media methods in CE programs on providing clinical community pharmacy services.

Table 1.

Definition of Media Methods¹⁷

Considering the importance of improving the quality of clinical community pharmacy services provided to patients, the primary aim of this study was to summarize scientific evidence of the effects of media methods in CE programs on providing clinical community pharmacy services. The secondary aim was to summarize methods used to evaluate the effectiveness of CE programs.

METHODS

A systematic review was performed according to recommendations of Preferred Reporting Items of Systematic Reviews and Meta-Analyses (PRISMA).¹⁸ Because this was a systematic review of published data, no approval was needed from the university’s institutional review board. Studies were included if they were original articles published in English, Portuguese, or Spanish from 1990 to 2013 that evaluated the effects of CE programs on providing clinical community pharmacy services. We searched Medline, Scopus, and Scientific Electronic Library Online (SciELO) databases. Additional articles were identified by searching the references of articles identified in these searches.

The search strategy for the databases is described in Appendices 1, 2, and 3. Search terms included: Continuing education (Title/Abstract); Continuing Professional development (Title/Abstract); Life-long learning (Title/Abstract); Professional development (Title/Abstract); Training (Title/Abstract); Postgraduate education (Title/Abstract); Adult learning (Title/Abstract); Competence development (Title/Abstract); Competence (Title/Abstract); Training and education (Title/Abstract); Recurrent education (Title/Abstract); or Practical work training (Title/Abstract) Study selection.

Two independent reviewers scanned the titles of the articles in parallel. If both reviewers felt a title was potentially eligible, then it was promoted to abstract review; if there were disagreements between reviewers, a third reviewer determined if the manuscript was promoted to abstract review. The abstract review phase was performed similarly to the title scans by the same reviewers. After the abstract review, the reviewers read the full article and determined whether it met the inclusion criteria. The articles proceeded to the data collection process if the two reviewers determined that an article was potentially eligible; if there was disagreements between reviewers, a third reviewer determined if the article was potentially eligible.

A data extraction sheet was developed, pilot-tested, and refined via discussion and consensus by three authors. Two independent reviewers conducted the data extraction using the data extraction sheet in parallel. Disagreement between reviewers was resolved by consensus. If no agreement could be reached, then a third reviewer made the determination. The data extracted from the articles included the first author’s name, year of publication, country where the study was conducted, study design, number of participants, content area of the CE program, media method used in the CE program, CE program description, methods of evaluation used to assess the effectiveness of the CE program.

The methodological quality of each study was assessed according to criteria based on Reed et al¹⁹ and Best Evidence Medical Education (BEME).²⁰ According to the BEME, the strength equates with critical appraisal and is a statement of one’s confidence that the results of the study are credible. The BEME proposes five levels of strength: (1) no clear conclusions can be drawn; not strong, (2) results ambiguous; there seems to be a trend, (3) conclusions can probably be based on the results, (4) results are clear and very likely to be true, and (5) results are unequivocal. Strength was evaluated independently by three reviewers, and differences between these reviewers about the strength assessment was resolved by discussion; interrater agreement (determined on the basis of the ratings before consensus was reached) was high (kappa 0.89; 95 percent confidence interval 0.82 to 0.99).

The BEME recommends evaluating how each study reports the outcomes of an intervention and then collecting these into a common format. The BEME defines this evaluation as “importance,” which was evaluated independently by three reviewers, and differences between their judgments was resolved by discussion; interrter agreement (determined on the basis of the ratings before consensus was reached) was high (kappa 0.88; 95 percent confidence interval 0.81 to 0.99). The BEME levels of importance are level 1: participation – covers learners’ views on the learning experience, its organization, presentation, content, teaching methods, aspects of the instructional organization, materials, and quality of instruction; level 2a: modification of attitudes or perceptions – outcomes at this level relate to changes in the reciprocal attitudes or perceptions between participant groups toward intervention or simulation; level 2b: modification of knowledge and skills – for knowledge, this relates to the acquisition of concepts, procedures, and principles and for skills, this relates to the acquisition of thinking and problem solving and psychomotor and social skills; level 3: behavioral change – documents the transfer of learning to the workplace or willingness of learners to apply new knowledge and skills; level 4a: change in organizational practice – wider changes in the organization or delivery of care, attributable to an educational program; and level 4b: benefits to patient or clients – any improvement in the health and well-being of patients and clients as a direct result of an educational program.

RESULTS

The initial search identified 3974 articles (988 from PubMed, 2986 from Scopus, 0 from SciELO). An additional 16 articles were added by reviewing the reference lists of these articles, yielding a total of 3990 articles. After eliminating duplicates, 3951 remained. A total of 111 articles remained after title scans. After abstract review, 31 articles remained. Finally, full-text articles were retrieved for review, and 19 articles met the eligibility criteria and were included in this systematic review (Figure 1).^21-39

Figure 1.

Flowchart of search strategy and study selection.

Most the articles were published from 2000 to 2013 (n=18). One study was published in 1997.³⁹ The studies were performed in countries on the following continents: four in North America,^27,28,36,30 six in Europe,^{22-24,27,30,36} four in Asia,^21,25,26,34 and five in Australia/Oceania.^{32,33,35,38,39} The sample sizes of these studies were generally small. Nine studies had a community pharmacists sample size of less than 50,^{23,27,31,33,34,36-39} seven studies had a community pharmacists sample size of 50 to 100,^{22,24,25,28-30,35} two studies had a community pharmacists sample size of 101 to 200,^26,32 one study had a community pharmacists sample size of more than 200.²¹ With regard to study design, 12 studies were before-and-after studies,^{21,24-27,31-37} and seven studies were controlled trials.^{22,23,28-30,38,39} The features of these studies are summarized in Tables 2, 3, and 4.

Table 2.

Summary of Studies that Used One Media Method in the Continuing Education Program

Table 3.

Summary of Studies that Used Two Media Methods in the Continuing Education Program

Table 4.

Summary of Study that Used Three Media Methods in the Continuing Education Program

Seventeen CE program content areas were identified (Tables 2-4). The content areas of the CE programs varied, from specific disease management (eg, asthma, childhood diarrhea, and depression) to general issues related to providing pharmaceutical services (eg, providing written drug information and information needed to identify and resolve drug-related problems). Most of the CE programs (n=13) had their content areas chosen by the researchers who conducted the studies without assessing gaps in knowledge or the self-reported educational needs of the community pharmacists.^{22,23,25,26,28-30,32-37}

Most of the studies (n=14)^21-34 used just one media method in the CE program. The live method was the most frequent (n=11),^21-31 followed by the Internet (nonreal time; n=2),^32,33 and print (n=1)³⁴ (Table 2). Four studies used two media methods in the CE programs.^35-38 Live methods were used in each of these combinations. The combinations included live computer-based and offline (n=1),³⁵ live Internet and nonreal time (n=1),³⁶ and live and print (n=2)^37,38 (Table 3). One study used a combination of three media methods in the CE program: live, video, and print³⁹ (Table 4).

Various outcome measures were used to assess the effectiveness of the CE programs. Most of the studies used nonvalidated instruments (n=11) to measure outcomes.^{21,28-30,32,33,35-39} Fifteen studies used more than one different outcome measure concomitantly (one study used four outcome measures;³⁹ seven studies used three outcome measures;^{21,28-30,32,33,38} seven studies used two outcome measures^{22,25-27,34-36}). The majority of the outcome measures focused on the effect of the CE program on modifying the knowledge and skills of the community pharmacists (n=14),^{21,23,25-30,32-35,38,39} modifying the attitudes or perceptions of the community pharmacists (n=14),^{21,22,25-28,30-34,36,38,39} and the learning experience of the community pharmacists (n=8).^{22,28,32,33,35,36,38,39} Four studies assessed the effects of the CE program on changes in community pharmacists behavior.^21,24,29,30 One study assessed the effects of the CE program on changes in organizational practice.²⁹ One study assessed the CE program’s benefits to patients or clients.³⁹

Almost every study reported that the CE program was effective with regard to the outcome measures used. Only the study by Watson et al found that the CE program was ineffective for the sale of over-the-counter antifungals.³⁰ Rouleau et al found that the CE program focused on asthma patient care was ineffective on two (interventions described by the community pharmacists in the study log and appropriateness of asthma medications that were used by patients during the study period) of three outcome measures.²⁹ These two studies that found that the CE program was ineffective on one or more outcome measures used the live media method as the sole educational approach.

DISCUSSION

The systematic review evaluated the effects of media methods in CE programs on providing clinical community pharmacy services. Awareness of the effectiveness of each media method may help governments, pharmacist associations, and CE providers develop higher-quality CE programs. The results provided no evidence regarding which media methods were the most effective. Most of the studies reported that their media methods improved the outcome measures chosen by the researchers.

The results of this systematic review suggest the need to standardize outcome measures used to evaluate CE programs. Without such standardization, it is impossible to compare the effectiveness of the different media methods and the strategies used in the different studies. Kirkpatrick et al recommended that evaluation of CE programs should consist of four levels: learner satisfaction – reaction (level 1), learner outcomes – learning (level 2), performance improvement – behavior (level 3), and patient or health outcomes – results (level 4).⁴⁰ Kirkpatrick’s model recommends that evaluations should begin with level 1 and then sequentially move through the other levels.⁴⁰ Each successive level represents a different measure of the effectiveness of CE programs and increases the clinical significance. We strongly recommend that future studies use Kirkpatrick’s model, which will allow comparisons of different CE programs and identification of which strategies are the most effective.

Although our systematic review and the studies’ results do not provide definitive answers regarding which media method is most effective, the findings may be useful for indicating which strategies may be worth pursuing in larger, longer-term trials with more standardized outcomes. The most thoroughly evaluated media method was live media. This method has been shown to improve community pharmacists´ participation, attitudes, knowledge, behavior (based on questionnaire surveys), simulated patient visits, and community pharmacy services based on direct observation.^21-31,35-38 However, the unique study evaluating the effects of CE programs that used live media on organizational practice (n=1)²⁹ and benefits to patients or clients (n=1)³⁹ found that the CE program was ineffective in these outcome measures.

Another important point in the evaluation of CE programs is the extent to which the knowledge, confidence, and skills obtained can be sustained. Patient benefits must be continuous and not limited to certain periods of time. Follow-up evaluations are needed to determine the duration of effectiveness of the educational interventions, strategies that can reduce the decline of the effectiveness of CE programs, and the time intervals at which CE programs should be implemented.^41,42 The studies in our sample did not report follow-up evaluations, so we were unable to assess such variables.

This systematic review has some limitations that must be mentioned. The restriction of languages in the inclusion criteria likely reduced our sample. We also only included studies that were indexed in the PubMed, Scopus, and SciELO databases.

CONCLUSION

We found no evidence regarding which media method is the most effective. The majority of the CE programs were reported to be effective based on the studies’ outcome measures. Standardizing outcome measures would make it possible to evaluate what media method is the most effective as would evaluating outcomes rather than satisfaction and knowledge of learners. The use of the Kirkpatrick’s model (evaluating every proposed level) could be a viable alternative to standardizing outcome measures and augmenting the outcomes evaluated.

Appendix 1. Search Strategy on Medline

Appendix 2. Search Strategy on Scopus

Appendix 3. Search Strategy on SciELO