Results of the Pre-Conference Survey: ACPE Invitational Conference on Advancing Quality in Pharmacy Education

INTRODUCTION

This report summarizes the results of a survey involving a broad array of pharmacy and health care leaders that was developed and administered by the Accreditation Council for Pharmacy Education (ACPE) prior to its invitational conference in Atlanta on September 12-14, 2012.¹ ACPE designed this consensus-seeking conference to ensure that the accreditation of Doctor of Pharmacy (PharmD) educational programs in the United States is well aligned with the future needs of the pharmacy profession. The findings and recommendations emerging from this conference will guide the revision process for ACPE accreditation standards and guidelines.^2,3

The purpose of the survey was to collect information to stimulate thought and discussion by conference participants and presenters. The survey assessed perceptions of: 1) the competency of current graduates in key practice areas, 2) the competencies needed for future practice, and 3) the effectiveness of current strategies for assessment of student attainment of expected competencies including the North American Pharmacist Licensure Examination (NAPLEX).

Surveys were distributed to those individuals who would be attending the conference (Participants) and to a larger, broader group of practitioners and educators (Stakeholders) using Survey Monkey ^® software. The following organizations agreed to invite individuals within their constituencies (Stakeholders) to complete the survey: American Association of Colleges of Pharmacy (AACP), American College of Clinical Pharmacy, Academy of Managed Care Pharmacy, American Pharmacists Association, American Society of Health-System Pharmacists, National Association of Boards of Pharmacy, National Alliance of State Pharmacy Associations, and National Community Pharmacists Association.

OVERALL FINDINGS

The Participant survey was sent to 93 expected conference attendees with 74 individuals returning the survey for a 79.5% response rate. The estimated number of individuals in the Stakeholder group who received the survey was 1932 with 761 individuals completing the survey (a response rate of about 40%).

Table 1 shows the demographic descriptors of the survey’s 835 respondents. Respondents represented a wide range of interests within pharmacy practice and education. The relative proportion of individuals within each sub-group appeared to be evenly distributed between Participants and Stakeholders groups except that a larger proportion of faculty appeared in the Stakeholder group and a higher proportion of Association Executives and Pharmacy Practice Managers within the Participant group.

Table 1.

Description of Survey Respondents (Conference Participants and Stakeholders)

Initial analysis found very few differences between Participant and Stakeholder responses so the results of the two groups were combined for purposes of this report. However, when important differences did appear between Participants and Stakeholders, they are noted in this report. Table 2, Table 3, Table 4, Table 5, and Table 6 list the respondent ratings of the survey items. Although the total pool of respondents was 835, the actual number of responses (see Total column) was somewhat different for each question because not all respondents answered all questions.

Table 2.

Responses of Both Participants and Stakeholders Regarding Current Graduate Preparation

Table 3.

Responses of Both Participants and Stakeholders Regarding Future Competencies

Table 4.

Responses of Both Participants and Stakeholders Regarding Assessment Effectiveness

Table 5.

Responses of Both Participants and Stakeholders Regarding ACPE Standards

Table 6.

Responses of Both Participants and Stakeholders Regarding NAPLEX and AACP Surveys

Competencies of Current Graduates

Respondents were asked questions regarding whether current PharmD graduates are competent in 27 key areas of practice. They recorded their perceptions using a 4 point Likert-type scale: strongly agree/agree/disagree/strongly disagree. Respondents also were given the option of declaring that they were “unable to answer” each question. Overall, respondents were in agreement with the majority of the areas; that is, graduates are competent in most of the 27 areas. To identify areas that could be improved, a criterion of 25% or higher level of disagreement (last column in Table 2) was established. Using this criterion, 11 areas (highlighted in Table 2) were identified as needing focused attention with the area of “conducting research and scholarship” having the highest level of disagreement (65.4% of respondents disagreed/strongly disagreed); “managing situations involving prescription drug abuse” having the next highest level (50.5%); and so on. Three general themes emerged as needing further attention: management (5 of the 11 areas), providing care (3), and research/learning (3). These perceptions were consistent across all groups (practitioners vs. academics, preceptors vs. non-preceptors, etc.).

Integration of Knowledge

The majority of respondents felt that recent graduates can indeed integrate what they learn in didactic coursework to actual practice - only 11.5% of respondents disagreed/strongly disagreed with this statement (Table 2). This perception was consistent across all respondent sub-groups.

Future Considerations

Respondents were asked to speculate whether the competency requirements for pharmacists would change in the future. Using the same 27 areas addressed earlier, respondents rated whether each competency area would: increase greatly, increase, stay the same, decrease or decrease greatly over the next 10 years. As seen in Table 3, most respondents indicated that competency requirements would be increasing or greatly increasing in 26 of the 27 areas. The one exception being Question 36 “dispensing and other traditional functions” where only 12% indicated that competency requirements related to dispensing and other traditional functions will be increasing in the future.

Areas where at least 25% of the respondents indicated that the area is likely to “increase greatly” over the next 10 years were identified for further analyses. Respondents identified 16 areas (highlighted in Table 3) with the area “using information technology effectively” receiving the most attention (55.9% of respondents indicated it will increase greatly); “contributing to patient care teams effectively” was next (52.7%); and so on.

To explore whether future graduates will be competent in the areas that will likely increase in importance, respondent perceptions of future importance were compared with their perceptions of current graduate competence in six areas (see Table 7). For example, about 56% of respondents felt that competency requirements in the “use of information technology’ would increase greatly in the future. At the same time, a large number of respondents (about 91%) agreed that current graduates are competent in this area. The same holds true for the remaining areas displayed in Table 7 indicating that, overall, respondents felt that current graduates are competent in the areas that may increase in importance over the next 10 years.

Table 7.

Increased Future Responsibilities and Current Graduate Competency

Quality of Assessment

Respondents were asked to rate how schools and colleges are assessing 17 key aspects of pharmacy education. Table 4 reveals that, in general, respondents felt that schools and colleges were performing well in most assessment areas. However, 25% or more of respondents felt that schools and colleges could do better in 9 of the 17 areas with assessment of “student skills in scholarship and research” receiving the most concern (46.5% of respondents disagreed/strongly disagreed); “student skills in self-directed life-long learning” the next (42.9%); and so on. Thus, it appears that the academy needs to enhance its assessment in several areas. Many of these are important to the future growth of the profession as noted earlier in the report; for example, students’ skills in research, cultural competency, and critical thinking. In addition, the effectiveness of preceptors and the performance of pharmacy students on patient care teams are especially critical to experiential learning programming. As competency requirements increase in the future, effective assessment mechanisms must be in place to monitor curricular performance.

VARIATIONS BETWEEN SUB-GROUPS

Ratings among sub-groups that differed by 10 percentage points or more in agreement/disagreement were flagged for further analysis.

Participants Compared to Stakeholders

When comparing Participant and Stakeholder responses regarding the competency of recent graduates, only 5 out of the 27 areas met the 10% criterion (Table 8). Perceptional differences were seen in: managing systems of patient care, providing population-based care, ensuring integrity of medication supply chain, participating in self-directed, life-long learning, and demonstrating cultural competency. In all five areas, more Participants were concerned with PharmD graduates’ preparation compared to the number of Stakeholders. Looking at future competency requirements (Table 9), perceptual differences occurred in just two areas: more Participants anticipated an increased need to 1) work with technicians (79% vs. 55%, respectively) and 2) evaluate the published literature (74% vs. 63%). In addition, more Participants had positive impressions of ACPE standards compared to Stakeholders (Table 10). This may possibly be due to the fact that as invited guests to the conference, they might have been more familiar with the accreditation process and more aware about the standards and their purpose.

Table 8.

Comparisons Between Participants and Stakeholders on Selected Current Competencies

Table 9.

Comparisons Between Participants and Stakeholders on Selected Future Competencies

Table 10.

Comparisons Between Participants and Stakeholders Regarding Selected Key Areas

Respondents within the Academy Compared to Respondents in Practice

Members of both the academy and practitioners rated the competency of current graduates in a similar manner (using the 10% criterion described above). Regarding future changes in competency requirements, both groups shared the same perception of future pharmacy activities with two exceptions. First, more practitioners (52%) felt that the length of training for general practice will increase compared to the respondents from the academy (42%). Second, more practitioners rated ACPE standards as being evidence based (83%) compared to academics (70%). In the area of assessment, more academics were critical of the assessment processes within the academy. For example, in the assessment of cultural competence, 34% of academicians disagreed compared to 24% of practitioners; scholarship 46% vs. 26%; and the ethical grounding of faculty 38% vs. 17% respectively.

Academic Administrators Compared to Faculty Members

Comparisons between responses from academic administrators and faculty members revealed that they shared similar perceptions with the following three noteworthy exceptions: more faculty (68% vs. 55%) agreed that a residency is essential for entry into direct patient care; more faculty (48% vs. 38%) felt that the length of education for general practice will increase in the next 10 years; and more administrators (53% vs. 43%) felt that the length of education/training for specialty practice will increase in the future.

SPECIFIC ISSUES

Importance of Residency Training

Respondents were asked to express their opinion whether or not residency training is essential to prepare new pharmacy graduates for direct patient care. A majority (63%; Table 2) of all respondents agreed that residencies are essential for preparing pharmacy graduates to provide direct patient care. More Stakeholders (65%) agreed than Participants (about 50%). Specialist practitioners had the highest proportion of supporters (80%), while academic administrators the lowest proportion (55%).

Future Length of Training and Pharmacists’ Roles

Respondents were asked if they felt that the length of training of either general practice or specialty practice will increase over the next 10 years. As seen in Table 3, a majority of respondents (64%) agreed that the length of training will increase for specialty practice; and about one-half (57%) stated that the length will remain about the same for general practitioners. Most respondents (91%) agreed that more pharmacists will be providing primary care services, much like other “physician extenders” within the next 10 years.

Respondent Perceptions of ACPE Standards, NAPLEX, and AACP Surveys

About one-third of the respondents (Table 5) had negative perceptions about the value of ACPE standards. For example, 30% of respondents felt that the standards were not evidence-based; 32% stated they did not foster innovation; and so on. More academics (30%) felt the standards are not evidence-based compared to practitioners (17%).

NAPLEX is one of the most commonly used outcome measures for graduating students and is monitored annually by ACPE. About one-half of all respondents (50%; Table 6) felt that this examination is not an adequate indicator. Analyses of all sub-groups (practitioners vs. academics, etc.) revealed similar results.

Respondents were asked to rate the value of AACP surveys (which capture perceptions of graduating students, faculty, preceptors, and alumni) as meaningful indicators of quality. Results (Table 6) indicate that two-thirds of all respondents (67%) agreed that they are meaningful indicators with comparisons between the various sub-groups yielding similar results.

WRITTEN COMMENTS

Respondents were offered opportunities to make written comments about the various survey items which provide additional insights into the various issues. Table 11 describes the recurrent themes that emerged from multiple written comments regarding both current and future competencies. For example, respondents repeatedly drew attention to the need for post-graduate training for new PharmD graduates to achieve competency in multiple areas. In addition, respondents identified the need to address the variability of introductory and advanced pharmacy practice experiences (IPPEs/APPEs) within PharmD curricula. Regarding future competency issues, respondents commented on the need to leverage technology, develop reimbursement mechanisms for cognitive services, collaborate with and at times compete against other health care professionals, and advance continuous professional development initiatives to enhance pharmacists’ impact on patient care. Examples of recurrent themes involving assessment included the need to assess the provision of collaborative patient care on interprofessional teams, and the need to enhance the subjective, non-standardized, volunteer-dependent nature of assessing student learning on APPEs.

Table 11.

Recurrent Themes within Written Responses for Current and Future Competencies

ACTION ITEMS

The survey results influenced the recommendations that emanated from the 2012 ACPE invitational conference.² In addition, below are some suggestions for the academic and practitioner communities based on the authors’ analysis of the survey findings.

For Schools and Colleges of Pharmacy

• 1. Academia needs to revise curricula and recruit appropriate faculty to teach the areas highlighted in this report; for example, skills related to information technology, management, and communication with patients, caregivers and other health care providers.

• 2. Academia needs to better prepare graduates to deliver primary patient care and to serve on interprofessional health care teams.

• 3. Graduates need to be better prepared to manage technicians as the role of the technician evolves in many areas of practice.

• 4. Graduates need to be better prepared to evaluate and interpret research results, but not necessarily conduct research themselves.

• 5. Schools and colleges must continue to develop effective assessment mechanisms to both report successes in new practice roles and also to identify areas where quality needs to be improved. Schools and colleges should collaborate in the development and use of these new assessment tools and methods.

• 6. Stronger assessments of preceptor and faculty effectiveness are needed.

• 7. Academia must consult with practitioner groups and with ACPE during the development of future assessment strategies.

For Practitioner Groups

• 1. Practitioner groups must continue to define the scope of pharmacy practice; that is, what are the knowledge, skills, and abilities needed by pharmacists to provide better patient care? Once identified, these expectations need to be communicated to academic programs so that they can be incorporated into revised curricula.

• 2. Similarly, practitioner groups need to continue to work with schools and colleges to make sure that academics and practitioners share the same perception of actual pharmacy practice.

• 3. The role of dispensing and other traditional activities within pharmacy practice will continue to change, and thus practice groups must determine what components are essential for general practice education and what elements for specialty practice education.

• 4. Practitioner groups should participate in ACPE’s accreditation standards review process to make sure elements of practice are addressed appropriately within the standards.

• 5. Residency training issues will continue to be discussed and debated, and thus practitioner organizations, in collaboration with AACP, need to clarify the patient-care competency-level targets for PharmD education versus those for PGY1 residency training.

SUMMARY

Survey results indicated that overall schools and colleges of pharmacy are graduating students who are competent in most critical areas of practice. In addition, a majority of respondents felt that ACPE standards are effective in assuring the quality of pharmacy education. However, as outlined above, respondents offered several suggestions on how academic pharmacy and ACPE could foster further innovation in the development, delivery, and assessment of Doctor of Pharmacy programs. A recurrent theme embedded within these results is the need for all groups (academia, practice, and ACPE) to continue to work together – that they cannot work in isolation - as they develop future enhancements to the profession and to PharmD programs. More specifically, all groups need to advance continuous professional development initiatives to enhance pharmacists’ impact on patient care; and to foster interprofessional health care delivery whenever possible.