Challenges to Implementation of the Co-Curriculum in Accredited Pharmacy Programs

Abstract

Objective. To determine areas of concern, and challenges to implementing and assessing the co-curriculum in accredited Doctor of Pharmacy programs, along with how confident programs are in their ability to meet the co-curriculum requirement as mandated by the Accreditation Council for Pharmacy Education (ACPE).

Methods. A survey was administered to all ACPE-accredited pharmacy programs to collect information regarding areas of concern, challenges, and confidence in their ability to meet the co-curriculum requirement. The frequency of responses to items are presented along with comparisons based on characteristics, including institution type, cohort size, most recent ACPE accreditation review, and supporting offices.

Results. The most common concerns centered on the documentation and assessment process. The most commonly reported challenges were lack of enthusiasm or buy-in from faculty, staff, and students; lack of a clear definition of co-curriculum; and faculty time and insufficient staff. Overall, programs had a high level of confidence in their ability to meet the requirements for co-curriculum. The only differences found were related to supporting offices and cohort size.

Conclusion. The results suggest that having supporting offices may reduce the co-curriculum burden. Similarly, student cohort size may have an impact on the challenges for some programs, particularly those with moderate-sized cohorts reporting challenges related to faculty and staff. Further research is needed to determine how programs address these critical issues, and to explore whether programs report differently on these areas after completing an accreditation review. The study results may be useful to members of the Academy when evaluating co-curriculum.

INTRODUCTION

The co-curriculum has become a significant topic of interest within the Academy.^1-12 At its core, the co-curriculum can be defined as anything outside of standard didactic and experiential coursework that helps students achieve the educational outcomes of the program.¹³ However, within pharmacy education, there is not a standard or universal definition of the co-curriculum available.² The most recent accreditation standards from the Accreditation Council for Pharmacy Education (ACPE) delineated that institutions must report how co-curricular activities are structured, assessed, and help students achieve educational outcomes.¹⁴ It is unclear how confident programs are that they are able to meet this requirement and what resources are needed.

While the explicit ACPE co-curriculum requirement may be relatively new, co-curricular and extracurricular activities have been typical components of the student experience within pharmacy education (eg, student organizations, community service, volunteering) even prior to the requirement.^1-12,15 Some studies have described and evaluated individual, institutional co-curriculum models that were developed based on activities and curricula in place prior to the ACPE requirement.^4,6,11 The requirement has led to many programs “formalizing” these activities, including providing approved activities, documenting student participation, assessing activities and learning, and providing oversight.²

A 2020 study of 107 ACPE-accredited US pharmacy programs examined how the co-curriculum accreditation requirement was being implemented and assessed, including activities, structural characteristics, learning outcomes, oversight, and assessment.¹² The study found that most programs offered similar activities in areas of patient care and non-patient care as part of the co-curriculum. Nearly all programs reported tracking hours or activities as well as the use of reflections. However, there was variation in how the co-curriculum was implemented, particularly related to which departments or offices had primary responsibility for the program, as well as how assessment and feedback were used.¹² The results of this study suggested that activities categorized as co-curricular were previously administered by multiple offices and subsequently may have remained under their oversight. Furthermore, additional tasks may be required by ACPE, and it was unclear whether programs have allocated additional resources to meet these demands.

Running parallel to the implementation of the co-curriculum requirement from ACPE, the overall landscape of pharmacy education has been changing. The admissions pipeline has experienced a significant decline in applicants, and it is uncertain how this change in demand will alter resources. The year 2014 was the last year in which an increase in first professional year student enrollment occurred while faculty vacancies have increased 22.2% from 2013-2014 to 2017-2018.^16,17 Thus, as resources become less available, programs may be forced to prioritize improvements based on a variety of factors, including evolving expectations from accrediting bodies, changes in the pharmacy practice environment, as well as shifts in the recognition of the importance of enhancing student and faculty personal well-being and professional development.^18,19

As pharmacy programs aim to provide high quality education experiences while meeting accreditation expectations, more information is needed to better understand what comprises a successful and sustainable co-curriculum. Given the dynamic environment surrounding accreditation requirements and lack of consistent definitions related to the co-curriculum, the authors sought to identify and describe the challenges facing pharmacy programs in implementing the co-curriculum in their programs and evaluate the level of confidence that programs have in meeting the new co-curriculum ACPE requirement.

METHODS

A survey was developed and administered electronically through the Qualtrics (SAP) survey platform to accredited programs between February and May 2018. The Cedarville University Institutional Review Board deemed this study as exempt from review.

The survey was developed to gather information about the co-curricular models and implementation. The first block of survey items focused on the activities, structural characteristics, learning outcomes, oversight, and assessment of the co-curriculum. Descriptive statistics on the results of these items were published previously and provide an overview of how co-curriculum models are being developed and implemented across the Academy.¹² The second block of items, which are the focus of this manuscript, explored programs’ concerns with satisfying the ACPE co-curriculum requirement, challenges to implementing co-curriculum, and confidence in meeting the ACPE co-curriculum requirement.

A list of prospective participants was obtained from the AACP Roster of Faculty and Professional Staff.²⁰ All those in the directory who had indicated that they had responsibility over assessment and were AACP members were identified. If more than one person was listed, the individual believed to have the highest academic rank was selected. An email that provided a brief overview of the research and an invitation to complete the survey online was sent to these prospective participants. The primary contact was encouraged to solicit input from others at the school with knowledge of the co-curriculum. A link to complete the survey online was provided, along with a document attached to make it easier to solicit feedback from others. Reminder emails were sent to nonrespondents at two, four, and eight weeks. Prospective participants who had not responded after eight weeks were contacted by phone by one of the investigators and asked to complete the survey using the online link.

Respondents were asked to identify any areas of concern for their programs in satisfying the ACPE co-curriculum requirement as well as challenges and barriers with respect to the design, implementation, and evaluation of their school’s co-curriculum. For survey items on areas of concern and challenges and barriers, a list of answer choices was provided and the respondent was able to select multiple responses. Another item asked the respondent to rate their confidence in their program’s ability to satisfy the ACPE co-curriculum requirement (1=unconfident, 10=very confident).

Institutional demographic data (ie, year founded, institution type, cohort size for academic year 2017-2018, and most recent ACPE accreditation review-focused or comprehensive) were collected from AACP and ACPE.^16,21 These data were merged with the co-curriculum survey responses. For the data analysis, the survey respondents’ programs were compared to the target population of all ACPE-accredited programs. Respondents were not required to complete all survey questions for their program’s data to be included in the dataset. Responses were included for all items with a valid response.

During the data analysis, the researchers created categories for the challenges and barriers reported by survey participants to help them understand the types of challenges and barriers reported. The frequency of responses to the areas of concern and challenges and barriers were examined. Comparisons were conducted to determine whether any characteristics of the programs or co-curriculums were related to areas of concern, challenges and barriers, and confidence. To accomplish this, group comparisons were conducted based on the program characteristics of institution type (public, private), cohort size (<80, 80-120, >120), most recent ACPE accreditation review (before 2016, 2016 or later), and number of offices supporting the co-curriculum (0, 1 or more). The data for type, cohort size, and most recent ACPE review were publicly available, while the number of offices supporting the co-curriculum was taken from survey results of a previous study conducted by the authors.¹² The survey first asked the respondent which office was primarily responsible for the co-curriculum, and then a second question asked what additional offices (eg, record keeping, support implementing activities, tracking hours) supported the co-curriculum. Respondents were instructed to select all responses that applied.

Comparisons were conducted to identify the top three concerns and challenges and barriers reported and the confidence rating for the results. The chi-square test was used for all comparisons of areas of concern and challenges and barriers. The confidence rating variable was ordinal; therefore, nonparametric analyses were conducted. The Mann-Whitney U test was used for comparisons with two groups (institution type and most recent ACPE accreditation review), and the Kruskal-Wallis test was performed for cohort size, which included three groups. For each comparison, the p value is presented along with the effect size. Because the comparisons were conducted as explorative research and the group sizes and effect sizes were small, a p value of .1 was considered significant in order to be able to recognize possible differences. Cohen d interpretation of effect size was used where values less than 0.2 are considered very small; greater than 0.2 and up to 0.5 are considered small; greater than 0.5 and up to 0.8 are considered medium; and greater than 0.8 are considered large.²² All analyses were conducted using SPSS, version 23 (IBM).

RESULTS

Information on the programs that participated in the survey is presented in Table 1, along with demographic data for all ACPE-accredited programs. Programs that responded to the survey (107 of 143, 74.8% response rate) were representative of all ACPE-accredited programs in the areas of founding year, type of institution, student cohort size, and recency of last ACPE review.

Table 1.

Demographic Information on Accredited Pharmacy Programs That Participated in a Survey to Identify Challenges to the Implementation of the Co-Curriculum

The number and frequency of responses to areas of concern with satisfying the ACPE co-curriculum requirement and challenges and barriers to implementing the co-curriculum are presented in Table 2. The mean number of concerns reported was 2.2 (SD=1.5). The most commonly reported areas of concern included closing the loop (50.5%), assessment (50.5%), and documentation (49.5%). Few reported having an inadequate number of activities available as a concern (8.4%).

Table 2.

Areas of Concern with Satisfying the ACPE Co-curriculum Requirement and Barriers to Implementing Co-Curriculum Reported by Pharmacy Programs (N=107)

The mean number of challenges and barriers reported for implementation of co-curriculum was 4.0 (SD=2.9). The barriers were grouped into higher categories of inadequate school or college resourcing, inadequate external resourcing, lack of support from stakeholders, and lack of experience. These categories were created by the researchers after data were collected to help make sense of the types of barriers reported. At least one barrier was selected from inadequate school or college resourcing by 80.4% of respondents, inadequate external resourcing by 43.9%, lack of support from stakeholders by 59.8%, and lack of experience by 28.0%.

Comparisons by groups of the top three highest-rated concerns for meeting the ACPE co-curriculum requirement are presented in Table 3. For most of the comparison groups, there were not significant differences on the concerns. One difference that was that programs with no supporting offices for the co-curriculum differed in whether they reported “assessment of co-curriculum activities/plan” as an area of concern (p=.07, d=.18), with programs with supporting offices being less likely to select this concern (46.7%) compared to programs with no supporting offices (70.6%).

Table 3.

Comparisons on Top Three Concerns with Satisfying the ACPE Co-curriculum Requirement Based on Pharmacy Program Characteristics

Comparisons between groups on the top three highest-rated barriers to co-curriculum implementation are presented in Table 4. There were no significant differences among the comparison groups in terms of the challenges and barriers they faced with the exception of cohort size. Cohort size was significantly related to two challenges and barriers. The first was “faculty members/staff/committees not having time to establish and oversee the co-curriculum,” where programs with the moderate cohort size of 80-120 students selected this as a challenge or barrier more frequently (74.2%) than programs with a smaller cohort size (46.9%), and programs with a larger cohort size (50.0%). The effect size (d=.24) was small. The second was “insufficient FTE personnel to design, implement, or evaluate the co-curriculum,” where programs with moderate (64.5%) and large cohort sizes (61.9%) selected this as a challenge or barrier more frequently than programs with a smaller cohort size (34.4%). The effect size was small (d=.27).

Table 4.

Comparisons on Top Three Challenges/Barriers to Implementing Co-Curriculum Based on Pharmacy Program Characteristics

The ratings of confidence that the program was able to meet the ACPE co-curriculum requirement ranged from 2 to 10, with a mean rating of 7.9 (SD=1.6). The median was 8.0. Comparisons by groups found no significant differences in confidence rating (Table 5). Although not significant different, the mean confidence rating for programs with no supporting offices (M=7.4, SD=1.5) was lower than that of programs with one or more supporting offices (M=8.0, SD=1.5). This difference approached significance, with a very small effect size. A post-hoc power analysis suggests an observed power of only 0.38 for this chi-square test, suggesting that larger group sizes would be needed to detect a significant difference of this effect size.

Table 5.

Comparisons on Confidence in Satisfying Co-Curriculum Requirement by Program Characteristics Based on Pharmacy Program Characteristics

DISCUSSION

Findings from this study highlight several important themes regarding barriers faced by PharmD programs in implementing the co-curriculum accreditation requirement. There were multiple challenges and barriers to implementing the co-curriculum commonly identified. Overall, however, programs reported a high level of confidence in their ability to meet the requirements of the 2016 ACPE Standards for co-curriculum.

Relatively few programs reported concerns about having an inadequate number of co-curriculum activities or that the activities provided were not linked or mapped to education outcomes; however, slightly more programs reported the establishment of deliberate and intentional activities as an issue. The lack of concern reported on these issues may reflect that such activities were present in the form of student organization events or other program-level activities, like legislative days, prior to the accreditation requirement. Previously published studies in pharmacy that describe individual institutional co-curriculum models are largely comprised of activities and curricula that were in place prior to the accreditation requirement, suggesting that other programs may have been able to capitalize on their previous activities as well.^4,6,11

The most common concerns that programs had centered on the co-curriculum documentation and assessment process. This is not surprising as the 2016 accreditation standards formalized the need for the co-curriculum within pharmacy programs to have a coherent structure. In our study, some programs reported having an inadequate system for documentation. The literature suggests programs are using a variety of systems, such as experiential management systems, electronic portfolios, learning management systems, cloud-based spreadsheets for documentation and tracking co-curriculum activities.^6,7,12,23

A majority of programs reported concerns with documenting how the experiences advanced learning, closing the loop, and assessing the activities or plan. Many programs have adopted reflections and reflection rubrics systematically within the program for assessment of co-curricular learning outcomes, or they have selectively integrated assessments that are validated for use with certain affective domains, such as professionalism, to provide data.^23-25 Other programs have incorporated a co-curricular mapping process to identify skills obtained through involvement in a professional organization and to effectively describe how it links to the curriculum.¹ Regardless, many programs are trying to determine how best to enact the co-curriculum, provide rigorous assessment data for accreditation, and appropriately provide feedback to students. The one significant difference that our study found in the group comparisons was that programs with a primary office implementing co-curriculum and no supporting offices were more likely to select assessment of co-curriculum activities or plans as a concern (70.6%) than those with additional offices supporting the co-curriculum (46.7%). This finding may reflect secondary offices, such as assessment and/or advisement, working together to create a cohesive plan, including assessment.

In our study, the commonly reported challenges and barriers to successful implementation of the co-curriculum pertained to the lack of enthusiasm or buy-in from faculty, staff, and students; lack of a clear definition of co-curriculum; and faculty time and insufficient staff. The lack of enthusiasm from faculty, staff, and students may be the result of the other reported barriers (ie, lack of a clear definition and insufficient time and staff). The lack of clarity in what constitutes co-curriculum is not surprising given that Patel and colleagues recently observed the absence of a universal definition for co-curriculum in pharmacy education.² Even within higher education, the definition of co-curricular experiences is unclear. Suskie proposed that co-curriculum is an experience outside of the curriculum which enriches academic learning, yet the potential for confusion with extracurricular experiences and the need for integration and overlap with required academic components creates the lack of clarity.²⁶

The most commonly reported challenges and barriers identified in this study pertained to faculty and staff time. A previous study found that two-thirds of programs have a standing or ad hoc committee, subcommittee, or task force overseeing the co-curriculum, and that many programs have multiple offices involved in implementation of the co-curriculum.¹² Further, the study found that most programs use self-assessment or reflection, which typically require providing written or verbal feedback, as part of the co-curriculum. Often an advisor, course faculty, or professional staff is tasked with providing this feedback, which subsequently could take significant time to provide at a high-quality level useful to the student. Regardless of its usefulness or original intent, this research provides evidence of the ways in which the co-curriculum requirement could increase workload and why this may be the greatest challenge or barrier for programs. Applications to PharmD programs have decreased year-over-year for the past few cycles across the United States, potentially putting a strain on budgets to hire additional personnel to handle the increased workload.²¹ Additionally, faculty vacancies at US pharmacy programs have increased 22.2% from 2013-2014 (the first year for which such data are available) to 2017-2018.^17,27 Results from the 2019 AACP faculty survey showed that, nationwide, 34.2% of faculty disagreed or strongly disagreed that there was a sufficient number of pharmacy faculty and that 34.1% disagreed or strongly disagreed that the there was a sufficient number of staff to effectively address programmatic needs.²⁸ Similarly, 22.8% of faculty who participated in the survey responded that they spent too much time on service, while 17.8% indicated that they spent too much time on teaching. Given the uncertain outlook for student enrollment in pharmacy programs, strain on resources and budgets is likely to continue. These conditions will require creativity and innovation on the part of programs to ensure that accreditation requirements are met, even while working to keep faculty and staff workloads manageable.

In this study, significant differences were found between the program’s cohort size and the endorsement of faculty/staff not having time to establish and oversee the curriculum and insufficient personnel as challenges and barriers to co-curriculum. In the first, programs with a moderate cohort size (80-120 students) were more likely to endorse this barrier than programs with smaller or larger cohort sizes. While in the second, programs with a smaller cohort size were less likely than the moderate or larger cohort size to endorse this barrier. Parallel to the new co-curriculum requirement from ACPE, the overall landscape of pharmacy education is changing, with an approximately 30% decrease in applications from 2013-14 to 2017-18. It is uncertain how this change will impact programs’ cohort sizes and resources.²¹ Some programs have experienced a drop in overall class sizes, which may limit available resources for implementing programmatic changes. Resource concerns may be reflective of the fact that accreditation requirements have continued to increase over time while the number of students, and subsequently revenue, have decreased.

The finding that most programs had a high level of confidence was of interest because it provided insight into how the programs regard their ability to address the co-curriculum. Although there were no significant differences found based on the program variables that were examined, the variable concerning a program having supporting offices approached significance and may have been influenced by a lack of power to detect a small difference. This lack of power was in part because of the small number of programs that reported not having supporting offices (n=17). Programs with no supporting offices had a lower mean score on ability to satisfy co-curriculum requirement than those with supporting offices. These items are important elements in ensuring compliance with the co-curriculum, as programs must document the impact of the co-curriculum on the affective domains (Standards 3 and 4). One indicator of the accuracy of their confidence could be compliance with these standards during ACPE reviews. The ACPE released information regarding the status of the 46 comprehensive reviews of established pharmacy programs that were conducted between fall 2016 and fall 2019 using the Standards 2016 Rubric.²⁹ On Standard 4, which covers personal and professional development (an important aspect of the co-curriculum), 34 (73.9%) of these 46 programs were rated as compliant, nine were rated as compliant with monitoring (19.6%), and three were rated as partially compliant (6.5%). It would be interesting to determine whether the programs that had higher confidence in meeting the standards and supporting offices were more likely to be compliant, and this could be a future research initiative.

As pharmacy programs face the changing landscape of pharmacy education, the added service-centered aspects of implementation and assessment of the co-curriculum is a challenge that may impact faculty and administrator workload. The AACP faculty survey data have consistently demonstrated that the amount of time faculty report spending on service elements is too much, ranging from 21.1% to 22.8% in 2018-2020. Furthermore, this area had the highest endorsement of all workload areas covered in the survey (ie, service, teaching, research, and clinical practice).^28,30,31 Organizing, assessing, and delivering the co-curriculum could constitute an additional burden on faculty and administrator time.

The availability of program resources regarding co-curriculum is critical. The AACP Assessment SIG organized a working group in 2018 to bring together those involved in various aspects of the co-curriculum and to research how programs are addressing the co-curriculum. As a result, presentations and webinars have been hosted to share the findings and to inform institutions about various approaches to meet ACPE accreditation requirements. More recently, in 2019 an AACP Co-Curricular Community was started to provide a forum specifically for co-curricular sharing of ideas and to house literature and tools related to the co-curriculum. This community is currently working on a toolkit that will compile co-curricular resources to share with other programs.

Finally, the true cost of accreditation may be contributing to challenges faced by institutions in successfully managing and assessing the co-curriculum. Many institutions are using multiple software platforms for various aspects of accreditation, including the co-curriculum. Although Maerten-Rivera and colleagues determined that about 20% of pharmacy programs have appointed at least one individual with the title of director or coordinator of co-curriculum, it is unclear whether programs have allocated resources to create new positions or simply added responsibilities to existing ones.¹² The latter appears likely as previous research by Rudolph and colleagues determined that nearly 90% of administrators at the assistant or associate dean level and 50% at the director or assistant director level with responsibility for assessment in US pharmacy programs are also faculty members.³² With declining enrollment and new challenges in the era of COVID-19 (eg, distance or remote education), pharmacy programs may face new barriers in resource allocation for the curriculum in general, which includes the co-curriculum.

There are several limitations associated with this project. While a majority of programs (74.8%) responded to the survey, not all are represented in the sample. Second, the survey was distributed to assessment personnel with the recommendation to include other individuals as appropriate to gather the data and discuss results however this may not have been done. More detailed information regarding some survey items also would have been helpful to ascertain for example, knowing the faculty to student ratio may have been more insightful than cohort size as it would provide a better measure of resources per student. Finally, the survey was administered in 2018 before many programs had completed a self-study and accreditation review under the new ACPE standards and programs may have changed policies and practices since responding to the survey. However, given the lack of data in this area and the amount of time institutions typically take to prepare for site visits (usually one to two years), we believe these results are valuable to PharmD programs.

CONCLUSION

This study examined the confidence of pharmacy programs in their ability to meet co-curriculum requirements, their concerns about satisfying the requirement, challenges and barriers they faced to implementing the co-curriculum, and program characteristics that may impact these. Generally, programs were confident in their ability to meet the requirement. The most common concerns reported were documenting how the experience advanced learning, closing the loop, and assessment of the activities and plan. We found consensus among the programs in the common challenges and barriers they faced, which included lack of enthusiasm or buy-in from faculty, staff, and students; lack of a clear definition of co-curriculum; and faculty time and insufficient staff. The results suggest that having a primary office involved in co-curriculum and supporting offices may reduce the burden in some areas. Similarly, student cohort size may have an impact on the challenges and barriers, particularly those related to faculty and staff. Further research is needed to determine if and how programs address these critical issues and to explore whether programs report differently on these areas after completing an accreditation review. The study results may be useful to members of the Academy when evaluating co-curriculum, particularly when considering the resources needed for a co-curriculum model.