The Development of the Canadian Physiotherapy Assessment of Clinical Performance (ACP) 2.0 – Alignment with the 2017 Competency Profile

Abstract

Purpose:

Clinical education and assessment of students’ performance during clinical placements are key components of Canadian entry-to-practice physiotherapy curriculum and important in developing entry-level physiotherapy practitioners. The Canadian Physiotherapy Assessment of Clinical Performance (ACP) is the measure currently used to assess physiotherapy student performance on clinical placements in most of the entry-to-practice physiotherapy programmes across Canada. The release of the 2017 Competency Profile by the National Physiotherapy Advisory Group resulted in a revision of the existing ACP. The purpose of this study is to report the process used to develop a revised version of the ACP based on the 2017 Competency Profile, henceforth called the ACP 2.0.

Method:

Using a multistage process, we sought input from Canadian clinical education academics, an expert consultant panel, as well as physiotherapists across Canada using a questionnaire, meetings, and an online survey, respectively.

Results:

Twelve of 15 clinical education academics responded to a questionnaire. The expert consultant panel (n = 12) met three times. There were 144 physiotherapists who initiated the national, online, survey and met the inclusion criteria; 84 completed the survey. In the ACP 2.0, rating scales and comments boxes were grouped, and additional text was added to 12 items for further clarification. The ACP 2.0 came to have 18 items and 9 comment boxes in addition to summative comments, in contrast to the original ACP’s 21 items and 9 comment boxes.

Conclusions:

In November 2020, Canadian clinical education academics reviewed the proposed draft ACP 2.0 and unanimously accepted it for implementation in Canadian physiotherapy university programmes.

In Canadian entry-to-practice physiotherapy programmes, clinical education, and assessment of students’ performance during clinical placements are key components of the curriculum and play a vital role in the development of an entry-level physiotherapy practitioner. Clinical education in Canadian entry-to-practice physiotherapy programmes is guided by two core documents: the Clinical Education Guidelines,¹ which identify learning needs to achieve entry-level status; and the Accreditation Standards for Physiotherapy Education Programs in Canada, established by Physiotherapy Education Accreditation Canada (PEAC), which set standards for the education programmes in physiotherapy.² The Guidelines recommend a minimum of 1025 hours of the academic programme must take place in clinical placements, and at least 80% of these hours must be in settings that provide direct patient care.

All Canadian entry-to-practice physiotherapy programmes have one or more individual(s) assigned to oversee the implementation and delivery of the clinical education component of the curriculum. Typically, these individuals are referred to as the Academic Coordinator for Clinical Education (ACCE) or Director of Clinical Education (DCE) (for the purposes of this paper, the terms ACCE and DCE will be used interchangeably). Nationally, the ACCEs/DCEs from each programme, along with physiotherapy regional coordinators from northern Ontario and from Newfoundland and Labrador, form the National Association for Clinical Education in Physiotherapy (NACEP), and work collaboratively to identify common resources and standards for physiotherapy clinical education. The NACEP is part of the Canadian Council of University Programs in Physiotherapy (CCPUP).

The Canadian Physiotherapy Assessment of Clinical Performance (ACP) is the measure currently used to assess physiotherapy student performance on clinical placements in the majority (13) of the 15 entry-to-practice physiotherapy programmes across Canada. The ACP was developed between 2012-2014³ and there is evidence to support the internal consistency reliability, construct validity, and practicality of the ACP.^4,5 It was implemented nationally in 2017. The original ACP was based on the Canadian Essential Competency Profile for Physiotherapists (ECP) published in 2009,⁶ and included a modified rating scale based on the Revised Physical Therapy (PT) Clinical Performance Instrument (CPI) (version 2006), used with permission from the American Physical Therapy Association.⁷

The ACP is available in both French and English, in both online and paper formats. A cross-sectional, national analysis of data collected from the ACP from 10 Canadian entry-to-practice physiotherapy programmes, accounting for 3,290 discrete placements, generated typical “performance profiles” (profiles) for each of the 21 evaluative items on the ACP⁸ In this study, the placement data (97% of all data), used to generate the profiles was generated from placements that occurred within 27 months of the student’s programme start.⁸ The trajectories for each of the 21 profiles revealed only subtle differences amongst items and showed that students trained in Canadian entry-to-practice physiotherapy programmes are generally successful in completing their placements at all levels, and that these students tend to meet entry-level competency by 24 months in their programme.⁸ This analysis provides support for the utility of the ACP in the assessment of student performance on clinical placements.

In 2017, the ECP⁶ was updated, and published as the Competency Profile for Physiotherapists in Canada (CPP).⁹ A roles-based framework was retained from the 2009 Essential Competency Profile and is used as the organizational principle of the CPP The Essential Competencies and Entry-to-Practice Milestones are set out under seven domains of physiotherapy practice.⁹ The CPP⁹ is now the foundational document that describes essential competencies required of a physiotherapist in Canada throughout their career, as well as milestones expected by a physiotherapist when entering the profession. Changes in the updated CPP include: concepts under the former role of advocate are now included within the domain leadership; entry-to-practice milestones replaced key competencies; all roles were replaced by domains (e.g., communicator replaced by communication; manager replaced by management, etc.).^6,9 The update of the ECP to the CPP resulted in the need to revise the existing ACP.

The purpose of this paper is to report on the multistage process of using the 2017 CPP to develop a revised version of the ACP henceforth called the ACP 2.0.

Method

The development of the ACP 2.0 included several steps described below and summarized in Table 1.

Table 1.

Summary of Steps to Develop the ACP 2.0

Step	Objective	Who	What
1	Gain consensus on guiding principles for updating the ACP	NACEP members	Questionnaire
2	Using the guiding principles from step 1, determine the items that should have a rating scale and the location of the comment boxes to develop a draft ACP 2.0	Expert consultant panel	Two virtual meetings in April 2020
3	Gather national input on the draft ACP 2.0	Physiotherapists across Canada who met the inclusion criteria	Online survey administered in summer and fall 2020
4	Review results of the national survey and make final changes to the ACP 2.0	Expert consultant panel	One virtual meeting in October 2020
5	Vote regarding the acceptance of the draft ACP 2.0	NACEP members	Virtual meeting November 2020

Step 1

In step 1, we requested feedback electronically in a questionnaire from NACEP members to gain consensus on guiding principles for updating the ACP in November 2018. The questionnaire asked if the rating scale, summative comment boxes, and grade recommendation options by the clinical instructor (CI) should remain the same in the ACP 2.0 as in the original ACP. The questionnaire also asked NACEP members to provide their perspectives about the number of items and comment boxes in the ACP 2.0 that would be sufficiently informative about student performance without overburdening the CIs.

Step 2

In the second step, we convened an expert consultant panel. Half the panel constituted of NACEP members and the other half site coordinators, CIs, and recent physiotherapy graduates. We sought to have national representation and varying areas of clinical practice represented in the panel. To develop the expert consultant panel, we invited NACEP members to volunteer to participate, as well as to provide suggestions for other panel members who would represent CIs, centre coordinators for clinical education, and individuals who worked in a variety of practice areas. Once we received NACEP volunteers and recommendations for other panel members, we identified there was a lack of representation from eastern and central Canada, as well as from private practice. To address these gaps in representation on the expert panel, the project lead (BM) emailed specific NACEP members asking for their participation or suggestions of other clinicians from private practice to help attain representation regarding geography and clinical areas of practice.

The primary purpose of the expert consultant panel was to consider the guiding principles for ACP revisions agreed upon by NACEP (step 1) and determine which items in the CPP should have an associated rating scale — essentially creating the draft ACP 2.0. Prior to the first meeting, we asked panel members to independently review the CPP and submit ideas for potential grouping of items. Following this, two virtual meetings were facilitated by BM and SW to review the aggregated results of each panel member’s individual submission on the ideas for grouping of items and comment boxes. Both meetings were structured to present the aggregated results for each domain of the CPP and followed by open discussion among panel members to arrive at consensus on how items and rating scales would be grouped and the location of comment boxes. The goal of step 2 was to generate a draft ACP 2.0 that would go forward to step 3.

Step 3

We sought national input on the draft ACP 2.0 using an online survey (Qualtrics Survey Software) administered in a similar manner to when national input for the original ACP was sought.⁴ The purpose of the survey was to gain input from practising physiotherapists across Canada regarding the groupings of items, rating scales, and comment boxes in the draft ACP 2.0, as well as questions regarding its overall use in student assessment.

To participate in the survey, individuals had to be a registered physiotherapist who had supervised and assessed a PT student in clinical practice with the ACP in the last 24 months. The survey had three sections. In Section 1, for each domain of practice, participants were asked to respond to the following question, with the response options strongly agree, agree, unsure, disagree, strongly disagree: “Consider the last student you supervised. Based on the groupings of the rating scales and comment boxes, I would be able to effectively assess my student’s performance for the Physiotherapy Expertise domain in clinical practice.”

Section 2 asked participants for their overall opinion about the draft ACP 2.0 and if it would allow them to adequately identify a student who was performing poorly or very well. Section 3 included demographic questions to allow us to describe the survey participants. There were opportunities to enter comments throughout the survey.

The online survey was available in English and French. Invitations to participate, which included the survey link, were distributed twice, through the Canadian Physiotherapy Association’s (CPA) newsletter “What’s Moving at CPA.” In the first distribution, the survey was available for four weeks from July 2-August 21, 2020, and a reminder was included in the newsletter after two weeks. For the second distribution, the survey was available from September 24-October 9, 2020. Step 3 was granted approval by the Health Sciences Research Ethics Board of the University of Toronto.

Step 4

In step 4, we invited the expert consultant panel to review the results from the national survey to make final revisions to a revised draft of the ACP 2.0. It was this version that would be proposed to NACEP for national approval in step 5.

Step 5

In November 2020, NACEP members were emailed the revised draft of the ACP 2.0 in advance of a virtual meeting. At this meeting, the lead investigator (BM) presented a brief overview on the steps taken to develop the ACP 2.0, invited open discussion about the APC 2.0, and a vote was called for the acceptance of the ACP 2.0.

Results

Step 1

We distributed a questionnaire to all NACEP members to establish agreement on the guiding principles for the development of the ACP 2.0. Twelve of 15 Canadian physiotherapy university programmes responded to the questionnaire that was available for 3 weeks in 2018. All responding NACEP members agreed the summative comment boxes at midpoint and final assessments, as well as the rating scale used in the original ACP, should remain unchanged. Additionally, all questionnaire respondents agreed the ACP 2.0 should not be longer than the current ACP. Most (91.7%) of the programmes also agreed the grade recommendations by the CI at the end of the ACP should remain unchanged. Participants agreed the ACP 2.0 should have a similar number (41.7%), fewer (25.0%), or similar or fewer (33.3%) items than the ACP. Similarly, participants agreed the ACP 2.0 should have a similar number (58.3%), fewer (8.3%), or similar or fewer (33.3%) comment boxes than the ACP.

Step 2

We gathered an expert consultant panel to guide the research team in the development of a draft ACP 2.0. In addition to the two authors (BM and SW), the expert consultant panel had 10 members: 4 from NACEP 2 physiotherapy clinical education site coordinators, 2 CIs, and 2 recent graduates. These individuals lived in British Columbia (1), Saskatchewan (1), Manitoba (1), Ontario (4), Quebec (1), Nova Scotia (1), and Newfoundland and Labrador (1). Six panel members worked in clinical environments: 5 from the public sector, and 1 from the private sector.

The expert consultant panel met twice for 90 minutes each time. During these meetings, the panel came to consensus on the grouping of items and comment boxes for each domain. In the first meeting, the first four domains of the CPP were discussed; in the second meeting, the remainder of the domains were reviewed. In addition, the panel developed text to enhance clarity regarding four items in the draft ACP 2.0 (Box 1).

Box 1:

Items with Clarifying Text (Italics)

• 1.6 Complete or transition care. Transitioning patients to weekend care, or another PT or health care provider, or transitioning patients from the student’s caseload back to the primary PT can also be included with this item.

• 3.4 Contribute to conflict resolution. “Conflict” can represent a variety of situations (e.g., setting discharge dates; a patient’s willingness to participate in treatment session; potential treatment approach).

• 4.5 Supervise others. This may include physiotherapist assistants, rehabilitation assistants, caregivers, family members or other health care professionals.

• 6.5 Contribute to the learning of others. This can include patients, families, caregivers, peers, and colleagues (e.g., as in a student presentation).

After the two meetings, the panel members electronically reviewed and approved a draft of the ACP 2.0, which included 18 rating scales and 9 comment boxes. These aligned with the guiding principles previously generated by NACEP members. This version of the draft ACP 2.0 moved to step 3.

Step 3

In step 3, we reviewed results from the national survey of physiotherapists who had used the ACP in the previous two years in a CI role. We obtained their input on the structure of the draft ACP 2.0, and its overall use in student assessment.

Describing the survey participants

It is difficult to determine how many people received or would have been forwarded the CPA newsletter. However, 161 individuals consented to participate (28 of them accessed the French version) in completing the survey, and 144 self-identified as meeting the inclusion criteria and started the survey. Twenty-seven participants who met the inclusion criteria (one person did not meet the inclusion criteria) accessed the French version of the survey. Of the 144 who met the inclusion criteria, 44 completed the survey section on the inclusion criteria, and 16 participants did not progress to answer any other questions. Comments entered in French were translated and reviewed by the researchers and expert consultant panel.

Eighty-four participants completed questions pertaining to the content of the ACP 2.0 and demographic information, of which 19 (22.6%) of the total sample completed the survey in French. The average age of survey participants was 40.9 years, and participants graduated an average of 16.2 years ago. The majority (57.1%) of participants had supervised between one and five students in the last five years. Geographically, 31.0% and 23.8% of participants were working in Ontario and Saskatchewan, respectively, and most (68.7%) participants reported working in a large urban centre. Of the participants who answered demographic questions, 40.7% worked in acute care hospitals, and a wide range of areas of practice were represented, including mixed caseload (21.8%), musculoskeletal (20.2%), and neurological (14.9%). Tables 2 and 3 provide additional details about survey participants.

Table 2.

Characteristics of Step 3 Survey Participants – Demographics

Age (n = 80) (mean ± standard deviation; minimum-maximum)
40.9 years ± 9.4; 26-73 years
Years since graduation (n = 84) (mean ± standard deviation; minimum-maximum)
16.2 years ± 9.7; less than one year-47 years
Number of students supervised in the last 5 years (n = 84)
1–5 students	48 (57.1%)
6–10 students	24 (28.6%)
11–15 students	5 (6.0%)
16–20 students	1 (1.2%)
More than 20 students	6 (7.1%)
Province you currently work in (n = 84)
Alberta	1 (1.2%)
British Columbia	12 (14.3%)
Saskatchewan	20 (23.8%)
Ontario	26 (31.0%)
Québec	16 (19.0%)
New Brunswick	2 (2.4%)
Prince Edward Island	2 (2.4%)
Nova Scotia	5 (6.0%)

Table 3.

Characteristics of the Step 3 Survey Participants — Type of Employment/Practice

Location of Where You Work (n = 83)
Large urban population centres (100,000+)	57 (68.7%)
Medium population centres (30,000–99,999)	11 (13.3%)
Small population centres (1,000–29,999)	13 (15.7%)
Remote (< 1,000)	2 (2.4%)
Age Group of Patients/Clients You Primarily Work With (n = 84)
0–18 years	12 (14.3%)
19–65 years	13 (15.5%)
66 and older	17 (20.2%)
Mixed ages	42 (50.0%)
Areas of Practice (n = 188; Responses in Multiple Categories were Permitted)
Mixed caseload	41 (21.8%)
Musculoskeletal or orthopaedics	38 (20.2%)
Neurology or neurosciences	28 (14.9%)
Cardiopulmonary	18 (9.6%)
General rehabilitation	17 (9.0%)
Pain	10 (5.3%)
Hand therapy or plastic surgery	5 (2.7%)
Oncology	4 (2.1%)
Burns and wound care	4 (2.1%)
Rheumatology	4 (2.1%)
Médecine Générale*	2 (1.1%)
Administration/Recherche*	1 (0.5%)
Other (Pediatrics (8), Critical care (2), ICU, Vestibular, Trauma, Enseignement Universitaire, Amputés du membre inférieur, blank)	16 (8.5%)
Your practice setting (n = 108; responses in multiple categories were allowed)
Acute care hospital	44 (40.7%)
Rehabilitation hospital or facility	25 (23.1%)
Private practice	15 (13.9%)
Long-term care facility	6 (5.6%)
Community care or home care	4 (3.7%)
Administration or research or education	2 (1.9%)
Community health centre / family health team	2 (1.9%)
Insurance sector	2 (1.9%)
Other (Charge de cours à l’Université, Community Accessible Rehabilitation, Child Development Centre, Pediatric Outpatients, Outpatient family practice and physiatry, Tertiary Rehab, CIUSSS (Centre Intégré Universitaire en Santé et Services Sociaux), blank)	8 (7.4%)

^* These response options were only available on the French version.

Survey responses

Figure 1 shows most CIs “strongly agreed” or “agreed” that the grouping of rating scales and comment boxes would allow them to effectively assess a student’s performance for the domains of physiotherapy expertise (92.0%), communication (95.7%), collaboration (84.6%), management (83.3%), leadership (64.4%), scholarship (90.8%), and professionalism (95.3%). Participants also “strongly agreed” or “agreed” the ACP 2.0 would allow them to identify a student who was performing poorly (84.5%), as well as a student who was performing well (88.1%) (Figure 2).

Figure 1.

Confidence in assessing the student for each domain.

Figure 2.

General impressions of the ACP 2.0.

In addition to collecting quantitative data, the survey also requested comments on how a CI’s ability to effectively assess the student’s performance for each domain could be improved. Survey respondents provided comments for each domain, with the leadership domain receiving the most feedback. Box 2 provides examples of the comments included in the national survey responses.

Box 2:

Direct Quote Comments From The National Survey

• Definitions for quality improvement strategies and use of resources. Difficult what is exactly meant by this/does it really apply to students (Management domain)

• This is a vague competency that I find difficult to assess due to its wording. The descriptors are using “buzz words”. Champion the health needs of clients, promote innovation in healthcare and embrace social responsibility as a health professional. As a student, I wouldn’t have a clue where to start, and as a preceptor, what things am I wanting to see from my student? Maybe if there are examples along with the descriptors. e.g., does embrace social responsibility mean behaving in public? not posting on social media? (Leadership domain)

• I agree with the need for this category as well as the groupings but find this area difficult to assess in a 5-week placement (Leadership domain)

• More specifics for evidence-informed inquiry e.g., using many different sources to inform practice, and being able to determine if the sources are credible (Scholarship domain)

• (Translated from French) I find the new goal 7.5 a bit touchy … It is somewhat of an interference in the personal lives of the trainees. I don’t see why it should be included as a goal (several physiotherapists would not even attain it … and I cannot see myself telling a student to stop their personal activities during their internship, for example … ) (Professionalism domain)

• If it [rating scale] is still in any way based off % of caseload I think it is a poor measure. % of caseload is not an accurate depiction of what a student’s skill set is. If I have a student in acute care, ICU paediatrics for only 5 weeks they could be the best student I’ve ever had but would score poorly due to % of caseload.

• There should be a “not applicable” mark on the rating scales.

Step 4

The national survey results, including the qualitative comments, were reviewed by the expert consultant panel, who integrated these results to develop the version of the ACP 2.0 that would be reviewed by NACEP members for final approval. Through discussion about the feedback received from the national survey, the expert consultant panel decided to add examples to clarify some additional items highlighted in Box 3. The panel also decided to add clarifying text for some words such as “champion” (item 5.1), “evidence-informed” (item 6.1), and “currency” (item 6.4).

Box 3:

Additional Items With Clarifying Text (Italics)

• 4.4 Engage in quality improvement activities. For a student in clinical education, this might include reorganizing the day to better meet patients’ needs and the needs of the team; sharing ideas that would improve the patient flow or improve team communication, developing or revising a patient education handout.

• 5.1 Champion (advocate for) the health needs of clients. (e.g., advocating for a diagnostic test for a patient; collaborating with the team for a discharge date/location that meets the patient’s needs; advocating for other health care professionals to be involved with patient; explaining the value of PT to clients new to PT, to referring and collaborating health care providers; identifying resources in the community to promote the health and physical activity of clients and client groups)

• 5.2 Promote innovation in healthcare. (e.g., sharing new approaches in response to a system change; use of virtual / telerehab care as required; liaising with community resources for programme development to address the health needs of a client group)

• 5.3 Contribute to leadership in the profession. (e.g., advocating for the physiotherapy profession, presenting in-services, educating other health care professionals, students or groups such as volunteers)

• 6.1 Use an evidence-informed approach in practice (e.g., multiple and credible sources to inform practice).

• 6.4 Maintain currency (up to date) with developments relevant to area of practice.

• 7.3 Embrace social responsibility as a health professional. (e.g., discussing how an individual’s social determinants of health impacts the care they are able to receive; making recommendations that are appropriate to the patient’s environmental, personal, or cultural factors; adhering to public health guidelines; identifying barriers for clients and contributing to solutions)

• 7.5 Maintain personal wellness consistent with the needs of practice. (e.g., approaches learning with a growth mindset, identifies when situations are stressful, asks for help when needed)

The panel did not change the grouping of rating scales and comment boxes but did decide to make scoring all items on the ACP 2.0 required as opposed to the original ACP where item 4.2 was not a mandatory item to be scored. The panel also added clarifying text in the instructions on how the rating scale applies to items beyond the physiotherapy expertise domain. The panel reviewed the version of the ACP 2.0 that incorporated the feedback from the national survey and approved this version to move forward to step 5.

Step 5

At the November 2020 NACEP meeting, 27 members were present, and all voted to accept the proposed version of the ACP 2.0 (Appendix 1) for implementation in physiotherapy university programmes in Canada.

Discussion

Clinical education is an essential component of entry-to-practice physiotherapy programmes.^1,2,10 The aim of clinical education in entry-to-practice preparation is to assist students in developing behaviours and skills that are requisites of the profession, and to prepare students to become competent and autonomously practicing entry-level practitioners.¹¹ The ACP was developed in 2015 to provide a standardized measure to capture the Canadian entry-level competencies for physiotherapists.³ At the time, it was developed based on the 2009 ECP for physiotherapists in Canada⁶ which described the essential competencies (i.e., the knowledge, skills, and attitudes) required by physiotherapists at the beginning of and throughout their careers. The ECP was also constructed to reflect the diversity of physiotherapy practice in Canada and support evolution of the profession in relation to the changing nature of practice environments and advances in evidence-informed practice.⁶

The ACP was implemented across the majority of Canadian entry-to-practice physiotherapy programmes, and has provided a mechanism to standardize the assessment of clinical placement performance across the country.⁸ The benefits of a unified approach to the assessment of physiotherapy placements have included: the ability to analyze and compare placement performance evaluation data on a national scale; increased portability of students to complete placements across Canada, as practitioners can apply the same evaluation tool with minimal additional training; and establishing a common language for ACCE/DCEs to use when meeting and discussing placement expectations.⁸

The multistaged approach used in this study to develop the ACP 2.0 ensured the revisions to this national tool were based on feedback from stakeholders from across Canada and from a variety of practice areas. Every attempt was made to ensure the individuals involved in the expert panel and national survey were reflective of the physiotherapy profession in Canada. For example, the expert panel included representation from most provinces, and the national survey was distributed on two different occasions by the CPA to invite participation. Data from the Canadian Institute for Health Information (CIHI) suggest the average age of practising physiotherapists across Canada is 42 years of age;¹² participants in our national survey were 40.9 years of age ± 9.4 years indicating a similarity between the national physiotherapy workforce and survey respondents. However, the members of our expert panel and sample of respondents to the national survey for the ACP 2.0 included more physiotherapists who worked in acute care or a rehabilitation hospital; this setting distribution is different from 2019 CIHI data showing that most Canadian physiotherapists practice in community settings.¹² As clinicians in the community are actively involved in supervising student placements, it is unclear why this demographic difference existed. It is possible that because the survey was administered during the COVID-19 pandemic, when many jurisdictions were working to establish if and how community-based physiotherapy services fit into the delivery of essential services, many physiotherapists who were practising in the community did not prioritize survey completion.

The engagement of NACEP members was instrumental to setting foundational principles on which these revisions were based. One of the earliest agreements by NACEP members in the renewal process was that the ACP 2.0 should be based on the updated Competency Profile, as this document guides curriculum development for entry-to-practice physiotherapy programmes in Canada. However, some responses from the national survey identified concerns with the language of the items in the tool developed to assess students in clinical education. The expert panel agreed that while this feedback from survey respondents was important to consider, re-wording of items from the Competency Profile was not within the scope of this project. The Competency Profile is a national document developed and vetted by a variety of stakeholders in physiotherapy⁹ and thus the wording of items in the ACP 2.0 was kept consistent with the Competency Profile. As such, the feedback about how to re-write items on the draft ACP 2.0 was instead used by the expert panel to generate clarifying text that was added throughout the ACP 2.0.

Survey respondents also recommended considering a “not applicable” rating option to the rating scale. However, the expert panel identified that due to the new grouping of items per rating scale in the ACP 2.0 at least one component per every evaluation scale would be applicable to the student’s performance at every clinical placement, and there were fewer items to rate on the ACP 2.0 compared to the ACP. Thus, the expert panel decided a “not applicable” was not warranted for any scale. However, the expert panel agreed that clarifying text added to the ACP 2.0 would help CIs understand each item’s applicability to all practice settings. There was also a discussion early in the renewal process by NACEP members and members of the expert panel in relation to modifying the rating scale to avoid having it be caseload dependent. However, the existing rating scale has been implemented widely in both the CPI⁷ used in the United States and Canada; evidence exists that supports the rating scale’s validity and reliability in the original ACP;^4,5 and there has already been a delay between the publishing of the Competency Profile and revising the ACP. Thus, NACEP members agreed early in the renewal process not to modify the caseload dependency component of the rating scale. Future research on the ACP 2.0 may include revisiting the rating scale and potentially developing and validating a new scale that does not include caseload within the descriptions for the rating levels.

However, to respond to concerns expressed from national survey respondents about the integration of caseload as an anchor for the rating scales, the expert panel added clarifying language to the scale anchors to indicate caseload expectations for students are based on expectations for early career clinicians in a specific setting and should not be compared to the caseload expectations for a senior or mid-career clinician with advanced clinical reasoning and caseload management skills. Having the opportunity to manage caseload expectations for physiotherapy students during clinical placements has been identified as a factor that influenced transition from student to new graduate physiotherapist.¹³ Specifically, new graduate physiotherapists in Australia identified opportunities to manage a larger caseload (when appropriate), as well as manage patients with complex conditions and prioritize caseloads would have facilitated the transition between student to new-graduate physiotherapist.¹³ As such, caseload management expectations during placements should not be under-emphasized in terms of importance in preparing students for transition to the workforce.

Now that the ACP 2.0 has been developed and approved by stakeholders across Canada, next steps will include developing training resources to support students and clinicians in learning how to integrate this tool into the assessment of clinical practice. Currently, resources to support the integration of the ACP include an asynchronous learning module available online (in French and English), sample comments and scenarios to guide clinical instructors on how to use the rating scale, as well as programme-specific resources, including expectations for minimum performance based on the student’s level of learning. NACEP members along with individual ACCEs/ DCEs will now need to establish a minimum set of training resources to ensure the effective implementation of the ACP 2.0.

Finally, a prospective plan for longitudinal data collection and analysis at the national level should be considered early in the implementation of the ACP 2.0 to allow for the development of growth curves or expected trajectories in student performance for students in entry-to-practice physiotherapy programmes in Canada. These curves could be helpful to a number of stakeholders, including students, so they are aware of how performance and expectations increase throughout entry-to-practice education; university programmes to help inform decisions about student progress and success; and regulatory bodies to understand the level of preparedness for practice that students in Canadian entry-to-practice physiotherapy programmes achieve by graduation. Once training resources are developed, the tool will be distributed nationally, and Canadian physiotherapy programmes invited to integrate the ACP 2.0 into the evaluation processes for clinical placements.

Limitations

Our project has limitations to be considered in the context of interpreting our results. First, the national survey requesting feedback on the draft ACP 2.0 was distributed during a global pandemic which may have influenced response rates and overall willingness of the physiotherapy workforce to respond. In the development of the original ACP, Mori and colleagues² had 259 responses to the national survey which requested feedback on the development of the tool. In this survey, only 84 participants responded to the invitation to provide feedback on the revisions to the ACP, and we had a lower representation from community-based physiotherapists. Step 3 of our project may specifically be at risk for respondent bias because the primary method of distribution was through the CPA, which may have excluded potential survey respondents who met the inclusion criteria but were not CPA members. Although we distributed this survey through the CPA’s newsletter, it is possible CPA members could have forwarded the invitation to physiotherapists who were not CPA members. Additionally, the physiotherapy university programmes in Canada could have forwarded the survey’s request for participation to their broader communities that would include a mix of CPA members and non-members. We are unable to discern if survey responses are representative of physiotherapists in Canada or only CPA members, because our survey did not ask if respondents were CPA members. However, the research team took many efforts to ensure feedback was solicited from a diverse sample, including repeating the national survey at two separate points, and actively recruiting members to the expert panel that represented diverse practice areas and different provinces across Canada.

Among those individuals who did respond to the national survey, there was also attrition between the start and end of survey completion. Specifically, there was a decrease in responses toward the end of the survey compared with the outset of the survey (i.e., a greater number of individuals completed the questions at the beginning compared to the end of the survey). However, despite the small sample size and attrition between the start to end of the survey, the inclusion criteria for survey completion were specific to individuals who had experience in clinical education, indicating those who completed the full survey were likely well informed about the need for robust assessment of student performance on clinical placements.

Conclusion

This paper presents the multistaged approach to revising the national tool used to assess physiotherapy student performance on clinical placements in Canada. The ACP 2.0 is based on the updated Competency Profile for physiotherapists in Canada⁷ and was generated with diverse stakeholder feedback. Training tools are being developed, so it is anticipated the ACP 2.0 will be widely available for programmes to use by January 2022.

Key Messages

What is already known on this topic

Assessment of students in clinical education is important. Evidence supports the internal consistency reliability, construct validity, and practicality of the ACP and thus the ACP was implemented nationally in 2017. The ACP has allowed us to identify typical performance profiles for PT students in university programmes across Canada.

What this study adds

The ACP 2.0 has been developed in a thorough manner which invited input from multiple stakeholders and reflects the current competency profile for physiotherapists in Canada.