Clinician's Commentary on Mori et al.

Epstein and Hundert² defined professional competence as “…the habitual and judicious use of communication, knowledge, technical skills, clinical reasoning, emotions, values, and reflection in daily practice for the benefit of the individual and community being served.” Professional competence includes dimensions such as cognitive, technical, integrative, context, relationship, affective (moral), and self-reflective dimensions, which needs to be demonstrated under highly variable circumstances.³ The CanMed model in Canada, similar to the original Competency Profile for Physiotherapists in Canada (CPP),⁴ describes the competencies of healthcare practitioners in terms of different roles namely, medical/physiotherapy expert, communicator, collaborator, leader, health advocate, scholar, and professional. Assessing professional competence, of which the dimensions also align with twenty-first century skills, poses a challenge to assessment in a real-life setting due to the complexity of it. Professional and clinical competence, therefore, involves human processes such as cognitive and social skills, which challenge conventional assessment approaches.⁵

Standardized testing enhances assessment reliability, but it compromises validity due to the controlled circumstances. Assessing competence under controlled conditions is not a valid indicator of performance under uncertain, changing, or challenging conditions graduates may encounter in practice. Longitudinal assessment of physiotherapy students is used in many institutions to assess performance of students under a range of circumstances, in different contexts, and dealing with different patient needs and spectra.³

The purpose of assessment in health care speaks to the trainee, the curriculum, the institution, and the public. For the trainee it provides useful feedback to foster future learning, habits of self-reflection and self-remediation, and promotes access to advanced training. In terms of the curriculum, assessment responds to lack of demonstrated competence, achievement of goals, foster course for curricular change, coherence, or cross-validation, and establishes the different levels of competence.² Clinical assessment also guides institutional and educational values and research and promotes faculty development. Most importantly, assessment has a public responsibility in the sense that it certifies the competence of graduates to enter practice and be in service of and take responsibility for individual patients and the health of the public.²

The importance of valid assessment tools and practices should therefore not be underestimated. A concern is that a recent systematic review by O’Connor and colleagues⁶ has emphasized the need and current lack of reporting on psychometric and edumetric properties of clinical assessment tools in physiotherapy. They suggested that existing tools should rather be adapted, developed, and validated, instead of creating new tools, in an effort to strengthen evidence timely and efficiently. Studies such as by Torres-Narváez and colleagues⁷ have started filling this gap by validating existing tools for clinical competency in physiotherapy. The study by Mori and colleagues¹ in 2022, published in Physiotherapy Canada recently, has also demonstrated how the process of development and adaptation of existing tools and evidence, can be used to ensure transferability of evidence-based practice into a diverse, advancing, and changing profession and context.

Development and adaptation are closely related and the distinction between the two is not always clearly defined. Researchers might use a hybrid approach to development and adaptation, while others suggest that adaptation to context should be happening during the development and evaluation phases.⁸ Adaptation of evidence-based procedures, programmes, or practices is often defined as the modification of existing procedures, programmes, or practices to meet the needs of the target population and/or new contextual circumstances. This makes the practice, programme, or procedure more feasible, responsive, and focused. It also increases ownership of the practice.⁸

Several adaptation frameworks exist, but still have their limitations.⁹ A new framework, the ADAPT framework, has recently been published to assist the systematic adaptation of interventions, procedures, programmes, policies, or practices to new contexts, and to encourage the transparent reporting to facilitate a synthesis on successful versus unsuccessful aspects of adaptation and implementation. The process of adaptation is broken down into five steps: assess, select, prepare, pilot, and implement.⁸

It was interesting to note how the study by Mori and colleagues¹ fit into this proposed framework of adaptation. The first step of assessment in the ADAPT framework involves input from stakeholders, understanding the current evidence-based practice, and determining the capacity to implement the proposed practice. It also compares new versus previous contexts. The publication of the updated version of the Canadian Essential Competency Profile for Physiotherapists (ECP) (upon which the ACP was based) as the Competency Profile for Physiotherapists in Canada (CPP) in 2017, necessitated an update of the Canadian Physiotherapy Assessment of Clinical Performance (ACP). There is evidence to substantiate the internal consistency, reliability, construct validity, and practicality of the original ACP. In the first step of the study by Mori and colleagues,¹ the authors consulted with stakeholders such as the National Association for Clinical Education in Physiotherapy (NACEP) to gain consensus on the guiding principles for updating the ACP. They also enquired regarding capacity, by determining the number of items and comment boxes to be included in the updated ACP 2.0 not to overburden clinical instructors with an unnecessary long tool for assessment.

The second step of the ADAPT framework refers to the decision to adopt, adapt, or select another practice to implement by consulting with the community or staff. In the study of Mori and colleagues,¹ this step overlapped with the third step of the ADAPT framework which involves the preparation or necessary changes to the chosen evidence-based practice. It involves determining what changes are needed based on the assessment step (aligning with Step 1 in Mori and colleagues’ study), making the necessary adjustments and keeping records of the process.⁸ These two steps of the ADAPT framework were well covered by Mori and colleague¹ by consulting with stakeholders from different geographical and clinical areas of practice to determine, by consensus, which items in the CPP should have an associated rating scale, and how the items and rating scales would be grouped (including the location of the comment boxes). National input sought in Step 3 of their study, as well as the expert consultant panel to review the results from the national survey (Step 4), ensured rigorous input on the internal logic of the process being adapted. Step 5 of Mori and colleagues’ study¹ involved the NACEP members to ensure adoption and buy-in of the ACP 2.0.

Common reasons for failure of transferability of evidence-based practices include contextual differences or adaptations which compromise important qualities, such as validity or reliability of the original ACP. Mori and colleagues¹ addressed contextual factors by involving stakeholders representing different geographical and clinical areas of practice. The findings from the first step, where most stakeholders agreed that the ACP 2.0 should not be longer or have more items or comment boxes than the original version, limited compromising the validity of the original version. The decision not to modify the caseload dependency of the rating scale, as well as the addition of clarifying text in Step 2 and Step 4 of the study, would most likely strengthen validity or limit compromising the validity of the ACP 2.0.

The next step of the ADAPT framework, as well as suggested by Mori and colleagues,¹ would be to build capacity, compile an implementation plan and do the necessary training of clinicians and students on how to integrate the ACP 2.0 into practice. Monitoring and evaluation, as proposed by the authors by means of growth curves or expected trajectories of student performance, will help to identify any area of improvement, or identify what is working well.

Systematic and accountable decision making, as well as the transparent reporting of the adaptation process as was portrayed in the study of Mori and colleagues,¹ can stimulate thinking, further research and innovation in other physiotherapy institutions, on how to ensure effective adaptation and transferability of existing evidence or assessment practices into new or advanced contexts, or diverse settings. Considering the advantages that adaptation frameworks pose, improvement of such frameworks should also be prioritized.⁹ Adaptation frameworks should be tested rigorously to assess their effectiveness and usability regarding uptake and improvement of implementation guidelines in different settings and for different evidence-based practices.