The assessment of clinical competence of physiotherapists during and after the COVID-19 pandemic

The coronavirus (COVID-19) pandemic has created significant disruptions to life and work for most industries, including healthcare. While the physiotherapy profession has moved to using telehealth for the assessment and treatment of many patients during the pandemic, there is understandable hesitation about using online media for assessing the competence of physiotherapists who wish to be registered to practise. This editorial seeks to: explore various existing models of remotely assessing clinical competence; consider their suitability with respect to the assessment of overseas-qualified physiotherapists to ensure that this can continue in times of disruption, as experienced during the pandemic; and pose questions about whether this should become the ‘new normal’.

The Australian Physiotherapy Council (the Council) rapidly adapted services in response to the COVID-19 pandemic. To maintain the Council's function of accrediting entry-level physiotherapy programs, on-site visits of physiotherapy programs at universities transitioned to virtual site visits, enabling universities to maintain their accreditation cycles as scheduled. This was vital to ensuring that the number of graduates qualifying during 2020 was unaffected. Similarly, the Council moved to deliver the computer-based, multiple choice assessment of overseas-qualified physiotherapists to remote invigilation, removing the need for candidates to physically attend assessment venues. However, transitioning the practical component (the clinical assessment) of overseas-qualified physiotherapists to an online medium requires thought and careful consideration because this assessment method has, for many years, involved demonstration of ‘hands-on’ application of clinical knowledge and decision-making through the physical assessment and treatment of a patient.

In recent years, the Council's clinical assessment underwent a major overhaul with the replacement of hospital-based assessment to a simulation model. Since April 2019, the clinical assessment has been conducted in a simulated environment with standardised patients (actors) using the Council's purpose-built facilities in Melbourne, Australia. The impact of this change has been overwhelmingly positive, as not only did this provide the Council with a sustainable and scalable, not to mention robust, assessment model, but within 6 months of implementing simulation, the Council's waiting list for the clinical assessment was dramatically reduced from 24 to 2 months.¹ Despite this, as well as impacting the Council's ability to deliver the clinical assessment in a timely manner, COVID-19 has exposed a significant flaw with its current model, namely: reliance on the ability of the Council to host assessments within physical premises, and that of candidates to travel there. This has led the Council to consider how the clinical assessment can be improved to be more resilient, not only in the face of global health emergencies such as COVID-19, but other potentially disruptive events such as civil unrest or environmental crises.

To this end, the Council conducted an extensive environmental scan and analysis of existing remote practical assessment models for their adaptability with respect to the clinical assessment. One such method is an Objective Structured Clinical Examination (OSCE) delivered remotely. Several medical training and assessment bodies use OSCE-style summative assessments as a means of standardising clinical skills. Specialist medical colleges, for example, often employ OSCEs to assess whether trainees have reached the standards required to progress to the next phase of training, or for specialisation in the relevant medical field resulting in eligibility for college fellowship.^{2 , 3} This is especially important where national standards regarding the training and assessment of workplace-based performance may not exist in the same way that the Assessment of Physiotherapy Practice (APP) tool does for the physiotherapy profession.4, 5, 6 As a result of COVID-19 some accreditation and assessment authorities have modified their OSCE to be delivered remotely – that is, without the candidate being in the same room as the assessors, patients or simulated patients.^{2 , 7} These modifications fall into two categories: one where attendance at a physical venue is required, and another that allows for assessment to be completed from any site, using videoconferencing facilities.

A number of such organisations have required candidates to attend a venue within their state, territory or region. Typical venues have included the candidate's current hospital workplace, a nearby hospital, or another facility.² For example, in response to COVID-19, the Royal Australasian College of Physicians has modified the long case component of the Divisional Clinical Examination, which was designed to test clinical examination skills, and involved examination of a patient followed by discussion with examiners. This transitioned to a virtual patient interaction, whereby the patient was in a different room/location to the candidate and assessors, the latter of whom were also able to attend virtually.² Similarly, the Australasian College of Dermatologists and the Royal Australian and New Zealand College of Psychiatrists also adapted their long case viva-style OSCEs, with candidates attending state-based venues and interacting with patients via videoconference.^{8 , 9} Given the virtual nature of the modified OSCEs, physical patient examination was not required. In the Royal Australasian College of Physicians' modified exam, patient information, including results from physical examination findings, was provided to the candidate at the beginning of the assessment, while the Australasian College of Dermatologists used clinical photos in place of a physical examination.^{2 , 8}

Some specialist medical colleges have allowed candidates to attend OSCEs from their own locations. Given the nature of the specialty, the Australian College of Rural and Remote Medicine has had an option in place for some time for candidates to attend the Structured Assessment using Multiple Patient Scenarios virtually.¹⁰ This is a viva-style examination, with each scenario involving introductory case information and three questions relating to the case¹¹; it does not appear to involve any patient interaction. In October 2020, the Royal Australian College of General Practitioners replaced their OSCE, the largest clinical examination in the Southern Hemisphere,¹² with the Remote Clinical Exam (RCE), delivered entirely online by videoconference.¹² A major difference is that in the OSCE, candidates were assessed on their ‘ability to perform an appropriate and systematic examination that is appropriately focused and not overly inclusive,’³ whereas in the RCE, candidates ‘interact with a simulated patient over videoconference, but demonstration of physical techniques is not required’.⁷ There are also differences between the types of tasks completed in each examination, which is to be expected given the varying modes of delivery. The OSCE typically included the performance of discrete practical procedures, actual physical examination, vivas and entire patient consultations.³ The RCE, on the other hand, includes assessments such as: a case-based discussion, remote consultation with a simulated standardised patient, case-based discussion with description of a physical examination or procedure, and critical appraisal and case-based discussion with reference to a research paper.⁷

Whether the remote OSCE is attended physically or virtually, the overwhelming issue with this method is that the physical examination of patients (either real or simulated) is not required, or no patient interaction is required at all. It is also important to note that, in these examples, the OSCE most often takes place at the conclusion of years of workplace-based training. Recognition as a specialist physician by the Royal Australasian College of Physicians, for example, requires the completion of a minimum 6-year training program in the clinical environment.¹³ There will therefore be many opportunities for specialist trainees to demonstrate their clinical skills in real-life settings. Indeed, several training programs necessitate the completion of mini clinical evaluation exercises in the medical environment as part of their formative assessment requirements.^{14 , 15} Conversely, for many assessment bodies of overseas-qualified professionals (including the Council), a ‘snapshot in time’ summative assessment is the only method by which practical and clinical reasoning skills are evaluated. It is therefore difficult to translate examinations requiring physical demonstration of clinical competence to virtual delivery using the remote OSCE models.

Even prior to the pandemic, telehealth had become an increasingly popular method for physiotherapists and other health professionals as ‘an alternate means of providing all aspects of care including the interview, physical assessment/diagnosis, intervention, maintenance activities, consultation, education, and training to clients at a remote location’.¹⁶ Since the early 2000s, telehealth has emerged as an effective assessment and rehabilitation tool for patients,¹⁷ and more recently, telemedicine OSCEs (TeleOSCEs) as a formative assessment strategy. Interestingly, a study conducted by Cantone and colleagues involving the implementation of a TeleOSCE for medical students as part of a required clinical placement found that the TeleOSCE ‘could be used for… national assessments such as licensing examinations’.¹⁸

It is notable that the clinical scenario of Cantone et al's TeleOSCE study was a patient with insomnia, with depression as the underlying cause.¹⁸ This, therefore, did not require a full physical examination. As with medicine, there will be some conditions requiring physiotherapy intervention that lend themselves particularly well to a TeleOSCE-type summative assessment. For example, Mani et al reported good concurrent validity and excellent reliability of internet-based physiotherapy assessment of musculoskeletal disorders when compared with in-person treatment, with the exception of some conditions where this was low to moderate (eg, lumbar spine posture, orthopaedic special tests, neurodynamic tests and scar assessment).¹⁹ Evidence of the efficacy of telehealth to identify and treat cardiorespiratory and neurological disorders, on the other hand, is less clear. Indeed, there will be some instances where assessment and treatment of patients cannot take place virtually using a videoconferencing model of telehealth. As Professor Jenny Freeman noted in a recent interview: ‘how can quality of movement and balance after falls, for example, be assessed without an in-person appointment?’²⁰ Similarly, manual assessment and treatment is often necessary to feel anatomical structures, tissues, tension and spasm, which cannot be replicated virtually.²¹ As was the case with the move to simulation-based assessment, careful consideration would be required of the suitability of conditions to be treated via telehealth, as well as the validity and reliability of telehealth itself as a summative assessment tool.

There are emerging educational philosophies driving change in this space. Tai et al defined evaluative judgement as ‘the capability to make decisions about the quality of work of oneself and others’.²² As a necessary capability of graduates, this aligns with the Physiotherapy Practice Thresholds in Australia and Aotearoa New Zealand, which devote an entire ‘role’ (out of a total of seven ‘practitioner roles’ it defines) to self-directed reflection and learning.²³ Therefore, it may be possible to design an online viva-like assessment through which higher-order cognitive abilities, such as clinical reasoning and problem-solving skills, are demonstrated orally, and which incorporates examination of candidates' evaluative judgement capability. Methods to achieve this could include incorporating student-centred assessment methods such as consensus assessment, which requires input from both assessor and student. Unlike a traditional assessment, grading cannot be completed without both parties engaging in a reflective conversation about competent performance.²⁴ This approach provides an opportunity to identify ‘false positives’,²⁴ that is: candidates who may receive a pass despite ‘a marginal performance with limited understanding of the key elements, providing the case outcomes [are] satisfactory’,²⁴ whereas those who may make a judgement error, given the stressful nature of such a high-stake exam, without the opportunity to explain and offer a more appropriate course of action, are likely to fail the assessment.²⁴ Similarly, an assessment model that includes elements of evaluative judgement may reward those who have made errors but who, given the opportunity, will be able to identify, rationalise and correct these.²⁴

But what of technical competence? It is broadly accepted that technical competence, or the ability to use manual skills and observation for assessment and treatment, is a lifelong journey. Additionally, practitioners are bound to ‘recognise and work within the limits of their competence and scope of practice’.²⁵ Would it be sufficient for a candidate to provide an oral description of various techniques? Or could the assessment be flipped such that the candidate observes and judges the performance of another person executing manual assessment and treatment tasks using video recordings? Again, this would provide an opportunity to assess evaluative judgement capability, hopefully providing a sense of the candidate's ability to perform these tasks themself. Candidates' understanding of a practitioner's responsibility to work within their areas of competence and scope of practice would also need to be tested, which could be performed orally.

Finally, there may be a technical solution to this predicament. Russell identified sensor-based technologies as well as virtual environments/virtual reality as additional ways that physiotherapy patients could be assessed and treated. The former of these involves the utilisation of sensor equipment (such as tilt switches, accelerometers and gyroscopes) to assess movement within three-dimensional space, and the latter entails use of computer-generated three-dimensional settings for a similar purpose.¹⁶ Regardless of whether appropriate technology already exists, scientific validation of the suitability for summative assessment purposes would be necessary. Practical and cost implications would also need to be considered.

At present, a model of remote assessment that can be readily adapted to evaluate clinical competence at the same level as the Council's simulation-based clinical assessment does not appear to exist – the requirement for physical demonstration of skill remains a significant barrier to remote delivery of this. Notwithstanding the rollout of a COVID-19 vaccine, the Council will continue to explore methods of remote clinical assessment of overseas-qualified physiotherapists to ensure that this can continue in the face of any disruption. Like past modifications to the assessment process, any proposed new method will be rigorously tested to ensure its efficacy as a summative assessment tool. As others rapidly exploit existing and emerging technologies or develop new ones for a post-COVID world, so too must accreditation bodies by working with all stakeholders to find a ‘new normal’ for assessment in this era.