Exploring Record Keeping, Clinical Reasoning, and Practice Context: Peer Assessment Findings from the Perspective of Situational Competence

Abstract

Purpose: The College of Physiotherapists of Ontario developed its peer practice assessment (PA) process on the basis of the statutory requirements for quality assurance. We previously reported outcomes from physiotherapists who had two PAs. The aims of the current research were to identify areas of sub-optimal performance on the assessments and explore any associations with the physiotherapists' practice context. Methods: We examined scores from the PAs of all physiotherapists who had two unrelated PAs between 2004 and 2012 (n=117), and we examined assessment reports for those who had specific patterns of sub-optimal outcomes (n=22). We conducted qualitative content analysis on the assessment reports to identify areas deemed to need improvement and the contexts in which the physiotherapists practised. Comparisons of proportions were carried out using Fisher's exact test. Results: The most common area of sub-optimal scores was record keeping, followed by clinical reasoning as assessed by means of chart-stimulated recall. Record-keeping deficits were related to either clinical care or administrative requirements (e.g., documenting patient consent in the manner required by law). At the second PA, record-keeping deficits were predominantly administrative. Physiotherapists with sub-optimal outcomes disproportionately worked in private practice contexts (p=0.026). Conclusions: Ontario physiotherapists generally maintain high-quality practice. Regulatory bodies may consider developing support strategies for meeting professional standards that take practice context into account.

A chief objective of health care regulation is to monitor practice and minimize risk to the public.^1–3 Regulatory bodies typically address this objective by having both quality improvement programmes to improve all practitioners' conduct and professional conduct programmes to address those whose conduct has been the subject of complaint or other serious inquiry. Regulatory bodies may use the findings of their quality improvement programmes to discern the most common aspects of practice in which practitioners fall below professional standards. They may also identify criteria or performance indicators that can be used to judge quality of practice or distinguish characteristics that increase risk.^2–4

Several authors have noted associations between lower quality of care or conduct by health care professionals and their practice contexts.^2–4 The Quality and Outcomes Framework (QOF), used to evaluate the quality of general practices in England's National Health Service, was introduced in 2004.³ Ashworth and colleagues³ analyzed 5 years of data to identify the risk factors for general practices to score in the lowest 10% of QOF scores over 4 consecutive years; two key factors were being a sole or small practice (fewer than three practitioners) and being an older physician.³ In Ontario, an analysis of peer assessment results from the College of Physicians and Surgeons (1997–2000) showed that organizational context and health care system context affected physicians' performance scores; lower scores were associated with settings that had a higher volume of patient visits and contexts in which health care services were less available.⁴

The risk associated with practice context has also been reported for pharmacists: An analysis of the characteristics of British pharmacists referred for discipline by the Royal Pharmaceutical Society of Great Britain's Disciplinary Committee identified the highest risk factor to be sector of work.² Among physiotherapists, demographic variables such as age and years since graduation have been shown not to correlate with outcome on peer assessment of clinical practice⁵ or with poorer performance on the written component of the Canadian Physiotherapy Competency Examination.⁶ However, we were unable to find any peer-reviewed studies that had investigated possible associations between physiotherapists' practice context and quality of care or conduct.

The idea that practice context factors such as setting, number of colleagues, and changing practice environment influence maintaining quality of care is congruent with the model of situational competence.^7,8 This view of competence is patient centred and contextual, applies across the practice lifespan, and assumes that competence is time and situation dependent.⁸ Quality assurance (QA) programmes that consider where and how practitioners currently work reflect the principles underlying the model of situational competence. They contrast with QA programmes that repeatedly assess practitioners according to entry-to-practice standards.

To our knowledge, there is no peer-reviewed literature regarding physiotherapists with respect to the outcomes of a regulatory body's assessments of situational competence other than our previous work.⁹ Nor were we able to find peer-reviewed literature on the risk factors for physiotherapists who were found to be providing quality of care that had slipped below professional standards. However, it is plausible that the factors that influence physiotherapists' risk are similar to those described for physicians and pharmacists. We therefore investigated these issues using administrative data from a physiotherapy regulatory body's QA programme.

The College of Physiotherapists of Ontario (CPO) is one of the regulatory authorities for physiotherapists in Canada. By statutory authority, it is required to have a QA programme that provides mechanisms to ensure the quality of physiotherapists' practice and ongoing clinical competence.¹⁰ The CPO's current QA program of practice assessment (PA) is largely unchanged from when it was introduced in 2004. Miller and colleagues⁵ established the content validity and reliability of this process and indicated the need to identify the factors associated with sub-optimal outcomes on PA.

In a previous report, we described the outcomes of 117 Ontario physiotherapists who had undergone two PAs, typically 5–7 years apart.⁹ The majority of the physiotherapists assessed had optimal outcomes on both PAs. Of particular interest for the present analysis, however, were physiotherapists with sub-optimal outcomes. The aims of this research were to identify areas of sub-optimal performance in the first and second PAs, to gain insight into the factors contributing to sub-optimal performance, and to identify practice contexts that might increase physiotherapists' risk for any outcome decision other than successful completion. In this article, we highlight the key areas of sub-optimal performance on the PA and relate those findings to the model of situational competence. These research aims were intended to create information to assist the CPO in developing resources to help physiotherapists successfully complete any PA.

Methods

Ethical considerations

The data were gathered as part of the CPO's QA activities, and the current analysis represents a secondary use of the CPO's QA data. The PA reports written by peer assessors do not contain any directly identifying information. Unique identifier numbers on the documents enabled us to link two PA reports to the same physiotherapist as well as to examine whether any assessors were responsible for writing more than one report. These quality management (QM) identifier numbers are unrelated to the physiotherapists' registration numbers, and the CPO keeps the links between physiotherapists' registration numbers and QM numbers confidential. Thus, all the records provided to us had been de-identified. The project received ethics approval from the Queen's University's General Research Ethics Board.

College of Physiotherapists of Ontario's on-site practice assessments

The CPO's on-site PAs evaluate physiotherapists' knowledge of professional practice standards, their implementation of the standards in clinical practice, and their ongoing clinical competence. The CPO's QM program randomly selects approximately 5% of registered physiotherapists for on-site assessment each year.¹¹ During the on-site PA, the assessor considers whether the physiotherapist is meeting professional standards in the following areas: practice issues, business practices, record keeping, chart-stimulated recall of decision making in patient care, and evidence of ongoing learning in the physiotherapist's portfolio.¹¹ Record keeping refers to adhering to the principles and requirements set out in the CPO's professional standard on record keeping. Chart-stimulated recall is the process of assessing clinical reasoning by discussing current and past patient cases.^5,12–14 A peer assessor conducts each PA. We described the selection and training of peer assessors in our previous article.⁹

After the assessment, the assessors provide reports to the CPO. In the summary section, they are required to provide a detailed description of the physiotherapist's practice setting and employment status, omitting names and geographical locations. In the subsequent sections, the assessors provide scores in specific areas and narrative comments. The descriptors for score levels changed during the study period. However, the principles remained constant in that the assessor rates and describes whether a physiotherapist met all applicable CPO-established standards of practice (score=1) and, if not, the extent to which the physiotherapist's practice was below the expected standard. Scores of 2, 3, or 4 reflect progressively greater improvement needed to achieve the expected standard. The assessor provides detailed comments in response to standardized questions; the comments substantiate the selected score. If the score is sub-optimal, the comments explain where deficits were noted.

If any scores are sub-optimal, the physiotherapist receives a copy of the report and is invited to respond to the scores and comments. The physiotherapist may provide evidence to rebut the assessor's comments or to indicate how his or her practice has changed since the assessment. The CPO's Quality Management Committee (QMC) reviews the assessors' reports and physiotherapists' submissions, if applicable, and determines the outcomes (see the next section).

Most physiotherapists complete the PAs successfully,⁹ although a minority do not. CPO annual reports over multiple years beginning in 2005–2006 demonstrate that record keeping and chart-stimulated recall are the two areas in which physiotherapists consistently have the greatest likelihood of not meeting CPO standards.^15-20

Tabulating scores and selecting reports for detailed review

We describe the main study in detail elsewhere.⁹ In brief, we obtained outcomes for registrants who had undergone two PAs in the period April 2004–December 2012 from the CPO's QM database. As previously reported, we examined the assessors' scores and PA outcome decisions for the first and second PAs for 117 physiotherapists.⁹ For this study, we tabulated the assessors' scores across the areas assessed in the PAs (see the column headings in Table 1). PA outcomes were categorized as follows: A, all scores were optimal, and the report was considered complete; B, some scores were sub-optimal, but a review by the QMC determined that the PA had been successfully completed; C, some scores were sub-optimal, and the QMC determined that the PA had been completed but provided recommendations; and D, some scores were sub-optimal, and the QMC required remediation, practice enhancement, or reassessment (see Box 1).

Table 1.

Counts of Sub-Optimal Scores on Practice Assessment

		Area of Assessment,† No.
PA	Score*	Practice Issues	Business Practices	Record Keeping	Chart-Stimulated Recall‡	PISA§	PD§
PAs reported by the CPO, 2009–2012 (N=985)¶
n/a	Total	50	21	196	77	5	14
Results from physiotherapists who had undergone 2 PAs (n=117)
1	2	6	6	26	27	3	2
	3	0	0	4	8	–	–
	4	0	2	0	2	–	–
	Total	6	8	30	37 (23)	3	2
2	2	4	0	12	3 (2)	0	2
	3	0	0	0	0	–	–
	4	0	0	0	0	–	–
	Total	4	0	12	3 (2)	0	2
Results from physiotherapists whose PA documents were reviewed (n=22)**
1	2	3	2	7	10	1	1
	3	0	0	2	7	–	–
	4	0	1	0	2	–	–
	Total	3	3	9	19 (11)	1	1
2	2	4	0	12	3 (2)	0	2
	3	0	0	0	0	–	–
	4	0	0	0	0	–	–
	Total	4	0	12	3 (2)	0	2

^*Sub-optimal scores (2, 3, 4) indicate the increasing levels of improvement required to meet the expected professional standard.

^†Another area of assessment is the summary report. It is excluded here because this score is an overall rating, summarizing the other ratings, and because count data for it are not available from the CPO's annual reports.

^‡In each PA, this process is conducted for four to eight patient charts, and a rating is assigned to each one. The number of processes per PA is determined by the assessor. When two numbers are shown, the first number indicates the total number of processes that were scored 2 or higher, and the number in parentheses indicates the number of physiotherapists whose PAs had at least one score of 2 or higher.

^§For this area, assessors assigned one of only two possible ratings: present/evident (i.e., meeting standards), coded as 1, or not present/evident, coded as 2.

^¶The PA outcomes provided in the CPO's annual reports give only the counts of sub-optimal scores; they are not broken down by level. Data in this row represent the total from three years of annually reported information after subtracting the PA2 score counts; otherwise, the latter would have been included in publicly reported annual totals.

^**The counts of sub-optimal scores for PA2 are identical to those for the larger group (n=117) because achieving sub-optimal scores was the reason for being included in the reviewed group.

PA=practice assessment; PISA=Professional Issues Self Assessment; PD=professional development; CPO=College of Physiotherapists of Ontario; n/a=not applicable.

Box 1.

Outcomes and Decisions

Study Code	Description	Information Communicated to Member	Outcome Reported in CPO's Annual Report
A	All scores were optimal (i.e., 1 or 0). No concerns arose during the staff review of all documents. The QMC approved the results of the staff review and deemed the PA to be complete.	Completed successfully	Completed successfully (staff reviewed)
B	One or more scores were sub-optimal. The member was invited to make a submission before the QMC's review. The QMC reviewed all documents and deemed the PA to be complete.	Completed successfully	Completed successfully (QMC reviewed)
C	One or more scores were sub-optimal. The member was invited to make a submission before the QMC reviewed the PA. The QMC reviewed all documents and made specific recommendations to be communicated to the member. Risks to patients were deemed to be minimal, and the member was not required to submit further documentation to the QMC. The QMC deemed the PA to be complete.	Completed with recommendations	Completed with recommendations
D	One or more scores were sub-optimal: multiple scores at level 2, at least 1 score at level 3 or 4, or both. The member was invited to make a submission before the QMC reviewed the assessment. The QMC reviewed all documents and concluded that the member was not adequately meeting standards in at least one area. Risks to patients were identified, and the QMC required follow-up.	The QMC's requirements for remediation, practice enhancement, and/or reassessment*	Remediation, practice enhancements, and reassessments

^*The terms and processes used for addressing sub-optimal outcomes evolved over the study period. Because the study identified so few physiotherapists with such outcomes, they were combined into one category, D.

CPO=College of Physiotherapists of Ontario; QMC=Quality Management Committee; PA=practice assessment.

For detailed review, we selected reports from physiotherapists who met the following criteria: One PA had led to outcome D (n=9); the outcome of both PAs had been B or C (n=4); or the outcome had been A for the first PA and C for the second (n=1). (See Figure 1, dark-shaded diamonds.) Thus, we did a detailed review of PA reports from 14 physiotherapists. The rationale for reviewing these reports was that the physiotherapists had the lowest possible outcome on one PA, a drop in outcome from one PA to the next, or a persistently sub-optimal outcome. In keeping with the aims of our research, we sought to explore what situational factors might be associated with those outcomes.

Figure 1.

Flowchart of selected physiotherapists' PA reports. Notes: The dark-shaded diamonds indicate the 14 selected physiotherapists in which a PA led to outcome D (n=9); the outcome at PA1 was B or C, and the physiotherapist subsequently had outcome B or C at PA2 (n=4); or the outcome was A at PA1 and C at PA2 (n=1). The stippled diamond indicates the 8 selected physiotherapists for whom sub-optimal scores occurred only on PA2: outcome A at PA1 and outcome B at PA2. PA=practice assessment; PA1=first practice assessment; PA2=second practice assessment. Source: Modified from Norman et al.⁹

As data analysis proceeded and recurring factors related to practice context were identified in these 14 physiotherapists (see data analysis), we extended our document review. We identified a further 8 physiotherapists who had sub-optimal scores only on their second PA—that is, outcome A at PA1 and outcome B at PA2. (See Figure 1, stippled diamond). For these 8 physiotherapists, we reviewed only the summary sections of the PA reports. The rationale for reviewing these reports was that the physiotherapists had a drop in outcome, albeit a minor one, and we sought to explore whether we could identify any associated situational factors.

Data analysis

We tabulated the sub-optimal scores for both PAs for all 117 physiotherapists. The CPO had published counts of sub-optimal scores among all Ontario physiotherapists who had received PAs between 2009 and 2012 in its 2012 annual report.²⁰ Thus, we were able to compare the scores of the second PA for the cohort of 117 with those for all Ontario physiotherapists assessed in the same time period. In previous years, the CPO's annual report had not included the same level of detail, and therefore it was not possible to compare the Ontario-wide results with the cohort's results from the first PA.

We applied Fisher's exact test to the results for the cohort of 117 to determine whether there was a significant association between a sub-optimal record-keeping score and a sub-optimal score in chart-stimulated recall for both PA1 and PA2. The rationale for this was that these are the most common areas for sub-optimal scores, according to the CPO's data.^15–20 We also used Fisher's exact test to determine whether there was a significant association between practice context and having a sub-optimal score in either record keeping or chart-stimulated recall. The threshold for significance was p=0.05.

For the 22 selected physiotherapists whose PA reports were reviewed, there were several additional levels of analysis. First, we counted how frequently each assessment area was found to be sub-optimal—that is, a repetition of the process just described, limited to the subset of 22 physiotherapists. Second, we examined the text of the assessment reports using qualitative content analysis. Our process was similar to what others have described as conventional content analysis,²¹ fundamental qualitative description,²² or qualitative content analysis.²³ In this process, the study starts with observation, and the codes are defined during data analysis, derived from the data. The frequency of occurrence of the codes is used as a method of describing patterns.^21–23

For the first 14 physiotherapists, we extracted and coded information about their practice context from the summary section of the PA reports. We also searched all other sections of the reports to find any recurrent themes and created codes for those themes. Two authors (MO and KN) independently reviewed the PA reports from 3 physiotherapists to identify recurrent themes and develop codes. We refined the coding system, independently re-reviewed the 3 physiotherapists' PA reports, and reviewed 6 additional physiotherapists' PA reports. We achieved consensus on the coding, and one author (MO) completed the review of the PA reports for the 5 remaining physiotherapists.

After we had identified recurrent themes, we considered whether any categorization was emerging from the themes. Because record keeping was the most common area determined to need improvement^5,15–20 and five of our six themes had a strong connection to record keeping (see the Results section), we identified an emergent two-way categorization of assessors' comments related to record keeping, specifically whether the deficiency was principally about administrative requirements or about clinical details. One author (MO) re-reviewed all sections of both reports from all 14 physiotherapists to code and count any comments that related to record keeping according to our two-way categorization. Another author (KN) reviewed the comments extracted and confirmed that they agreed with our categorization.

For the other 8 physiotherapists, we also extracted and coded information about their practice context at each PA from the summary section of the PA reports. We also extracted and coded any of the assessor's comments in the summary section that pertained to the two-way categorization described earlier. The author who coded the PA reports for the 14 physiotherapists also coded the PA reports from these 8 physiotherapists, and the coding was again confirmed by the same other author.

Results

Figure 1 shows the number of physiotherapists in each outcome category at each PA. Physiotherapists with B, C, or D outcomes by definition had sub-optimal scores. Table 1 shows the sub-optimal scores (2, 3, or 4, whereby increasing levels of improvement were required to meet the expected professional standard) for all PAs performed by the CPO in 2009–2012, as well as for both categories of physiotherapists in our study for both PAs. Among all PAs conducted by the CPO between 2009 and 2012 (N=985), the most common areas receiving sub-optimal scores were record keeping and chart-stimulated recall, which yielded 75% of the sub-optimal scores (see Table 1, first section).^18–20 The predominance of sub-optimal scores in these areas is consistent with the findings for the 117 physiotherapists who had undergone two PAs: At PA1, these two areas yielded 67 of 86 (78%) sub-optimal scores (see Table 1, second section).

Among the 22 physiotherapists for whom PA reports were reviewed, record keeping and chart-stimulated recall yielded 28 of 36 (78%) sub-optimal scores at PA1 (see Table 1, third section). At PA2, record keeping and chart-stimulated recall were again the predominant areas receiving sub-optimal scores: 15 of 21 (71%). (For a full list of scores and outcome categories from both PA reports for all physiotherapists, see the online Appendix.)

At PA1, physiotherapists who had sub-optimal record-keeping results were also more likely to have at least one sub-optimal chart-stimulated recall result (Fisher's exact test, p=0.0026). However, at PA2, there was no significant association between sub-optimal scores in those two areas. (See Table 2.)

Table 2.

Association between Sub-Optimal Scores in Chart-Stimulated Recall and Record keeping

Practice Assessment	All Chart-Stimulated Recall Scores=1, No. (%)	At Least 1 Chart-Stimulated Recall Score>1, No. (%)	Fisher's Exact Test Result, p-value
1
Record-keeping score=1	76 (87.4)	11 (12.6)	0.0026
Record-keeping score>1	18 (60.0)	12 (40.0)
2
Record-keeping score=1	103 (98.1)	2 (1.9)	1.0000
Record-keeping score>1	12 (100.0)	0 (0.0)

For the 22 physiotherapists whose PA documents were reviewed, the 44 PAs were conducted by 36 assessors. No physiotherapist had the same assessor twice. Of the 22 physiotherapists, 5 self-referred to PA1 and were randomly selected for PA2. The other 17 were randomly selected on both occasions.

Qualitative content analysis of the full PA1 and PA2 reports from the first 14 physiotherapists uncovered six recurring themes leading to sub-optimal scores on the PA. One theme related to billing practices; five related to record keeping, chart-stimulated recall, or both. The billing practices theme was that physiotherapists inadequately monitored the billings submitted by their employers for work the physiotherapists had done. The five recurring themes associated with record keeping and chart-stimulated recall were inadequate documentation of consent, inadequate documentation of identifiers in the patient record, lack of evidence or lack of documentation of thorough assessment and ongoing reassessment, lack of documentation of an explicit analysis or clinical impression, and insufficient detail regarding interventions.

The two emergent categories identified for assessors' comments about deficiencies in record keeping were administrative requirements and clinical details. Administrative requirements included the first two of our five themes as well as any other administratively oriented comments. Clinically oriented details included the three other themes as well as any other related comments, such as those about patient goals. Of the physiotherapists who had sub-optimal record-keeping scores at PA1 (2004–2006), most had clinically oriented record-keeping comments; a few had administratively oriented comments (see Table 3, upper half). Fewer of those physiotherapists had a sub-optimal score on record keeping at PA2. Of the additional eight physiotherapists for whom only summary data were reviewed, seven had a sub-optimal record-keeping score at PA2. In six of the seven reports, the assessor had cited the reason for the sub-optimal score in the summary report; more of them had to do with administrative requirements than with clinically oriented details (see Table 3, lower half).

Table 3.

Categories of Assessors' Comments in Record keeping or Chart-Stimulated Recall

	PA
Category	1	2
Review of full reports from both PAs (n=14)
Any deficiencies leading to a sub-optimal score	12/14	7/14
Administrative requirements	3	2
Clinically oriented details	9	1
Both administrative requirements and clinically oriented details		4
Review of summary section of both PA reports (n=8)
Any deficiencies leading to a sub-optimal score	0 out of 8	7 out of 8
Administrative requirements		3
Clinically oriented details		1
Both administrative requirements and clinically oriented details		2
Unable to discern the type		1

PA=practice assessment

Among the 22 physiotherapists, 64% worked in a private sector context (see Table 4). Of the 13 physiotherapists to whom the QMC had directed explicit practice recommendations or required remediation or practice enhancement after either or both PAs, 85% worked in a private sector context. This contrasts with the CPO's published data for the same time period, which indicate that only 40%–50% of physiotherapists reported such contexts as their primary employment site.^15–20 Thus, physiotherapists working in a private sector context were more likely to have a sub-optimal outcome on the PA than those working in a public sector context (Fisher's exact test, p=0.026). From the descriptions, it seemed that some of the physiotherapists had changed practice sites between PA1 and PA2, but none seemed to have crossed between the private and public sectors. It could not always be determined from the assessors' descriptions whether a private sector context was a small company with few employees or a larger corporation with potentially more administrative support.

Table 4.

Practice Context at Practice Assessments 1 and 2

Practice Context	No. (%) in Private Practice*	No. (%) in Publicly Funded Facility†	Fisher's Exact Test Result, p-value
Requiring recommendations or remediation at either practice assessment	11 (84.6)	2 (15.4)	0.0260
Successful at both practice assessments (but with some sub-optimal scores)	3 (33.3)	6 (66.7)

^*Examples are owners, private contractors, and employees working in an outpatient clinic, long-term care facility, or home care.

^†Examples are hospitals and children's treatment centres.

Discussion

Most physiotherapists who participate in a CPO PA complete it successfully, and they are especially likely to do so if they have previously completed a PA.⁹ In the current study, we focused on the reasons for sub-optimal outcomes and whether there were any discernible features of the physiotherapists' practice associated with risk for sub-optimal outcomes. The most common areas for sub-optimal scores among the cohort of 117 physiotherapists who had two PAs were record keeping and clinical reasoning as assessed by chart-stimulated recall. In light of the published results for all PAs conducted by the CPO during the time frame of our study (2004–2012),^15–20 the cohort was similar to all physiotherapists undergoing PAs except that the physiotherapists in the cohort were more likely to have a successful outcome on PA2 in 2009–2012 than other physiotherapists undergoing their first PA in the same time period.⁹ Analyzing documents from a subset of the cohort produced two additional key findings. First, there were a variety of record-keeping deficiencies, some related to administrative matters and others related to clinical aspects of the patient records evaluated. Second, the frequency of sub-optimal outcomes was higher for physiotherapists in private practice contexts than for those in public sector contexts.

PAs in some form are mandated for practitioners in all Ontario colleges of self-regulating health care professions under the Regulated Health Professions Act.¹⁰ In an on-site assessment, the CPO expects physiotherapists “to demonstrate competence within the context of their practice environment and role description.”²⁴ Competence has variously been understood to be the attainment of specific knowledge, the ability to perform certain skills, the ability to meet standardized benchmarks on assessment scales, the ability to demonstrate a self-reflective process in practice, and, finally, the ability to meet specific quality standards in practice.²⁵

The model of situational competence, described by Klass⁸ and Handfield-Jones and colleagues,⁷ can provide insights into the CPO's PAs that yielded sub-optimal scores. In contrast to models of competence that predict stability or gradual decline over time, the model of situational competence describes how competence is context specific and influenced by many factors.⁷ Traditionally, health care practitioners have been considered to be competent when they complete their entry-to-practice examinations because their education has led them to acquire all the knowledge and skills they need to successfully provide competent care.^7,8 However, given the constantly changing and updating health care environment, the level of knowledge and skill that a practitioner has attained at entry to practice may eventually be insufficient. Competence will potentially fall below acceptable levels (see Figure 2a).⁷

Figure 2.

Models of competence: (a) the trajectory, or ballistic (attributional), model;^7,8 (b) the “on-the-ground voyage” model; (c) the voyage model relative to expected levels; (d) the voyage model for an area of central importance to a practitioner; (e) the voyage model for an area of minimal interest; and (f) the voyage model with an abrupt shift in the minimum professionally acceptable level.

Source: Figures 2a, 2b, and 2c modified from Handfield-Jones and colleagues.⁷ Reproduced with permission from John Wiley & Sons, Inc. © Blackwell Science Ltd. Figures 2d, 2e, and 2f were created by the authors.

As others have noted, competence more likely fluctuates across the practice lifespan as practitioners' practice situation and the health care environment change (see Figure 2b).⁷ Handfield-Jones and colleagues⁷ described the influence of critical incidents in which a practitioner was surprised or a clinical error may have occurred or been narrowly averted. Practitioners are motivated to augment their knowledge or skill by experiencing critical incidents or by identifying gaps, leading them to maintain at least a minimum professionally acceptable level of performance (see Figure 2c).⁷ By reflecting on challenging cases in their practice, for example, physiotherapists can recognize gaps in their knowledge base or treatment skills, which they can address by reading, consulting colleagues, or taking courses. In these ways, they can improve their professional performance in areas in which they lack sufficient expertise or knowledge. Ideally, competence is always maintained by ensuring that performance is above the minimum professionally acceptable level.⁷

More important, competence can be area specific, in that a practitioner may not be evenly competent across multiple skill areas. Professional interests may lead him or her to develop a particularly high level of performance in one or more areas (see Figure 2d). An example would be a physiotherapist who pursues extra post-professional training in orthopaedic manual therapy and ultimately achieves the designation of Fellow of the Canadian Academy of Manipulative Physiotherapy.

Low need or interest may indicate a lower level of performance in other areas (see Figure 2e). Physiotherapists working in environments in which they are unlikely to be called on to give first aid may have a lower level of performance in cardiopulmonary resuscitation than physiotherapists who work at sporting events. Equally, a physiotherapist who has a low interest in using electrophysical agents in treating injuries will have a lower level of performance when using them than a physiotherapist who uses them routinely. In addition, high performance may not be feasible or beneficial for either physiotherapist or patient; Figure 2e depicts such a situation. For example, the administrative aspect of record keeping is an area of performance in which the minimum personally acceptable level is unlikely to be far above the minimum professionally acceptable level.

Finally, changes in practice environments can cause abrupt shifts in the minimum professionally acceptable level of competence, and a practitioner can potentially fall below standards if he or she is not aware of those changes or does not take action to meet the new requirements (see Figure 2f). For example, changes in privacy legislation that alter requirements for documentation can result in a shift in the minimum professionally acceptable level for the administrative aspects of record keeping. If a practitioner does not pay attention to this shift, he or she may be determined to not meet the new standard.

When we found deficits in record keeping in PA2, they seemed more likely to be administrative than clinical. In 2007—that is, between PA1 and PA2 for the cohort under study—the CPO's professional standard for record keeping underwent a substantial revision.²⁶ This change in professional standard can be considered an abrupt shift in the minimally professionally acceptable level, as depicted in Figure 2f. In 2009–2012 when the 117 physiotherapists underwent PA2, most had adjusted their record keeping to meet the new standard; however, for those who did not, the most common deficit was a shortfall in the administrative details of record keeping, determined to need only minor improvement.

The situational competence model can also explain the improvement from PA1 to PA2 in all assessed areas (see Table 1). The first PA can be thought of as a formative experience, one that enabled the physiotherapists to calibrate their self-perception of performance and their minimum personally acceptable level with the minimum professionally acceptable level. It presumably increased the physiotherapists' awareness of what areas of their practice behaviour might have needed changing to keep up with changing professional standards. They would then be more likely to succeed in obtaining all optimal scores at PA2.

From the perspective of situational competence, we also considered the practice context in which physiotherapists worked. The majority of this subset of the cohort, all of whom had some sub-optimal PA outcomes, worked in a private sector context, some in sole or small practice settings. This finding is consistent with those of Ashworth and colleagues,³ who demonstrated that small general physician practices (fewer than three practitioners) were at higher risk of lower quality-of-care outcomes than practices with three or more general physicians practising together. Somewhat similarly, Phipps and colleagues² found that pharmacists working in community settings, in contrast to hospital settings, were disproportionately represented in a group of those referred for disciplinary action. In addition, Wenghofer and colleagues⁴ found that organizational factors such as a walk-in practice and high volume of patient visits were associated with lower physician performance, as were system factors such as a low availability of diagnostic tests and a low physician-to-population ratio.

In private practice, especially very small settings, administrative support for physiotherapists to meet documentation standards may be limited. Because the CPO's PA reports are created without identifying details to maintain confidentiality, we were unable to determine the size of the work settings of most of the physiotherapists whose PAs we reviewed. However, we postulate that in smaller organizations with only a few employees, a physiotherapist would have to take time away from direct patient care to become familiar with and institute record-keeping and chart audit processes and adapt them as requirements changed. Larger, multi-clinic corporations would likely have more administrative support to keep up with best practices. In addition, in smaller practice contexts, there is likely less opportunity for physiotherapists to receive feedback from their peers regarding documentation or to view their peers' documentation and potentially identify weaknesses in their own. Physiotherapists in smaller, private sector settings may find it more challenging to ensure that their record keeping always stays abreast of legal and regulatory requirements and professional standards.

Also in a private practice setting, physiotherapists may think that others are unlikely to read their patient files,²⁷ whereas in a public institution, where physiotherapists interact with a multidisciplinary team, documentation is a form of inter-professional communication. Thus, the importance of record keeping as being necessary and directly related to care may be more evident to physiotherapists working in a public sector context. Moreover, we speculate that standardized charting forms and routine chart audits are more likely to be mandated in a public sector context.

If corroborated by future research, the results of this study have implications for both physiotherapists and regulatory bodies of health care professionals. Physiotherapists in private practice may benefit from more familiarity with record-keeping standards, especially in anticipation of a PA. Regulators may consider providing support to health care practitioners that is specific to certain practice contexts to optimize the practitioners' ability to maintain professional standards in areas that are commonly found to be more challenging. During the PA, regulators should aim to capture more specific information about the practice context to explore in greater depth its potential influence on maintaining professional standards. Ultimately, regulators who use a random-draw method for selecting practitioners for PA may want to use a more targeted approach, such as adding stratification to their randomization procedures. Such stratification could be designed to increase the chances of selecting practitioners who are practising in contexts that put them at higher risk of not maintaining professional standards.

The findings of this study echo those of Wenghofer and colleagues.⁴ They also support the researchers' suggestion that regulators consider not only personal attributes but also the contribution of organizational and system factors,⁴ similar to what we call practice context, to a practitioner's ability to maintain competence.

This study has two limitations. First, the cohort was small from a statistical analysis perspective. Although it represented the entire population of eligible participants—physiotherapists registered with the CPO who had undergone two PAs between 2004 and 2012 with specified patterns of sub-optimal outcomes—it may not represent all Ontario physiotherapists or physiotherapists in other jurisdictions. Second, the document analysis included only information that could be revealed in de-identified forms that protected privacy. As a result, we were unable to examine demographic variables or detailed practice information that may be associated with the risk of not maintaining continuing competence.

Conclusions

Record keeping and clinical reasoning, assessed through chart-stimulated recall, were the most common areas determined to need at least minor improvement when physiotherapists were undergoing PAs by a peer in conjunction with a regulatory body's QA program. In an initial PA, these two areas were significantly associated with one another. In a repeat PA 5–7 years later, record keeping was again the most common area needing improvement, but it was not significantly associated with any clinical reasoning concerns that would have been elicited during chart-stimulated recall. Many of the assessors' comments in the second PA related to administrative omissions or errors in record keeping.

Physiotherapists who had sub-optimal outcomes on their PAs practised more often in private sector settings than those who did not. Our model of situational competence expands on the model by Handfield-Jones and colleagues⁷ to include recognition that health care practitioners may not be evenly competent across all areas of practice and that practice context and environmental factors may influence level of competence. To our knowledge, this is the first review of practice context associated with physiotherapists' outcomes on PA. Our findings are consistent with similar reports of physician and pharmacist PA outcomes.

Key Messages

What is already known on this topic

When assessed by peers, most physiotherapists are found to maintain all required professional standards, but those who do not are often deemed to need improvement in record keeping. In other health care professions, practice context and the presence of peers are related to the likelihood that practitioners will maintain high standards. Models of situational competence have been proposed to describe a professional's competence over time.

What this study adds

Ontario physiotherapists practising in private sector contexts were more likely than those practising in public sector contexts to be deemed by a peer assessor to need improvements in their practice. We extended the model of situational competence developed by others to explain the relatively frequent occurrence of record keeping needing minor improvement to meet standards. Record keeping may represent an area in which a practitioner's personally acceptable level may be at or near the minimum professionally acceptable level.

Appendix

Practice Assessment Scores* from the 22 Physiotherapists Reviewed in Detail

	Practice Assessment 1							Practice Assessment 2
No.	PI	BP	RK	CSR† (n)	PISA‡	PD‡	O§	PI	BP	RK	CSR† (n)	PISA‡	PD‡	O§
First 14 physiotherapists
1	1	2	2	1 (6)	1	1	D	1	1	1	1 (4)	1	1	A
2	1	1	3	3 (5); 4 (1)	1	1	D	1	1	1	1 (4)	1	1	A
3	1	1	3	1 (3); 2 (1); 3 (2)	1	1	D	1	1	1	1 (4)	1	1	A
4	1	2	2	1 (4); 2 (1)	1	1	D	1	1	1	1 (4)	1	1	A
5	1	1	1	1 (3); 2 (3)	1	1	D	1	1	1	1 (4)	1	1	A
6	2	4	2	1 (4); 4 (1)	1	1	D	1	1	1	1 (4)	1	1	A
7	1	0	1	1 (6)	2	2	D	1	1	1	1 (4)	1	1	A
8	1	1	2	1 (5); 2 (1)	1	1	D	1	1	1	1 (2); 2 (2)	1	1	C
9	1	1	2	1 (6)	1	1	C	1	1	2	1 (4)	1	1	B
10	1	1	2	1 (5)	1	1	C	1	1	2	1 (4)	1	1	C
11	2	1	1	1 (6)	1	1	C	2	1	2	1 (4)	2	1	D
12	2	1	1	1 (5); 2 (1)	1	1	B	1	1	2	1 (4)	1	1	B
13	1	1	2	1 (4); 2 (1)	1	1	B	2	1	2	1 (4)	1	1	C
14	1	1	1	1 (6)	1	1	A	1	1	2	1 (4)	1	1	C
Next 8 physiotherapists
15	1	1	1	1 (6)	1	1	A	1	1	2	1 (4)	1	1	B
16	1	1	1	1 (6)	1	1	A	1	1	2	1 (4)	1	1	B
17	1	1	1	1 (6)	1	1	A	1	1	2	1 (4)	1	1	B
18	1	1	1	1 (6)	1	1	A	1	1	2	1 (4)	1	1	B
19	1	1	1	1 (6)	1	1	A	1	1	2	1 (4)	2	1	B
20	1	1	1	1 (6)	1	1	A	1	1	2	1 (4)	1	1	B
21	1	1	1	1 (6)	1	1	A	2	1	1	1 (4)	1	1	B
22	1	1	1	1 (6)	1	1	A	2	1	1	1 (3); 2 (1)	1	1	B

Note: Each row shows the data for one physiotherapist. The numbers in the cells are the scores awarded by the assessors and the letters are the outcomes.

^*Scores: 1=meets professional standard in that area; 2, 3, 4=the increasing levels of improvement required to meet the expected professional standard; 0=the item was deemed not applicable.

^†In each practice assessment, the chart-stimulated recall process is conducted for four to eight patient charts, and a rating is assigned to each one. The number of chart-stimulated recall processes per assessment is determined by the assessor.

^‡For this area, assessors assigned one of only two possible ratings: present/evident (i.e., meeting standards), coded as 1; or not present/evident, coded as 2.

^§See Box 1 for explanations of outcomes.

PI=practice issues; BP=business practices; RK=record keeping; CSR=chart-stimulated recall; PISA=Professional Issues Self Assessment; PD=professional development; O=outcome.