Test ordering by GP trainees

Abstract

Objective

To assess the effectiveness of an educational intervention on test-ordering attitudes and intended practice of GP trainees, and any associations between changes in test ordering and trainee characteristics.

Design

Preworkshop and postworkshop survey of attitudes to test ordering, intended test-ordering practices for 3 clinical scenarios (fatigue, screening, and shoulder pain), and tolerance for uncertainty.

Setting

Three Australian regional general practice training providers.

Participants

General practice trainees (N = 167).

Intervention

A 2-hour workshop session and an online module.

Main outcome measures

Proportion of trainees who agreed with attitudinal statements before and after the workshop; proportion of trainees who would order tests, mean number of tests ordered, and number of appropriate and inappropriate tests ordered for each scenario before and after the workshop.

Results

Of 167 trainees, 132 (79.0%) completed both the preworkshop and postworkshop questionnaires. A total of 122 trainees attended the workshop. At baseline, 88.6% thought that tests can harm patients, 84.8% believed overtesting was a problem, 72.0% felt pressured by patients, 52.3% believed that tests would reassure patients, and 50.8% thought that they were less likely to be sued if they ordered tests. There were desirable changes in all attitudes after the workshop. Before the workshop, the mean number of tests that trainees would have ordered was 4.4, 4.8, and 1.5 for the fatigue, screening, and shoulder pain scenarios, respectively. After the workshop there were decreases in the mean number of both appropriate tests (decrease of 0.94) and inappropriate tests (decrease of 0.24) in the fatigue scenario; there was no change in the mean number of appropriate tests and a decrease in inappropriate tests (decrease of 0.76) in the screening scenario; and there was an increase in the proportion of trainees who would appropriately not order tests in the shoulder pain scenario. There were no significant associations between changes in test ordering and trainee demographic characteristics or tolerance for uncertainty subscale scores.

Conclusion

General practice trainees have conflicting attitudes to test ordering and demonstrate nonrational test ordering in 3 common scenarios. A workshop on rational test ordering led to desirable changes in attitudes and more rational intended test ordering. Our findings inform the development of appropriate educational interventions that address nonrational testing in family medicine.

The use of medical investigations is increasing worldwide.¹ While some of this increase might be appropriate, emerging literature suggests that overtesting is becoming a considerable problem.² Overtesting is costly, can precipitate a cascade of further tests,³ and in turn can lead to patient harm.

As primary care providers, GP or family medicine practitioners are responsible for initiating most pathology tests.⁴ Overtesting is especially problematic in family medicine, a setting where the pretest probability of serious disease is low. Australian general practice data suggest that pathology testing is often inconsistent with clinical guidelines, with 25% to 75% of tests unsupported by evidence or expert opinion.⁴

Vocational training is a critical period in the development of appropriate patterns of test-ordering behaviour. A low tolerance for uncertainty has been described as a causative factor in overtesting⁵ and is likely to be particularly problematic for early career GPs.

We aimed to assess the effectiveness of an educational intervention for GP trainees on rational test ordering, including any change in attitudes and intended test-ordering practices, and any associations with such change.

METHODS

Study population

Participants were GP trainees with 3 of Australia’s 17 regional training providers. Trainees were in terms 1 and 2 (first 12 months) of 3 mandatory GP training terms. We used trainee demographic data from the larger ReCEnT (Registrar Clinical Encounters in Training) cohort study.⁶

Workshop session

As part of the routine GP trainee educational release workshop program, we developed a 2-hour workshop session on rational test ordering, including large and small group activities and didactic presentations. It was supported by an online module on screening.

Content and delivery methods were based on a review of the literature and were refined by a local group of medical educators. The workshop discussed “drivers” and risks of overtesting; the importance of pretest probabilities in interpreting test results; principles of screening; and barriers to, and strategies for, rational test ordering.⁷

The workshop was delivered in July 2014 and January and March 2015.

Survey instruments

We developed a preworkshop instrument comprising 5 attitudinal questions, a self-rated scale of tolerance for uncertainty, and 3 clinical scenarios. It was delivered 1 month before the workshop.

Trainees rated their agreement with 5 attitudinal statements on a 4-point Likert scale (strongly disagree to strongly agree). The statements are listed in Box 1.

Box 1. Attitude statements and scenarios.

Attitude statements

• I believe that overtesting (ordering too many pathology and imaging tests) is a problem in Australian general practice

• I believe that tests can lead to harm to patients

• I feel that my patients will be reassured by me ordering tests

• I feel pressured by patients to order tests

• I feel that I am less likely to be sued if I order more tests

Scenario 1

Milla is a previously well 29-year-old woman who presents with a 2-month history of being “tired all the time.” She is married with no children and works as a full-time registered nurse. She has no other symptoms to report and there are no red flags. She is not taking any regular medications or other drugs. There have not been any important changes in her life that she can recall, and she cannot explain why she feels so fatigued. She denies feeling depressed. Physical examination findings are unremarkable. Findings from her mental state examination are essentially normal—she reports her mood as “tired.”

Scenario 2

Bob, a 53-year-old man, presents stating that he has been sent in by his wife for a health check. His wife attends the practice but he has never been seen here before. He is asymptomatic, denies relevant PMH, takes no medications, does not drink alcohol, and has no important family history. He is not overweight. He has had no blood tests for at least 5 years.

Scenario 3

June, a 77-year-old retired bookkeeper living with her husband, presents with a 6-week history of discomfort in her right shoulder and deltoid region while sleeping, and difficulty doing her hair, putting on her coat, doing up her bra, and reaching up to high shelves. On examination there is tenderness over the lateral aspect of the shoulder and pain on shoulder abduction in the mid range, but a normal range of movement. The remainder of the findings on physical examination are normal. There is no history of trauma. She has been previously well with no history of serious illness. A previous GP prescribed a 2-week course of NSAIDs, which did not provide any relief.

NSAID—nonsteroidal anti‑inflammatory drug, PMH—past medical history.

Trainees completed the Physicians’ Reactions to Uncertainty scale, a validated set of 4 subscales assessing tolerance for uncertainty in clinical practice.⁸ The subscales are anxiety due to uncertainty, concern about bad outcomes, reluctance to disclose uncertainty to patients, and reluctance to disclose mistakes to physicians.

Trainees responded to 3 scenarios (Box 1) by nominating tests they would order in clinical practice. The scenarios represented common GP diagnostic and screening presentations. All were uncomplicated with no red flags for serious disease. Responses were free text in order to avoid prompting.

Trainees completed, and returned by mail, a follow-up questionnaire approximately 2 months after the workshop. This comprised the same attitudinal questions and clinical scenarios. Responses were matched by a unique identifier.

Data analysis

Responses to the 5 attitudinal questions were collapsed into 2 variables: agree (agree and strongly agree) and disagree (disagree and strongly disagree). Proportions were calculated with 95% CIs.

The proportion of trainees who would order tests and the mean number of tests, when tests were ordered, were calculated with 95% CIs for each scenario.

For scenarios 1 and 2, outcome factors were the number of appropriate (guideline-congruent) and inappropriate (guideline-incongruent) tests ordered. Appropriate tests were those recommended by evidence-based Australian guidelines for fatigue⁹ and screening, respectively.¹⁰ For scenario 2, we also examined the number of fecal occult blood tests (FOBTs) suggested before and after the workshop.

For scenario 3, the outcome factor was whether any tests were ordered. Australian guidelines recommend against any investigations for acute shoulder pain, the clinical presentation in this scenario.¹¹

Analyses.

Independent factors in analyses were trainee age, sex, training term, and place of medical qualification (Australia or international), as well as the 4 Physicians’ Reactions to Uncertainty subscale scores.

For preworkshop and postworkshop comparisons, analyses employed McNemar tests (categorical data) and Wilcoxon signed rank tests (continuous data).

For associations with change in outcome before and after the workshop, Χ² analysis and Wilcoxon rank sum tests were used for categorical and continuous outcomes, respectively.

We employed “intention-to-educate” analyses (all trainees who completed preworkshop and postworkshop questionnaires were included, irrespective of workshop attendance).

Significance was set at α < .05. All analyses were performed using STATA, version 13.1.

Ethical considerations

Ethical approval was provided by the University of Newcastle Human Research Ethics Committee.

RESULTS

Of 167 eligible trainees, 132 (79.0%) completed both preworkshop and postworkshop questionnaires. A total of 122 trainees attended 1 of the 4 workshops (Figure 1). Characteristics of the trainees who completed preworkshop and postworkshop questionnaires and their practices are outlined in Table 1.¹²

Figure 1.

Flow chart of trainee recruitment and participation

Table 1.

Participating trainee and practice characteristics: N = 132; not all data were available for all respondents.

CHARACTERISTIC	VALUE	95% CIs
Sex, n (%)
• Male	49 (38.6)	30.4–47.4
• Female	78 (61.4)	52.6–69.6
Australian qualified, n (%)
• No	20 (15.4)	10.1–22.8
• Yes	110 (84.6)	77.2–89.9
Mean (SD) age, y	32.0 (5.6)	NA
Training term, n (%)
• Term 1	44 (33.3)	25.7–41.9
• Term 2	87 (65.9)	57.3–73.6
• Term 3	1 (0.8)	0.1–5.3
Worked at the practice previously, n (%)
• No	95 (74.2)	65.8–81.1
• Yes	33 (25.8)	18.7–34.2
Works full time, n (%)
• No	29 (22.8)	16.3–31.1
• Yes	98 (77.2)	68.9–83.7
Bulk billing practice,* n (%)
• No	104 (81.2)	73.4–87.2
• Yes	24 (18.8)	12.8–26.6
No. of GPs in practice, n (%)
• 1–5	43 (33.9)	26.1–42.6
• ≥ 6	84 (66.1)	57.4–73.9
Rurality of practice, n (%)
• Major city	88 (68.8)	60.1–76.3
• Inner regional	33 (25.8)	18.9–34.2
• Outer regional, remote, or very remote	7 (5.5)	2.6–11.1
Mean (SD) SES of practice†	5.8 (2.9)	NA
RTP code
• 1	52 (39.4)	31.3–48.1
• 2	11 (8.3)	4.6–14.5
• 3	69 (52.3)	43.7–60.8

NA—not applicable, RTP—regional training provider, SES—socioeconomic status.

^*No financial cost to patient.

^†Based on the Socio-Economic Index for Area.¹²

Attitudes about test ordering

At baseline, most trainees believed that tests can harm patients (88.6%, 95% CI 81.9% to 93.1%) and that overtesting was a problem in general practice (84.8%, 95% CI 77.6% to 90.1%). Nearly three-quarters (72.0%, 95% CI 63.6% to 79.0%) stated that they felt pressured by patients to order tests, 52.3% (95% CI 43.6% to 60.9%) thought that tests would reassure patients, and 50.8% (95% CI 42.1% to 59.4%) believed that they were less likely to be sued after ordering tests.

After the workshop, significantly fewer trainees believed that ordering tests would reassure patients (absolute decrease 11.6%), and fewer thought they were less likely to be sued (absolute decrease 12.4%). There were nonsignificant trends toward a greater proportion of trainees stating that tests can harm patients (absolute increase 5.3%), a greater proportion stating that they felt overtesting was a problem (absolute increase 6.0%), and fewer feeling pressured by patients (absolute decrease 9.2%) (Figure 2).

Figure 2.

Trainees’ attitudes to test ordering before and after the workshop

Scenario 1: fatigue.

At baseline, at least 1 test was ordered by 94.9% (95% CI 88.3% to 97.0%) of trainees, and the mean (SD) number of tests requested (when tests were ordered) was 4.4 (2.0). The most commonly ordered tests were complete blood count, thyroid function tests, and iron studies or ferritin testing.

After the workshop, the number of appropriate tests requested per trainee decreased by a median (inter-quartile range [IQR]) of 1 (0 to 2) and mean (SD) of 0.94 (1.93) (P < .0001). The number of inappropriate tests decreased by a median (IQR) of 0 (0 to 1) and mean (SD) of 0.24 (1.02) (P = .0396) (Figure 3).

Figure 3.

Trainees’ intended test ordering for scenario 1 (fatigue) before and after the workshop

CBC—complete blood count; CRP—C-reactive protein; ESR—erythrocyte sedimentation rate; EUC—electrolytes, urea, creatinine; LFT—liver function test; TSH—thyroid-stimulating hormone.

Scenario 2: screening.

Before the workshop, at least 1 test was ordered by 97.7% (95% CI 93.1% to 99.3%) of trainees, and the mean (SD) number of tests requested (when tests were ordered) was 4.8 (1.8). The most commonly ordered tests were lipid measurement and blood glucose level (BGL); FOBTs were requested by 23.5% (95% CI 17.0% to 31.6%) of trainees.

The number of appropriate tests requested per trainee did not change after the intervention (P = .6516). The number of inappropriate tests decreased by a median (IQR) of 0 (0 to 2) and a mean (SD) of 0.76 (1.62) after the intervention (P < .0001). There was a significant (P = .0010) increase in the proportion of trainees who would order an FOBT (absolute increase 15.9%) (Figure 4).

Figure 4.

Trainees’ intended test ordering for scenario 2 (screening) before and after the workshop

CBC—complete blood count; ECG—electrocardiogram; EUC—electrolytes, urea, creatinine; FOBT—fecal occult blood test; LFT—liver function test; PSA—prostate-specific antigen; TFT—thyroid function test.

Scenario 3: acute shoulder pain.

Before the workshop, at least 1 test was ordered by 76.5% (95% CI 68.4% to 83.0%) of trainees and the mean (SD) number of tests requested (when tests were ordered) was 1.5 (1.6). The most commonly ordered tests were shoulder ultrasound (65.9%, 95% CI 57.2% to 73.9%) and shoulder x-ray scans (33.3%, 95% CI 25.4% to 42.1%).

After the intervention, there was a significant (P < .0001) increase in the proportion of trainees who would not order any tests (absolute increase 25.7%) (Figure 5).

Figure 5.

Trainees’ intended test ordering for scenario 3 (shoulder pain) before and after the workshop

CBC—complete blood count; CRP—C-reactive protein; ESR—erythrocyte sedimentation rate; EUC—electrolytes, urea, creatinine; LFT—liver function test.

Associations

There were no significant associations between changes in test ordering and trainee demographic characteristics or tolerance for uncertainty subscale scores.

DISCUSSION

We found that GP trainees have conflicting attitudes about test ordering and demonstrate nonrational test ordering in 3 common GP scenarios. We also found that the delivery of a workshop for GP trainees on rational test ordering led to desirable changes in attitudes around test ordering and to significant reductions in inappropriate test ordering in 2 of 3 clinical scenarios. There was, however, also a reduction in appropriate test ordering in scenario 1 (the fatigue scenario) and no significant change in appropriate test ordering in scenario 2 (the screening scenario).

To our knowledge, this is the first time that the attitudes and intended test-ordering practices of GP trainees, and the effect of an educational intervention on these, has been described.

Comparison with previous literature

Attitudes.

Our findings suggest that trainees generally appreciate the issue and consequences of overtesting. However, they also have concerns regarding patient expectations and legal implications. Doctors’ personal beliefs and attitudes have previously been shown to influence test ordering, including believing that test ordering will reassure patients,¹³ feeling pressured by patients to order tests,¹⁴ and medicolegal fears.¹⁵

Scenarios.

Fatigue is the most common undifferentiated presentation in general practice¹⁶ and it frequently leads to multiple “superfluous” investigations.¹³ While most tests requested by trainees for scenario 1 were congruent with guidelines, our findings also suggest nonrational test ordering. This includes low rates of appropriate tests (eg, BGL and erythrocyte sedimentation rate) and ordering of less-appropriate tests (eg, vitamin B12).

Health checks are common in general practice¹⁷ and a source of high rates of test ordering. Australian data suggest that nearly 50% of screening tests are inappropriate.⁴ While some recommended tests were commonly ordered by trainees (eg, lipid levels and BGL), there were high rates of nonrecommended tests, including complete blood count (74.4%) and liver function tests (55.6%). In contrast, despite an FOBT being recommended by guidelines for this scenario, the rate of FOBT ordering was only 23.1%. Low rates of FOBT screening by GPs have previously been reported.¹⁸

The inappropriate use of imaging for acute shoulder pain has been widely reported.¹⁹ We found high rates of imaging ordering by trainees, as well as nonrecommended pathology testing, consistent with studies of established GPs.¹⁹

Associations.

A low tolerance for uncertainty has been described as a causative factor in overtesting.⁵ We found no associations between trainee characteristics, including tolerance for uncertainty, and changes in test-ordering attitudes or intended behaviour after the intervention.

Strengths and limitations

Study strengths include the high response rate and participants having similar demographic characteristics to the national GP trainee cohort.²⁰ Our workshop session was time efficient (only 2 hours). Furthermore, our analyzing all eligible registrars, irrespective of receipt of the intervention, is consistent with a “real-world,” pragmatic trial of an educational intervention.

A limitation of the study is that our outcome factor was expressed as intention to order tests rather than actual test ordering. Although vignettes have been found to be a valid tool for measuring the quality of clinical practice,²¹ they are a proxy measure of true test ordering.

Another limitation is that our postworkshop survey was administered after only 2 months, and longer-term changes in attitudes and intended practice have not been measured.

Implications for policy and practice

Trainees usually enter general practice after exclusively hospital-based experience, a setting with a much greater focus on investigation and diagnostic certainty. Vocational training provides a unique opportunity for learning rational test ordering, and, indeed, critical use of investigations is a core learning outcome of Australian GP training.²²

Education and training on appropriate pathology test use has positively influenced knowledge and test-ordering patterns of primary health care physicians and trainees.^23,24 Despite this, Australian GP trainees have a relative lack of training in high-quality use of pathology testing.²⁵

Our findings inform the development of appropriate educational interventions. This comprises both a focus on overtesting and an increase in ordering appropriate tests (eg, FOBTs for bowel cancer screening). Training needs to address identified barriers to rational test ordering. Similarly, the specific test-ordering patterns for each scenario support targeted education around use of guidelines.

Australian GP training is based on the apprenticeship model, where trainees train under the supervision of accredited GP supervisors. Trainees have previously been found to be influenced by the decisions of their supervisors, at least for prescribing.²⁶ Our findings support appropriate professional development for GP supervisors in relation to rational test ordering.²⁷

Finally, our findings of a reduction in some appropriate, as well as inappropriate, test ordering are important. While trainees responded to the simple message that overtesting in general practice needs to be reduced, we found that they did not significantly increase appropriate testing in either scenarios 1 or 2. In fact, appropriate test ordering decreased in scenario 1. Thus, trainees might not have fully assimilated the more nuanced educational message that rational test ordering reflects decisions about appropriate as well as inappropriate testing. This result might reflect the relatively short duration of the intervention or the content and specific focus on inappropriate test ordering.

Future research in this field could investigate the longer-term effects on test-ordering practices of such an educational intervention, for both trainees as well as established GPs.

EDITOR’S KEY POINTS

• Overtesting, which is costly and can precipitate a cascade of further tests and lead to patient harm, is a considerable problem. Primary care providers are responsible for initiating a large proportion of tests, in a setting where the pretest probability of serious disease is low. Further, appropriate tests are sometimes not ordered. Therefore, it is important that GP trainees are educated about rational test ordering. This study aimed to assess the effectiveness of a workshop on rational test ordering for GP trainees.

• The workshop led to desirable changes in trainees’ attitudes and to statistically significant reductions in inappropriate test ordering in 2 of 3 common clinical scenarios. However, there was also a reduction in appropriate test ordering in one scenario and no improvement in appropriate test ordering in another. While trainees responded to the message that overtesting needs to be reduced, they might not have fully appreciated that rational test ordering reflects decisions about appropriate as well as inappropriate testing.

POINTS DE REPÈRE DU RÉDACTEUR

• Prescrire trop d’examens constitue un problème important non seulement en raison des coûts, mais aussi parce que cela peut déclencher une série de tests additionnels ou même nuire au patient. Même quand le patient présente peu de possibilités d’avoir une maladie grave, les soignants de première ligne prescrivent souvent un grand nombre d’examens initiaux. De plus, certains tests appropriés sont parfois omis. Il est donc important d’enseigner aux futurs MF la façon rationnelle de prescrire des examens. Cette étude voulait vérifier l’efficacité d’un atelier de formation ayant pour but que les demandes d’examens des futurs MF soient plus appropriées.

• L’atelier a entraîné des changements d’attitude chez les étudiants et une diminution significative des demandes d’examens inappropriés, et ce, dans 2 scénarios cliniques courants sur 3. Dans un scénario toutefois, on a observé une réduction des demandes d’examens appropriés et dans un autre, aucune amélioration des prescriptions de tests appropriés. Même s’ils étaient d’accord sur la nécessité d’éviter de demander trop de tests, les étudiants pourraient n’avoir pas bien compris que la décision de prescrire concerne non seulement les tests appropriés mais aussi ceux qui ne le sont pas.