Factors contributing to inappropriate ordering of tests in an academic medical department and the effect of an educational feedback strategy

Abstract

Aims

To identify factors contributing to laboratory overutilisation in an academic medical department, and to assess the effect of an educational feedback strategy on inappropriate test‐ordering behaviour.

Methods

The records of 426 patients admitted during a 6‐month period were reviewed. The usefulness of 25 investigations (haematology, basic biochemistry and arterial blood gases) was assessed according to implicit criteria. Trainees' acquaintance with investigation costs was assessed via a multiple‐choice questionnaire. The medical staff was informed about their test‐ordering behaviour, cost awareness and the factors associated with overuse of diagnostic tests. The test‐ordering behaviour of the same doctors was reassessed on 214 patients managed during 6 months after the intervention.

Results

Overall, 24 482 laboratory tests were ordered before the intervention (mean 2.96 tests/patient/day). Among those, 67.9% were not considered to have contributed towards management of patients (mean avoidable 2.01 tests/patient/day). Patient age ⩾65 years, hospitalisation beyond 7 days and increased case difficulty (death or inability to establish a diagnosis) were factors independently associated with overuse of laboratory tests. Senior trainees ordered more laboratory examinations, but the percentage of avoidable tests requested by junior trainees was higher. A moderate and disparate level of trainees' awareness about the cost of common laboratory examinations was disclosed. The avoidable tests/patient/day were significantly decreased after the intervention (mean 1.58, p = 0.002), but containment of unnecessary ordering of tests gradually waned during the semester after the intervention.

Conclusion

Repeated audit, continuous education and alertness of doctors, on the basis of assessment of factors contributing to laboratory overutilisation, result in restraining the redundant ordering of tests in the hospital setting.

The overuse of laboratory investigations is widely prevalent in hospital practice, including academic departments.¹,²,³ Reasons for excessive ordering of tests by doctors include defensive behaviour and fear or uncertainty, lack of experience, the use of protocols and guidelines, “routine” clinical practice, inadequate educational feedback and clinician's unawareness about the cost of examinations.⁴,⁵,⁶ Inappropriate testing causes unnecessary patient discomfort, entails the risk of generating false‐positive results, leads to overloading of the diagnostic services, wastes valuable healthcare resources and is associated with other inefficiencies in healthcare delivery, undermining the quality of health services.¹,⁵ Interventions on inappropriate testing aim to reduce costs, along with improving the quality of care provided. Results, however, are not always consistent, usually owing to inherent limitations of the strategies proposed.⁵,⁶

In this study, we assessed the appropriateness of routine ordering of laboratory tests of the trainees in an academic internal medicine department, as well as their awareness about the examination cost; we applied a feedback strategy for reducing the unnecessary ordering of tests, on the basis of the identification of factors associated with inappropriate laboratory utilisation, and reassessed the trainee's test‐ordering behaviour after the intervention.

Materials and methods

The medical records of 426 consecutive patients admitted in the wards via the emergency or the outpatient department for a period of 6 months (March–August 2003) were retrospectively reviewed. Patients were excluded from the analysis if (a) they were discharged in <48 h after admission; (b) they were admitted for a reason other than investigation (ie, patients diagnosed with cancer admitted for chemotherapy); (c) their medical records were incomplete or did not contain information adequate for evaluating the rationale for and the usefulness of the ordered tests; and (d) if they had hospitalisation prolonged for social reasons unrelated to their disease course. Table 1 presents the main characteristics of the patients whose records were analysed during the study.

Table 1 Main characteristics of the patients whose medical records were analysed during the study.

	Before intervention (March–August 2003)	After intervention (November 2003–April 2004)	p Value
Patients, n	426	214
Mean age (range)	67.8 (15–98)	66.1 (16–104)	NS
Hospitalisation days (mean, range)	3845 (9.0, 2–50)	1797 (8.4, 2–47)	NS
Males (%)	212 (49.7)	110 (51.4)	NS
Multiple problems* (%)	50 (11.7)	22 (10.3)	NS
Died or undiagnosed (%)	41 (9.6)	27 (12.6)	NS
Emergency department† (%)	341 (80)	176 (82.2)	NS

*Patients presenting with >1 active medical conditions that prompted investigation.

†Patients admitted via the emergency department.

The clinical usefulness of 25 laboratory blood tests was assessed: full blood count (FBC), prothrombin time, activated partial thromboplastin time, plasma glucose, urea, creatinine, sodium, potassium, calcium, transaminases, γ‐glutamylic transferase, alkaline phosphatase, bilirubin, total protein, albumin, creatin kinase, lactate dehydrogonase, total cholesterol, low‐density lipoprotein‐cholesterol, high‐density lipoprotein‐cholesterol, triglycerides, uric acid, amylase and arterial blood gases (ABGs). Chest x ray and electrocardiogram were not analysed, as they are routinely performed for all patients admitted to our hospital.

To assess the utility of laboratory tests ordered, an effort was made to determine whether they were ordered in logical combinations or sequences. For this reason, each patient's symptoms and diagnoses were identified during a review of the medical records, to determine the appropriateness of the test according to previously validated, uniform implicit criteria. In this context, ordering of a laboratory test was regarded as avoidable, when the test was not relevant to the patient's symptoms and provisional diagnosis, when a normal result was not used to exclude a suspected diagnosis, when a repeated test was not used for monitoring treatment, and when the test result did not make any difference to the course of patient care and careful review of the patient's chart and hospital course did not indicate any change in the clinical status that could potentially dictate for ordering new laboratory tests at the given stage. In the same way, an abnormal result of a laboratory test was considered to be a case finding when there was no medical record documentation of clinical conditions associated with an abnormal test. For such case‐finding tests, patients' charts were reviewed to determine the consequences of the abnormality; those tests were judged as inappropriate when they were not considered in planning for subsequent evaluation by the doctors ordering the test.

In retrospectively defining a laboratory test to be clinically indicated, both current illnesses and chronic problems were taken into account. Initial ordering of each blood test at the admission site (emergency or outpatient department) was analysed separately, as this was considered a reasonable minimum standard of medical practice.

All medical records were initially reviewed by two reviewers (GK, and ML) independently. When the two reviewers gave discordant opinions on the usefulness of a test, the case was reviewed by a third, senior investigator (SM). A preliminary analysis of data from 25 randomly selected medical records, performed before the beginning of the study showed that agreement between the two initial reviewers (GK and ML), as well as between the initial and the senior (SM) reviewer was significantly beyond chance (κ = 0.76, and 0.67, respectively; p<0.05 in both instances).

Trainees' awareness of the costs of selected laboratory investigations was assessed using an anonymous multiple‐choice questionnaire regarding the estimated cost of the following examinations: FBC, serum glucose, serum sodium, alkaline phosphatase, creatine kinase and ABGs. Each question contained four possible answers with the estimated cost range, one of which corresponded to the cost that the relevant department (ie, haematology and clinical chemistry) charges for the examination in question. Therefore, the questionnaire contained 24 possible answers, 6 of which (one for each question) were correct.

The results from both the analysis of appropriateness of the test and the cost‐awareness questionnaire were presented, along with a review of the literature, during a session attended by the entire medical and senior nursing staff of the department. The session was followed by an open discussion, proposing strategies for reducing unnecessary tests. Next, the medical charts of 214 patients admitted during a 6‐month period (November 2003–April 2004) after the intervention were similarly reviewed. All the patients under the care of the doctors who were also staff members during the first study period were selected for analysis. The same laboratory investigations were included in the analysis, and appropriateness of the diagnostic tests was assessed using the same criteria by the same reviewers. Apart from the investigators, the staff was unaware that a study was being carried out to assess their test‐ordering behaviour after the presentation of the results of the first part.

Approval to conduct this research was obtained from our local institutional review board.

Statistical analysis

For each patient, ordering of tests was adjusted for the length of hospitalisation by dividing the number of examinations (overall, as well as unnecessary) by the number of days in hospital. For categorical data, the χ² test was used to examine univariate correlations; for continuous variables, Student's t test or one‐way analysis of variance with Bonferroni correction for multiple comparisons were used, as appropriate. Variables distributed in a non‐parametric fashion were examined using the Mann–Whitney U test or the Kruskal–Wallis test, as appropriate. Normality was tested using the Kolmogorov–Smirnov test. Variables showing significant univariate correlation with excessive ordering of tests were next introduced in a multiple logistic regression analysis model, using a forward stepwise selection algorithm with entry and removal criteria of p = 0.05 and 0.10, respectively, for identification of independent correlates. All tests of significance were two tailed, and p<0.05 was considered significant. For each comparison, 95% confidence intervals (95% CIs) were calculated. The statistical analysis was conducted using the SPSS software V.12.0.

Results

Figure 1 summarises the distribution of the tests ordered in each of the two parts of the study. For the 426 patients analysed before the intervention, 24 482 laboratory tests were ordered overall (fig 1, left). Among those tests, 4701 were performed at the site of presentation (emergency or outpatients department), and thus were not included in further analysis. In all, 6545 tests were ordered in the wards on the day of admission, 1873 (28.6%) among which were considered to be avoidable. By contrast, 9175 (69.3%) of the 13 236 tests ordered beyond the first day of hospitalisation could have been avoided, without any effect on patient management.

Figure 1 Flow chart of the study. ED, emergency department; OPD, outpatient department

A review of the medical records of 214 patients admitted after the intervention showed a total of 10 297 laboratory tests ordered (fig 1, right), 2570 of which had been ordered during the initial assessment, and thus they were excluded from further analysis. Of the 2463 tests ordered in the wards during the day of admission, 657 (26.7%) were judged as avoidable, whereas of the 5264 tests ordered beyond the first day of hospitalisation, 3326 (63.2%) could have been avoided. The proportion of avoidable tests among those ordered beyond the first day of hospitalisation was significantly decreased after the intervention (p<0.005, odds ratio (OR) 0.760; 95% CI 0.710 to 0.812), whereas the proportion of avoidable tests among those ordered on the first day in hospital failed to show a significant change (p = 0.067, OR 0.907; 95% CI 0.818 to 1.007).

Table 2 shows the laboratory tests ordered/patient/day in the wards. The mean total number of laboratory tests performed/patient/day was 2.96, whereas the mean number of avoidable tests/patient/day was 2.01. Thus, 67.9% of the laboratory tests ordered did not seem to have contributed towards the management of patients. Analysis of patient groups showed that overall, as well as avoidable, ordering of tests was higher for patients ⩾65 years in comparison with the younger patients (p<0.005 in both instances), for patients staying in hospital for >1 week (p<0.005 for both overall and avoidable tests) and for patients who died or were discharged without a definite diagnosis (p<0.005 for overall, and p = 0.033 for avoidable tests). Patients cared for by senior trainees underwent more laboratory examinations overall in comparison with those managed by junior trainees (p = 0.043). However, the number of avoidable tests ordered did not differ significantly between senior and junior trainees (table 2).

Table 2 Laboratory tests (total and avoidable) adjusted/patient/day in hospital, for the patients analysed before the intervention (March–August 2003).

		Sex		Age (years)		Hospitalisation (days)		Diagnosis		Trainee's eexperience
		All patients (n = 426)	Male (n = 214)	Female (n = 212)	<65 (n = 138)	⩾65 (n = 288)	⩽7 days (225)	>7 days (n = 201)	Diagnosed on discharge (n = 385)	Died or undiagnosed (n = 41)	Junior (n = 230)	Senior (n = 196)
Mean	Total tests/patient/day	2.96	2.99	2.92	2.27	3.28*	2.22	3.79*	2.72	5.21*	2.69	3.27**
	Avoidable tests/patient/day	2.01	2.03	1.99	1.56	2.23*	1.49	2.60*	1.91	2.94*	1.86	2.20

*p<0.005; **p = 0.045.

The number of patients within each group are in parentheses.

Multivariate analysis identified prolonged hospitalisation (correlation coefficient 4.02, 95% CI 2.65 to 6.10), unfavourable outcome (defined as death or lack of diagnosis; 2.98, 1.42 to 6.28) and advanced age (1.82, 1.16 to 2.86) as independent risk factors for excessive ordering of tests.

The burden of unnecessary ordering of tests was higher for serum potassium (mean 0.163 unnecessary tests/patient/day), sodium (0.159) and glucose (0.155) levels, and lower for serum lipids (mean total cholesterol 0.011 unnecessary tests/patient/day), serum amylase (0.012), ABGs (0.017), coagulation tests (activated partial thromboplastin time 0.018; prothrombin time 0.023) and FBC (0.081). The percentage of the avoidable tests among the tests ordered was higher for serum calcium (84% of the measurements that were performed were judged as avoidable), transaminases (alanine transaminase: 82%; aspartate transaminase: 77%), potassium (77%) and glucose (77%) levels, and lower for ABGs (16%), GBC (34%) and prothrombin time (40%; data for all individual tests not shown).

Comparison of the ordering of tests between patients admitted before and after the intervention disclosed a significant decrease in the number of both total tests/patient/day (from 2.96 to 2.56, p = 0.029) and avoidable tests/patient/day (from 2.01 to 1.58, p = 0.002, fig 2), the percentage of avoidable tests in the overall testing decreasing from 67.9% (before intervention) to 61.7% (after intervention). Avoidable testing was significantly decreased for all patient groups initially identified as prone to diagnostic test overuse—that is, patients ⩾65 years (from 2.23 to 1.74 avoidable tests/patient/day, p = 0.003), patients remaining in hospital >1 week (from 2.60 to 1.93 p<0.001) and patients who died or remained undiagnosed (from 2.94 to 1.70, p = 0.027; fig 2).

Figure 2 Comparison of the avoidable tests/patient/day between patients admitted before (black columns) and after (white columns) the intervention. The squares below each patient group represent the number of patients in the group.

We identified 80 laboratory tests during the period before intervention (0.3% of the before intervention tests), the results of which showed abnormalities that required attention, despite tests being initially ordered without any apparent clinical context to dictate for the ordering. A similar percentage (0.2%) was found for the tests of the period after intervention (18 tests). Finally, there were 16 pre‐intervention, and 8 post‐intervention cases, where the results of tests appropriately ordered for some other reason unexpectedly showed an unknown comorbidity.

Figure 3 shows changes in avoidable ordering of tests (expressed in mean avoidable tests/patient/day) from the period previous the intervention throughout the three 2‐month periods of the semester that followed the intervention. In comparison with the before intervention period (mean avoidable tests/patient/day = 2.01), there was a progressive decrease in the avoidable tests/patient/day ordered starting from the first 2 months (mean 1.85), and reaching significance in the third and fourth months after intervention (mean avoidable tests/patient/day = 1.08, p<0.005). However, the avoidable test rate returned close to the pre‐intervention levels by the third bimonthly post‐intervention period (months 5 and 6, mean avoidable tests/patient/day = 1.80).

Figure 3 Avoidable ordering of tests (expressed as mean avoidable tests/patient/day) for the period before the intervention, and for the three 2‐month periods of the semester after the intervention.

In all, 19 trainees responded to the questionnaire on the estimation of the cost of six laboratory investigations commonly ordered, giving a total of 114 answers. Of the 114 answers, 57 (50%) indicated the range of the actual cost correctly. None of the trainees replied correctly to all six questions. Figure 4A shows the numbers of correct answers given by trainee. Regarding the cost of individual tests, most of the trainees were aware of the cost of serum glucose (18/19 correct answers), of FBC and of serum sodium level (11 correct answers on each). By contrast, only 3 of the 19 responders estimated correctly the cost of ABGs, and most of them were unaware of the cost of alkaline phosphatase and creatinine phosphokinase (7/19 correct answers on each; fig 4B).

Figure 4 (A) Number of trainees with correct answers on the estimation of the cost of six common laboratory investigations. (The numbers of correct estimations are represented by different shaded patterns in the pie. None of the trainees could estimate correctly the cost of all six examinations requested.) (B) Number of correct (white columns) and wrong (black columns) answers by 19 trainees, on the estimated cost of six common laboratory investigations. ABGs, arterial blood gases; ALP, alkaline phosphatase; CPK, creatine phosphokinase; FBC, full blood count; Glc, serum glucose; serum sodium.

Discussion

Our findings indicate that almost 68% of the laboratory tests commonly ordered in an academic internal medicine department could have been avoided, without any adverse effect on patient management; this figure corresponds to 2.01 unnecessary tests ordered/patient during each day of their hospitalisation. Several previous studies have also shown an overuse of laboratory examinations in routine hospital practice, with percentages of inappropriate tests ordered in the medical wards of university hospitals ranging from 40% to 65%,⁷,⁸,⁹ but also as high as 95% when more stringent assessment criteria were applied.¹⁰ In addition, it has been estimated that only 1–5% of laboratory examinations ordered during the management of patients result in action.¹¹ High rates of unnecessary laboratory tests have been recorded in paediatric,¹² surgical¹³ and even emergency departments,¹⁴,¹⁵ as well as in intensive care units,¹⁶ implying that redundant ordering of tests is a universal phenomenon in the hospital setting.

Several strategies have been proposed for rationalising laboratory utilisation,²,⁵,⁶,¹⁷,¹⁸ including rationing of tests, remodelling of request forms, unbundling—but sometimes also constructing—serum panel tests, financial incentives, education about the characteristics or the cost of investigations, computer‐based decision support systems, implementation of guidelines or protocols, peer reviews by experts and feedback via utilisation audits (at personal or team levels). It has been postulated that multidimensional interventions are more likely to be successful than those aimed at one level.¹⁸ In the era of evidence‐based medicine, however, evidence that supports the conclusions of studies on ordering of tests is still weak.²,¹⁸ This is not surprising, as the investigation of individual patients is influenced by many factors other than just scientific evidence.¹⁹ Studies directly comparing different interventions are missing, and with the strategies most used, both successful and unrewarding interventions have been reported. It seems that the most decisive factor for the success of a test‐ordering containment strategy is not the nature of the intervention itself, but rather its design and implementation in a given hospital and medical staff setting.

In this study, containment of undue laboratory utilisation was attempted by using a feedback‐based intervention. Previous studies have shown that feedback strategies are effective when the information provided is directly useful in daily practice²; when doctors, as well as the expert peer, are asked personally, they acknowledge the need for improvement in their current practice and are able to act immediately on the receiving feedback.⁵,¹⁷ All these conditions were met in the present context. In addition, the medical staff was informed about factors (increased age, long hospitalisation, adverse outcome or inability to establish the diagnosis) found to predispose to unnecessary tests in the department. To our best knowledge, this is the first attempt to identify case factors that entail the risk of excessive ordering of tests. Interestingly, inappropriate ordering of tests was significantly decreased in all high‐risk patient groups after the intervention, confirming the rule according to which the higher is the risk, the greater the benefit from measures aiming to reduce it. We note that, although statistically significant, the reduction in unnecessary laboratory tests after intervention was small, in absolute numbers, hence the clinical significance related to this might be questionable. On the other hand, even small differences achieved/patient/day in departments handling large numbers of hospitalised patients may be translated into important cost and resource savings.

The design of this study allowed for concomitant evaluation of all routine haematology and clinical chemistry investigations. Redundant ordering of tests was less marked for haematology investigations (GBC, prothrombin time) compared with biochemistry. This is consistent with previous findings from paediatric¹² and emergency¹⁴ departments. In this study, knowledge of the trainees regarding the cost of the tests they ordered was rather limited. Cost unawareness undoubtedly contributes to inappropriate laboratory utilisation²⁰,²¹; in our study, however, the lowest level of cost awareness was recorded with regard to ABGs, but at the same time this examination was also the one that was least often overused. In view of the fact that ABG sampling is a laborious and time‐consuming task for the clinician, this finding points out that lack of ownership by clinicians (who regard ordering of tests as a laboratory issue) and the ease with which tests can be requested are important reasons for the rising workload and costs in laboratories.²² Interestingly, the request forms in our department do not require ticking boxes for ordering tests, but writing the abbreviations of the tests requested; this again points out that thoughtless ease of ordering tests contributes to the overutilisation of laboratory resources.

Although the senior trainees in our department requested more investigations compared with the juniors, the percentage of avoidable examinations requested by the juniors was actually higher. This is in line with previous suggestions that supervision by senior medical staff, good leadership and medical training are of paramount importance for rationalising laboratory utilisation.⁵,²²

It is interesting to note that almost two thirds of the investigations ordered beyond the first 24 h of hospitalisation did not seem to have contributed to diagnosis, whereas only approximately one fourth from the tests ordered on the first day in hospital, seemed to be redundant. The percentage of tests ordered on the first day, for which not adequate rationale was provided, is striking, and can be explained only in the context of reports on laboratory overuse from emergency departments.¹⁴,¹⁵ Repetition is undoubtedly one of the main factors contributing to unnecessary ordering of tests⁶,⁷; yet, the optimal time interval between repetitions of laboratory tests assessing organ functions (as the ones we analysed) in various circumstances is far from being concluded.¹,³,⁷,⁸

The decrease in the unnecessary ordering of tests after the intervention was not sustained: the effectiveness waned at the end of the semester after the intervention. Similar results have been obtained with other forms of intervention²³,²⁴,²⁵,²⁶; perpetuation seems, therefore, to be critical for any effect to last.²,²⁷ The rotation of junior doctors in hospital departments also makes it unlikely that the effect of any intervention of this kind is sustained over time, unless audits are repeated at regular intervals, combined with continuous education of the medical roster on the reasons and consequences of inappropriate ordering of tests.

The absence of a post‐intervention control group is one of the limitations of our study. However, contamination of a control group would be unavoidable under the working conditions prevailing in our institution. To overcome the lack of a control group, we included in the comparison only those doctors who were practising in the department during both periods. Furthermore, the inpatient population of our department is homogeneous over time, and as a result, patient characteristics were comparable on both occasions. Incomplete or inaccurate documentation in the medical records could also confuse the evaluation of the necessity of the investigations. Although every effort was made to minimise this effect (by excluding from analysis all those records where incomplete documentation was repeated), such a confounder seems partially inevitable in studies of this kind. In any case, evaluation based on the reasoning as documented in the medical records seems more informative than administrative data²⁸ or than data derived from request forms, where documentation of the reasoning is incomplete (at times absent) and often detached.¹

Conclusion

Assessment of a large number of redundant laboratory investigations ordered during routine practice by the trainees of an academic medical department was used to identify factors predisposing to laboratory overutilisation. A feedback approach based on the results of this assessment led to a major, yet transient restriction on inappropriate test‐ordering behaviour.

Key points

• Two thirds of common laboratory investigations ordered during hospitalisation of patients did not influence management decisions.

• Factors in the case profile independently associated with overuse of diagnostic tests included prolonged stay in hospital, increased age and unfavourable outcome or inability to establish the diagnosis.

• Redundant ordering of tests occurred less often for haematology investigations compared with biochemistry; examination of arterial blood gas was least often overused test.

• Trainees had a low and disparate level of awareness about the cost of laboratory examinations that they order routinely.

• An intervention including audit, education and alertness of doctors, which was based on assessment of factors contributing to laboratory overutilisation, resulted in a marked decrease in the unnecessary ordering of tests; however, this containment gradually waned during the semester after intervention.

Abbreviations

ABGs - arterial blood gases

FBC - full blood count