Abstract
Background
Repetitive inpatient laboratory testing in the face of clinical stability is a marker of low-value care. However, for commonly encountered clinical scenarios on medical units, there are no guidelines defining appropriate use criteria for laboratory tests.
Objective
This study seeks to establish consensus-based recommendations for the utilization of common laboratory tests in medical inpatients.
Design
This study uses a modified Delphi method. Participants completed two rounds of an online survey to determine appropriate testing frequencies for selected laboratory tests in commonly encountered clinical scenarios. Consensus was defined as agreement by at least 80% of participants.
Participants
Participants were 36 experts in internal medicine across Canada defined as internists in independent practice for ≥ 5 years with experience in medical education, quality improvement, or both. Experts represented 8 of the 10 Canadian provinces and 13 of 17 academic institutions.
Main Measures
Laboratory tests and clinical scenarios included were those that were considered common on medical units. The final survey contained a total of 45 clinical scenarios looking at the utilization of six laboratory tests (complete blood count, electrolytes, creatinine, urea, international normalized ratio, and partial thromboplastin time). The possible frequency choices were every 2–4 h, 6–8 h, twice a day, daily, every 2–3 days, weekly, or none unless there was specific diagnostic suspicion. These scenarios were reviewed by two internists with training in quality improvement and survey methods.
Key Results
Of the 45 initial clinical scenarios included, we reached consensus on 17 scenarios. We reached weak consensus on an additional 19 scenarios by combining two adjacent frequency categories.
Conclusions
A Canadian expert panel of internists has provided frequency recommendations on the utilization of six common laboratory tests in medical inpatients. These recommendations need validation in prospective studies to assess whether restrictive versus liberal laboratory test ordering impacts patient outcomes.
KEY WORDS: laboratory tests, utilization, consensus-based recommendations, modified Delphi, internal medicine
INTRODUCTION
Laboratory testing is an important contributor to health care expenditure,1–3 and yet up to 42% of laboratory testing could be considered wasteful.4–7 Redundant testing has been estimated to waste up to 5 billion USD annually in the USA8–10 and Canadians receive over 1 million unnecessary tests each year.4 Unnecessary diagnostic testing performed without appropriate consideration of pretest probability can generate false-positive results, which drives further unnecessary tests and wasted health care dollars.3 Therefore, redundant laboratory testing could result in substantial downstream costs even if the individual tests themselves are relatively inexpensive. The motivation to improve testing practices extends beyond cost savings. Excessive blood work may result in additional unintended consequences which include patient discomfort, hospital-acquired anemia, unnecessary transfusions, prolonged hospitalizations, over-investigation of false positives, and increased mortality for patients with cardiopulmonary diseases.3,11,12 Efforts to reduce the frequency of laboratory tests can improve patient satisfaction and reduce costs without worsening patient outcomes, readmission rates, critical care utilization, or mortality.3,13,14
Several organizations attempt to address unnecessary laboratory testing. Choosing Wisely is a campaign to help clinicians and patients engage in conversations about reducing unnecessary tests, treatments, and procedures.15 Several of its recommendations focus on appropriate utilization of laboratory testing. The American Association of Blood Banks recommends against performing serial blood counts on clinically stable patients.16 The Society for the Advancement of Blood Management recommends against performing laboratory blood testing unless clinically indicated or necessary for diagnosis/management in order to avoid iatrogenic anemia.17 Choosing Wisely Canada recommends internists to avoid ordering repeated complete blood count and chemistry testing in the face of clinical and lab stability in the inpatient setting.18 Although there is some evidence to guide this in the perioperative setting, there is minimal existing evidence in the medical inpatient population to guide either indications or frequencies for use of common laboratory tests.19
Without clear guidance on how frequently to order common laboratory tests and under what circumstances, optimization of inpatient laboratory testing is difficult. This is particularly relevant for complex patients admitted under internal medicine who are often responsible for significant resource use in teaching hospitals.20 In addition, there is substantial inter-physician variability with respect to practices and patterns of testing even within the same practice setting.21 The aim of this study was to develop consensus-based frequency recommendations for the use of common laboratory tests in routinely encountered clinical scenarios on general medical units. This contributes to filling the current gap of evidence-based guidelines for laboratory testing.
METHODS
Study Design
To reach consensus on frequency recommendations on common laboratory tests, we used the modified Delphi22 approach. Consensus was defined as > 80% of agreement by experts on the same frequency choice. This cutoff is in keeping with current recommendations for consensus-based studies.23 Consensus was considered weak when it was reached by combining two adjacent frequency categories. We determined a priori to conduct no more than three rounds of voting.24 All rounds of this closed survey were conducted using an online survey tool25 between November 2017 and March 2018. Each expert who completed all rounds of the survey received $50 CAD in honorarium.
Participants
An expert considered for inclusion in this study was an internist who had been in independent practice in a Canadian medical unit for at least 5 years and who had made significant contributions to the fields of quality improvement, medical education, or both. This included serving as an examiner at the Royal College of Physicians and Surgeons of Canada, authoring peer-reviewed publications in either area, and/or holding educational or quality improvement leadership roles within their hospitals. Medical units considered for this study are Canadian clinical teaching units, which are non-critical care general internal medicine inpatient teaching wards.26 Our expert panel was limited to this setting because similar types of patients are cared for in a similar fashion across the country. This homogeneity assists with consensus building given the shared mental model held by experts across the country.
As only a small subset of academic internists in Canada have known expertise in the fields of quality improvement and/or medical education, we used a non-probabilistic snowball technique to form our expert panel, by first targeting experts known to our authorship team.27 Targeting a panel size of at least 10, consistent with guidelines on consensus methods,21 and assuming a response rate of 25%, we sought to invite a minimum of 40 experts to participate in this panel through an e-mail invitation.
Laboratory Test Selection and Survey Development
We performed a review of laboratory test use in medical units in four adult tertiary care hospitals in Western Canada to identify the highest cost contributors to laboratory test expenditure (Appendix Table 4). We decided to focus on the top contributors, i.e., complete blood count and differential (CBC), electrolytes, renal studies (creatinine and urea), extended electrolytes (calcium, magnesium, phosphate), and coagulation studies [international normalized ratio (INR) and partial thromboplastin time (PTT)]. Although not a high-cost contributor, we also included creatine kinase in our survey because its narrow range of utilization makes it a good candidate for an attempt to derive consensus-based recommendations for use. Research team members who were general internists with expertise in quality improvement and survey design/consensus methods (A.A. and I.M. respectively), with input from local internists, compiled an initial survey draft of commonly encountered clinical scenarios (total 123) where the above laboratory tests may be ordered on medical units. The scenarios referred only to general medical units in Canada and did not include scenarios which might require an intensive care unit admission.
Table 4.
Individual Contributions of the Top 15 Contributors to Laboratory Test Expenditure on Medical Units from the Period of 2015–2018 in Four Adult Tertiary Care Hospitals in Western Canada
| Laboratory test | % contribution to total expenditure on laboratory testing |
|---|---|
| Complete blood count and differential | 15.7 |
| Electrolytes (sodium/potassium/chloride/bicarbonate) | 9.2 |
| Creatinine and urea | 8.5 |
| Extended electrolytes (calcium/magnesium/phosphate) | 6.2 |
| Coagulation studies (PT INR/PTT) | 5.7 |
| Liver studies (ALT/total and direct bilirubin/AST/ALP/GGT) | 4.9 |
| Blood gas arterial | 3.2 |
| Respiratory infection panel (viral) | 3.2 |
| MRSA swab | 2.9 |
| Blood culture | 2.8 |
| Anti-GBM (GBM, ANCA, MPO, PR3, ANA) | 2.1 |
| Vancomycin level | 1.3 |
| Troponin | 1.3 |
| CK | 1.2 |
| Alpha-1 antitrypsin | 1.1 |
Italicized tests are those that were included in the draft survey
PT INR prothrombin time international normalized ratio, PTT partial thromboplastin time, ALT alanine aminotransferase, AST aspartate aminotransferase, ALP alkaline phosphatase, GGT gamma-glutamyltransferase, MRSA methicillin-resistant Staphylococcus aureus, GBM glomerular basement membrane, ANCA antineutrophil cytoplasmic antibodies, MPO myeloperoxidase, PR3 proteinase 3, ANA antinuclear antibody, CK creatine kinase
We piloted the initial draft survey on 12 internists who were not part of the expert panel and solicited feedback on survey usability and technical functionality. Based on their input regarding common clinical scenarios, we reduced the number of scenarios to 45 and the number of laboratory tests to six in the final survey to optimize survey length while still focusing on common scenarios for the most utilized tests. These six tests were CBC (13 scenarios), electrolytes (14 scenarios), creatinine (7 scenarios), urea (3 scenarios), INR (5 scenarios), and PTT (3 scenarios). For each of the 45 clinical scenarios, we asked experts how frequently they would recommend ordering the associated laboratory test on a time scale that included the following selections: every 2–4 h, every 6–8 h, twice a day, daily, every 2–3 days, weekly, once for diagnostic workup, or not indicated.
Participants were provided space for written feedback in the survey. Scenarios where expert comments demonstrated a requirement for more contextual clarity were modified and included in round 2. For each scenario, frequency range choices that had received no votes in round 1 were removed for round 2. We provided statistical group response feedback to participants between rounds including quantitative results (% agreement in prior round for each scenario).24
RESULTS
Sixty-four experts were invited to participate in this panel. A total of 36 members participated representing 13 of 17 (76%) Canadian academic institutions and 8 of the 10 Canadian provinces (Table 1). The majority (n = 31, 86%) of the experts were specialists in internal medicine or general internal medicine and the remaining (n = 5, 14%) had additional training in other medicine subspecialties (Table 1).
Table 1.
Demographic of the 36 Members of the Expert Panel Convened for the Study
| Demographic | N (%) |
|---|---|
| Academic institution | |
| University of British Columbia | 6 (17) |
| University of Calgary | 1 (3) |
| University of Alberta | 1 (3) |
| University of Saskatchewan | 1 (3) |
| University of Manitoba | 1 (3) |
| Western University | 1 (3) |
| McMaster University | 2 (6) |
| University of Toronto | 9 (25) |
| Queen’s University | 1 (3) |
| University of Ottawa | 6 (17) |
| McGill University | 4 (11) |
| Université de Sherbrooke | 1 (3) |
| Dalhousie University | 2 (6) |
| Province | |
| British Columbia | 6 (17) |
| Alberta | 2 (6) |
| Saskatchewan | 1 (3) |
| Manitoba | 1 (3) |
| Ontario | 19 (53) |
| Québec | 5 (14) |
| New Brunswick | 1 (3) |
| Nova Scotia | 1 (3) |
| Gender | |
| Male | 19 (53) |
| Female | 17 (47) |
| Subspecialty* | |
| Internal medicine/general internal medicine | 31 (86) |
| Critical care | 2 (6) |
| Infectious disease | 1 (3) |
| Nephrology | 1 (3) |
| Respirology | 1 (3) |
| Medical biochemistry | 1 (3) |
| Years of independent practice in internal medicine | |
| 5 to 7 years | 5 (14) |
| 8 to 10 years | 4 (11) |
| 11 to 15 years | 4 (11) |
| 16 to 20 years | 6 (17) |
| 21 years or more | 17 (47) |
| Average number of weeks on the medical teaching unit per year over the past 5 years | |
| 1 to 10 weeks | 8 (22) |
| 11 to 20 weeks | 21 (58) |
| 21 to 30 weeks | 5 (14) |
| 31 or more weeks | 2 (6) |
*Some respondents indicated more than one specialty choice. One expert had a combined respirology/critical care specialty and is represented in both
Round One
Of the 45 clinical scenarios included, consensus was reached in nine clinical scenarios, weak consensus was reached for 20 scenarios, and no consensus was reached for the remaining 16 scenarios (Fig. 1, Appendix Table 5).
Figure 1.
Flow diagram of the consensus process summary.
Table 5.
Survey Results of Round 1; Number (%) of 36 Members Who Voted in Each Scenario
| Scenarios per laboratory test | Number of experts (%) who voted for each frequency range | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Complete blood count | Q2–4 h | Q6–8 h | BID | Daily | Q2–3 days | Weekly | Once for diagnosis | Not indicated | Total | |
| 1a | General surveillance for a stable in-patient | 0 (0) | 0 (0) | 0 (0) | 0 (0) | 4 (11) | 12 (34)b | 10 (29)b | 9 (26)b | 35 |
| 2 | Monitoring an actively bleeding patient requiring hemodynamic support | 15 (44)b | 18 (53)b | 1 (3) | 0 (0) | 0 (0) | 0 (0) | 0 (0) | 0 (0) | 34 |
| 3a | Monitoring an actively bleeding patient not requiring hemodynamic support | 1 (3) | 9 (26)b | 19 (54)b | 6 (17) | 0 (0) | 0 (0) | 0 (0) | 0 (0) | 35 |
| 4a | Monitoring new anemia (hemoglobin < 70 g/L)—no active bleeding | 0 (0) | 0 (0) | 0 (0) | 27 (75)b | 5 (14)b | 2 (6) | 2 (6) | 0 (0) | 36 |
| 5a | Monitoring new anemia (hemoglobin ≥ 70 g/L)—no active bleeding | 0 (0) | 0 (0) | 0 (0) | 16 (44)b | 16 (44)b | 1 (3) | 2 (6) | 1 (3) | 36 |
| 6a | Monitoring chronic asymptomatic anemia (hemoglobin 70–120 g/L) | 0 (0) | 0 (0) | 0 (0) | 0 (0) | 7 (19)b | 14 (39)b | 9 (25)b | 6 (17) | 36 |
| 7a | Monitoring in patient with known inflammatory state (sepsis, rheumatological flare-up) | 0 (0) | 0 (0) | 0 (0) | 11 (31)b | 16 (44)b | 3 (8)b | 3 (8) | 3 (8) | 36 |
| 8a | Monitoring in patient with leukocytosis not yet diagnosed | 0 (0) | 0 (0) | 0 (0) | 24 (67)b | 9 (25)b | 0 (0) | 2 (6) | 1 (3) | 36 |
| 9 | Monitoring new thrombocytopenia (platelets < 50 × 109/μL) | 0 (0) | 0 (0) | 1 (3) | 33 (92)b | 2 (6) | 0 (0) | 0 (0) | 0 (0) | 36 |
| 10 | Monitoring new thrombocytopenia (platelets 50–150 × 109/μL) | 0 (0) | 0 (0) | 0 (0) | 19 (53)b | 15 (42)b | 1 (3) | 1 (3) | 0 (0) | 36 |
| 11 | Monitoring chronic asymptomatic thrombocytopenia (platelets < 50 × 109/μL) | 0 (0) | 0 (0) | 0 (0) | 5 (14)b | 16 (44)b | 10 (28)b | 3 (8) | 2 (6) | 36 |
| 12 | Monitoring chronic asymptomatic thrombocytopenia (platelets 50–150 × 109/μL) | 0 (0) | 0 (0) | 0 (0) | 0 (0) | 10 (28)b | 16 (44)b | 7 (19)b | 3 (8) | 36 |
| 13 | Monitoring on treatment associated with CBC abnormalities (heparin, antibiotics, etc.) | 0 (0) | 0 (0) | 0 (0) | 14 (40)b | 13 (37)b | 6 (17)b | 1 (3) | 1 (3) | 35 |
| Electrolytes | ||||||||||
| 14a | General surveillance for an in-patient | 0 (0) | 0 (0) | 0 (0) | 2 (6) | 10 (28)b | 7 (19)b | 9 (25)b | 8 (22)b | 36 |
| 15 | Monitoring in DKA being treated with IV insulin infusion | 29 (81)b | 7 (19) | 0 (0) | 0 (0) | 0 (0) | 0 (0) | 0 (0) | 0 (0) | 36 |
| 16 | Monitoring on treatment associated with electrolyte abnormalities (e.g., new diuresis) | 0 (0) | 1 (3) | 1 (3) | 30 (83)b | 4 (11) | 0 (0) | 0 (0) | 0 (0) | 36 |
| 17 | Monitoring acute severe hyponatremia (< 120 mmol/L) | 19 (53)b | 14 (39)b | 3 (8) | 0 (0) | 0 (0) | 0 (0) | 0 (0) | 0 (0) | 36 |
| 18a | Monitoring acute hyponatremia (120–134 mmol/L) | 1 (3) | 9 (25)b | 12 (33)b | 14 (39)b | 0 (0) | 0 (0) | 0 (0) | 0 (0) | 36 |
| 19 | Monitoring severe chronic hyponatremia (< 120 mmol/L) | 4 (11) | 3 (8)b | 9 (25)b | 15 (42)b | 4 (11)b | 1 (3) | 0 (0) | 0 (0) | 36 |
| 20 | Monitoring chronic hyponatremia (120–134 mmol/L) | 0 (0) | 1 (3) | 3 (9) | 12 (34)b | 13 (37)b | 3 (9)b | 1 (3) | 2 (6) | 35 |
| 21 | Monitoring hypernatremia requiring hypotonic IV fluids | 5 (14) | 17 (47)b | 7 (19)b | 7 (19)b | 0 (0) | 0 (0) | 0 (0) | 0 (0) | 36 |
| 22 | Monitoring hypernatremia not requiring hypotonic IV fluids | 0 (0) | 0 (0) | 4 (11)b | 28 (78)b | 3 (8) | 0 (0) | 0 (0) | 1 (3) | 36 |
| 23 | Initial monitoring of acute hyperkalemia requiring shifting | 22 (61)b | 13 (36)b | 1 (3) | 0 (0) | 0 (0) | 0 (0) | 0 (0) | 0 (0) | 36 |
| 24 | Initial monitoring of mild/moderate acute hyperkalemia not requiring shifting | 0 (0) | 3 (8) | 21 (58)b | 12 (33)b | 0 (0) | 0 (0) | 0 (0) | 0 (0) | 36 |
| 25a | Monitoring stable hyperkalemia (5.5–6.5 mmol/L) | 0 (0) | 1 (3) | 3 (8) | 27 (75)b | 4 (11)b | 1 (3) | 0 (0) | 0 (0) | 36 |
| 26 | Monitoring new hypokalemia requiring IV/PO replacement | 0 (0) | 3 (8) | 16 (44)b | 17 (47)b | 0 (0) | 0 (0) | 0 (0) | 0 (0) | 36 |
| 27 | Monitoring new hypokalemia not requiring potassium replacement | 0 (0) | 0 (0) | 1 (3) | 24 (69)b | 9 (26)b | 0 (0) | 0 (0) | 1 (3) | 35 |
| Creatinine | ||||||||||
| 28a | General surveillance for an inpatient | 0 (0) | 0 (0) | 0 (0) | 1 (3) | 6 (17) | 11 (31)b | 12 (33)b | 6 (17)b | 36 |
| 29 | Monitoring in patient with sepsis | 0 (0) | 0 (0) | 2 (6) | 34 (94)b | 0 (0) | 0 (0) | 0 (0) | 0 (0) | 36 |
| 30 | Initial monitoring with use of new nephrotoxic agents/therapies (contrast, diuretics, antibiotics, large volume paracentesis) | 0 (0) | 0 (0) | 0 (0) | 34 (94)b | 2 (6) | 0 (0) | 0 (0) | 0 (0) | 36 |
| 31 | Monitoring in patient with worsening AKI | 0 (0) | 0 (0) | 4 (11) | 32 (89)b | 0 (0) | 0 (0) | 0 (0) | 0 (0) | 36 |
| 32 | Monitoring in patient with improving AKI | 0 (0) | 0 (0) | 0 (0) | 24 (67)b | 12 (33)b | 0 (0) | 0 (0) | 0 (0) | 36 |
| 33a | Monitoring in patient on chronic dialysis | 0 (0) | 0 (0) | 0 (0) | 1 (3) | 8 (22)b | 8 (22)b | 3 (8)b | 16 (44)b | 36 |
| Urea | ||||||||||
| 34 | Monitoring in stable chronic kidney disease | 0 (0) | 0 (0) | 0 (0) | 0 (0) | 10 (29)b | 15 (43)b | 5 (14)b | 5 (14) | 35 |
| 35a | General surveillance for an inpatient | 0 (0) | 0 (0) | 0 (0) | 0 (0) | 2 (6) | 3 (9) | 5 (14)b | 25 (71)b | 35 |
| 36 | Diagnosis of gastrointestinal bleed | 0 (0) | 0 (0) | 0 (0) | 3 (9) | 1 (3) | 0 (0) | 14 (40)b | 17 (49)b | 35 |
| 37a | Diagnosis of uremic encephalopathy | 0 (0) | 0 (0) | 0 (0) | 7 (20) | 3 (9)b | 0 (0)b | 12 (34)b | 13 (37)b | 35 |
| International normalized ratio (INR) | ||||||||||
| 38 | General surveillance for an inpatient | 0 (0) | 0 (0) | 0 (0) | 0 (0) | 0 (0) | 4 (11) |
11 (31) b |
20 (57) b |
35 |
| 39 | Diagnosis of a bleeding diathesis (congenital/acquired, e.g., disseminated intravascular coagulation, liver disease) | 0 (0) | 1 (3) | 0 (0) | 14 (40)b | 3 (9)b | 1 (3)b | 16 (46)b | 0 (0) | 35 |
| 40 | Monitoring in patient on warfarin (new start/new dose/concurrent antibiotics | 0 (0) | 0 (0) | 0 (0) | 25 (71)b | 10 (29)b | 0 (0) | 0 (0) | 0 (0) | 35 |
| 41 | Monitoring in patient on stable dose warfarin with therapeutic INRs | 0 (0) | 0 (0) | 0 (0) | 2 (6) | 13 (37)b | 16 (46)b | 3 (9) | 1 (3) | 35 |
| 42 | Prior to invasive procedures where INR check is recommended | 0 (0) | 0 (0) | 0 (0) | 4 (12) | 0 (0) | 0 (0) | 30 (88)b | 0 (0) | 34 |
| Partial thromboplastin time (PTT) | Per local heparin infusion protocol | Daily | Q2–3 days | Weekly | Once for diagnosis | Not indicated | Total | |||
| 43 | General surveillance for an in-patient | 0 (0) | 0 (0) | 0 (0) | 0 (0) | 8 (23)b | 27 (77)b | 35 | ||
| 44 | Diagnosis of a bleeding diathesis (congenital or acquired) | 0 (0) | 4 (11) | 1 (3) | 1 (3) | 28 (80)b | 1 (3) | 35 | ||
| 45 | Monitoring in patient on intravenous heparin infusion | 33 (97)b | 0 (0) | 0 (0) | 0 (0) | 0 (0) | 1 (3) | 34 | ||
Q every, BID twice daily, AKI acute kidney injury
aScenarios that were modified for round 2
bFrequency choices that in combination represent 80% of votes
Round Two
All 36 members participated in this round. A total of 18 scenarios were included (Fig. 1, Appendix Table 6). Of the 20 scenarios that had reached weak consensus in round 1, 6 were modified for round 2. Of the 16 scenarios that had not reached consensus in round 1, 4 were modified and an additional 4 split into two each for round 2. For this round, the frequency options were the following: every 2–4 h, every 6–8 h, twice a day, daily for 3 days followed by reassessment, every 2–3 days, weekly, and none unless diagnostic suspicion.
Table 6.
Survey Results of Round 2; Number (%) of 36 Members Who Voted in Each Scenario
| Scenarios per Laboratory Tests | Number of experts (%) who voted for each frequency range | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Complete blood count | Q2–4 h | Q6–8 h | BID | Daily × 3 then reassess | Q2–3 days | Weekly | None unless diagnostic suspicion | Total | |
| 1a | General surveillance for a stable inpatient awaiting rehabilitation/transition/placement | N/A | N/A | N/A | 0 (0) | 0 (0) | 2 (6) | 34 (94)a | 36 |
| 1b | General surveillance for a stable inpatient (admitted for > 3 days) with resolving medical issues | N/A | N/A | N/A | 0 (0) | 13 (36)a | 15 (42)a | 8 (22)a | 36 |
| 3 | A slowly bleeding patient NOT requiring crystalloids and/or blood products | 0 (0) | 1 (3) | 13(36)a | 22 (61)a | N/A | N/A | N/A | 36 |
| 4 | New diagnosis of anemia in newly admitted patient without overt bleeding (hemoglobin < 70 g/L) | N/A | N/A | N/A | 34 (94)a | 2 (6) | 0 (0) | 0 (0) | 36 |
| 5 | New diagnosis of anemia in newly admitted patient without overt bleeding (hemoglobin > 70 g/L) | N/A | N/A | N/A | 15 (42)a | 21 (58)a | 0 (0) | 0 (0) | 36 |
| 6a | New admission with chronic asymptomatic anemia (hemoglobin 70–120 g/L) | N/A | N/A | N/A | 2 (6) | 18 (51)a | 9 (26)a | 6 (17)a | 35 |
| 6b | Stable inpatient (admitted > 3 days) with chronic asymptomatic anemia (hemoglobin 70–120 g/L) | N/A | N/A | N/A | 0 (0) | 5 (14) | 17 (47)a | 14 (39)a | 36 |
| 7 | New admission with acute inflammatory state | N/A | N/A | N/A | 25 (69)a | 10 (28)a | 0 (0) | 1 (3) | 36 |
| 8 | Patient being worked up for new leukocytosis | N/A | N/A | N/A | 34 (94)a | 2 (6) | 0 (0) | 0 (0) | 36 |
| Electrolytes | |||||||||
| 14a | Stable inpatient awaiting rehabilitation/transition/placement | N/A | N/A | N/A | 0 (0) | 0 (0) | 6 (17) | 30 (83)a | 36 |
| 14b | Stable inpatient (admitted > 3 days) with resolving medical issues | N/A | N/A | N/A | 2 (6) | 17 (47)a | 11 (31)a | 6 (17)a | 36 |
| Creatinine | |||||||||
| 28a | Stable inpatient awaiting rehabilitation/transition/placement | N/A | N/A | N/A | 0 (0) | 0 (0) | 3 (9) | 32 (91)a | 35 |
| 28b | Stable inpatient (admitted > 3 days) with resolving medical issues | N/A | N/A | N/A | 1 (3) | 19 (54)a | 9 (26)a | 6 (17) | 35 |
| 33 | Patient on chronic hemodialysis with no residual renal function | N/A | N/A | N/A | 0 (0) | 2 (7) | 3 (10) | 26 (84)a | 31 |
| Electrolytes | 120–122 mmol/L | 123–125 mmol/L | 126–128 mmol/L | 129–131 mmol/L | |||||
| 18a | For a patient with acute hyponatremia (120–134 mmol/L), above what sodium level would you be comfortable with only daily testing (not more frequently) of sodium? | 0 (0) | 8 (23) | 22 (63) | 5 (14) | 35 | |||
| Electrolytes | Yes, I agree | No, I disagree | |||||||
| 25 | For a patient with stable hyperkalemia (5.5–6.0 mmol/L), would you in general be comfortable with only daily monitoring (not more frequently) of potassium as an inpatient? | 36 (100)a | 0 (0) | 36 | |||||
| Urea | |||||||||
| 35 | Urea testing is NOT indicated for most stable inpatients without specific diagnostic suspicion | 33 (94)a | 2 (6) | 35 | |||||
| 37 | Urea level is NOT indicated in the diagnosis and/or follow up of uremic encephalopathy? | 21 (60) | 14 (40) | 35 | |||||
Q every, BID twice daily, N/A frequency option was not available in round 2 of survey
aFrequency selections that together represent 80% of votes
Of the six scenarios that had been modified from the “weak consensus” pool from round 1, four scenarios now reached consensus4,8,25,28 while the other two remained in the weak consensus category.3,5 Of the total of 12 scenarios that had entered round 2 from the no consensus pool from round 1, we arrived at consensus for four, weak consensus for three, and no consensus for the remaining five scenarios (Fig. 1).
Summary of Consensus
We started with 45 clinical scenarios and for round 2 split up scenarios 1, 6, 14, and 28 leading to a total of 49 surveyed scenarios. Of these, we arrived at consensus on frequency recommendations for 17 scenarios (Table 2), weak consensus for 19 scenarios (Table 3), and no consensus for 13 scenarios (Table 3).
Table 2.
Consensus-Based Frequency Recommendation for Scenarios That Reached Consensus or Weak Consensus (over Two Adjacent Frequency Categories)
| Clinical scenarios by laboratory test | Final frequency recommendation | Recommendation strength | |
|---|---|---|---|
| Complete blood count (CBC) | |||
| *1a | Stable inpatient awaiting rehabilitation/transition/placement | None unless diagnostic suspicion | Consensus |
| 2 | Monitoring an actively bleeding patient requiring hemodynamic support | Q2-8h | Weak consensus |
| *3 | A slowly bleeding patient NOT requiring crystalloids and/or blood products | BID—daily × 3 then r/a | Weak consensus |
| *4 | New diagnosis of anemia in newly admitted patient without overt bleeding (Hemoglobin <70 g/L) | Daily × 3 then reassess | Consensus |
| *5 | New diagnosis of anemia in newly admitted patient without overt bleeding (Hemoglobin>70 g/L) | Daily × then r/a—q2–3 days | Weak consensus |
| *6b | Stable inpatient (admitted >3 days) with chronic asymptomatic anemia (Hemoglobin 70-120 g/L) | Weekly—none unless diagnostic suspicion | Weak consensus |
| *7 | New admission with acute inflammatory state | Daily × 3 then r/a—q2–3 days | Weak consensus |
| *8 | Patient being worked up for new leukocytosis | Daily × 3 then reassess | Consensus |
| 9 | Monitoring new thrombocytopenia (Platelets <50 × 109/μL) | Daily | Consensus |
| 10 | Monitoring new thrombocytopenia (Platelets 50–150 × 109/μL) | Daily—q2–3 days | Weak consensus |
| Electrolytes | |||
| *14a | Stable in-patient awaiting rehabilitation/transition/placement | None unless diagnostic suspicion | Consensus |
| 15 | Monitoring in DKA being treated with IV insulin infusion | Q2-4h | Consensus |
| 16 | Monitoring on treatment associated with electrolyte abnormalities (e.g. new diuresis) | Daily | Consensus |
| 17 | Monitoring acute severe hyponatremia (<120 mEq/L) | Q2-8h | Weak consensus |
| 22 | Monitoring hypernatremia not requiring hypotonic IV fluids | BID—daily | Weak consensus |
| 23 | Initial monitoring of acute hyperkalemia requiring shifting | Q2-8h | Weak consensus |
| 24 | Initial monitoring of mild/moderate acute hyperkalemia not requiring shifting | BID—daily | Weak consensus |
| *25 | For a patient with stable hyperkalemia (5.5–6.0 mmol/L) would you in general be comfortable with only daily monitoring (not more frequently) of potassium as an in-patient? | Daily | Consensus |
| 26 | Monitoring new hypokalemia requiring IV/PO replacement | BID—daily | Weak consensus |
| 27 | Monitoring new hypokalemia not requiring potassium replacement | Weak consensus | |
| Creatinine | |||
| *28a | Stable in-patient awaiting rehabilitation/transition/placement | None unless diagnostic suspicion | Consensus |
| *28b | Stable in-patient (admitted >3 days) with resolving medical issues | Q2–3 days—weekly | Weak consensus |
| 29 | Monitoring in patient with sepsis | Daily | Consensus |
| 30 | Initial monitoring with use of new nephrotoxic agents/therapies (contrast, diuretics, antibiotics, large volume paracentesis) | Daily | Consensus |
| 31 | Monitoring in patient with worsening AKI | Daily | Consensus |
| 32 | Monitoring in patient with improving AKI | Daily—q3 days | Weak consensus |
| *33 | Monitoring in patient on chronic dialysis with no residual renal function | None unless diagnostic suspicion | Consensus |
| Urea | |||
| *35 | Urea testing is NOT indicated for most stable inpatients without specific diagnostic suspicion | None unless diagnostic suspicion | Consensus |
| 36 | Diagnosis of gastrointestinal bleed | Once or not indicated | Weak consensus |
| INR | |||
| 38 | General surveillance for an in-patient | Once for diagnosis—not indicated | Weak consensus |
| 40 | Monitoring in patient on warfarin (new start/new dose/concurrent antibiotics | Daily—q2–3 days | Weak consensus |
| 41 | Monitoring in patient on stable dose warfarin with therapeutic INRs | Q2 days—weekly | Weak consensus |
| 42 | Prior to invasive procedures where INR check is recommended | Once for diagnosis | Consensus |
| PTT | |||
| 43 | General surveillance for an in-patient | Once for diagnosis or not indicated | Weak consensus |
| 44 | Diagnosis of a bleeding diathesis (congenital or acquired) | Once for diagnosis | Consensus |
| 45 | Monitoring in patient on intravenous heparin infusion | Per local heparin infusion | Consensus |
DKA diabetic ketoacidosis, IV intravenous, PO per oral, AKI acute kidney injury, BID twice daily, Q every
*Indicates scenario was reworded between rounds 1 and 2 for clarity
Table 3.
Survey Results on Scenarios Where No Consensus Was Reached
| Clinical scenarios by laboratory test | |
|---|---|
| Complete blood count (CBC) | |
| 1b | Stable inpatient (admitted for > 3 days) with resolving medical issues |
| 6a | New admission with chronic asymptomatic anemia (hemoglobin 70–120 g/L) |
| 11 | Monitoring chronic asymptomatic thrombocytopenia (platelets < 50 × 109/μL) |
| 12 | Monitoring chronic asymptomatic thrombocytopenia (platelets 50–150 × 109/μL) |
| 13 | Monitoring on treatment associated with CBC abnormalities (heparin, antibiotics, etc.) |
| Electrolytes | |
| 14b | Stable inpatient (admitted > 3 days) with resolving medical issues |
| 18 | Monitoring acute hyponatremia (120–134 mEq/L) |
| 19 | Monitoring severe chronic hyponatremia (< 120 mEq/L) |
| 20 | Monitoring chronic hyponatremia (120–134 mEq/L) |
| 21 | Monitoring hypernatremia requiring hypotonic IV fluids |
| Urea | |
| 34 | Monitoring in stable chronic kidney disease |
| 37 | Diagnosis of uremic encephalopathy |
| INR | |
| 39 | Diagnosis of a bleeding diathesis (congenital/acquired, e.g., disseminated intravascular coagulation, liver disease) |
DISCUSSION
In this study, for ordering laboratory investigations on the medical inpatient unit, frequency recommendations were reached for 17 common clinical scenarios and weak consensus was reached for 19 clinical scenarios. Our experts seemed to agree on scenarios that require urgent, daily, or no blood work at all. For example, they agreed that patients with diabetic ketoacidosis should get electrolytes tested every 2–4 h, but for several other conditions daily testing sufficed (e.g., daily creatinine for patients with sepsis, acute kidney injury, or those on nephrotoxic agents; daily electrolytes when abnormalities were anticipated and daily CBC for workup for severely abnormal cell counts). For stable inpatients who are awaiting rehabilitation/transition/placement, our experts agreed that regular blood work was not indicated. In addition, urea was generally thought to be unnecessary for most patients. Lastly, for diagnostic purposes, our experts felt that testing of coagulation parameters once during the hospital stay was sufficient, to be repeated only in the setting of anticipated invasive procedure.
Our experts were less able to agree in the scenarios where testing needs to be done either more than daily or less than daily. For instance, in patients with gastrointestinal bleeding or acute electrolyte abnormalities (sodium/potassium), experts generally agreed that testing should be done more frequently than daily. However, a number of experts pointed out that the exact frequency would depend on numerous contextual factors that the scenario description cannot capture. In scenario 2, looking at the use of CBC for monitoring an actively bleeding patient requiring hemodynamic support, our experts indicated that the exact frequency would depend on the starting hemoglobin, the rate of bleeding, requirements for and response to transfusion, and timing and success of planned interventions. Thus, the suggested frequency may range between every 2–8 h. Similarly in scenarios involving patients with resolving issues, those with less severe or chronic anemia, less severe or chronic thrombocytopenia, in patients on warfarin, and those with stable chronic kidney disease, experts agree that testing should generally be less than daily. Again the exact frequency would depend on other factors. Often, the scenarios where the experts desired more clinical information and context were the ones where there was either weak or no consensus.
Our expert panel, comprised predominantly of general internists, was not able to agree on the utility of urea in diagnosis/monitoring of gastrointestinal bleeding and uremic encephalopathy.
Although we only arrived at consensus in 17 scenarios, the results from the other scenarios (including weak and no consensus) still provide insight into expert recommendations of laboratory test utilization. For instance, in scenario 1b (Appendix Table 6), for a stable inpatient with resolving medical issues, while experts did not agree on how frequently CBC should be performed, all agreed that it should be done no more frequently than every 2–3 days. Hence, even though there was no consensus on a specific frequency window for several scenarios, the frequency range can still be helpful to guide optimization efforts.
Unnecessary blood work is often ordered daily in many institutions.29,30 Several studies acknowledge the paucity of consensus on what comprises appropriate laboratory testing in this population and recognize a need for guideline development.28,31–35 However, little has been done to establish appropriate testing frequencies in medical inpatients possibly because there are too many variables that make the task onerous. We used a geographically diverse expert panel of internists in Canada to provide frequency recommendations for the utilization of six laboratory among medical inpatients. These six tests are known to be highly utilized, both from local data (Appendix Table 4) and in the test utilization literature.13,14,20,29 Even though we arrived at consensus in only 17 scenarios, we believe that knowledge of the range of frequency selections by our expert group can still help with professional development and guide quality improvement efforts to standardize practices. Intervention bundles used to optimize laboratory testing often include an educational component.3 These recommendations could be incorporated into the Choosing Wisely Toolkit and help standardize the educational component of these bundles and be used to set benchmarks for audit and feedback.
There are several limitations to our study. First, our group is composed entirely of internists who work on Canadian medical units. This limits the generalizability of our recommendations outside the Canadian medical teaching unit context. However, medical units serve as teaching units for Canadian internists. We also know that spending habits picked up during residency can persist for years.36 Hence, optimization of laboratory test use in this population and setting may impact the practice pattern of current residents and future internists. Second, a disadvantage of using non-probabilistic sampling strategy is that it may be difficult to assess how representative our sample is compared to all possible relevant experts in the field. Third, the scope of our study was such that we were able to focus only on the common scenarios for highly utilized tests. We did not attempt to comprehensively define all possible indications for each laboratory test. The description of clinical scenarios was general and could not possibly capture all relevant contextual features. Thus, any ensuing recommendations cannot replace nuanced clinical judgment. Fourth, our recommendations are only based on expert opinion-based consensus. We did not grade the strength of our recommendations nor conduct a systematic review on all applications. However, given the lack of evidence on the ideal testing frequency for most of these scenarios, we believe that consensus-based recommendations from experts serve as an important starting point. As additional evidence becomes available on appropriate use of laboratory tests, the current recommendations will need to be updated. Fifth, the frequency recommendations are limited to scenarios commonly seen on general medical units and may not be applicable to specific scenarios encountered on specialized services. Sixth is the issue of representation; we deliberately sought to seek general internists as experts in the field of laboratory test management in the inpatient setting. However, we notice the ambiguity in certain areas (e.g., use of urea for diagnosis of gastrointestinal bleeding or uremic encephalopathy) where additional medical subspecialty representation may have been useful. Lastly, our recommended test order frequency do not take into account automatic test bundling that may be in place for a variety of reasons in specific institutions. For example, it may not be possible to order an INR without a PTT or a sodium and potassium without extended electrolytes. Individual physicians will need to take these practice constraints into consideration.
In conclusion, our expert panel consensus-based recommendations highlight considerate and indication-driven utilization of laboratory testing in the inpatient setting. They are not intended to replace clinical judgment. In the setting of limited evidence in this area, consensus-based recommendations are an important intermediate step as we move towards evidence-based guidelines directing appropriate use of laboratory tests. These recommendations can guide future clinical trials of restrictive versus liberal frequency of laboratory testing to assess their impact on patient-oriented outcomes.
Acknowledgments
We would like to acknowledge the members of our expert panel for their participation in the survey.
Appendix
Funders
This study was funded by Alberta Health Services. The funding body played no role in the design of the study; collection, analysis, and interpretation of the data; and the decision to approve publication of the finished manuscript.
Compliance with Ethical Standards
Conflict of Interest
The authors declare that they do not have a conflict of interest.
IRB Approval
This study was approved by the Conjoint Health Research Ethics Board at the University of Calgary (REB17-0702).
Footnotes
Prior Presentations
This study was presented as a poster presentation at the Canadian Society for Internal Medicine Conference Oct 14, 2018 in Banff, Alberta, Canada
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
References
- 1.Berwick DM. A user’s manual for the IOM’s ‘Quality Chasm’ report. Health affairs (Project Hope). 2002;21(3):80–90. doi: 10.1377/hlthaff.21.3.80. [DOI] [PubMed] [Google Scholar]
- 2.Berwick DM, Hackbarth AD. Eliminating waste in US health care. Jama. 2012;307(14):1513–6. doi: 10.1001/jama.2012.362. [DOI] [PubMed] [Google Scholar]
- 3.Eaton KP, Levy K, Soong C, Pahwa AK, Petrilli C, Ziemba JB, et al. Evidence-Based Guidelines to Eliminate Repetitive Laboratory Testing. JAMA Intern Med. 2017;177(12):1833–9. doi: 10.1001/jamainternmed.2017.5152. [DOI] [PubMed] [Google Scholar]
- 4.Canadian Institute for Health Information Unnecessary Health Care in Canada. Unnecessary Health Care in Canada. 2017. https://www.cihi.ca/en/unnecessary-care-in-canada-infographic. Accessed April 17 2018.
- 5.Fuchs VR. The gross domestic product and health care spending. New Engl J Med. 2013;369(2):107–9. doi: 10.1056/NEJMp1305298. [DOI] [PubMed] [Google Scholar]
- 6.Huck A, Lewandrowski K. Utilization management in the clinical laboratory: an introduction and overview of the literature. Clin Chim Acta; Int J Clin Chem. 2014;427:111–7. doi: 10.1016/j.cca.2013.09.021. [DOI] [PubMed] [Google Scholar]
- 7.McConnell TS, Berger PR, Dayton HH, Umland BE, Skipper BE. Professional review of laboratory utilization. Hum Pathol. 1982;13(4):399–403. doi: 10.1016/S0046-8177(82)80229-3. [DOI] [PubMed] [Google Scholar]
- 8.Jha AK, Chan DC, Ridgway AB, Franz C, Bates DW. Improving safety and eliminating redundant tests: cutting costs in U.S. hospitals. Health Affairs (Project Hope). 2009;28(5):1475–84. doi: 10.1377/hlthaff.28.5.1475. [DOI] [PubMed] [Google Scholar]
- 9.Baird G. The laboratory test utilization management toolbox. Biochem Med. 2014;24(2):223–34. doi: 10.11613/bm.2014.025. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Bates DW, Boyle DL, Rittenberg E, Kuperman GJ, Ma'Luf N, Menkin V, et al. What proportion of common diagnostic tests appear redundant? Am J Med. 1998;104(4):361–8. doi: 10.1016/S0002-9343(98)00063-1. [DOI] [PubMed] [Google Scholar]
- 11.Lin RJ, Evans AT, Chused AE, Unterbrink ME. Anemia in general medical inpatients prolongs length of stay and increases 30-day unplanned readmission rate. South Med J. 2013;106(5):316–20. doi: 10.1097/SMJ.0b013e318290f930. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Thavendiranathan P, Bagai A, Ebidia A, Detsky AS, Choudhry NK. Do blood tests cause anemia in hospitalized patients? The effect of diagnostic phlebotomy on hemoglobin and hematocrit levels. J Gen Intern Med. 2005;20(6):520–4. doi: 10.1111/j.1525-1497.2005.0094.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.May TA, Clancy M, Critchfield J, Ebeling F, Enriquez A, Gallagher C, et al. Reducing unnecessary inpatient laboratory testing in a teaching hospital. Am J Clin Pathol. 2006;126(2):200–6. doi: 10.1309/wp59-ym73-l6ce-gx2f. [DOI] [PubMed] [Google Scholar]
- 14.Wertheim BM, Aguirre AJ, Bhattacharyya RP, Chorba J, Jadhav AP, Kerry VB, et al. An Educational and Administrative Intervention to Promote Rational Laboratory Test Ordering on an Academic General Medicine Service. Am J Med. 2017;130(1):47–53. doi: 10.1016/j.amjmed.2016.08.021. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Choosing Wisely. Philidelphia, PA. 2014. http://www.choosingwisely.org. Accessed March 25 2019.
- 16.American Association of Blood Banks . Five Things Physicians and Patients Should Question. 2014. [Google Scholar]
- 17.Choosing Wisely Clinician Lists: Laboratory Recommendations. 2017. http://www.choosingwisely.org/clinician-lists/#keyword=laboratory. Accessed August 1 2018.
- 18.Canadian Society of Internal Medicine Choosing Wisely Canada Internal Medicine: Five Things Physicians and Patients Should Question. 2018. https://choosingwiselycanada.org/internal-medicine/. Accessed April 25 2018.
- 19.Bock M, Fritsch G, Hepner DL. Preoperative Laboratory Testing. Anesthesiol Clin. 2016;34(1):43–58. doi: 10.1016/j.anclin.2015.10.005. [DOI] [PubMed] [Google Scholar]
- 20.Iwashyna TJ, Fuld A, Asch DA, Bellini LM. The impact of residents, interns, and attendings on inpatient laboratory ordering patterns: a report from one university’s hospitalist service. Acad Med. 2011;86(1):139–45. doi: 10.1097/ACM.0b013e3181fd85c3. [DOI] [PubMed] [Google Scholar]
- 21.McDonald EG, Saleh RR, Lee TC. Mindfulness-Based Laboratory Reduction: Reducing Utilization Through Trainee-Led Daily ‘Time Outs’. Am J Med. 2017;130(6):e241–e4. doi: 10.1016/j.amjmed.2017.01.011. [DOI] [PubMed] [Google Scholar]
- 22.Dalkey N. An experimental study of group opinion. Futures. 1969;1(5):408–426. doi: 10.1016/S0016-3287(69)80025-X. [DOI] [Google Scholar]
- 23.Diamond IR, Grant RC, Feldman BM, Pencharz PB, Ling SC, Moore AM, et al. Defining consensus: a systematic review recommends methodologic criteria for reporting of Delphi studies. J Clin Epidemiol. 2014;67(4):401–9. doi: 10.1016/j.jclinepi.2013.12.002. [DOI] [PubMed] [Google Scholar]
- 24.Humphrey-Murto S, Varpio L, Gonsalves C, Wood TJ. Using consensus group methods such as Delphi and Nominal Group in medical education research() Med Teach. 2017;39(1):14–9. doi: 10.1080/0142159X.2017.1245856. [DOI] [PubMed] [Google Scholar]
- 25.SurveyMonkey Inc San Mateo C, USA,. San Mateo, California, USA. www.surveymonkey.com.
- 26.Gupta S, Detsky AS. Prioritizing continuity in Canadian clinical teaching units. Can Med Assoc J. 2014;186(10):800. doi: 10.1503/cmaj.140565. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Goodman LA. Snowball Sampling. Ann Math Stat. 1961;32(1):148–70. doi: 10.1214/aoms/1177705148. [DOI] [Google Scholar]
- 28.Dhanani JA, Barnett AG, Lipman J, Reade MC. Strategies to reduce inappropriate laboratory blood test orders in intensive care are effective and safe: a before-and-after quality improvement study. Anaesth Intensive Care. 2018;46(3):313–20. doi: 10.1177/0310057X1804600309. [DOI] [PubMed] [Google Scholar]
- 29.Miyakis S, Karamanof G, Liontos M, Mountokalakis TD. Factors contributing to inappropriate ordering of tests in an academic medical department and the effect of an educational feedback strategy. Postgrad Med J. 2006;82(974):823–9. doi: 10.1136/pgmj.2006.049551. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Stuebing EA, Miner TJ. Surgical vampires and rising health care expenditure: reducing the cost of daily phlebotomy. Arch Surg (Chicago, Ill. : 1960). 2011;146(5):524–7. doi: 10.1001/archsurg.2011.103. [DOI] [PubMed] [Google Scholar]
- 31.Corson AH, Fan VS, White T, Sullivan SD, Asakura K, Myint M, et al. A multifaceted hospitalist quality improvement intervention: Decreased frequency of common labs. J Hosp Med. 2015;10(6):390–5. doi: 10.1002/jhm.2354. [DOI] [PubMed] [Google Scholar]
- 32.Gupta SS, Voleti R, Nyemba V, Demir S, Lamikanra O, Musemwa N, et al. Results of a Quality Improvement Project Aimed at Eliminating Healthcare Waste by Changing Medical Resident Test Ordering Behavior. J Clin Med Res. 2017;9(12):965–9. doi: 10.14740/jocmr3210w. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Konger RL, Ndekwe P, Jones G, Schmidt RP, Trey M, Baty EJ, et al. Reduction in Unnecessary Clinical Laboratory Testing Through Utilization Management at a US Government Veterans Affairs Hospital. Am J Clin Pathol. 2016;145(3):355–64. doi: 10.1093/ajcp/aqv092. [DOI] [PubMed] [Google Scholar]
- 34.Melendez-Rosado J, Thompson KM, Cowdell JC, Sanchez Alvarez C, Ung RL, Villanueva A, et al. Reducing unnecessary testing: an intervention to improve resident ordering practices. Postgrad Med J. 2017;93(1102):476–9. doi: 10.1136/postgradmedj-2016-134513. [DOI] [PubMed] [Google Scholar]
- 35.Rudolf JW, Dighe AS, Coley CM, Kamis IK, Wertheim BM, Wright DE, et al. Analysis of Daily Laboratory Orders at a Large Urban Academic Center: A Multifaceted Approach to Changing Test Ordering Patterns. Am J Clin Pathol. 2017;148(2):128–35. doi: 10.1093/ajcp/aqx054. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Poonam S, Wachter A, Ming D, Cohen L, Noppon S. The Hospitalist. 2015. Physician Spending Habits During Residency Training Can Persist for Years. [Google Scholar]