ABSTRACT
Health care-associated infection (HAI) rates are subject to public reporting and are linked to hospital reimbursement from the Centers for Medicare and Medicaid Services (CMS). The increasing pressure to lower HAI rates comes at a time when advances in the clinical microbiology laboratory (CML) provide more-precise and -sensitive tests, altering HAI detection in ways that may increase reported HAI rates. I review how changing CML practices can impact HAI rates and how the financial implications of HAI metrics may produce pressure to change diagnostic testing practices. Finally, I provide suggestions for how to respond to this rapidly changing environment.
KEYWORDS: clinical microbiology, infection prevention, nosocomial, pay for performance, health care-associated infections
COMMENTARY
Consider the following three scenarios related to the role of the clinical microbiology laboratory (CML) in health care-associated infection (HAI) surveillance and prevention. In scenario 1, during an investigation of rising rates of central-line-associated bloodstream infection (CLABSI), the infection prevention program (IPP) notes that some CLABSIs were due to organisms that grew in only one of several blood cultures and which, prior to the institution of matrix-assisted laser desorption–ionization time of flight (MALDI-TOF) mass spectrometry for organism identification, would have been classified as contaminants. An example included a CLABSI attributed to an unusual species of Actinomyces that previously would have been categorized as a “diphtheroid.” Your hospital leadership requests that you either revert to former identification methods or change your reporting in a way that prevents these events from being classified as CLABSIs.
In scenario 2, since you changed to a nucleic acid amplification test (NAAT) for Clostridium difficile detection, your positivity rate has increased by over 100%. The rate of hospital-onset C. difficile infection (HO-CDI) has increased similarly. During an investigation of the increase in CDI, you find that many samples positive by NAAT are toxin negative by enzyme immunoassay (EIA) and that many patients with a low pretest probability of disease are being tested. However, after an initiative to improve testing practices, your HO-CDI lab identification (LabID) event rate remains high (standardized infection ratio [SIR], >1). Concerned about how this rate will impact the value-based purchasing (VBP) and health care-associated condition (HAC) scores, your hospital leadership asks you to consider a change back to EIA for CDI diagnosis.
In scenario 3, even after a campaign to reduce your hospital's rate of catheter-associated urinary tract infection (CAUTI) through a reduction in catheter use and improvement in catheter placement and care, the CAUTI rate remains unacceptably high. Your hospital epidemiologist sets up a meeting with you to discuss how to reduce the number of urine cultures ordered or performed by the laboratory as another way to reduce both the CAUTI rate and unnecessary antibiotic use. She is open to almost any idea to reduce culture ordering, from using “reflex testing” (where culture is performed only when urinalysis reveals evidence for inflammation) to requiring laboratory director approval for every urine culture ordered from catheterized inpatients.
The above scenarios are all based upon actual situations confronted by clinical microbiologists in practice. In each case, the laboratory was asked to change its testing approach in order to reduce the number of cases meeting a current National Healthcare Safety Network (NHSN) HAI definition (1). There is no question that a renewed focus on HAI reduction in U.S. hospitals is being driven in part by the relationship between HAI rates and hospital reimbursement via the VBP and HAC reduction programs (2, 3). Both the VBP and HAC reduction programs are administered by the Centers for Medicare and Medicaid Services (CMS) in an effort to link hospital payments to improvements in quality of care (2, 3). Table 1 describes characteristics of each of these pay-for-performance programs, with a focus on the HAI metrics that are included. All told, up to 3% of CMS payments are at risk, which can amount to millions of dollars for a large medical center. Despite recent studies that suggest VBP and HAC measures are not good indicators of hospital quality (4, 5), these programs (or something similar) are likely to remain in place for the foreseeable future.
TABLE 1.
Characteristics of programs that tie reimbursement to HAI metrics for fiscal year 2017
| Program | Money at risk | Incentive possible? | HAIs included |
|---|---|---|---|
| Value-based purchasing | 2% of payments | Yes, the program reallocates funds from low to high performers | CAUTI, CLABSI, SSI (colon/hysterectomy),a C. difficile infection,b MRSA bacteremiab |
| Hospital-acquired condition reduction | 1% of payments | No, the incentive is a penalty only for the worst quartile | CAUTI, CLABSI, SSI (colon/hysterectomy),a C. difficile infection,b MRSA bacteremiab |
SSI, surgical-site infection rates for colon surgery and abdominal hysterectomy.
The metric is the NHSN LabID event.
Of the five HAI metrics included in the VBP and HAC programs, four of them (CAUTI, CLABSI, CDI, and hospital-onset methicillin-resistant Staphylococcus aureus [MRSA] bloodstream infection) depend heavily on the results of a diagnostic test performed in CMLs. For two of the metrics (CDI and MRSA), the NHSN definition includes only the positive test result, the admission date, and unit (LabID events for CDI and MRSA bacteremia), with no additional patient-level clinical information (6). In an effort to reduce subjectivity and facilitate electronic reporting, future definition changes will likely rely more on objective results from electronic medical records, such as laboratory test results, and less on clinical symptoms.
UNINTENDED CONSEQUENCES FOR CMLs
Meanwhile, diagnostic technology is improving, allowing for greater precision and sensitivity than ever before (7). These advances, while they have great potential to improve patient care and outcomes, may also have the effect of increasing reportable HAI rates. In scenario 1 above, the greater precision in species identification provided by MALDI-TOF mass spectrometry may change the classification of some positive blood cultures from contaminants to CLABSIs. The most common scenario leading to this result is the finding of one blood culture positive (out of two or more) for an organism that can now be identified to the species level and that previously would have been reported with less precise terminology (e.g., “diphtheroid”). In such cases, if the organism identified is not on the NHSN list of common commensals, but the genus is included in the “all organisms” list, the episode counts as a CLABSI (8). In scenario 2 above, the increased sensitivity of the C. difficile NAAT results in a predictable increase in CDI rates of 50% or more (9, 10). Although the NHSN includes a “test method” variable in the risk adjustment formula for HO-CDI rate calculations (11), the risk adjustment is clearly not adequate to fully account for the increased sensitivity of the NAAT. For example, at our center, we used an algorithm for CDI testing that allowed us to determine that our SIR using the NAAT is nearly twice what it would be if we used an EIA alone (0.95 versus 0.5) (12).
In an environment in which the stakes of each HAI are so high and in which the only acceptable HAI rate is zero, inclusion of diagnostic test results into HAI definitions creates perverse incentives for hospitals to influence laboratory testing for reasons not directly related to improved patient care. One incentive, as outlined above, is to choose less sensitive tests (e.g., the EIA rather than the NAAT for CDI diagnosis), which has the obvious drawback of failing to diagnose some patients who would benefit from treatment. Another incentive is to reduce the number of diagnostic tests ordered. In scenario 3, for example, the lab director is faced with choosing the best option for limiting the ordering of urine cultures in catheterized inpatients as part of an effort to reduce the CAUTI rate.
Reducing diagnostic test utilization may not always be detrimental to patient care; indeed, improved stewardship of diagnostic tests can benefit the patient as well as assist in reducing reported HAI rates. The best example of this is CAUTI. Many institutions have reported success in CAUTI reduction via approaches that have the effect of reducing the number of urine cultures ordered or performed (13–15). In some cases the interventions include application of guideline-driven culture practices (13, 15), while in other cases reflex testing involves culturing urine only if evidence of inflammation is found upon urinalysis (UA) (14). Because most positive urine cultures in catheterized patients represent catheter-associated asymptomatic bacteriuria (CA-ASB) and not CAUTI, because most CA-ASB does not progress to CAUTI (16), and because there are so many other causes of fever in hospitalized patients, reducing urine culture ordering in catheterized inpatients is a laudable goal and can both reduce the NHSN-defined CAUTI rate and reduce unnecessary antibiotic use (14, 15). The same principle (restricting test ordering to those patients with the highest pretest probability of disease) can and should be applied to CDI testing. Rather than adopting less sensitive tests for CDI, the CML director should work with clinical partners to implement policies to limit more sensitive testing (e.g., NAAT) to those truly at risk (e.g., antibiotic exposure, frequent liquid stools, etc.). Up to half of hospitalized patients tested for CDI do not have significant diarrhea, and over 40% in some studies received laxatives (17). Testing patients with low pretest probability of CDI thus increases the likelihood that a positive test represents C. difficile colonization rather than disease (18, 19). As for CAUTI, limiting test ordering for CDI may not only reduce NHSN-defined HAI rates but may also improve patient care and improve antibiotic stewardship (18, 19).
In other situations, efforts to limit diagnostic testing may be misguided and detrimental to patient care. In an effort to lower their CAUTI rate, one hospital implemented a policy of treating all catheterized patients with a third-generation cephalosporin if their UA revealed inflammatory cells, performing culture only if signs and symptoms developed later (personal communication). A recent publication from a different health care system quotes a house officer describing his approach to a possible CLABSI: “There's like the central-line infection protocols…. If you suspect that anybody has any type of bacteremia, you do not do a blood culture, you just do a urine culture and pull the lines…. we just do not even test for it because the quality improvement then like marks you off” (20). These limitations on diagnostic testing are obviously inappropriate and dangerous, subjecting the patient to unnecessary antibiotic therapy, misdiagnosis, or worse. The CDC and CMS have heard enough reports similar to those described above that they jointly published a letter warning hospitals that “depart[ing] from standard diagnostic practices to avoid reporting infections to NHSN” can “put patients at risk,” leading to “use of antibiotics that is not necessary, such as treatment for bacterial colonization rather than infection, or antibiotic treatment that is not informed by culture results” (21).
At the other end of the diagnostic-testing spectrum, some hospitals have established protocols that include ordering tests upon admission for patients who have no signs or symptoms of an infection. Examples include ordering urine culture or a C. difficile NAAT on all admitted patients in order to detect asymptomatic bacteriuria or the C. difficile carrier state. This ostensibly allows the hospital to claim that any subsequent infection was present on admission and thus not reportable to NHSN as an HAI (21). Needless to say, these practices also increase the risk to patients by exposing some to unnecessary antibiotic therapy, in addition to increasing hospital costs. Such approaches also belie a fundamental misunderstanding of HAI surveillance and prevention. Most HAIs are due to organisms that are part of the patient's flora prior to the infection, and the fact that an organism colonizes a patient's urine, stool, or nares does not mean the hospital is not responsible for preventing it from later causing disease.
Finally, even in the absence of attempts to change practices in order to lower HAI rates, there is substantial background variation in diagnostic practices that makes interfacility comparison (which is what the VBP and HAC programs are based upon) problematic. One study of 16 pediatric intensive care units found major differences across units in several aspects of diagnosis of CLABSI, including basic blood culture practices (e.g., volume, number, sites, and frequency). The investigators then devised a “surveillance aggressiveness score,” which (unsurprisingly) correlated with the unit's CLABSI rate (“the harder you look, the more you find”) (22). As described above, these diagnostic practice variations become even more problematic when public reporting and financial penalties are introduced into the HAI prevention equation. As described by Dixon-Woods and Perencevich,
policy moves have converted a locally useful surveillance measure into what social scientists call a “reactive” measure: the kind of measure that modifies the phenomenon under study and in the process changes the thing being measured. Put bluntly, the more that organizations are incentivized by the prospect of shaming or financial penalties to decrease sensitivity—and thus not to find cases—the less certain it is that they are reporting a valid assessment of their infection rate (23).
USE AND MISUSE OF SURVEILLANCE DEFINITIONS
It is useful to step back and consider the main purpose for which hospitals and health care systems perform HAI surveillance, which is to help inform local infection prevention efforts. Some important attributes of a good surveillance definition include the use of objective data when possible, high interrater reliability, and consistent application over time (24). Sensitivity is favored over specificity, so as not to miss potentially preventable events, and some degree of misclassification is expected (which will even out over time, provided that the definitions are applied consistently). Ideally, the data produced from such surveillance would be used at the local level to detect outbreaks, to measure changes that result from new HAI prevention initiatives, and to help set HAI prevention priorities during annual risk assessments.
Unfortunately, financial penalties based upon interfacility comparisons place a great deal of pressure on HAI surveillance metrics, which changes the metrics by distorting the incentives associated with them. This phenomenon is not limited to the health care setting. Whenever extreme pressure (in the form of financial rewards or penalties) is placed on a metric, human nature guarantees that complications will follow; recent well-publicized examples include widespread gaming that occurs when law enforcement is under pressure to lower crime rates (25) or when teachers are under pressure to improve student test scores (26). This phenomenon is known as Goodhart's law: “when a measure becomes a target, it ceases to be a good measure” (23, 27, 28).
Thus, one can argue that the best solution to this problem is to no longer tie interfacility comparisons of HAI rates to financial penalties. However, understandable consumer and payer pressure to improve patient safety makes this outcome unlikely. Therefore, as clinical microbiologists, we need to adjust to this high-pressure environment in a way that limits unintended adverse consequences.
RECOMMENDATIONS
I offer the following suggestions for approaching this high-stakes environment.
(i) CML leadership should select diagnostic approaches with the goal of improving individual patient outcomes. At times, this will align with efforts to reduce reportable HAI rates (e.g., reducing urine cultures among catheterized inpatients and limiting CDI testing to those with a high pretest probability of disease), and at other times, it will not (e.g., introduction of MALDI-TOF mass spectrometry for organism identification and adoption of the NAAT for CDI detection).
(ii) Hospital and IPP leadership should not pressure CMLs to alter diagnostic practices based on the need to demonstrate lower HAI rates for pay-for-performance measures. Regulatory and accrediting agencies (e.g., the CMS, state agencies, and The Joint Commission) likewise should be alert to changes in diagnostic practice that are associated with changes in HAI rates.
(iii) Public health authorities like the CDC and NHSN must be proactive in adjusting HAI metrics to changing CML technology. Rapid updating of the master organism lists used by the NHSN to define commensals is needed to “catch up” to the increased precision offered by MALDI-TOF mass spectrometry. Similarly, measures that are driven by diagnostic test results require risk adjustment when new technology that changes test performance significantly is introduced. Tertiary care teaching hospitals are often the earliest adopters of such technology, putting them at a disadvantage in interfacility comparisons that are used in pay-for-performance programs.
(iv) The CMS should reconsider the use of laboratory-identified event metrics in pay-for-performance programs. Laboratory-identified event metrics do not take into account any clinical information beyond admission date and location, may vary substantially based upon the diagnostic technology applied, and for nonsterile sites may conflate colonization with disease. Although measuring LabID events has the advantage of being less labor-intensive and more objective, they are still subject to gaming and to interfacility variation in diagnostic practice.
(v) Measures of diagnostic aggressiveness should be developed and validated for selected HAIs. Rates of blood culture utilization and other aspects of blood culture practice might help inform the adjustment of CLABSI rates for interfacility comparison. Alternatively, diagnostic-test-independent clinical syndromes might be further developed and validated. For example, the rate of health care-associated clinical sepsis or systemic inflammatory response syndrome (SIRS) might help interpret the significance of changes in CLABSI rates. If a hospital reduced its CLABSI rate by 90% but saw no change in health care-associated sepsis or SIRS events, the implication is that the change in CLABSI may be related to changes in diagnostic aggressiveness or the application of surveillance definitions.
(vi) CML leadership should be represented on the infection prevention committee and should advocate for CMLs as an integral part of the infection prevention program. Only by closely collaborating with the infection prevention program can CML leadership inform the IPP and hospital leaders regarding the impact of changing diagnostic practices on reportable HAI rates. In so doing, they can also explain any unintended adverse consequences that may arise from attempts to reduce reportable HAI events via changes in diagnostic practices.
In conclusion, the CML is an essential partner in the diagnosis, management, and prevention of health care-associated infections (29). Increased pressure to improve HAI prevention metrics (for interfacility comparison and pay-for-performance metrics) must never interfere with optimal diagnostic strategies. Close collaboration between the CML, the IPP, and hospital leadership, along with some adjustments to current HAI definitions and pay-for-performance programs, can help ensure that the focus remains firmly on the patient and can provide confidence that declining HAI rates are indeed a reflection of safer care.
ACKNOWLEDGMENTS
I receive research funding from bioMérieux, Inc. I am currently cochair of the CDC Healthcare Infection Control Practices Advisory Committee (HICPAC).
The views expressed in this work are mine and do not necessarily reflect the views of the CDC or HICPAC.
The views expressed in this Commentary do not necessarily reflect the views of the journal or of ASM.
Funding Statement
No external funding was received for this work.
REFERENCES
- 1.Centers for Disease Control and Prevention. 2016. National Healthcare Safety Network. CDC, Atlanta, GA: https://www.cdc.gov/nhsn/ Accessed 17 December 2016. [Google Scholar]
- 2.Centers for Medicare and Medicaid Services. 2016. Hospital value-based purchasing. Centers for Medicare and Medicaid Services, Baltimore, MD: https://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/hospital-value-based-purchasing/index.html. [Google Scholar]
- 3.Centers for Medicare and Medicaid Services. 2016. Hospital Acquired Conditions Reduction Program. Centers for Medicare and Medicaid Services, Baltimore, MD: https://www.cms.gov/Medicare/Medicare-Fee-for-Service-Payment/AcuteInpatientPPS/HAC-Reduction-Program.html. [Google Scholar]
- 4.Figueroa JF, Tsugawa Y, Zheng J, Orav EJ, Jha AK. 2016. Association between the value-based purchasing pay for performance program and patient mortality in US hospitals: an observational study. BMJ 353:i2214. doi: 10.1136/bmj.i2214. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Rajaram R, Chung JW, Kinnier CV, Barnard C, Mohanty S, Pavey ES, McHugh MC, Bilimoria KY. 2015. Hospital characteristics associated with penalties in the CMS HAC reduction program. JAMA 314:375–383. doi: 10.1001/jama.2015.8609. [DOI] [PubMed] [Google Scholar]
- 6.Centers for Disease Control and Prevention/National Healthcare Safety Network. 2016. Multidrug-resistant organism and Clostridium difficile infection (MDRO/CDI) module. Centers for Disease Control and Prevention, Atlanta, GA: https://www.cdc.gov/nhsn/PDFs/pscManual/12pscMDRO_CDADcurrent.pdf. [Google Scholar]
- 7.Caliendo AM, Gilbert DN, Ginocchio CC, Hanson KE, May L, Quinn TC, Tenover FC, Alland D, Blaschke AJ, Bonomo RA, Carroll KC, Ferraro MJ, Hirshhorn LR, Joseph WP, Karchmer T, MacIntyre AT, Reller LB, Jackson AF. 2013. Better tests, better care: improved diagnostics for infectious diseases. Clin Infect Dis 57(S3):S139–S170. doi: 10.1093/cid/cit578. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Centers for Disease Control and Prevention. NHSN organism list. Centers for Disease Control and Prevention, Atlanta, GA: www.cdc.gov/nhsn/xls/master-organism-com-commensals-lists.xlsx Accessed 11 February 2017. [Google Scholar]
- 9.Koo HL, Van JN, Zhao M, Ye X, Revell PA, Jiang ZD, Grimes CZ, Koo DC, Lasco T, Kozinetz CA, Garey KW, Dupont HL. 2014. Real-time polymerase chain reaction detection of asymptomatic Clostridium difficile colonization and rising C. difficile-associated disease rates. Infect Control Hosp Epidemiol 35:667–673. doi: 10.1086/676433. [DOI] [PubMed] [Google Scholar]
- 10.Moehring RW, Lofgren ET, Anderson DJ. 2013. Impact of change to molecular testing for Clostridium difficile infection on healthcare facility-associated incidence rates. Infect Control Hosp Epidemiol 34:1055–1061. doi: 10.1086/673144. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Dudeck MA, Weiner LM, Malpiedi PJ. 2013. Risk adjustment for healthcare facility-onset C. difficile and MRSA bacteremia laboratory-identified event reporting in NHSN. Centers for Disease Control and Prevention, Atlanta, GA: http://www.cdc.gov/nhsn/pdfs/mrsa-cdi/RiskAdjustment-MRSA-CDI.pdf. [Google Scholar]
- 12.Marra AR, Edmond MB, Ford BA, Herwaldt LA, Algwizani AR, Diekema DJ. 17 October 2016. Failure of risk-adjustment by test method for C. difficile laboratory-identified event reporting. Infect Control Hosp Epidemiol doi: 10.1017/ice.2016.227. [DOI] [PubMed] [Google Scholar]
- 13.Mullin KM, Kovacs CS, Fatica C, Einloth C, Neuner EA, Guzman JA, Kaiser E, Menon V, Castillo L, Popovich MJ, Manno EM, Gordon SM, Fraser TG. 17 November 2016. A multifaceted approach to reduction of catheter-associated urinary tract infections in the intensive care unit with an emphasis on “stewardship of culturing.” Infect Control Hosp Epidemiol doi: 10.1017/ice.2016.266. [DOI] [PubMed] [Google Scholar]
- 14.Sarg M, Waldrop GE, Beier MA, Heil EL, Thom KA, Preas MA, Johnson JK, Harris AD, Leekha S. 2016. Impact of changes in urine culture ordering practice on antimicrobial utilization in intensive care units at an academic medical center. Infect Control Hosp Epidemiol 37:448–454. doi: 10.1017/ice.2015.334. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Trautner BW, Grigoryan L, Petersen NJ, Hysong S, Cadena J, Patterson JE, Naik AD. 2015. Effectiveness of an antimicrobial stewardship approach for urinary catheter associated asymptomatic bacteriuria. JAMA Intern Med 175:1120–1127. doi: 10.1001/jamainternmed.2015.1878. [DOI] [PubMed] [Google Scholar]
- 16.Hooton TM, Bradley SF, Cardenas DD, Colgan R, Geerlings SE, Rice JC, Saint S, Schaeffer AJ, Tampayh PA, Tenke P, Nicolle LE. 2010. Diagnosis, prevention, and treatment of catheter-associated urinary tract infection in adults: 2009 international clinical practice guidelines from the Infectious Diseases Society of America. Clin Infect Dis 50:625–663. doi: 10.1086/650482. [DOI] [PubMed] [Google Scholar]
- 17.Buckel WR, Avdic E, Carroll KC, Gunaseelan V, Hadhazy E, Cosgrove SE. 2015. Gut check: Clostridium difficile testing and treatment in the molecular testing era. Infect Control Hosp Epidemiol 36:217–221. doi: 10.1017/ice.2014.19. [DOI] [PubMed] [Google Scholar]
- 18.Dubberke ER, Burnham CA. 2015. Diagnosis of Clostridium difficile infection: treat the patient, not the test. JAMA Intern Med 175:1801–1802. doi: 10.1001/jamainternmed.2015.4607. [DOI] [PubMed] [Google Scholar]
- 19.Polage CR, Gyorke CE, Kennedy MA, Leslie JL, Chin DL, Wang S, Nguyen HH, Huang B, Tang YW, Lee LW, Kim K, Taylor S, Romano PS, Panacek EA, Goodell PB, Solnick JV, Cohen SH. 2015. Overdiagnosis of Clostridium difficile in the molecular test era. JAMA Intern Med 175:1792–1801. doi: 10.1001/jamainternmed.2015.4114. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Butler JM, Anderson KA, Supiano MA, Weir CR. 8 November 2016. “It feels like a lot of extra work”: resident attitudes about quality improvement and implications for an effective learning healthcare system. Acad Med doi: 10.1097/ACM.0000000000001474. [DOI] [PubMed] [Google Scholar]
- 21.Bell BP, Conway P. 2015. Adherence to the CDC's infection definitions and criteria is needed to ensure accuracy, completeness, and comparability of infection information. Centers for Disease Control and Prevention, Atlanta, GA: http://www.cdc.gov/nhsn/cms/cms-reporting.html. [Google Scholar]
- 22.Niedner MF. 2010. The harder you look, the more you find: catheter-associated bloodstream infection surveillance variability. Am J Infect Control 38:585–595. doi: 10.1016/j.ajic.2010.04.211. [DOI] [PubMed] [Google Scholar]
- 23.Dixon-Woods M, Perencevich EN. 2013. When counting central line infections counts. Infect Control Hosp Epidemiol 34:555–557. doi: 10.1086/670630. [DOI] [PubMed] [Google Scholar]
- 24.Talbot TR, Bratzler DW, Carrico RM, Diekema DJ, Hayden MK, Huang SS, Yokoe DS, Fishman NO. 2013. Public reporting of health care associated surveillance data: recommendations from the Healthcare Infection Control Practices Advisory Committee. Ann Intern Med 159:631–635. doi: 10.7326/0003-4819-159-9-201311050-00011. [DOI] [PubMed] [Google Scholar]
- 25.Francescani C. 3 September 2012. NYPD report confirms manipulation of crime stats. Reuters, New York, NY: http://www.reuters.com/article/us-crime-newyork-statistics-idUSBRE82818620120309 Accessed 19 December 2016. [Google Scholar]
- 26.Rich M. 2 April 2013. Scandal in Atlanta reignites debate over test's role. New York Times, New York, NY: http://www.nytimes.com/2013/04/03/education/atlanta-cheating-scandal-reignites-testing-debate.html Accessed 19 December 2016. [Google Scholar]
- 27.Bevan G, Hood C. 2006. What's measured is what matters: targets and gaming in the English public health care system. Public Admin 84:517–538. doi: 10.1111/j.1467-9299.2006.00600.x. [DOI] [Google Scholar]
- 28.Strathern M. 1997. “Improving ratings”: audit in the British university system. Eur Rev 5:305–321. doi: 10.1017/S1062798700002660. [DOI] [Google Scholar]
- 29.Diekema DJ, Saubolle MA. 2011. Clinical microbiology and infection prevention. J Clin Microbiol 49:S57–S60. doi: 10.1128/JCM.00690-11. [DOI] [Google Scholar]
