Abstract
Background. The clinical presentation of severe acute respiratory syndrome (SARS) resembles that of other etiologies of community-acquired pneumonia, making diagnosis difficult. Hematological and biochemical abnormalities, particularly lymphopenia, are common in patients with SARS.
Methods. With the use of 2 databases, we compared the ability of the absolute lymphocyte count, absolute neutrophil count, lactate dehydrogenase level, creatine kinase level, alanine aminotransferase level, and serum calcium level at hospital admission to discriminate between cases of SARS and cases of community-acquired pneumonia. The SARS database contained data for 144 patients with SARS from the 2003 Toronto SARS outbreak. The community-acquired pneumonia database contained data for 8044 patients with community-acquired pneumonia from Edmonton, Canada. Patients from the SARS database were matched to patients from the community-acquired pneumonia database according to age, and receiver operating characteristic curves were constructed for each laboratory variable.
Results. The areas under the receiver operating characteristic curves (AUCs) demonstrated fair to poor discriminatory ability for all laboratory variables tested except absolute neutrophil count, which had an AUC of 0.80, indicating good discriminatory ability (although there was no cutoff value of the absolute neutrophil count at which reasonable sensitivity or specificity could be obtained). Combinations of any 2 tests did not perform significantly better than did the absolute neutrophil count alone.
Conclusions. Routine laboratory tests, including determination of absolute lymphocyte count, should not be used in the diagnosis of SARS or incorporated into current case definitions of SARS. The role of the absolute neutrophil count in SARS diagnosis is likely limited, but it should be assessed further.
Severe acute respiratory syndrome (SARS) is a new respiratory illness that emerged in China in February 2003, causing outbreaks in Hong Kong; Toronto, Ontario; Singapore; Taiwan; and Vietnam [1]. SARS is caused by the SARS-associated coronavirus (SARS-CoV) [2–4].
Diagnosis of SARS—and distinguishing SARS-CoV infection from other etiologies of community-acquired pneumonia (CAP)—was one of the major challenges of the global SARS outbreak. The difficulty of diagnosing SARS resulted from the nonspecific clinical and radiographic presentation of SARS, which is similar to that of other causes of CAP [5–10].
Given the nonspecific presentation of SARS, the World Health Organization (WHO), the Centers for Disease Control and Prevention (CDC), and other groups proposed case definitions that combined clinical features (e.g., fever or respiratory symptoms) with epidemiological features (e.g., direct contact with another patient with SARS or travel to a SARS-affected area) to improve diagnostic accuracy [11, 12]. Unfortunately, these epidemiological factors are also nonspecific and had poor positive predictive values during the global outbreak, particularly in areas where the incidence of SARS was low [13–15].
Patients with SARS frequently have hematological and biochemical abnormalities that are revealed by routine laboratory testing, including lymphopenia (in 54%–98%), thrombocytopenia (37%–55%), elevated lactate dehydrogenase levels (35%–87%), elevated creatine kinase levels (14%–36%), hypocalcemia (60%), and elevated alanine aminotransferase levels (29%) [3, 5–9, 16–23]. Several case series and reviews of SARS diagnosis, as well as a comparative study, suggest that routine hematological and biochemical abnormalities (particularly lymphopenia) can aid in SARS diagnosis [7, 8, 16, 21, 24–26], and current WHO recommendations also suggest measuring hematological and biochemical indices as part of the diagnostic workup for SARS [27]. Despite these recommendations, no study has systematically examined the ability of these tests to accurately discriminate between cases of SARS and cases of CAP due to other causes. This study was intended to assess the ability of routine hematological and biochemical tests to distinguish between SARS-CoV infection and other etiologies of CAP.
Methods
Design, setting, and patients. To determine the ability of 6 common hematological and biochemical tests, measured at hospital admission, to differentiate between patients with SARS and patients with CAP due to other causes, we compared the results from a database of SARS patients with those from a database of CAP patients. The SARS database contained information on 144 patients with SARS from the first phase of the Toronto SARS outbreak, which occurred from March through May 2003. A description of this database has been published previously [8]. Cases were identified by clinicians at all Toronto-area hospitals caring for ⩾10 patients with SARS and were entered into the database if they met the 4 April 2003 CDC case definition for suspect or probable SARS. The definition for suspected SARS consisted of the presence of fever, respiratory symptoms, and an exposure history; the definition for probable SARS included suspected SARS plus abnormal chest radiograph findings consistent with pneumonia [1]. Ethics approval for this study was obtained through the research ethics boards at all participating hospitals and from the University of Toronto.
The CAP database contained information for 8044 patients with CAP who presented to emergency departments at 7 hospitals in Edmonton, Canada, from 1 November 2000 through 31 October 2002. Patients were included in this database if they presented to an emergency department at a participating hospital in Edmonton with ⩾2 symptoms and signs of CAP and had evidence of pneumonia revealed by their initial chest radiograph, as interpreted by the emergency department physician or internal medicine consultant. The symptoms and signs of CAP used to determine inclusion in this database included cough (productive or nonproductive), pleuritic chest pain, shortness of breath, fever (temperature, ⩾38.0°C), or crackles on auscultation. In addition, patients had to be >17 years of age. Pregnant or nursing patients, patients with cystic fibrosis, and patients receiving immunosuppressive drugs that reduced their leukocyte counts to <1.0 × 109 cells/L were excluded. Patients with other causes of leukopenia were not excluded. In addition, patients admitted directly to the intensive care unit from the emergency department were excluded. Data for all patients were entered into the database from 1 November 2000 through 31 October 2002 as part of a study of pneumonia management pathways. Ethics approval for this study was obtained from the research ethics boards at all participating hospitals and the University of Alberta.
Patients from either database were included in this study if they presented to an emergency department with respiratory symptoms and had abnormal chest radiograph findings. Patients in the SARS database who met the exclusion criteria for the CAP database were excluded, with 1 exception, from the analysis. Patients from the SARS database who were admitted directly to an intensive care unit were not excluded from the analysis, because the decision to admit patients with SARS to an intensive care unit was based on different criteria from those usually used for CAP, such as the need for negative-pressure isolation rooms and the need to intubate patients with SARS early in the course of disease and under controlled conditions to minimize the risk of transmission.
Because serological tests for SARS-CoV were not performed for all patients in the SARS database, it was decided, a priori, to conduct the analysis twice, once using both the seropositive patients and those patients for whom definitive convalescent-phase serological testing was never performed and once using only seropositive patients. If the results of both analyses were similar, the first analysis was to be considered the primary analysis.
Measurement of laboratory variables. Routine laboratory tests were ordered at the discretion of the treating physicians and were conducted using standard methods at local laboratories serving hospitals in Toronto and Edmonton. Because reference ranges varied between laboratories, results were standardized by dividing individual results by the upper range of normal for the laboratory at which the test was performed. Following analysis, results were converted back into standard units on the basis of the reference range used for all of the patients in the CAP database.
Serological testing. Serological status was determined by a combination of ELISA and immunofluorescent antibody methods for detection of antibodies against SARS-CoV [28]. Serological status was considered negative if convalescent serum samples obtained ⩾28 days after onset of symptoms had negative serological test results, and serological status was considered positive if ⩾1 method detected IgM or IgG antibodies at predefined dilutions.
Statistical analysis. A separate analysis was performed for each laboratory variable. For absolute lymphocyte count (ALC), all patients from the SARS database that had a baseline ALC recorded were included. The largest possible sample of patients from the CAP database who had also had their baseline ALC recorded was then selected, such that the age distribution by decade matched that of the sample from the SARS database. Age-matching was performed to ensure that the presence of differences in laboratory values was because of differences in pathological processes and not simply because of the expected age difference between the 2 cohorts. This process was then repeated de novo for the absolute neutrophil count (ANC), lactate dehydrogenase (LDH) level, creatine kinase (CK) level, alanine aminotransferase (ALT) level, and serum calcium level.
Medians and interquartile ranges were calculated as summaries of continuous variables. The Wilcoxon rank sum test was used to compare the distributions of each laboratory variable between the SARS database and CAP database patients [29].
Receiver operator characteristic (ROC) curves were constructed and the area under the ROC curve (AUC) was calculated as a measure of discriminative ability [30, 31]. Higher AUC values were considered to demonstrate better discriminatory abilities as follows: excellent discrimination, AUC of ⩾0.90; good discrimination, 0.80 ⩽ AUC < 0.90; fair discrimination, 0.70 ⩽ AUC < 0.80; and poor discrimination, AUC of <0.70.
The values for each laboratory variable were divided into quintiles on the basis of data for the patients with SARS, such that ∼20% of patients with SARS would fall within each of the 5 ranges. Likelihood ratios for the diagnosis of SARS were then determined for values falling within each range.
After examining each laboratory test in isolation, all paired combinations of laboratory tests were then compared to determine whether any pair of laboratory variables would have significantly greater discriminatory ability than that of any of the individual variables.
Results
Of the 144 patients in the SARS database, 52 were excluded from the analysis because of normal chest radiograph findings at presentation (25 patients), lack of respiratory symptoms at presentation (16), or both (11). of the other 92 patients, the results of serological testing for SARS-CoV were available for 67 and were positive for 65 (97%). Given the high proportion of patients who were seropositive and because no significant differences were seen between the primary analysis (patients with positive serological status or patients who did not have testing of convalescent-phase serum samples performed) and secondary analysis (seropositive patients only), only the results of the primary analysis are presented.
Of the 8044 patients in the CAP database, 255 were excluded because of missing age (212 patients) or age <17 years (43 patients).
The number of patients that had each laboratory variable measured and the number of patients available after age-matching for each laboratory variable are shown in table 1. Because each laboratory value was assessed individually, numbers vary from test to test.
The median LDH level (289 U/L vs. 215 U/L; P < .001), CK level (196 U/L vs. 96 U/L; P < .001), and ALT level (35 U/L vs. 23 U/L; P < .001) were significantly elevated in patients with SARS, compared with patients with CAP, whereas the median ANC (3.85 × 109 cells/L vs. 8.3 × 109 cells/L; P < .001) and serum calcium level (2.09 mmol/L vs. 2.22 mmol/L; P < .001) were lower in patients with SARS, compared with patients with CAP (table 2). There was no statistically significant difference in the median ALC between patients with SARS and patients with CAP (1.1 × 109 cells/L vs. 1.2 × 109 cells/L; P = .25) (table 2).
Despite statistically significant differences in median values for all but the ALC, interquartile ranges demonstrated significant overlap between the distributions for SARS and CAP for all laboratory variables (table 2 and figures 1). This is reflected in the lack of discriminatory power identified through analysis of the ROC curves and AUC (figures 1–3). ALC (AUC, 0.54), LDH level (AUC, 0.68), ALT level (AUC, 0.67), and serum calcium level (AUC, 0.68) had poor discriminatory ability, CK level (AUC, 0.71) had fair discriminatory ability, and only the ANC (AUC, 0.80) demonstrated good discriminatory ability.
When ROC curves and AUC were calculated for all paired combinations of laboratory tests, only trivial improvements in AUC were noted, with the maximum AUC being 0.84 (for the paired combination of ANC and ALT levels; data not shown). The likelihood ratios for the diagnosis of SARS for ANC ranged from 7.0 for the lowest quintile (ANC, <2.25 × 109 cells/L) to 0.28 for the highest quintile range (ANC, >6.00 × 109 cells/L). For all other laboratory variables examined, the maximum likelihood ratio for the diagnosis of SARS was 3.3 (LDH level, 334–486 U/L), and the minimum likelihood ratio was 0.41 (ALT level, <22 U/L; data not shown). Furthermore, no specific cutoff value for the ANC could be identified that provided a reasonable sensitivity or specificity. To obtain a sensitivity of 90%, an ANC cutoff value of 7.6 × 109 cells/L provided a specificity of only 55%; conversely, to obtain a specificity of 90%, an ANC cutoff value of 3.6 × 109 cells/L provided a sensitivity of only 46%.
Discussion
Hematological and biochemical abnormalities occur frequently in patients with SARS. This study also demonstrates that several of these abnormalities occur more frequently in patients with SARS than in patients with CAP. However, to be useful diagnostically, the overlap in the distribution of the results for a given test for patients with SARS and those with CAP must be small. Thus, although statistically significant differences in median values were found for all laboratory variables examined except the ALC, the discriminative ability of these tests was limited by the overlap in their distributions, as seen in the box plots and ROC curves (figure 1–3) and quantitatively demonstrated by the low AUC seen for all variables tested except the ANC.
The identification of the ANC as the most discriminatory test assessed, on the basis of an AUC of 0.80, was an unexpected finding and has not, to the best of our knowledge, been previously reported. The discriminatory ability of the ANC is likely caused by elevated ANCs seen in cases of bacterial CAP rather than by any abnormality of the ANC in patients with SARS. The value of the ANC in clinical practice will be limited for several reasons. First, no cutoff ANC was identified that was both sensitive and specific for the diagnosis of SARS. Second, in clinical practice, it will be necessary to distinguish not only between SARS and CAP but also between SARS and other causes of fever and respiratory symptoms, including influenza, other viral respiratory tract infections, and congestive heart failure. Because these other conditions, unlike bacterial CAP, do not typically cause elevations in the ANC, it is unlikely that the low normal ANC typically seen in cases of SARS (median ANC, 3.85 × 109 cells/L) will be capable of distinguishing between SARS and nonbacterial causes of CAP.
The present study has several limitations. First, serological confirmation of the diagnosis of SARS was not obtained for all patients. However, serological test results were positive for 97% of patients tested, and the results of the analysis did not differ when patients without serological results were excluded. Second, because the CAP database did not measure laboratory values over time, we were unable to examine trends in laboratory values after hospital admission. Therefore, it remains possible that greater differences between the laboratory values in patients with SARS, compared with patients with CAP, might occur later in the illness. However, laboratory tests performed late in the course of illness would not help in distinguishing cases of SARS from cases of CAP at the time of initial assessment, when decisions regarding hospital admission, isolation, and treatment must be made. Furthermore, the patients with SARS in this cohort presented to the hospital at a median of 5 days (interquartile range, 3–7 days) after the onset of their symptoms. Given the widespread awareness of SARS, in future outbreaks, patients will likely present earlier, further limiting the value of tests that may not be discriminatory until late in the course of the illness. Third, we examined each test individually but were unable to perform a multivariable analysis because an insufficient number of patients in the CAP database had data for all 6 laboratory values. However, because no individual test demonstrated good discriminatory ability, because the results of several of these tests are unlikely to be independent, and because testing of all paired combinations of tests did not significantly improve discriminatory ability, it is unlikely that any aggregate measurement of these tests is likely to be useful in the diagnosis of SARS.
Since the global SARS outbreak was declared over on 12 June 2003, sporadic cases of animal-to-human transmission of SARS-CoV—like viruses have continued in southern China, and 11 cases of SARS have resulted from laboratory transmission in Singapore, Taiwan, and China [32]. Because the confirmation of a diagnosis of SARS now depends on either serological testing for SARS-CoV (the results of which may not be positive for up to 28 days) or PCR assays for the detection of SARS-CoV (which continue to have limited sensitivity and specificity early in the course of disease), initial assessment of potential SARS cases continues to rely heavily on clinical, radiographic, and epidemiological factors. However, the assessment of patients from southern China with unexplained febrile respiratory illness is complicated by the difficulty of identifying direct epidemiological links in the absence of a recognized outbreak. Therefore, physicians and public health authorities continue to struggle with the difficult task of assessing patients for SARS when they present with unexplained febrile respiratory illnesses, particularly if the patients are travelers from or residents of southern China. As a result, national and international public health authorities have continued to develop and revise case definitions in an attempt to improve their diagnostic accuracy. Some have advocated for the inclusion of routine hematological or biochemical test data (in particular, the ALC) in the case definition, given the frequency with which lymphopenia has been reported in association with SARS [27].
Our results do not support this approach, because we were unable to demonstrate significant discriminative ability for any of the laboratory tests that have been suggested for use in the diagnosis of SARS. In particular, the ALC is a poor discriminator between SARS and CAP. Although these results are derived from a single retrospective study, we believe that additional studies assessing the discriminative ability of laboratory tests should be performed before the use of such laboratory tests is recommended in either formal or informal algorithms for SARS diagnosis.
Acknowledgments
We thank Chris Booth and Larissa Matukas, for their efforts in the creation of the SARS database. We would also like to acknowledge the assistance of the Ontario Laboratory Working Group for the Rapid Diagnosis of Emerging Infections in collection of the serological results.
Financial support. Funding for the establishment of the CAP database was provided through grants from the Capital Health Authority, the Alberta Heritage Foundation for Medical Research, Abbott Canada, Pfizer Canada, and Jansen Ortho Canada. M.P.M. is supported by a research fellowship from Bristol-Myers Squibb.
Potential conflicts of interest. All authors: no conflicts.
References
- 1.Centers for Disease Control and Prevention Outbreak of severe acute respiratory syndrome—worldwide, 2003. MMWR Morb Mortal Wkly Rep. 2003;52:226–8. [PubMed] [Google Scholar]
- 2.Ksiazek TG, Erdman D, Goldsmith CS, et al. A novel coronavirus associated with severe acute respiratory syndrome. N Engl J Med. 2003;348:1953–66. doi: 10.1056/NEJMoa030781. [DOI] [PubMed] [Google Scholar]
- 3.Peiris JS, Lai ST, Poon LL, et al. Coronavirus as a possible cause of severe acute respiratory syndrome. Lancet. 2003;361:1319–25. doi: 10.1016/S0140-6736(03)13077-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Drosten C, Gunther S, Preiser W, et al. Identification of a novel coronavirus in patients with severe acute respiratory syndrome. N Engl J Med. 2003;348:1967–76. doi: 10.1056/NEJMoa030747. [DOI] [PubMed] [Google Scholar]
- 5.Poutanen SM, Low DE, Henry B, et al. Identification of severe acute respiratory syndrome in Canada. N Engl J Med. 2003;348:1995–2005. doi: 10.1056/NEJMoa030634. [DOI] [PubMed] [Google Scholar]
- 6.Tsang KW, Ho PL, Ooi GC, et al. A cluster of cases of severe acute respiratory syndrome in Hong Kong. N Engl J Med. 2003;348:1977–85. doi: 10.1056/NEJMoa030666. [DOI] [PubMed] [Google Scholar]
- 7.Lee N, Hui D, Wu A, et al. A major outbreak of severe acute respiratory syndrome in Hong Kong. N Engl J Med. 2003;348:1986–94. doi: 10.1056/NEJMoa030685. [DOI] [PubMed] [Google Scholar]
- 8.Booth CM, Matukas LM, Tomlinson GA, et al. Clinical features and short-term outcomes of 144 patients with SARS in the greater Toronto area. JAMA. 2003;289:2801–9. doi: 10.1001/jama.289.21.JOC30885. [DOI] [PubMed] [Google Scholar]
- 9.Choi KW, Chau TN, Tsang O, et al. Outcomes and prognostic factors in 267 patients with severe acute respiratory syndrome in Hong Kong. Ann Intern Med. 2003;139:715–23. doi: 10.7326/0003-4819-139-9-200311040-00005. [DOI] [PubMed] [Google Scholar]
- 10.Wong KT, Antonio GE, Hui DS, et al. Severe acute respiratory syndrome: radiographic appearances and pattern of progression in 138 patients. Radiology. 2003;228:401–6. doi: 10.1148/radiol.2282030593. [DOI] [PubMed] [Google Scholar]
- 11.World Health Organization . Case definition for surveillance of severe acute respiratory syndrome (SARS) Available at: http://www.who.int/csr/sars/casedefinition/en. Accessed 20 August 2004. [Google Scholar]
- 12.Centers for Disease Control and Prevention Revised U.S. surveillance case definition for severe acute respiratory syndrome (SARS) and update on SARS cases—United States and worldwide, December 2003. MMWR Morb Mortal Wkly Rep. 2003;52:1202–6. [PubMed] [Google Scholar]
- 13.Schrag SJ, Brooks JT, Van Beneden C, et al. SARS surveillance during emergency public health response, United States, March-July 2003. Emerg Infect Dis. 2004;10:185–94. doi: 10.3201/eid1002.030752. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Tambyah PA, Singh KS, Habib AG. SARS: understanding the coronavirus: accuracy of WHO criteria was similar in a “non-SARS” hospital in Singapore. BMJ. 2003;327:620. doi: 10.1136/bmj.327.7415.620-a. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Rainer TH, Cameron PA, Smit D, et al. Evaluation of WHO criteria for identifying patients with severe acute respiratory syndrome out of hospital: prospective observational study. BMJ. 2003;326:1354–8. doi: 10.1136/bmj.326.7403.1354. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Cui W, Fan Y, Wu W, Zhang F, Wang JY, Ni AP. Expression of lymphocytes and lymphocyte subsets in patients with severe acute respiratory syndrome. Clin Infect Dis. 2003;37:857–9. doi: 10.1086/378587. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Vu HT, Leitmeyer KC, Le DH, et al. Clinical description of a completed outbreak of SARS in Vietnam, February-May 2003. Emerg Infect Dis. 2004;10:334–8. doi: 10.3201/eid1002.030761. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Jang TN, Yeh DY, Shen SH, Huang CH, Jiang JS, Kao SJ. Severe acute respiratory syndrome in Taiwan: analysis of epidemiological characteristics in 29 cases. J Infect. 2004;48:23–31. doi: 10.1016/j.jinf.2003.09.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Wong RS, Wu A, To KF, et al. Haematological manifestations in patients with severe acute respiratory syndrome: retrospective analysis. BMJ. 2003;326:1358–62. doi: 10.1136/bmj.326.7403.1358. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Hsu LY, Lee CC, Green JA, et al. Severe acute respiratory syndrome (SARS) in Singapore: clinical features of index patient and initial contacts. Emerg Infect Dis. 2003;9:713–7. doi: 10.3201/eid0906.030264. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Yuen E, Kam CW, Rainer TH. Role of absolute lymphocyte count in the screening of patients with suspected SARS. Emerg Med (Fremantle) 2003;15:395–6. doi: 10.1046/j.1442-2026.2003.00486_3.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Tsui PT, Kwok ML, Yuen H, Lai ST. Severe acute respiratory syndrome: clinical outcome and prognostic correlates. Emerg Infect Dis. 2003;9:1064–9. doi: 10.3201/eid0909.030362. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Zhao Z, Zhang F, Xu M, et al. Description and clinical treatment of an early outbreak of severe acute respiratory syndrome (SARS) in Guangzhou, PR China. J Med Microbiol. 2003;52:715–20. doi: 10.1099/jmm.0.05320-0. [DOI] [PubMed] [Google Scholar]
- 24.Jernigan JA, Low DE, Hefland RF. Combining clinical and epidemiologic features for early recognition of SARS. Emerg Infect Dis. 2004;10:327–33. doi: 10.3201/eid1002.030741. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Hui DS, Wong PC, Wang C. SARS: clinical features and diagnosis. Respirology. 2003;8:S20–4. doi: 10.1046/j.1440-1843.2003.00520.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Wong GW, Hui DS. Severe acute respiratory syndrome (SARS): epidemiology, diagnosis and management. Thorax. 2003;58:558–60. doi: 10.1136/thorax.58.7.558. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.World Health Organization . Management of severe acute respiratory syndrome (SARS) Available at: http://www.who.int/csr/sars/management/en. Accessed 20 August 2004. [Google Scholar]
- 28.Tang P, Louie M, Richardson SE, et al. Interpretation of diagnostic laboratory tests for severe acute respiratory syndrome: the Toronto experience. CMAJ. 2004;170:47–54. [PMC free article] [PubMed] [Google Scholar]
- 29.Conover WJ. Practical non-parametric statistics. New York, NY: John Wiley and Sons; 1980. [Google Scholar]
- 30.Hanley JA. Receiver operating characteristic (ROC) methodology: the state of the art. Critical Reviews in Diagnostic Imaging. 1989;29:307–35. [PubMed] [Google Scholar]
- 31.Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982;143:29–36. doi: 10.1148/radiology.143.1.7063747. [DOI] [PubMed] [Google Scholar]
- 32.World Health Organization . Severe acute respiratory syndrome. Available at: http://www.who.int/csr/don/archive/disease/severe_acute_respiratory_syndrome/en/. Accessed 20 August 2004. [Google Scholar]