An official website of the United States government
Here's how you know
Official websites use .gov
A
.gov website belongs to an official
government organization in the United States.
Secure .gov websites use HTTPS
A lock (
) or https:// means you've safely
connected to the .gov website. Share sensitive
information only on official, secure websites.
As a library, NLM provides access to scientific literature. Inclusion in an NLM database does not imply endorsement of, or agreement with,
the contents by NLM or the National Institutes of Health.
Learn more:
PMC Disclaimer
|
PMC Copyright Notice
1Department of Microbiology, The University of Hong Kong, Queen Mary Hospital, Hong Kong
2Department of Microbiology, The University of Hong Kong, University Pathology Building, Queen Mary Hospital, and, HKU-Pasteur Research Centre, Hong Kong
1Department of Microbiology, The University of Hong Kong, Queen Mary Hospital, Hong Kong
2Department of Microbiology, The University of Hong Kong, University Pathology Building, Queen Mary Hospital, and, HKU-Pasteur Research Centre, Hong Kong
1Department of Microbiology, The University of Hong Kong, Queen Mary Hospital, Hong Kong
2Department of Microbiology, The University of Hong Kong, University Pathology Building, Queen Mary Hospital, and, HKU-Pasteur Research Centre, Hong Kong
1Department of Microbiology, The University of Hong Kong, Queen Mary Hospital, Hong Kong
2Department of Microbiology, The University of Hong Kong, University Pathology Building, Queen Mary Hospital, and, HKU-Pasteur Research Centre, Hong Kong
1Department of Microbiology, The University of Hong Kong, Queen Mary Hospital, Hong Kong
2Department of Microbiology, The University of Hong Kong, University Pathology Building, Queen Mary Hospital, and, HKU-Pasteur Research Centre, Hong Kong
1Department of Microbiology, The University of Hong Kong, Queen Mary Hospital, Hong Kong
2Department of Microbiology, The University of Hong Kong, University Pathology Building, Queen Mary Hospital, and, HKU-Pasteur Research Centre, Hong Kong
In a recent article, Weig et al. evaluated an enzyme-linked immunosorbent assay (ELISA), using recombinant mitogillin for the detection of antibodies in patients with Aspergillus fumigatus-associated diseases (3). The authors reported a good correlation between production of immunoglobulin G (IgG) antibody against mitogillin and clinical diseases, with sensitivities of 100, 64, and 60% for aspergilloma (AO), invasive pulmonary aspergillosis (IPA), and invasive disseminated aspergillosis (IDA), respectively, where the corresponding specificity of the ELISA was 95.4%, using a cutoff value of the mean optical density (OD) of 307 normal blood donor sera plus 2 standard deviations (SD).
Although the apparently high sensitivities of the ELISA look encouraging, we think that the interpretation of the results was misleading due to the choice of a low cutoff value. For rare diseases such as AO, IPA, and IDA, in order to eliminate the false positives, high specificities (over 99%) are desirable. In our recently published articles on ELISA for antibody and antigen detection in patients with penicilliosis marneffei, a cutoff value of the mean plus 10 SD was chosen, for a specificity of 100% (1, 2). However, in the study of Weig et al., a low cutoff value of the mean plus 2 SD was chosen, leading to a specificity of just 95.4% (3).
The ELISA would have much lower sensitivities if the mean plus 3 SD, the mean plus 4 SD, or the mean plus 5 SD were chosen as the cutoff value. Since an index of 0.75 (serum sample no. 96) was regarded as positive and an index of 0.73 (serum sample no. 63) was regarded as negative, the value obtained by dividing the mean plus 2 SD by the mean plus 3 SD should lie between 0.73 and 0.75. If we assume that the mean plus 2 SD divided by the mean plus 3 SD equals 0.74, 0.26 times the mean would equal 0.22 of the SD. Hence, the mean plus 4 SD divided by the mean plus 3 SD and the mean plus 5 SD divided by the mean plus 3 SD can be calculated as 1.26 and 1.52, respectively. Using the IgG OD index values of the serum samples shown on pages 1725 to 1727 of reference 3, a scattergram can be generated (Fig. 1) and the corresponding sensitivities of the ELISA using the mean plus 3 SD, the mean plus 4 SD, and the mean plus 5 SD as the cutoff values can be calculated (Table 1). In order to achieve a high specificity (over 99%), a cutoff value of at least the mean plus 4 SD or the mean plus 5 SD has to be chosen. At these cutoff values, the corresponding sensitivity of the ELISA for AO would fall to below 90%, whereas those for IPA and IDA would be less than 20%. In fact, it is not surprising to have low sensitivities for IPA and IDA, as patients suffering from invasive aspergillosis are usually severely immunocompromised (4) and production of antibodies is greatly impaired.
Scattergram showing the IgG OD index values of serum samples obtained from patients with AO, IPA, or IDA as measured by the ELISA using recombinant mitogillin.
TABLE 1.
Sensitivities of recombinant mitogillin-based ELISA at different cutoff values
Cutoff index
Specificity (%)
No. of serum samples with positive results (sensitivity)
Values in parentheses are percentages. For AO, IPA, and IDA, n equals 32, 42, and 40, respectively.
REFERENCES
1.Cao L, Chen D L, Lee C, Chan C M, Chan K M, Vanittanakom N, Tsang D N, Yuen K Y. Detection of specific antibodies to an antigenic mannoprotein for diagnosis of Penicillium marneffeipenicilliosis. J Clin Microbiol. 1998;36:3028–3031. doi: 10.1128/jcm.36.10.3028-3031.1998. [DOI] [PMC free article] [PubMed] [Google Scholar]
2.Cao L, Chan K M, Chen D, Vanittanakom N, Lee C, Chan C M, Sirisanthana T, Tsang D N, Yuen K Y. Detection of cell wall mannoprotein Mp1p in culture supernatants of Penicillium marneffeiand in sera of penicilliosis patients. J Clin Microbiol. 1999;37:981–986. doi: 10.1128/jcm.37.4.981-986.1999. [DOI] [PMC free article] [PubMed] [Google Scholar]
3.Weig M, Frosch M, Tintelnot K, Haas A, Groß U, Linsmeier B, Heesemann J. Use of recombinant mitogillin for improved serodiagnosis of Aspergillus fumigatus-associated diseases. J Clin Microbiol. 2001;39:1721–1730. doi: 10.1128/JCM.39.5.1721-1730.2001. [DOI] [PMC free article] [PubMed] [Google Scholar]
4.Yuen K Y, Woo P C Y, Ip M S M, Liang R H S, Chiu E K W, Siau H, Ho P L, Chen F F, Chan T K. Stage-specific manifestation of mold infections in bone marrow transplant recipients: risk factors and clinical significance of positive concentrated smears. Clin Infect Dis. 1997;25:37–42. doi: 10.1086/514492. [DOI] [PubMed] [Google Scholar]
A test specificity of >99%, as has been suggested by Woo et al., seems to be an unjustified postulation for the recombinant mitogillin enzyme-linked immunosorbent assay (ELISA), as the test is not designed to screen a large unbiased population (low prevalence of the disease). Instead, invasive aspergillosis (IA) and aspergilloma (AO) are diseases that occur in well-defined risk patients (high prevalence of the disease in the population studied) (1-2). Approximately one-third of the patients with acute IA initially show no definitive signs, though a fatal progression rate of the disease becomes manifest (1-1). Unfortunately, the diagnostic tools developed to date have low sensitivities (1-2) and antifungal therapy is initiated empirical and belated. The very high mortality rate of IA (>80 to 90%) could be reduced when a sensitive test system leads to an early diagnosis and well-timed therapy.
The usefulness of the mitogillin ELISA for a given patient at risk depends largely, besides on its specificity and sensitivity, on its positive and negative predictive values and the probability for false-positive and for false-negative test results. The latter test quality-defining parameters are dependent on the prevalence of the disease in the population studied.
The ability of a test to discriminate diseased cases from healthy controls is evaluated using receiver operating characteristic (ROC) curve analysis (1-3). An index of the test accuracy is the area under the ROC curve. Each point on the ROC plot represents a sensitivity-specificity pair corresponding to a particular cutoff value. A widely used method to determine the cutoff point based on a ROC curve is to maximize the Youden index (1-5). The ROC plot data allow calculation of the most effective cutoff value, when the prevalence of the disease is ascertained. The frequency of IA is known to be high in risk patients, but it varies substantially from risk group to risk group and from center to center (1-1). Since there is no “gold standard” in the laboratory diagnosis of IA, the true prevalence of IA in different risk groups is difficult to estimate (1-2).
ROC curve analyses of the recombinant anti-mitogillin IgG ELISA were done with data from blood donors (n = 307) and patients suffering from aspergillosis (n = 114); the analytic data were generated using SPSS software (version 10.0.7; SPSS Inc. Headquarters, Chicago, Ill.) (Fig. 1-1 and Table 1-1). The ROC curve analyses of our preliminary data show that our simplified approach (the cutoff was the mean optical density [OD] plus 3 SD standard deviations [SD], calculated from blood donors; the borderline was the mean plus 2 SD) was adequate for a first evaluation of the test. For the group of AO patients, the maximal Youden index lies exactly between the positive and borderline cutoff values used in our study. The results of the analysis confirm the usefulness of the recombinant anti-mitogillin antibody ELISA for the diagnosis of A. fumigatus-related diseases (area under ROC curve > 0.9). The analysis indicate further that it might be possible to define different cutoff values for different patient groups (e.g., OA versus IA) in order to increase the diagnostic accuracy of the test (data not shown). However, in our opinion this should be done in a later study with an extended number of patients from whom reliable data (proven disease) are available.
ROC curve. See the text and Table 1-1 for explanations of the data used to generate the curve. The solid arrow marks the point with the maximal Youden index. The cutoff value used in the study of Weig et al. (1-4) is indicated with a dashed arrow, and the cutoff value proposed by Woo et al. is indicated with a dotted arrow.
TABLE 1-1.
Sensitivity-specificity pairs, with Youden indices corresponding to a particular cutoff value1-a
Data are from samples from aspergillosis patients (AO, IPA, and IDA patients [n = 114]) compared to those from healthy volunteers (n = 307) and were generated using the recombinant mitogillin IgG ELISA as the discriminating variable, i.e., a value was considered positive when the test result was greater than the OD value.
1-b
Maximized Youden index.
1-c
Youden index for borderline test results proposed by Weig et al.
1-d
Youden index for the cutoff value proposed by Weig et al.
1-e
Youden index for the cutoff value proposed by Woo et al.
REFERENCES
1-1.Denning D W. Aspergillus species. In: Mandell G L, Bennett J E, Dolin R, editors. Principles and practice of infectious diseases. Philadelphia, Pa: Churchill Livingstone; 2000. pp. 2674–2685. [Google Scholar]
1-3.Metz C E. Basic principles of ROC analysis. Semin Nucl Med. 1978;14:109–121. doi: 10.1016/s0001-2998(78)80014-2. [DOI] [PubMed] [Google Scholar]
1-4.Weig M, Frosch M, Tintelnot K, Haas A, Groß U, Linsmeier B, Heeseman J. Use of recombinant mitogillin for improved serodiagnosis of Aspergillus fumigatus-associated diseases. J Clin Microbiol. 2001;39:1721–1730. doi: 10.1128/JCM.39.5.1721-1730.2001. [DOI] [PMC free article] [PubMed] [Google Scholar]
1-5.Youden W J. Index for rating diagnostic tests. Cancer. 1950;3:32–35. doi: 10.1002/1097-0142(1950)3:1<32::aid-cncr2820030106>3.0.co;2-3. [DOI] [PubMed] [Google Scholar]
