Abstract
Background
Most diagnostic tests for acute uncomplicated urinary tract infections (UTIs) have been previously studied in so-called single-test evaluations. In practice, however, clinicians use more than one test in the diagnostic work-up. Since test results carry overlapping information, results from single-test studies may be confounded. The primary objective of the Amsterdam Cystitis/Urinary Tract Infection Study (ACUTIS) is to determine the (additional) diagnostic value of relevant tests from patient history and laboratory investigations, taking into account their mutual dependencies. Consequently, after suitable validation, an easy to use, multivariable diagnostic rule (clinical index) will be derived.
Methods
Women who contact their GP with painful and/or frequent micturition undergo a series of possibly relevant tests, consisting of patient history questions and laboratory investigations. Using urine culture as the reference standard, two multivariable models (diagnostic indices) will be generated: a model which assumes that patients attend the GP surgery and a model based on telephone contact only. Models will be made more robust using the bootstrap. Discrimination will be visualized in high resolution histograms of the posterior UTI probabilities and summarized as 5th, 10th, 25th 50th, 75th, 90th, and 95th centiles of these, Brier score and the area under the receiver operating characteristics curve (ROC) with 95% confidence intervals. Using the regression coefficients of the independent diagnostic indicators, a diagnostic rule will be derived, consisting of an efficient set of tests and their diagnostic values.
The course of the presenting complaints is studied using 7-day patient diaries. To learn more about the natural history of UTIs, patients will be offered the opportunity to postpone the use of antibiotics.
Discussion
We expect that our diagnostic rule will allow efficient diagnosis of UTIs, necessitating the collection of diagnostic indicators with proven added value. GPs may use the rule (preferably after suitable validation) to estimate UTI probabilities for women with different combinations of test results. Finally, in a subcohort, an attempt is made to identify which indicators (including antibiotic treatment) are useful to prognosticate recovery from painful and/or frequent micturition.
Background
General background
An acute uncomplicated urinary tract infection (UTI, cystitis) is defined as an infection of the lower urinary tract (bladder) in an otherwise healthy, nonpregnant, adult woman without known anatomical or functional abnormalities of the urinary tract. [1-4] A U.S. study reported a 60% lifetime risk of physician-diagnosed UTI.[5]
The main symptoms suggesting a UTI are dysuria and urinary frequency, whereas vaginal irritation and discharge have been reported to reduce the probability of a UTI being present.[6] Besides a UTI, an important cause of dysuria and frequency is a Chlamydia urethritis. Other infectious diseases in the differential diagnosis of a UTI are other types of urethritis (e.g. gonorrhoea), vaginitis, interstitial cystitis and genital herpes. [6-8] Since a UTI differs from these diseases in its natural course and treatment, a correct diagnosis is essential.
Empirical treatment of symptomatic women has been reported to be cost-effective.[9,10] However, only half of symptomatic women are found to have a UTI if defined by ≥ 105 colony-forming units (CFU)/ml.[11,12] Moreover, bacterial resistance to commonly used antibiotics (e.g. trimethoprim and fluoroquinolones) is rising, suggesting the need for more evidence-based prescribing. [13-16] Better diagnosis of UTIs might prevent women from being unnecessarily treated with antibiotics and be even more cost-effective.
Diagnostic work-up
Diagnosis of UTIs in general practice consists of various medical history questions and laboratory investigations, of which nitrite and leukocyte esterase (LE) dipstick tests, microscopic examination of urinary sediment, and dipslide are the ones most widely used.
Most diagnostic tests for UTIs have been studied before in single-test evaluations, implying that a test is compared to the urine culture, not taking into account the distribution of other test results in the population studied. [17-19] In practice, however, the diagnostic work-up is inherently multivariable, and test results are mutually dependent. [20-36] Therefore it cannot be generally expected that a test's sensitivity is invariant over different degrees of prior testing. For example, the sensitivity of the nitrite dipstick test will be affected by the information contained in the diagnostic profile resulting from prior test results.[37]
The primary objective of the Amsterdam Cystitis/Urinary Tract Infection Study (ACUTIS) is to determine the (additional) diagnostic value of relevant tests from patient history and laboratory investigations, taking into account their mutual dependencies. Next, an easy to use, multivariable diagnostic rule (clinical index) will be derived. After suitable (external) validation, GPs may use the (validated) rule to estimate UTI probabilities for women with different combinations of test results.
Little et al. and McIsaac et al. described similar work in a UK and a Canadian population, respectively.[38,39] However, different inclusion criteria were used (see further) and urine samples were transported as in routine practice before cultures were made, whereas in ACUTIS, to assure reliability of the cultures (= reference standard), urine samples are being refrigerated until being cultured. Furthermore, both earlier studies did not assess diagnostic values of the sediment and dipslide investigations.
Other similar studies.[40,41] were performed before the European Urinalysis Guidelines.[42] recommended a cut-off value of 103 cfu/ml for significant Escherichia coli bacteriuria and therefore only used the traditional cut-off value of 105 cfu/ml. Besides, they did not assess dipstick investigations and/or were not performed in a primary care setting.[41]
Prognosis and natural history
Placebo arms of randomized controlled trials suggest that 25 to 50% of women presenting with a combination of UTI-symptoms and bacteriuria will have recovered in one week without using antibiotics. [43-45] Using data from ACUTIS, we will try to make a prognostic index. Using the prognostic index, a GP can estimate the 7-day course for women with different combinations of test results and decide which women are most likely to benefit from antibiotic treatment.
Methods
Setting
ACUTIS is performed in general practices and primary health care centers in Amsterdam and surroundings, which altogether comprise 20 practices of average size (total source population = 46,000 patients).
Design
In a prospective cohort, each participating patient will be followed for 7 days. At baseline, potentially relevant signs and symptoms will be recorded and potentially relevant laboratory tests (dipstick, sediment and dipslide) will be performed, along with a urine culture as reference test. Details are described below.
Study population
Eligible are female patients over 12 years of age, contacting their GP with painful and/or frequent micturition. The symptoms may have been present for a maximum of 7 days. Although not being adult and therefore not strictly meeting the definition of acute uncomplicated urinary tract infection, patients between the age of 12 and 18 are eligible, as their urogenital tract is expected to be anatomically similar to that of adult women.
Exclusion criteria are: pregnancy, lactation, signs of pyelonephritis, being immunocompromised (e.g. receiving radiotherapy or immunosuppressants, being HIV infected) with the exception of diabetes mellitus, having used antibiotics for another indication than UTI or having undergone a urological procedure in the past two weeks, and known anatomical or functional abnormalities of the urogenital tract.
To ensure a clinical domain in which the diagnostic index may be expected to perform as reported and to enhance generalizability in time and place, we use clearly formulated eligibility criteria. This differs from similar studies, in which patients were eligible when GPs "suspected" a UTI, based on their personal judgement.[38,39]
Informed consent
A letter with information about the study is provided to all patients. Participating women must provide written informed consent prior to enrollment. For patients under the age of 18 written parental authorization is required.
Sample size
When the events per variable (EPV) rule.[46,47] is applied on a number of 15 possible diagnostic indicators and taken into account that some indicators will have to be modeled as dummy variables, about 200 women with a UTI are needed for this study (although a more recent simulation study suggests that the traditional EPV figure may be an overestimate.[48], so that less patients may be needed for our study). Assuming that the probability of a UTI (if defined by ≥ 105 CFU/ml) in eligible women is approximately 50%.[11,12] (range 40–60%), between 333 and 500 women will have to be included in the study. According to a UTI incidence in The Netherlands of 68 per 1000 women per year.[49], 157 women per year are diagnosed with a UTI in a general practice of average size. Assuming that 80% of all eligible women will not take part in the study, 16–17 practices of average size that include patients for a period of 1 year are needed (500/(157 × (1-0.8)) = 16.03).
Clinical assessments (see table 1)
Table 1.
GP or GP assistant | Patient | |
day 1 (baseline) | Check availability | |
Obtain informed consent | ||
Collect urine at GP surgery | ||
Perform dipstick and dipslide (and optionally sediment) | Fill in questionnaire | |
Send urine to laboratory (refrigerated) for culture and sediment | ||
day 2 | Examine dipslide | |
day 1 till day 7 | Record symptoms in diary | |
day 8 | If patient did not use antibiotic: collect urine and send for culture |
After eligibility has been confirmed and informed consent has been obtained, urine is collected in a sterile container. Instructions for the collection method may be given at the GP's discretion, but are not necessary, as they have been reported to have no consequences for the extent of contamination. [50-52] We insist that the urine sample is collected at the GP surgery, since collection at home will cause at least some specimens to remain non-refrigerated for too long, thus rendering urinalysis unreliable.[53]
After urine collection, patients are asked to fill in a detailed questionnaire to record presence and severity of signs and symptoms (see additional file 1). Furthermore, they are given a special diary, in which they are to record clinical symptoms and any use of antibiotics for 7 days.
The GP or GP assistant performs a dipstick test (Multistix® 5, Siemens Medical Solutions Diagnostics™) and a dipslide test (Uricult® classic, Orion diagnostica™). Dipsticks will be assessed automatically by a Clinitek Status® analyzer (Siemens Medical Solutions Diagnostics™) to reduce inter-observer variation.
Immediately after dipstick and dipslide investigations have been performed, urine samples are stored in a refrigerator at the GP surgery. Within 8 hours they are transported to two laboratories (see below) at 4°C by a specialized courier service.
GPs are free to perform a urinary sediment investigation, the results of which will then be documented. However, in the statistical analysis we will assess the diagnostic value of the urinary sediment investigation performed by experts at the Laboratory for Clinical Chemistry of the Academic Medical Center in Amsterdam. Only if the sediment investigation proves to be useful under the optimal conditions of the laboratory, we will repeat it under daily practice conditions in a sub-study to be carried out after the main study has been completed (see further).
The Laboratory for Medical Microbiology of the Academic Medical Center in Amsterdam performs a urine culture, which serves as the reference standard. Trained laboratory technicians, who have no information on previous test results from history taking and/or urinalysis, will make a culture according to the standard loop method.[53], interpret the culture results and determine resistance patterns.
Patients who have not used antibiotics since their study entry will be asked to provide another urine sample for culturing at day 7 (see below).
Urine samples with a culture result ≥ 102 CFU/ml will be stored refrigerated for possible later use (e.g. identification of bacterial strains).
Data entry
The information obtained from patient history and laboratory investigations will be collected and entered into a structured database by a qualified commercial data-entry service, which guarantees a 97% accuracy at character level. For crucial variables (e.g. culture results) there will be double (independent) entry (99.7% accuracy at character level).
Statistical analysis
Data screening
The data will be screened for data entry errors, extreme values, and missing values using simple tables, plots, and dedicated commands to identify (multivariable) outliers. Errors will be corrected. Instead of deleting extreme cases, we shall consider to suitably truncate extreme values where their influence on models appears too strong.[54]
Missing values
Multiple imputation using chained equations will be used to impute missing values.[55,56]
Dependent variable
The results of the urine culture will be used as the dependent variable. A cut-off value of 103 CFU/ml will be used to define a positive culture, as is recommended by the European Urinalysis Guidelines.[42] We will explore how sensitive the model is to the use of different cut-off values for defining a positive culture.[6,12,57-60]
Theoretically, false positive reference standard results may occur if women with asymptomatic bacteriuria are included in the study when they visit their GP with complaints similar to UTI symptoms but caused by a different disease (e.g. vaginitis). In the community, the prevalence of asymptomatic bacteriuria has been reported to be 1 to 5% in premenopausal, nonpregnant women, growing with increasing age to >10% in women over 70 years of age.[61]
Variable selection strategy
General objectives
We aim to generate two models (diagnostic indices). First, a model which assumes that patients attend the GP surgery. Second, a model based on telephone contact only, since in daily routine this is very often the first and only contact with the patient.
Finally, using different threshold probabilities for 'no further action' (e.g. advice to increase the intake of water, 'wait and see') and prescription of an antibiotic, respectively, we will develop a diagnostic test algorithm such that patients may undergo the least number of possible tests. That is, if, regardless of its results, a certain test can be shown not to take a patient across one of the abovementioned thresholds for specific clinical action, that test may be informative, but not sufficiently so to change management. Therefore it is superfluous. It is possible that a 'telephone only' model provides sufficient certainty for patients with specific combinations of patient history items, while for other patients urinalysis at the GP surgery is needed to decide on management correctly.
Variable selection
At the time of writing, a failsafe model selection strategy on which all statisticians and other experts in diagnostic and predictive modeling agree does not appear to exist. However, in his book 'Regression modeling strategies (2001)'[62], Harrell made some general proposals for researchers to tailor to the specific circumstances. We intend to use penalized logistic regression, and Tibshirani's lasso (least absolute shrinkage and selection operator)[63] in particular to combine the requirements of counteracting over-optimism (shrinkage of regression coefficients) while leaving the opportunity that some coefficients are set to zero, which serves the requirement of a parsimonious model. Bootstrapping will be used to estimate the penalization coefficient.[62] As a form of sensitivity analysis, we shall also explore bootstrapped stepwise regression[64,65] to see how well these approaches concur. We will avoid univariable preselection of predictors. The linearity assumption will be checked for all continuous predictors.
Discrimination of the model(s) will be visualized in high resolution histograms and summarized as 5th, 10th, 25th 50th, 75th, 90th, and 95th centiles of these, Brier score and the area under the receiver operating characteristics curve (ROC) with 95% confidence intervals (overall discrimination).[66]
Using the regression coefficients of the independent diagnostic indicators, an easy to use, multivariable diagnostic rule (clinical index) will be derived, consisting of relevant tests and their diagnostic values.
Substudies
Prognosis
Using our diagnostic rule, the probability of a positive culture at baseline can be estimated. For a clinician it may also be relevant to learn more about a patient's prognosis and how this is affected by antibiotic medication. In previous studies 25 to 50% of women presenting with UTI symptoms and/or bacteriuria have been reported to recover spontaneously within one week. [43-45] It would be useful to know whether these patients, for whom antibiotics are not indicated, can be identified at baseline based on their clinical profiles.
In ACUTIS, all patients are asked to record their symptoms and possible antibiotic use in a special diary for 7 days. To learn more about the natural course of UTI, we will offer eligible women the opportunity to postpone the use of antibiotics. For patients who have not used antibiotics after 7 days, we shall try to secure a second urine specimen for culture. As a result, two prognostic models may be developed, with clinical and bacteriological cure after one week serving as dependent variables, respectively. Regression analysis will be used to assess the effects of antibiotic treatment using all cogent potential confounders to counteract possible confounding by indication as much as possible. If in specific subdomains of women no effect of antibiotics is found, randomised trials may be indicated to confirm such findings. If, on the other hand, an effect is found, the effect may be non-existent in particular subgroups. However, the power to detect such subgroup effects (or a lack of it) is probably somewhat limited. Women with milder symptoms may be more likely to defer antibiotics. We consider this not to be a great problem, since this is the subdomain of patients in which doubt about the value of antibiotics is greatest. Placebo-controlled randomized clinical trials in the subdomain of patients with severe dysuria and/or frequency are likely to incur ethical problems. If our results are promising, however, an RCT may be performed in the future (possibly by using a delayed prescription approach. [67-69]).
Sediment
As mentioned before, urinary sediment investigation will be performed under protocollized circumstances at the Laboratory for Clinical Chemistry. It could be argued that this should be done in the GP surgery, as this is more in accordance with common practice. However, if urinary sediment proves to be useful under the optimal conditions of the laboratory, it will be repeated under daily practice conditions in a sub-study after the main study has been completed. The originally obtained sediment data can then be replaced in our model by the daily practice results, using Deming (orthogonal) regression analysis. [70-76] Briefly, this is a method that can be used to estimate the relationship between two measurements with proportional errors, in this case sediment under optimal conditions and sediment under daily practice conditions. The data collection of this sediment sub-study will take place during a specially organized training session, where sediment scoring results of GPs and/or GP assistants will be measured before and after training by experts. Sediment investigation on the same samples will be performed by experts under optimal conditions (= reference standard). The before-training measurements as well as the after-training measurements will be compared with the expert measurements using Deming regression.
Chlamydia/gonorrhoea
Apart from a UTI, dysuria and urinary frequency in sexually active women may be a result of a urethritis caused by Chlamydia trachomatis or Neisseria gonorrhoeae. Therefore a polymerase chain reaction (PCR) for Chlamydia trachomatis and Neisseria gonorrhoeae will be done in each urine sample. An attempt will be made to extend our diagnostic rule for Chlamydia trachomatis and Neisseria gonorrhoeae infections in women with painful and/or frequent micturition. To attain this, we will perform a polytomous logistic regression analysis, a technique that is used to simultaneously estimate probabilities of multiple diagnoses. [77-79] As a result, a diagnostic model with three outcomes (UTI, Chlamydia trachomatis, and Neisseria gonorrhoeae) can be derived.
Discussion
Whereas most previous studies on diagnosis of UTIs are single-test evaluations, we will perform a multivariable analysis and develop an easy to use diagnostic rule for general practice. This will allow efficient diagnosis of UTIs, using only those diagnostic indicators which have proven added value to rule in or rule out UTI. For a large number of women with painful and/or frequent micturition, additional urine investigations might prove to add no useful information to what is already known from specific patient history questions.
This implies that in these cases the diagnostic procedure can be completed by telephone, which makes life easier for GP and patient alike. Another possibility is that patients, before contacting their GP, use a simplified questionnaire (based on the study results), available e.g. from the Internet or from their GP's office. As a result, the diagnostic work-up will be more cost-effective and less time-consuming. Besides, since use of dipslide and culture might be reduced, there may be less time delay in UTI diagnosis.
In sub-studies we will try to extend our diagnostic rule to diagnosis of Chlamydia trachomatis or Neisseria gonorrhoeae. Furthermore, prognostic indicators (potentially including antibiotics) of the 7-day course of painful and/or frequent micturition will be assessed in our cohort study.
We expect to be able to submit the first results of ACUTIS no later than by the end of 2009.
Competing interests
The authors declare that they have no competing interests.
Authors' contributions
BK participated in the design of the study, drafted the manuscript and will coordinate data collection, statistical analysis and reporting of the results. PB, SG, EM and GtR participated in the design, helped to draft the manuscript and will support or participate in data collection, statistical analysis and reporting of the results. GtR conceived the basic design and the main objectives of the study.
Pre-publication history
The pre-publication history for this paper can be accessed here:
Supplementary Material
Acknowledgments
Acknowledgements
We thank Lucas Bachmann (Horten Centre, University of Zurich), Thierry Christiaens (Ghent University, Department of General Practice and Primary Health Care), Bart van Pinxteren (Dutch College of General Practitioners), Hans Severens (Maastricht University, Department of Health Organization, Policy and Economics), Ellen Stobberingh (University Hospital Maastricht, Department of Medical Microbiology) and Koos Zwinderman (Academic Medical Center – University of Amsterdam, Department of Clinical Epidemiology and Biostatistics) for their advice on the study design.
Multistix® 5 dipsticks and Clinitek Status® analyzers are supplied by Siemens Medical Solutions Diagnostics™. Uricult® dipslides and Cultura® M incubators are produced by Orion diagnostica™ and supplied by Mediphos Medical Supplies™. Data entry is performed by Service Point™.
Contributor Information
Bart J Knottnerus, Email: b.j.knottnerus@amc.uva.nl.
Patrick JE Bindels, Email: p.j.bindels@amc.uva.nl.
Suzanne E Geerlings, Email: s.e.geerlings@amc.uva.nl.
Eric P Moll van Charante, Email: e.p.mollvancharante@amc.uva.nl.
Gerben ter Riet, Email: g.terriet@amc.uva.nl.
References
- Foxman B. Epidemiology of urinary tract infections: incidence, morbidity, and economic costs. Dis Mon. 2003;49:53–70. doi: 10.1016/S0011-5029(03)90000-9. [DOI] [PubMed] [Google Scholar]
- Gonzalez CM, Schaeffer AJ. Treatment of urinary tract infection: what's old, what's new, and what works. World J Urol. 1999;17:372–382. doi: 10.1007/s003450050163. [DOI] [PubMed] [Google Scholar]
- van Haaren KAM, Visser HS, van Vliet S, Timmermans AE, Yadava R, Geerlings SE, ter Riet G, van Pinxteren B. NHG-Standaard Urineweginfecties (tweede herziening) Huisarts en wetenschap. 2005;8:341–352. [Google Scholar]
- Warren JW, Abrutyn E, Hebel JR, Johnson JR, Schaeffer AJ, Stamm WE. Guidelines for antimicrobial treatment of uncomplicated acute bacterial cystitis and acute pyelonephritis in women. Clinical Infectious Diseases. 1999;29:745–758. doi: 10.1086/520427. [DOI] [PubMed] [Google Scholar]
- Foxman B, Barlow R, D'Arcy H, Gillespie B, Sobel JD. Urinary Tract Infection: Self-Reported Incidence and Associated Costs. Annals of epidemiology. 2000;10:509–515. doi: 10.1016/S1047-2797(00)00072-7. [DOI] [PubMed] [Google Scholar]
- Bent S, Nallamothu BK, Simel DL, Fihn SD, Saint S. Does this woman have an acute uncomplicated urinary tract infection? Jama-Journal of the American Medical Association. 2002;287:2701–2710. doi: 10.1001/jama.287.20.2701. [DOI] [PubMed] [Google Scholar]
- Komaroff AL. Acute dysuria in women. N Engl J Med. 1984;310:368–375. doi: 10.1056/NEJM198402093100607. [DOI] [PubMed] [Google Scholar]
- Kurowski K. The women with dysuria. Am Fam Physician. 1998;57:2155–2170. [PubMed] [Google Scholar]
- Barry HC, Ebell MH, Hickner J. Evaluation of suspected urinary tract infection in ambulatory women: a cost-utility analysis of office-based strategies. J Fam Pract. 1997;44:49–60. [PubMed] [Google Scholar]
- Fenwick EA, Briggs AH, Hawke CI. Management of urinary tract infection in general practice: a cost-effectiveness analysis. Br J Gen Pract. 2000;50:635–639. [PMC free article] [PubMed] [Google Scholar]
- Sobel JD, Kaye D. Urinary Tract Infections. In: Mandell GL, Douglas RG Jr, Bennett JE, Dolin R, editor. Mandell, Douglas, and Bennett's Principles and practice of infectious diseases. Philadelphia [etc.] : Churchill Livingstone; 2000. pp. 773–805. [Google Scholar]
- Stamm WE, Counts GW, Running KR, Fihn S, Turck M, Holmes KK. Diagnosis of coliform infection in acutely dysuric women. N Engl J Med. 1982;307:463–468. doi: 10.1056/NEJM198208193070802. [DOI] [PubMed] [Google Scholar]
- Chomarat M. Resistance of bacteria in urinary tract infections. Int J Antimicrob Agents. 2000;16:483–487. doi: 10.1016/S0924-8579(00)00281-8. [DOI] [PubMed] [Google Scholar]
- Gupta K, Hooton TM, Stamm WE. Increasing antimicrobial resistance and the management of uncomplicated community-acquired urinary tract infections. Annals of Internal Medicine. 2001;135:41–50. doi: 10.7326/0003-4819-135-1-200107030-00012. [DOI] [PubMed] [Google Scholar]
- Hooton TM, Besser R, Foxman B, Fritsche TR, Nicolle LE. Acute uncomplicated cystitis in an era of increasing antibiotic resistance: A proposed approach to empirical therapy. Clinical Infectious Diseases. 2004;39:75–80. doi: 10.1086/422145. [DOI] [PubMed] [Google Scholar]
- Kahlmeter G. An international survey of the antimicrobial susceptibility of pathogens from uncomplicated urinary tract infections: the ECO.SENS Project. Journal of Antimicrobial Chemotherapy. 2003;51:69–76. doi: 10.1093/jac/dkg028. [DOI] [PubMed] [Google Scholar]
- Deville WL, Yzermans JC, van Duijn NP, Bezemer PD, Windt DA van der, Bouter LM. The urine dipstick test useful to rule out infections. A meta-analysis of the accuracy. BMC Urol. 2004;4:4. doi: 10.1186/1471-2490-4-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Medina-Bombardo D, Segui-Diaz M, Roca-Fusalba C, Llobera J. What is the predictive value of urinary symptoms for diagnosing urinary tract infection in women? Family Practice. 2003;20:103–107. doi: 10.1093/fampra/20.2.103. [DOI] [PubMed] [Google Scholar]
- Winkens R, Nelissen-Arets H, Stobberingh E. Validity of the urine dipslide under daily practice conditions. Family Practice. 2003;20:410–412. doi: 10.1093/fampra/cmg412. [DOI] [PubMed] [Google Scholar]
- Begg CB. Biases in the assessment of diagnostic tests. Stat Med. 1987;6:411–423. doi: 10.1002/sim.4780060402. [DOI] [PubMed] [Google Scholar]
- Dawid AP. Properties of diagnostic data distributions. Biometrics. 1976;32:647–658. doi: 10.2307/2529753. [DOI] [PubMed] [Google Scholar]
- Detrano R, Guppy KH, Abbassi N, Janosi A, Sandhu S, Froelicher V. Reliability of Bayesian probability analysis for predicting coronary artery disease in a veterans hospital. J Clin Epidemiol. 1988;41:599–605. doi: 10.1016/0895-4356(88)90065-0. [DOI] [PubMed] [Google Scholar]
- Feinstein AR. Clinical biostatistics. XXXIX. The haze of Bayes, the aerial palaces of decision analysis, and the computerized Ouija board. Clin Pharmacol Ther. 1977;21:482–496. doi: 10.1002/cpt1977214482. [DOI] [PubMed] [Google Scholar]
- Hlatky MA, Pryor DB, Harrell FE, Jr, Califf RM, Mark DB, Rosati RA. Factors affecting sensitivity and specificity of exercise electrocardiography. Multivariable analysis. Am J Med. 1984;77:64–71. doi: 10.1016/0002-9343(84)90437-6. [DOI] [PubMed] [Google Scholar]
- Isenberg DA, Hammond L, Fisher C, Griffiths M, Stewart J, Bottazzo GF. Predictive value of SS-B precipitating antibodies in Sjoogren's syndrome. Br Med J (Clin Res Ed) 1982;284:1738–1740. doi: 10.1136/bmj.284.6331.1738. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lachs MS, Nachamkin I, Edelstein PH, Goldman J, Feinstein AR, Schwartz JS. Spectrum bias in the evaluation of diagnostic tests: lessons from the rapid dipstick test for urinary tract infection. Ann Intern Med. 1992;117:135–140. doi: 10.7326/0003-4819-117-2-135. [DOI] [PubMed] [Google Scholar]
- Ledley RS, Lusted LB. Reasoning foundations of medical diagnosis; symbolic logic, probability, and value theory aid our understanding of how physicians reason. Science. 1959;130:9–21. doi: 10.1126/science.130.3366.9. [DOI] [PubMed] [Google Scholar]
- Lofgren RP. The dynamic nature of sensitivity and specificity. J Gen Intern Med. 1987;2:452–453. doi: 10.1007/BF02596381. [DOI] [PubMed] [Google Scholar]
- Miettinen OS, Caro JJ. Foundations of medical diagnosis: what actually are the parameters involved in Bayes' theorem? Stat Med. 1994;13:201–209. doi: 10.1002/sim.4780130302. [DOI] [PubMed] [Google Scholar]
- Miettinen OS, Henschke CI, Yankelevitz DF. Evaluation of diagnostic imaging tests: diagnostic probability estimation. J Clin Epidemiol. 1998;51:1293–1298. doi: 10.1016/S0895-4356(98)00127-9. [DOI] [PubMed] [Google Scholar]
- Miettinen OS. The modern scientific physician: 4. The useful property of a diagnostic. CMAJ. 2001;165:910–911. [PMC free article] [PubMed] [Google Scholar]
- Moons KG, Harrell FE. Sensitivity and specificity should be de-emphasized in diagnostic accuracy studies. Acad Radiol. 2003;10:670–672. doi: 10.1016/S1076-6332(03)80087-9. [DOI] [PubMed] [Google Scholar]
- Moons KG, Biesheuvel CJ, Grobbee DE. Test research versus diagnostic research. Clin Chem. 2004;50:473–476. doi: 10.1373/clinchem.2003.024752. [DOI] [PubMed] [Google Scholar]
- Pozen MW, D'Agostino RB, Selker HP, Sytkowski PA, Hood WB., Jr A predictive instrument to improve coronary-care-unit admission practices in acute ischemic heart disease. A prospective multicenter clinical trial. N Engl J Med. 1984;310:1273–1278. doi: 10.1056/NEJM198405173102001. [DOI] [PubMed] [Google Scholar]
- Rozanski A, Diamond GA, Berman D, Forrester JS, Morris D, Swan HJ. The declining specificity of exercise radionuclide ventriculography. N Engl J Med. 1983;309:518–522. doi: 10.1056/NEJM198309013090902. [DOI] [PubMed] [Google Scholar]
- ter Riet G, Kessels AGH, Bachmann LM. Systematic reviews of evaluations of diagnostic and screening tests – Two issues were simplified. Br Med J. 2001;323:1188. doi: 10.1136/bmj.323.7322.1188. [DOI] [PubMed] [Google Scholar]
- Bachmann LM, ter Riet G, Weber WE, Kessels AG. Multivariable adjustments counteract spectrum and test review bias in accuracy studies. J Clin Epidemiol. 2008 doi: 10.1016/j.jclinepi.2008.02.007. [DOI] [PubMed] [Google Scholar]
- Little P, Turner S, Rumsby K, Warner G, Moore M, Lowes JA, Smith H, Hawke C, Mullee M. Developing clinical rules to predict urinary tract infection in primary care settings: sensitivity and specificity of near patient tests (dipsticks) and clinical scores. Br J Gen Pract. 2006;56:606–612. [PMC free article] [PubMed] [Google Scholar]
- McIsaac WJ, Low DE, Biringer A, Pimlott N, Evans M, Glazier R. The impact of empirical management of acute cystitis on unnecessary antibiotic use. Archives of Internal Medicine. 2002;162:600–605. doi: 10.1001/archinte.162.5.600. [DOI] [PubMed] [Google Scholar]
- Leibovici L, Alpert G, Laor A, Kalter-Leibovici O, Danon YL. A clinical model for diagnosis of urinary tract infection in young women. Arch Intern Med. 1989;149:2048–2050. doi: 10.1001/archinte.149.9.2048. [DOI] [PubMed] [Google Scholar]
- Wigton RS, Hoellerich VL, Ornato JP, Leu V, Mazzotta LA, Cheng IH. Use of clinical findings in the diagnosis of urinary tract infection in women. Arch Intern Med. 1985;145:2222–2227. doi: 10.1001/archinte.145.12.2222. [DOI] [PubMed] [Google Scholar]
- European Urinalysis Guidelines: Summary. Scandinavian Journal of Clinical and Laboratory Investigation. 2000;60:1–96. doi: 10.1080/00365510050184985. [DOI] [PubMed] [Google Scholar]
- Christiaens TCM, De Meyere M, Verschraegen G, Peersman W, Heytens S, De Maeseneer JM. Randomised controlled trial of nitrofurantoin versus placebo in the treatment of uncomplicated urinary tract infection in adult women. British Journal of General Practice. 2002;52:729–734. [PMC free article] [PubMed] [Google Scholar]
- Ferry SA, Holm SE, Stenlund H, Lundholm R, Monsen TJ. The natural course of uncomplicated lower urinary tract infection in women illustrated by a randomized placebo controlled study. Scand J Infect Dis. 2004;36:296–301. doi: 10.1080/00365540410019642. [DOI] [PubMed] [Google Scholar]
- Richards D, Toop L, Chambers S, Fletcher L. Response to antibiotics of women with symptoms of urinary tract infection but negative dipstick urine test results: double blind randomised controlled trial. British Medical Journal. 2005;331:143–146B. doi: 10.1136/bmj.38496.452581.8F. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Harrell FE, Jr, Lee KL, Matchar DB, Reichert TA. Regression models for prognostic prediction: advantages, problems, and suggested solutions. Cancer Treat Rep. 1985;69:1071–1077. [PubMed] [Google Scholar]
- Peduzzi P, Concato J, Kemper E, Holford TR, Feinstein AR. A simulation study of the number of events per variable in logistic regression analysis. J Clin Epidemiol. 1996;49:1373–1379. doi: 10.1016/S0895-4356(96)00236-3. [DOI] [PubMed] [Google Scholar]
- Vittinghoff E, McCulloch CE. Relaxing the rule of ten events per variable in logistic and Cox regression. Am J Epidemiol. 2007;165:710–718. doi: 10.1093/aje/kwk052. [DOI] [PubMed] [Google Scholar]
- Version 3111. Bilthoven: RIVM; 2007. National Public Health Compass.http://www.nationaalkompas.nl [Google Scholar]
- Baerheim A, Laerum E. Home-voided urine specimens in women. Diagnostic agreement with clean-catch midstream specimens. Scand J Prim Health Care. 1990;8:207–211. doi: 10.3109/02813439008994960. [DOI] [PubMed] [Google Scholar]
- Leisure MK, Dudley SM, Donowitz LG. Does a clean-catch urine sample reduce bacterial contamination? N Engl J Med. 1993;328:289–290. doi: 10.1056/NEJM199301283280420. [DOI] [PubMed] [Google Scholar]
- Lifshitz E, Kramer L. Outpatient urine culture – Does collection technique matter? Archives of Internal Medicine. 2000;160:2537–2540. doi: 10.1001/archinte.160.16.2537. [DOI] [PubMed] [Google Scholar]
- Graham JC, Galloway A. The laboratory diagnosis of urinary tract infection. Journal of Clinical Pathology. 2001;54:911–919. doi: 10.1136/jcp.54.12.911. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Schouten HJA. Klinische statistiek: een praktische inleiding in methodologie en analyse. 2. Houten: Bohn Stafleu Van Loghum; 1999. [Google Scholar]
- Donders AR, Heijden GJ van der, Stijnen T, Moons KG. Review: a gentle introduction to imputation of missing values. J Clin Epidemiol. 2006;59:1087–1091. doi: 10.1016/j.jclinepi.2006.01.014. [DOI] [PubMed] [Google Scholar]
- van Buuren S. Multiple imputation of discrete and continuous data by fully conditional specification. Stat Methods Med Res. 2007;16:219–242. doi: 10.1177/0962280206074463. [DOI] [PubMed] [Google Scholar]
- Arav-Boger R, Leibovici L, Danon YL. Urinary tract infections with low and high colony counts in young women. Spontaneous remission and single-dose vs multiple-day treatment. Arch Intern Med. 1994;154:300–304. doi: 10.1001/archinte.154.3.300. [DOI] [PubMed] [Google Scholar]
- Kass EH. Bacteriuria and the diagnosis of infections of the urinary tract; with observations on the use of methionine as a urinary antiseptic. AMA Arch Intern Med. 1957;100:709–714. doi: 10.1001/archinte.1957.00260110025004. [DOI] [PubMed] [Google Scholar]
- Kunin CM, White LV, Hua TH. A Reassessment of the Importance of Low-Count Bacteriuria in Young-Women with Acute Urinary Symptoms. Annals of Internal Medicine. 1993;119:454–460. doi: 10.7326/0003-4819-119-6-199309150-00002. [DOI] [PubMed] [Google Scholar]
- Stamm WE, Wagner KF, Amsel R, Alexander ER, Turck M, Counts GW, et al. Causes of the acute urethral syndrome in women. N Engl J Med. 1980;303:409–415. doi: 10.1056/NEJM198008213030801. [DOI] [PubMed] [Google Scholar]
- Nicolle LE. Asymptomatic bacteriuria: when to screen and when to treat. Infect Dis Clin North Am. 2003;17:367–394. doi: 10.1016/S0891-5520(03)00008-4. [DOI] [PubMed] [Google Scholar]
- Harrell FE., Jr Regression modeling strategies. 2001.
- Tibshirani R. The lasso method for variable selection in the Cox model. Stat Med. 1997;16:385–395. doi: 10.1002/(SICI)1097-0258(19970228)16:4<385::AID-SIM380>3.0.CO;2-3. [DOI] [PubMed] [Google Scholar]
- Royston P, Sauerbrei W. Multivariable model-building: A pragmatic approach to regression analysis based on fractional polynomials for modelling continuous variables. Chichester: Wiley; 2008. [Google Scholar]
- Sauerbrei W, Schumacher M. A bootstrap resampling procedure for model building: application to the Cox regression model. Stat Med. 1992;11:2093–2109. doi: 10.1002/sim.4780111607. [DOI] [PubMed] [Google Scholar]
- Harrell FE, Jr, Lee KL, Mark DB. Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Statistics in medicine. 1996;15:361–387. doi: 10.1002/(SICI)1097-0258(19960229)15:4<361::AID-SIM168>3.0.CO;2-4. [DOI] [PubMed] [Google Scholar]
- Dowell J, Pitkethly M, Bain J, Martin S. A randomised controlled trial of delayed antibiotic prescribing as a strategy for managing uncomplicated respiratory tract infection in primary care. Br J Gen Pract. 2001;51:200–205. [PMC free article] [PubMed] [Google Scholar]
- Little P, Gould C, Williamson I, Moore M, Warner G, Dunleavey J. Pragmatic randomised controlled trial of two prescribing strategies for childhood acute otitis media. BMJ. 2001;322:336–342. doi: 10.1136/bmj.322.7282.336. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Spiro DM, Tay KY, Arnold DH, Dziura JD, Baker MD, Shapiro ED. Wait-and-see prescription for the treatment of acute otitis media: a randomized controlled trial. JAMA. 2006;296:1235–1241. doi: 10.1001/jama.296.10.1235. [DOI] [PubMed] [Google Scholar]
- Cornbleet PJ, Gochman N. Incorrect least-squares regression coefficients in method-comparison analysis. Clin Chem. 1979;25:432–438. [PubMed] [Google Scholar]
- Deming WE. Statistical adjustment of data. New York: John Wiley & Sons; 1943. [Google Scholar]
- Kendall MG, Stuart A. The advanced theory of statistics. London: Charles Griffin; 1973. [Google Scholar]
- Linnet K. Estimation of the linear relationship between the measurements of two methods with proportional errors. Stat Med. 1990;9:1463–1473. doi: 10.1002/sim.4780091210. [DOI] [PubMed] [Google Scholar]
- Linnet K. Evaluation of regression procedures for methods comparison studies. Clin Chem. 1993;39:424–432. [PubMed] [Google Scholar]
- Martin RF. General deming regression for estimating systematic bias and its confidence interval in method-comparison studies. Clin Chem. 2000;46:100–104. [PubMed] [Google Scholar]
- Parvin CA. A direct comparison of two slope-estimation techniques used in method-comparison studies. Clin Chem. 1984;30:751–754. [PubMed] [Google Scholar]
- Begg CB, Gray R. Calculation of polychotomous logistic regression parameters using individualized regressions. Biometrika. 1984;71:11–18. doi: 10.2307/2336391. [DOI] [Google Scholar]
- Biesheuvel CJ, Vergouwe Y, Steyerberg EW, Grobbee DE, Moons KG. Polytomous logistic regression analysis could be applied more often in diagnostic research. J Clin Epidemiol. 2008;61:125–134. doi: 10.1016/j.jclinepi.2007.03.002. [DOI] [PubMed] [Google Scholar]
- Wijesinha A, Begg CB, Funkenstein HH, McNeil BJ. Methodology for the differential diagnosis of a complex data set. A case study using data from routine CT scan examinations. Med Decis Making. 1983;3:133–154. doi: 10.1177/0272989X8300300202. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.