Skip to main content
PMC home page
Health Services Research logoLink to Health Services Research
. 2008 Dec;43(6):2223–2238. doi: 10.1111/j.1475-6773.2008.00873.x

Accuracy of Physician Billing Claims for Identifying Acute Respiratory Infections in Primary Care

Geneviève Cadieux, Robyn Tamblyn
PMCID: PMC2614002  PMID: 18665858

Abstract

Objective

To assess the accuracy of physician billing claims for identifying acute respiratory infections in primary care.

Study Setting

Nine primary care physician practices in Montreal, Canada (2002–2005).

Study Design

A validation study was carried out to compare diagnoses in 3,526 physician billing claims with diagnoses documented in the corresponding patient medical records.

Data Collection

In-office medical record abstraction.

Principal Findings

Claims had a high positive predictive value (PPV), negative predictive value, and specificity for identifying respiratory infections; however, their sensitivity was below 50 percent. Large variation in sensitivity and PPV was observed among physicians.

Conclusions

Because claims data are now routinely used to monitor antibiotic prescribing in primary care, future research should determine if acute respiratory infection diagnoses are missing from claims at random, or if bias is present.

Keywords: Validation studies, databases, health services, International Classification of Diseases, respiratory tract infections

Several randomized, placebo-controlled trials of antibiotic use have shown that antibiotics do not provide clinical benefit to children or adults with upper respiratory tract infections (Hoaglund et al. 1950; Cronk et al. 1954; Howie and Clark 1970; Stott and West 1976; Verheij, Hermans, and Mulder 1994; Kaiser et al. 1996; Arroll 2005) and fail to prevent complicated bacterial infections (Gadomski 1993; Heikkinen et al. 1995). Yet, 75 percent of oral antibiotics prescribed to ambulatory patients are for pharyngitis, otitis media, sinusitis, bronchitis, common cold, and unspecified upper respiratory tract infection of likely viral etiology (McCaig and Hughes 1995), and 22–49 percent are estimated to be unnecessary (Kozyrskyj et al. 2004; Cadieux et al. 2007). Inappropriate use of antibiotics for respiratory infections promotes antibiotic resistance (Seppala et al. 1997; Austin, Kristinsson, and Anderson 1999; Pihlajamaki et al. 2001), increases health services utilization and costs (Little et al. 1997), and increases the risk of preventable drug-related adverse events (Classen et al. 1991). To enable the development of effective interventions to reduce inappropriate antibiotic use in primary care, determinants of inappropriate antibiotic prescribing and accurate methods for monitoring antibiotic use need to be identified.

Monitoring antibiotic prescribing in primary care is challenging because well-developed measures of antibiotic prescribing are scarce, often inaccurate, and may not reflect real prescribing practices. Studies of antibiotic prescribing in primary care have relied on physician self-reported prescribing (Mangione-Smith et al. 1999; Nash et al. 2002; Steinman, Landefeld, and Gonzales 2003), chart review or audit (Hueston, Jenkins, and Mainous 2000; Hutchinson et al. 2001; Mangione-Smith et al. 2002), or prescription claims (Mainous, Hueston, and Clark 1996; Majeed and Moser 1999; Wang et al. 1999; Steinke et al. 2000; Gill and Roalfe 2001; Kozyrskyj et al. 2004; Cadieux et al. 2007). Self-reported antibiotic prescribing was shown to underestimate actual antibiotic prescribing by about 30 percent (Mangione-Smith et al. 2002), and the cost of chart review is too high for wide-scale use. Prescription claims data avoid self-report bias, do not require additional data collection, and because they involve financial transactions, they are carefully audited by payers and have been found to be highly accurate (Tamblyn et al. 1995). Owing to these advantages, prescription claims are now used routinely to monitor antibiotic prescribing for respiratory infections in primary care (Mainous, Hueston, and Clark 1996; Majeed and Moser 1999; Wang et al. 1999; Steinke et al. 2000; Gill and Roalfe 2001; McCaig, Besser, and Hughes 2002; Kozyrskyj et al. 2004; Cadieux et al. 2007).

However, an important limitation of using prescription claims to monitor antibiotic prescribing is that treatment indication is not recorded on prescription claims. Treatment indication is required to determine the appropriateness of antibiotic prescribing; therefore, it must be inferred from other sources of information, such as physician billing claims for patient visits. If prescription claims are to be used to monitor antibiotic prescribing, then the accuracy of using diagnostic information in physician billing claims to infer the indication for antibiotic treatment needs to be assessed.

Two previous studies have assessed the accuracy of physician billing claims for identifying respiratory infection diagnoses, and both have shown promising results. The first was a study of administrative claims data from seven health insurance providers in Colorado, and it found that 79 percent of bronchitis diagnoses and 83 percent of pharyngitis diagnoses in administrative claims had a corresponding diagnosis in the written medical record (PPV; Maselli and Gonzales 2001). However, this study did not investigate what proportion of bronchitis and pharyngitis diagnoses documented in patient medical records was accurately documented in physician billing claims (sensitivity and specificity). The second study assessed the accuracy of Research Patient Data Repository (RPDR) claims from nine primary care clinics in the Brigham and Women's Primary Care Practice-Based Research Network in Boston and reported that 86 percent of respiratory infection diagnoses in RPDR claims had a corresponding diagnosis in the electronic health record (Linder et al. 2006). However, sensitivity and specificity estimates were not corrected for the verification bias introduced by over-sampling claims with a diagnosis of respiratory infection relative to claims without such a diagnosis (i.e., the study design inflated the prevalence of respiratory infection in the sample, relative to the true population prevalence; Begg and Greenes 1983; Irwig et al. 1994).

The objective of this study was to assess the accuracy of physician billing claims for identifying episodes of acute respiratory infection in primary care. In particular, we sought to estimate the sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of physician billing claims.

METHODS

Study Design and Population

A validation study was carried out to assess the sensitivity, specificity, PPV, and NPV of physician billing claims for identifying episodes of respiratory infection, as compared with the patient medical record. The study population comprised 34 Montreal region family physicians and 17,002 of their patients who were participating in the MOXXI electronic medication management trial (Tamblyn et al. 2006) in 2002–2005. All patients participating in the MOXXI trial had previously consented to share their medical records and provincial health insurance (RAMQ) data with researchers. These data were available for a period starting 1 year before patient enrollment date (2001 or later) until 2005, when the present study was conducted. From the available physician billing claims, we identified those with a diagnostic code (International Classification of Diseases, 9th Revision, or ICD-9) for laryngitis/tracheitis (464), common cold (460), influenza (480, 487), acute unspecified upper respiratory infection (465), pharyngitis/tonsillitis (462, 463, 034), otitis media (381, 382), sinusitis (461), acute bronchitis (466), or bacterial pneumonia (481–486); all decimal place suffixes of these ICD-9 codes were included. We purposefully selected 10 physicians who had been enrolled in the MOXXI trial for at least 2 years and had the most MOXXI-consenting patients (and therefore also had the most physician billing claims available for research purposes), and requested their consent.

Sample of Physician Billing Claims

Among the 10 physicians selected, we identified all MOXXI-consenting patients who had at least one physician billing claim with a diagnosis of acute respiratory infection during the study period, and randomly sampled 635 of those patients. We also identified all patients without any physician billing claim with a diagnosis of acute respiratory infection during the study period, and randomly sampled 94 of those patients. To improve the efficiency of data collection, we over-sampled patients with at least one diagnosis of acute respiratory infection, relative to those with no diagnosis of acute respiratory infection (Begg and Greenes 1983; Irwig et al. 1994). For each of the 729 patients sampled, we identified all physician billing claims generated during the study period (i.e., from 2001 or later, depending on the enrollment date, until 2005) and validated each one against the paper-based patient medical record. Information available in the physician billing claims included the patient's lifelong RAMQ personal identifier, physician license number, visit date, and ICD-9 diagnostic code.

Medical Record Abstraction

Once the physician billing claims had been sampled, a list of sampled patients' names and RAMQ personal identifiers was generated and sent to each consenting physician's office. The selected patients' paper-based medical records were retrieved by the office staff and reviewed by one of the authors (G. C.). For each sampled physician billing claim, the corresponding visit was identified in the medical record, the date of the visit was recorded, and the presence or absence of all acute respiratory infections under study was ascertained from the clinical notes. At the time of the medical record abstraction, the reviewer was blinded to the ICD-9 code in the corresponding physician billing claim. Information abstracted from patient medical records was entered directly in an electronic, structured chart abstraction form and stored in an MS Access database, which had been prepopulated with participating physicians' license numbers, sampled patients' RAMQ identifiers, and visit dates according to physician billing claims. Intra-rater reliability was measured on 25 randomly selected visits that were reviewed a second time, and the percent agreement between the first and second review was 100 percent.

Linkage of Physician Billing Claims and Data Abstracted from Medical Records

Data retrieved from patient medical records were linked directly to physician billing claims using the patient lifelong RAMQ personal identifier, physician license number, and visit date ±1 day.

Analyses

For each type of acute respiratory infection under study, a 2 × 2 table of diagnoses abstracted from patient medical records versus diagnoses obtained from the corresponding physician billing claims was generated using SAS statistical software (Version 9.3, SAS Institute Inc., Cary, NC). Information retrieved from the patient medical record was treated as a gold standard. The prevalence, PPV, and NPV of physician billing claims for identifying acute respiratory infections were estimated. Sensitivity and specificity estimates were corrected for the over-sampling of claims with a diagnosis of acute respiratory infection relative to claims without such a diagnosis (Begg and Greenes 1983) using MS Excel 2003 (Version 5.1). To investigate between-physician variation in physician billing claim diagnosis accuracy, these analyses were repeated for each physician individually, combining all nine types of acute respiratory infection under study (because each physician contributed too few of each type of acute respiratory infection to analyze each type individually).

Because we sampled several claims (and medical record visits) per patient, we conducted a sensitivity analysis to assess the effect of clustering of claims within patients on our estimates of sensitivity, specificity, PPV, and NPV. We did this by generating 100 random samples of one claim per patient (n=729 claims) from our total sample of 3,526 claims, and averaging the estimates of sensitivity, specificity, PPV, and NPV over all 100 random samples, which is similar to bootstrapping methodology (Efron and Tibshirani 1994).

RESULTS

Ten physicians participating in the MOXXI trial were purposefully selected for this study, and nine agreed to participate. Among these nine physicians' patients enrolled in the MOXXI trial, we randomly selected 635 patients who had at least one claim with a diagnosis of acute respiratory infection, and 94 patients without any claims with a diagnosis of acute respiratory infection. These 729 patients made 3,526 visits to their respective MOXXI physicians during the study period (duration of 1–5 years, depending on the date of enrollment), for an average of 4.8 visits per patient. The medical records of all 729 sampled patients were abstracted, and written documentation for each of the 3,526 visits identified from physician billing claims was found in the corresponding patient's medical record. In all, 1,173 (33.3 percent) of sampled claims were positive for respiratory infection (Figure 1). Sixty-six percent of sampled patients were women, and the mean age of sampled patients was 47.6 (SD 21.0, range <1–90 years). The characteristics of patients enrolled in the MOXXI trial, as compared with those of the general population, have been discussed previously (Bartlett et al. 2005).

Figure 1.

Figure 1

Open in a new tab

Diagnoses from the Sampled Physician Billing Claims and Corresponding Patient Medical Records.

The agreement between the diagnosis in the medical record and the ICD-9 code in the physician billing claim is shown in Table 1, where shaded areas indicate concordant diagnoses. For example, there were 63 physician billing claims with a diagnosis of laryngitis, and for all 63 claims, a diagnosis of laryngitis was also documented in the medical record at the corresponding date; however, an additional 16 diagnoses of laryngitis were documented in medical records that were not documented in physician billing claims. The overall percent agreement for the presence of any acute respiratory infection was 72.5 percent, which is the sum of all diagnoses of respiratory infection present in the physician billing claim and the corresponding medical record (969) divided by the sum of all diagnoses of respiratory infection documented in medical records (1,337).

Table 1.

Concordance of Diagnoses in Physician Billing Claims and Patient Medical Records

Number of Physician Billing Claims Number of Visits in Patient Medical Records
Laryngitis Common Cold Influenza Unspecified Acute URI Pharyngitis Otitis Media Sinusitis Acute Bronchitis Pneumonia No Respiratory Infection Total
Laryngitis 63 0 0 0 0 0 0 0 0 0 63
Common cold 1 66 1 0 2 0 0 0 0 4 74
Influenza 2 1 85 7 1 1 6 6 5 15 129
Unspecified acute URI 0 1 1 154 3 0 2 4 0 22 187
Pharyngitis 2 1 1 2 93 1 1 0 0 6 107
Otitis media 0 0 0 5 1 116 1 0 1 8 132
Sinusitis 1 1 1 0 0 0 138 3 0 9 153
Acute bronchitis 6 3 1 11 3 3 15 173 9 15 239
Pneumonia 0 0 0 0 0 0 2 2 81 4 89
No respiratory infection 4 29 5 28 12 28 19 28 11 2,189 2,353
Total 79 102 95 207 115 149 184 216 107 2,272 3,526
Uncorrected sensitivity* 0.80 0.65 0.89 0.74 0.81 0.78 0.75 0.80 0.76
Open in a new tab

Note: Shaded numbers indicate concordance.

*

The uncorrected sensitivity estimate is inflated due to the purposeful over-sampling of physician billing claims with a diagnosis of acute respiratory infection relative to claims without such a diagnosis (Begg and Greenes 1983; Irwig et al. 1994).

URI, upper respiratory infection.

The proportion of physician billing claims with a diagnosis of acute respiratory infection confirmed in the patient medical record (PPV) was 0.93, 95 percent CI (0.91, 0.94), for all acute respiratory infection combined, and 0.84, 95 percent CI (0.81, 0.88), for respiratory infections of likely viral etiology (Table 2). The PPV for acute respiratory infections of potentially bacterial etiology was 0.89, 95 percent CI (0.87, 0.92), and ranged from 0.72, 95 percent CI (0.67, 0.78), for acute bronchitis to 0.91, 95 percent CI (0.85, 0.97), for bacterial pneumonia. Sensitivity of physician billing claims for all acute respiratory infections combined was 0.49, 95 percent CI (0.45, 0.53). With the exception of influenza, sensitivity was markedly lower for viral respiratory infections than for bacterial ones. Specificity was 0.99 or higher for all types of acute respiratory infection studied.

Table 2.

Sensitivity, Specificity, and Positive and Negative Predictive Values of the RAMQ Physician Billing Claims for Identifying Episodes of Acute Respiratory Infection

RAMQ Physician Billing Claims
Prevalence per 1,000 (95% CI) Sensitivity* (95% CI) Specificity* (95% CI) PPV† (95% CI) NPV‡ (95% CI)
All respiratory infections 67.3 (65.6, 69.0) 0.49 (0.45, 0.53) 0.99 (0.99, 1.00) 0.93 (0.91, 0.94) 0.93 (0.92, 0.94)
All likely viral respiratory infections 16.4 (15.6, 17.3) 0.30 (0.26, 0.34) 1.00 (1.00, 1.00) 0.84 (0.81, 0.88) 0.97 (0.96, 0.97)
 Laryngitis/tracheitis 1.2 (0.9, 1.4) 0.20 (0.13, 0.30) 1.00 (1.00, 1.00) 1.00 (1.00, 1.00) 1.00 (0.99, 1.00)
 Common cold 1.4 (1.2, 1.7) 0.11 (0.08, 0.15) 1.00 (1.00, 1.00) 0.89 (0.82, 0.96) 0.99 (0.99, 0.99)
 Influenza 3.6 (3.2, 4.0) 0.45 (0.30, 0.60) 1.00 (1.00, 1.00) 0.66 (0.58, 0.74) 1.00 (1.00, 1.00)
 Unspecified acute URI 10.2 (9.6, 10.9) 0.35 (0.29, 0.42) 1.00 (1.00, 1.00) 0.82 (0.77, 0.88) 0.98 (0.98, 0.99)
All potentially bacterial respiratory infections 50.9 (49.4, 52.3) 0.51 (0.47, 0.56) 0.99 (0.99, 0.99) 0.89 (0.87, 0.92) 0.95 (0.95, 0.96)
 Pharyngitis/tonsillitis 5.3 (4.9, 5.8) 0.42 (0.32, 0.53) 1.00 (1.00, 1.00) 0.87 (0.81, 0.93) 0.99 (0.99, 1.00)
 Otitis media 8.6 (7.9, 9.2) 0.44 (0.35, 0.53) 1.00 (1.00, 1.00) 0.88 (0.82, 0.93) 0.99 (0.99, 0.99)
 Sinusitis 12.5 (11.7, 13.2) 0.46 (0.38, 0.53) 1.00 (1.00, 1.00) 0.90 (0.85, 0.95) 0.99 (0.98, 0.99)
 Acute bronchitis 19.5 (18.6, 20.4) 0.52 (0.46, 0.59) 0.99 (0.99, 0.99) 0.72 (0.67, 0.78) 0.99 (0.98, 0.99)
 Pneumonia 5.0 (4.5, 5.5) 0.38 (0.28, 0.47) 1.00 (1.00, 1.00) 0.91 (0.85, 0.97) 0.99 (0.99, 1.00)
Open in a new tab
*

Adjusted for over-sampling of physician billing claims with a diagnosis of acute respiratory infection relative to claims without such a diagnosis (Begg and Greenes 1983; Irwig et al. 1994).

†

Positive predictive value.

‡

Negative predictive value.

URI, upper respiratory infection.

The prevalence of acute respiratory infection diagnoses in physician billing claims varied between physicians from 19.5 to 111.4 per 1,000 claims (Table 3). Sensitivity and PPV varied between physicians from 1.00, 95 percent CI (1.00, 1.00), to 0.19, 95 percent CI (0.06, 0.47), and from 0.98, 95 percent CI (0.96, 1.00), to 0.70, 95 percent CI (0.53, 0.87), respectively. The accuracy of physician billing claims for identifying acute respiratory infections did not appear to be higher among physicians who diagnosed more acute respiratory infections.

Table 3.

Sensitivity, Specificity, and Positive and Negative Predictive Values of the RAMQ Physician Billing Claims Database for Identifying All Acute Respiratory Infections Combined, by Study Physician

Study Physician RAMQ Physician Billing Claims
Number of Claims with a Diagnosis of Acute RI Total Number of Claims Prevalence per 1,000 (95% CI) Sensitivity* (95% CI) Specificity* (95% CI) PPV† (95% CI) NPV‡ (95% CI)
1 1,324 16,264 81.4 (77.1, 85.6) 0.85 (0.78, 0.91) 1.00 (0.99, 1.00) 0.96 (0.94, 0.98) 0.99 (0.98, 1.00)
2 2,041 18,319 111.4 (106.9, 116.0) 0.39 (0.33, 0.44) 0.98 (0.98, 0.99) 0.89 (0.86, 0.92) 0.82 (0.78, 0.86)
3 522 4,815 108.4 (99.6, 117.2) 0.42 (0.35, 0.49) 1.00 (0.99, 1.00) 0.97 (0.95, 1.00) 0.84 (0.80, 0.88)
4 356 9,614 37.0 (33.3, 40.8) 0.34 (0.24, 0.47) 1.00 (1.00, 1.00) 0.98 (0.96, 1.00) 0.93 (0.90, 0.96)
5 163 5,553 29.4 (24.9, 33.8) 0.37 (0.22, 0.56) 1.00 (0.99, 1.00) 0.89 (0.81, 0.97) 0.95 (0.92, 0.99)
6 159 6,905 23.0 (19.5, 26.6) 0.57 (0.16, 0.90) 0.99 (0.99, 1.00) 0.77 (0.64, 0.90) 0.99 (0.98, 1.00)
7 752 6,754 111.3 (103.8, 118.8) 0.72 (0.45, 0.89) 0.98 (0.96, 1.00) 0.82 (0.66, 0.98) 0.96 (0.92, 1.00)
8 94 4,815 19.5 (15.6, 23.4) 1.00 (1.00, 1.00) 0.99 (0.99, 1.00) 0.70 (0.53, 0.87) 1.00 (1.00, 1.00)
9 365 18,319 19.9 (17.9, 21.9) 0.19 (0.06, 0.47) 1.00 (0.99, 1.00) 0.97 (0.95, 1.00) 0.92 (0.84, 1.00)
Open in a new tab
*

Adjusted for over-sampling of respiratory infections relative to other diagnoses (Begg and Greenes 1983; Irwig et al. 1994).

†

Positive predictive value.

‡

Negative predictive value.

RI, respiratory infection.

Our sensitivity analysis using only one claim per patient yielded estimates for sensitivity (0.55, 95 percent CI 0.45, 0.64), specificity (0.99, 95 percent CI 0.99, 1.00), PPV (0.93, 95 percent CI 0.90, 0.96), and NPV (0.94, 95 percent CI 0.91, 0.97) that were similar to the estimates obtained when all visits were used. The confidence intervals from the sensitivity analysis are wider because the sample size is smaller: 729 claims (one per patient) were used instead of all 3,526. This shows that the effect of within-patient clustering of claims on sensitivity, specificity, PPV, and NPV estimates is small.

DISCUSSION

The PPV of physician billing claims was high for all types of acute respiratory infection studied. Our study was the first to estimate the prevalence, sensitivity, and specificity of physician billing claims for identifying chart-documented acute respiratory infections in primary care. For all but one type of acute respiratory infection investigated, our sensitivity estimates were below 0.50. Our study was also the first to look at between-physician variation in physician billing claim diagnosis accuracy. We found that prevalence of respiratory infections in physician billing claims varied widely between primary care physicians. We also observed large unexplained between-physician variation in sensitivity and PPV of physician billing claims for identifying acute respiratory infections.

If physician billing claims had many false-positive diagnoses of respiratory infection, they would not be a useful data source for monitoring antibiotic prescribing. Therefore, a high PPV, or a high likelihood that diagnoses of respiratory infection in physician billing claims are also present in the corresponding patient medical record, provides support for using health administrative data for monitoring antibiotic prescribing. For most of the acute respiratory infection diagnoses investigated, our estimates of PPV were similar to those previously reported in the literature (Maselli and Gonzales 2001; Linder et al. 2006). However, our PPV estimate for influenza (0.66, 95 percent CI 0.58, 0.74) was much higher than the 0.20 reported by Linder et al. (2006), but the latter was aberrantly low as compared with other PPV estimates in the same study.

Previous studies have emphasized the high PPV of health administrative data for identifying episodes of respiratory infection, but have overlooked the importance of sensitivity (Maselli and Gonzales 2001; Linder et al. 2006). A high sensitivity is desirable because it suggests that the data capture a majority of visits for respiratory infections. A low sensitivity is problematic because it suggests that several visits for respiratory infections are not documented in health administrative data. Nondocumentation of visits for respiratory infections may or may not be associated with antibiotic prescribing, which may result in bias when using health administrative data to monitor antibiotic prescribing.

Our study estimated the sensitivity of physician billing claims for identifying acute respiratory infections. Our sensitivity estimates were below 0.50 for all types of acute respiratory infection studied except acute bronchitis, which raises concerns about the potential for bias. Whereas one previous study estimated the sensitivity of claims for identifying respiratory infections (Linder et al. 2006), the authors did not correct their sensitivity estimate for the verification bias introduced by over-sampling claims with a diagnosis of acute respiratory infection relative to claims without such a diagnosis (Begg and Greenes 1983; Irwig et al. 1994); consequently, they greatly overestimated sensitivity. For example, if we had not corrected our estimates for verification bias, our estimate of the sensitivity of physician billing claims for identifying laryngitis would have been 0.80, as compared with the corrected sensitivity estimate of 0.20.

We were first to investigate between-physician variation in physician billing claim diagnosis accuracy for acute respiratory infections. We found almost a sixfold variation between physicians in the prevalence of acute respiratory infections. We observed similar between-physician variation in the sensitivity and PPV of physician billing claims for identifying acute respiratory infections. We expected that claims submitted by physicians who diagnosed more acute respiratory infections would be more accurate for identifying acute respiratory infections, but we found that neither frequency nor prevalence of acute respiratory infections seemed to be related to physician billing claim diagnosis accuracy. This finding suggests that other factors are likely responsible for the observed between-physician variation in physician billing claim diagnosis accuracy.

A limitation of our study is that medical records may not represent a true gold standard for identifying acute respiratory infections diagnosed in primary care. The use of a single rater was also a limitation of our study, and systematic misclassification of acute respiratory infection diagnoses may have occurred as a result. Another limitation of our study was its small convenience sample of primary care physicians. Whereas physicians participating in the MOXXI trial are generally similar to other eligible physicians in the Montreal region, they tend to be younger than MOXXI nonparticipants. If physician billing claim diagnosis accuracy is related to physician age or practice experience, then our study results may not be applicable to older or more experienced physicians. Also, the MOXXI trial involves physicians practicing in urban and suburban areas, and our results may not be generalizable to physicians practicing in rural areas. Furthermore, patients enrolled in the MOXXI trial tend to differ from nonparticipating patients in that they are generally older, with more complex health status, and have more visits to the MOXXI physician (Bartlett et al. 2005). Younger, healthier patients may be underrepresented in our study sample. Future research should involve a large random sample of primary care physician from both urban and rural areas, and a stratified random sample of patients from each physician's practice population.

Because physician billing claims and prescription claims are now routinely used to monitor antibiotic prescribing for acute respiratory infections in primary care (Mainous, Hueston, and Clark 1996; Majeed and Moser 1999; Wang et al. 1999; Steinke et al. 2000; Gill and Roalfe 2001; Kozyrskyj et al. 2004; Cadieux et al. 2007), it is important for future research to determine whether half of all acute respiratory infections diagnoses are missing from physician billing claims at random, or whether bias is present. If bias is present, future research should also focus on identifying determinants of physician billing claim diagnosis accuracy, so that appropriate corrections for the resulting bias can be developed and applied when physician billing claims are used to infer treatment indication for antibiotic prescribing. As suggested by the large between-physician variation observed in this study, physician characteristics may be associated with physician billing claim diagnosis accuracy. The effect of physician characteristics, as well as patient, encounter, practice, and billing characteristics, on physician billing claim diagnosis accuracy should be assessed.

Acknowledgments

Joint Acknowledgement/Disclosure Statement: Operational funding for this study was provided by the Canadian Institutes of Health Research. We would like to thank the MOXXI field coordinator, Rola El Halabieh, for her help in contacting physicians to participate in this study, and the MOXXIIT team (Jimmy Fragos and Marc-André Larochelle) for their help with database management.

Disclaimers: None.

Disclosures: None.

Supplementary material

The following supplementary material for this article is available online:

Appendix SA1

Verification Bias Correction.

hesr0043-2223-SD2.xls (21.5KB, xls)
Appendix SA2

Author matrix.

hesr0043-2223-SD1.doc (83KB, doc)

This material is available as part of the online article from: http://www.blackwell-synergy.com/doi/abs/10.1111/j.1475-6773.2008.00873.x (this link will take you to the article abstract).

Please note: Blackwell Publishing is not responsible for the content or functionality of any supplementary materials supplied by the authors. Any queries (other than missing material) should be directed to the corresponding author for the article.

REFERENCES

  1. Arroll B. Antibiotics for Upper Respiratory Tract Infections: An Overview of Cochrane Reviews. Respiratory Medicine. 2005;99(3):255–61. doi: 10.1016/j.rmed.2004.11.004. [DOI] [PubMed] [Google Scholar]
  2. Austin D J, Kristinsson K G, Anderson R M. The Relationship between the Volume of Antimicrobial Consumption in Human Communities and the Frequency of Resistance. Proceedings of the National Academy of Sciences of the United States of America. 1999;96(3):1152–6. doi: 10.1073/pnas.96.3.1152. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Bartlett G, Tamblyn R, Kawasumi Y, Poissant L, Taylor L. Non-Participation Bias in Health Services Research Using Data from an Integrated Electronic Prescribing Project: The Role of Informed Consent. Acta Bioethica. 2005;11(2):145–59. [Google Scholar]
  4. Begg C B, Greenes R A. Assessment of Diagnostic-Tests When Disease Verification Is Subject to Selection Bias. Biometrics. 1983;39(1):207–15. [PubMed] [Google Scholar]
  5. Cadieux G, Tamblyn R, Dauphinee D, Libman M. Predictors of Inappropriate Antibiotic Prescribing among Primary Care Physicians. Canadian Medical Association Journal. 2007;177(8):877–83. doi: 10.1503/cmaj.070151. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Classen D C, Pestotnik S L, Evans R S, Burke J P. Computerized Surveillance of Adverse Drug Events in Hospital Patients. Journal of the American Medical Association. 1991;266(20):2847–51. [PubMed] [Google Scholar]
  7. Cronk G A, Naumann D E, Mcdermott K, Menter P, Swift M B. A Controlled Study of the Effect of Oral Penicillin-G in the Treatment of Non-Specific Upper Respiratory Infections. American Journal of Medicine. 1954;16(6):804–9. doi: 10.1016/0002-9343(54)90444-1. [DOI] [PubMed] [Google Scholar]
  8. Efron B, Tibshirani R J. Monographs on Statistics and Applied Probability. Boca Raton, FL: Chapman & Hall; 1994. An Introduction to the Bootstrap. [Google Scholar]
  9. Gadomski A M. Potential Interventions for Preventing Pneumonia among Young-Children—Lack of Effect of Antibiotic-Treatment for Upper Respiratory-Infections. Pediatric Infectious Disease Journal. 1993;12(2):115–20. doi: 10.1097/00006454-199302000-00002. [DOI] [PubMed] [Google Scholar]
  10. Gill P S, Roalfe A. Antibiotic Prescribing by Single Handed General Practitioners: Secondary Analysis of Data. Journal of Clinical Pharmacy and Therapeutics. 2001;26(3):195–9. doi: 10.1046/j.1365-2710.2001.00345.x. [DOI] [PubMed] [Google Scholar]
  11. Heikkinen T, Ruuskanen O, Ziegler T, Waris M, Puhakka H. Short-Term Use of Amoxicillin-Clavulanate during Upper Respiratory-Tract Infection for Prevention of Acute Otitis-Media. Journal of Pediatrics. 1995;126(2):313–6. doi: 10.1016/s0022-3476(95)70569-4. [DOI] [PubMed] [Google Scholar]
  12. Hoaglund R J, Dietz E N, Myers P W, Cosand H C. Aureomycin in the Treatment of the Common Cold. New England Journal of Medicine. 1950;243:773–5. doi: 10.1056/NEJM195011162432003. [DOI] [PubMed] [Google Scholar]
  13. Howie J G, Clark G A. Double-Blind Trial of Early Demethylchlortetracycline in Minor Respiratory Illness in General Practice. Lancet. 1970;2(7683):1099–1102. doi: 10.1016/s0140-6736(70)92294-4. [DOI] [PubMed] [Google Scholar]
  14. Hueston W J, Jenkins R, Mainous A G., III Does Drug Treatment of Patients with Acute Bronchitis Reduce Additional Care Seeking? Evidence from the Practice Partner Research Network. Archives of Family Medicine. 2000;9(10):997–1001. doi: 10.1001/archfami.9.10.997. [DOI] [PubMed] [Google Scholar]
  15. Hutchinson J M, Jelinski S, Hefferton D, Desaulniers G, Parfrey P S. Role of Diagnostic Labeling in Antibiotic Prescription. Canadian Family Physician. 2001;47:1217–24. [PMC free article] [PubMed] [Google Scholar]
  16. Irwig L, Glasziou P P, Berry G, Chock C, Mock P, Simpson J M. Efficient Study Designs to Assess the Accuracy of Screening-Tests. American Journal of Epidemiology. 1994;140(8):759–69. doi: 10.1093/oxfordjournals.aje.a117323. [DOI] [PubMed] [Google Scholar]
  17. Kaiser L, Lew D, Hirschel B, Auckenthaler R, Morabia A, Heald A, Benedict P, Terrier F, Wunderli W, Matter L, Germann D, Voegeli J, Stalder H. Effects of Antibiotic Treatment in the Subset of Common-Cold Patients Who Have Bacteria in Nasopharyngeal Secretions. Lancet. 1996;347(9014):1507–10. doi: 10.1016/s0140-6736(96)90670-4. [DOI] [PubMed] [Google Scholar]
  18. Kozyrskyj A L, Dahl M E, Chateau D G, Mazowita G B, Klassen T P, Law B J. Evidence-Based Prescribing of Antibiotics for Children: Role of Socioeconomic Status and Physician Characteristics. Canadian Medical Association Journal. 2004;171(2):139–45. doi: 10.1503/cmaj.1031629. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Linder J A, Bates D W, Williams D H, Connolly M A, Middleton B. Acute Infections in Primary Care: Accuracy of Electronic Diagnoses and Electronic Antibiotic Prescribing. Journal of the American Medical Informatics Association. 2006;13(1):61–6. doi: 10.1197/jamia.M1780. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Little P, Gould C, Williamson I, Warner G, Gantley M, Kinmonth A L. Reattendance and Complications in a Randomised Trial of Prescribing Strategies for Sore Throat: The Medicalising Effect of Prescribing Antibiotics. British Medical Journal. 1997;315(7104):350–2. doi: 10.1136/bmj.315.7104.350. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Mainous A G, III, Hueston W J, Clark J R. Antibiotics and Upper Respiratory Infection: Do Some Folks Think There Is a Cure for the Common Cold. Journal of Family Practice. 1996;42(4):357–61. [PubMed] [Google Scholar]
  22. Majeed A, Moser K. Age- and Sex-Specific Antibiotic Prescribing Patterns in General Practice in England and Wales in 1996. British Journal of General Practice. 1999;49(446):735–6. [PMC free article] [PubMed] [Google Scholar]
  23. Mangione-Smith R, Elliott M N, McDonald L, McGlynn E A. An Observational Study of Antibiotic Prescribing Behavior and the Hawthorne Effect. Health Services Research. 2002;37(6):1603–23. doi: 10.1111/1475-6773.10482. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Mangione-Smith R, McGlynn E A, Elliott M N, Krogstad P, Brook R H. The Relationship between Perceived Parental Expectations and Pediatrician Antimicrobial Prescribing Behavior. Pediatrics. 1999;103(4 Pt 1):711–8. doi: 10.1542/peds.103.4.711. [DOI] [PubMed] [Google Scholar]
  25. Maselli J H, Gonzales R. Measuring Antibiotic Prescribing Practices among Ambulatory Physicians: Accuracy of Administrative Claims Data. Journal of Clinical Epidemiology. 2001;54(2):196–201. doi: 10.1016/s0895-4356(00)00269-9. [DOI] [PubMed] [Google Scholar]
  26. McCaig L F, Besser R E, Hughes J M. Trends in Antimicrobial Prescribing Rates for Children and Adolescents. Journal of the American Medical Association. 2002;287(23):3096–102. doi: 10.1001/jama.287.23.3096. [DOI] [PubMed] [Google Scholar]
  27. McCaig L F, Hughes J M. Trends in Antimicrobial Drug Prescribing among Office-Based Physicians in the United-States. Journal of the American Medical Association. 1995;273(3):214–9. [PubMed] [Google Scholar]
  28. Nash D R, Harman J, Wald E R, Kelleher K J. Antibiotic Prescribing by Primary Care Physicians for Children with Upper Respiratory Tract Infections. Archives of Pediatrics and Adolescent Medicine. 2002;156(11):1114–9. doi: 10.1001/archpedi.156.11.1114. [DOI] [PubMed] [Google Scholar]
  29. Pihlajamaki M, Kotilainen P, Kaurila T, Klaukka T, Palva E, Huovinen P, Kostiala-Thompson A, Renkonen R, Muotiala A, Vaara M, Sivonen A, Jousimies-Somer H, Laitinen K, Korpela J, Kaukoranta-Tolvanen S S, Hiekkaniemi H, Hirvonen P, Nissinen A, Ruuska P, Ahonen E, Jagerroos H, Raisanen S, Larinkari U, Forsblom B, Katila M L, Farkkainen U, Sarkkinen H, Karpanoja P, Kauppinen M, Paltemaa S, Karkkainen P, Silvennoinen-Kassinen S, Lantto K, Koskela M, Pietarinen I, Klossner M L, Pajarre S, Oinonen S, Ratia V, Gronroos P, Vuento R, Liimatainen O, Siro M R, Eerola E, Maninen R, Meurman O, Luukkonen S, Stranden P, Lager K. Macrolide-Resistant Streptococcus pneumoniae and Use of Antimicrobial Agents. Clinical Infectious Diseases. 2001;33(4):483–8. doi: 10.1086/322735. [DOI] [PubMed] [Google Scholar]
  30. Seppala H, Klaukka T, Vuopio-Varkila J, Muotiala A, Helenius H, Lager K, Huovinen P. The Effect of Changes in the Consumption of Macrolide Antibiotics on Erythromycin Resistance in Group A Streptococci in Finland. Finnish Study Group for Antimicrobial Resistance. New England Journal of Medicine. 1997;337(7):441–6. doi: 10.1056/NEJM199708143370701. [DOI] [PubMed] [Google Scholar]
  31. Steinke D T, Bain D J, MacDonald T M, Davey P G. Practice Factors That Influence Antibiotic Prescribing in General Practice in Tayside. Journal of Antimicrobial Chemotherapy. 2000;46(3):509–12. doi: 10.1093/jac/46.3.509. [DOI] [PubMed] [Google Scholar]
  32. Steinman M A, Landefeld C S, Gonzales R. Predictors of Broad-Spectrum Antibiotic Prescribing for Acute Respiratory Tract Infections in Adult Primary Care. Journal of the American Medical Association. 2003;289(6):719–25. doi: 10.1001/jama.289.6.719. [DOI] [PubMed] [Google Scholar]
  33. Stott N C, West R R. Randomised Controlled Trial of Antibiotics in Patients with Cough and Purulent Sputum. British Medical Journal. 1976;2(6035):556–9. doi: 10.1136/bmj.2.6035.556. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Tamblyn R, Huang A, Kawasumi Y, Bartlett G, Grad R, Jacques A, Dawes M, Abrahamowicz M, Perreault R, Taylor L, Winslade N, Poissant L, Pinsonnikault A, Pinsonneault A. The Development and Evaluation of an Integrated Electronic Prescribing and Drug Management System for Primary Care. Journal of the American Medical Informatics Association. 2006;13(2):148–59. doi: 10.1197/jamia.M1887. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Tamblyn R, Lavoie G, Petrella L, Monette J. The Use of Prescription Claims Databases in Pharmacoepidemiological Research: The Accuracy and Comprehensiveness of the Prescription Claims Database in Quebec. Journal of Clinical Epidemiology. 1995;48(8):999–1009. doi: 10.1016/0895-4356(94)00234-h. [DOI] [PubMed] [Google Scholar]
  36. Verheij T J, Hermans J, Mulder J D. Effects of Doxycycline in Patients with Acute Cough and Purulent Sputum: A Double Blind Placebo Controlled Trial. British Journal of General Practice. 1994;44(386):400–4. [PMC free article] [PubMed] [Google Scholar]
  37. Wang E E, Einarson T R, Kellner J D, Conly J M. Antibiotic Prescribing for Canadian Preschool Children: Evidence of Overprescribing for Viral Respiratory Infections. Clinical Infectious Diseases. 1999;29(1):155–60. doi: 10.1086/520145. [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Appendix SA1

Verification Bias Correction.

hesr0043-2223-SD2.xls (21.5KB, xls)
Appendix SA2

Author matrix.

hesr0043-2223-SD1.doc (83KB, doc)

Articles from Health Services Research are provided here courtesy of Health Research & Educational Trust

RESOURCES