Skip to main content
PMC home page
PLOS ONE logoLink to PLOS ONE
. 2023 Jan 20;18(1):e0280151. doi: 10.1371/journal.pone.0280151

Evaluating the cost-effectiveness of testing pregnant women for penicillin allergy

Viengneesee Thao 1, Emily E Sharpe 2, Ruchita Dholakia 1, Hannah H Ahn 1, James P Moriarty 1, Bijan J Borah 1,3, Margaret C Gill 4, Regan N Theiler 5,*
Editor: Tefera Chane Mekonnen6
PMCID: PMC9858404  PMID: 36662778

Abstract

Introduction

True penicillin allergy is rare and is commonly incorrectly reported. In fact, less than five percent of patients who report a penicillin allergy will have a currently active clinically-significant IgE- or T-cell-mediated hypersensitivity when appropriately tested. Penicillin is the agent of choice for intrapartum antibiotic prophylaxis to reduce the risk of group B streptococcus early-onset disease in the newborn. Inaccurate penicillin allergy status may lead to inappropriate antibiotic use, as most alternative drugs are more expensive and broader spectrum than penicillin. Penicillin allergy testing has been found to be safe in pregnancy and cost-effective in other patient populations.

Objective

To evaluate the cost-effectiveness of penicillin allergy testing and appropriate antibiotic treatment (test then treat strategy) compared to usual care among pregnant women.

Methods

We developed a decision tree to evaluate the cost of providing appropriate care via a test then treat strategy for pregnant women who report a penicillin allergy, compared to usual care.

Results

Using the test then treat strategy the additional cost to ensure appropriate care for all pregnant women who report a penicillin allergy, was $1122.38 per person. Adopting a test then treat strategy increased the number of appropriate antibiotic use from 7,843/10,000 to 10,000/10,000 simulations.

Conclusion

Our results show that a test then treat strategy for pregnant women who report a penicillin allergy is a good-value intervention.

Introduction

Penicillin allergy is the most commonly reported allergy in pregnant women. The reported rate of unconfirmed penicillin allergy of about 8.0% in pregnant women is lower than commonly reported rates in outpatients using healthcare or other hospitalized populations [1, 2]. In hospitalized patients, penicillin allergy is also the most commonly reported antibiotic allergy, occurring in 10–25% of patients [3–6]. When reported adverse reactions to antibiotics are not explored by prescribing providers, it often results in the use of alternative broader spectrum treatment than is required. This practice contributes to the overuse of broad-spectrum agents which has added to the emergence of multiple drug-resistant microbes. Interestingly, even among patients with a history of antibiotic-associated anaphylaxis, only 20–50% will have a positive skin test when tested within 3 months of the reported anaphylaxis [7]. Ultimately, less than five percent of patients who report a penicillin allergy actually have a clinically significant IgE- mediated or T- lymphocyte mediated hypersensitivity when tested [2].

Penicillin is the agent of choice for intrapartum antibiotic prophylaxis to reduce the risk of Group B streptococcus (GBS) early-onset disease in the newborn because it has a narrow, targeted spectrum of antimicrobial activity and is highly effective. An alternative antibiotic to penicillin includes first-generation cephalosporins such as cefazolin for women with a low-risk penicillin allergy. In women with high-risk penicillin allergy or a positive allergy test, clindamycin is recommended; however, GBS isolates can be resistant to clindamycin and therefore susceptibility testing is necessary. When a GBS isolate is not susceptible to clindamycin, vancomycin is the only validated option for intrapartum prophylaxis in patients with a penicillin allergy. GBS is the most common infectious cause of morbidity and mortality in neonates [8]. The primary risk factor for early-onset GBS infection in neonates is colonization of the maternal rectum or genital tract, and between 10–30% of women are colonized with GBS [9–12]. Because of the significant morbidity and mortality associated with neonatal GBS infection, both the Centers for Disease Control and Prevention and American College of Obstetricians and Gynecologists (ACOG) recommend universal culture-based GBS screening for all prenatal patients between 36 0/7 and 37 6/7 weeks of gestation [13]. GBS remains susceptible to both penicillin and ampicillin, and penicillin is the treatment of choice for GBS infections and prophylaxis because of its low cost, effectiveness, and narrow antimicrobial spectrum [8]. For this reason, ACOG has recommended penicillin allergy testing for pregnant women who report a penicillin or amoxicillin allergy [13]. This recommendation aligns with the American Academy of Allergy, Asthma and Immunology’s 2016 recommendation that penicillin allergy testing should be routinely performed on all patients with a listed allergy to penicillin, ampicillin, or amoxicillin [14]. Using economic modeling, Sousa-Pinto et al. evaluated penicillin allergy testing techniques to evaluate cost-savings in 24 decision models [15]. They found penicillin allergy testing to be cost saving, with a net benefit ranging from $256 to $6745 for inpatients and outpatients, respectively. Protocols for penicillin allergy testing are well established, the cost of testing has been previously characterized in detail [16], and testing has been shown to be safe for pregnant women [17–19]. Considering pregnancy care and the delivery hospitalization, including GBS antimicrobial susceptibility testing, we compared the cost of usual care to the cost of a test then treat strategy for all pregnant women who report a penicillin allergy.

Methods

We report our methods using the Consolidated Health Economic Evaluation Reporting Standards 2022 (CHEERS 2022) guidelines. In this institutional review board exempt study, we used billing data from a tertiary care center and previously published literature. Our decision tree compares the additional cost of testing for penicillin allergy to no test (usual care), which ensures appropriate antibiotic use for all women in the test then treat strategy. Our population of interest includes all pregnant women who report a penicillin allergy. We excluded outcomes for neonatal care and/or maternal post-cesarean surgical site infections.

Decision tree model

To estimate the cost-effectiveness of a test then treat strategy for pregnant women who self-report a penicillin allergy, we constructed a decision tree. The decision tree compared the test then treat strategy to usual care. We selected appropriate or inappropriate antibiotic use as the endpoint of the model. Because of lack of published differential effectiveness outcomes for penicillin vs. alternative antibiotics, our model does not include neonatal or maternal post-cesarean surgical site infection outcomes. A diagram of our decision tree is shown in Fig 1.

Fig 1. Decision tree.

Fig 1

Open in a new tab

Briefly, women under the test then treat strategy are first tested for a penicillin allergy and then treated according to their test results, while those under usual care are treated as if their self-reported allergy is confirmed. No antibiotics are routinely indicated for GBS negative women who give birth vaginally, but penicillin prophylaxis is indicated for those who are colonized with GBS. The second line antibiotic for GBS prophylaxis is clindamycin, with vancomycin reserved for those with clindamycin-resistant bacterial isolates. Therefore, women who have a penicillin allergy and whose rectal-vaginal cultures are positive for GBS should have additional testing of GBS isolates for clindamycin susceptibility. In those GBS positive patients attempting vaginal birth, vancomycin is prescribed when the GBS isolate is not susceptible to clindamycin. Alternately, for patients with no penicillin allergy, cefazolin is prescribed when giving birth via cesarean (planned or unplanned), regardless of GBS status. For those with a low-risk penicillin allergy, cefazolin may still be used at cesarean delivery, but for those with a history of anaphylaxis to beta lactams, clindamycin and gentamycin are used for surgical prophylaxis [20].

Model inputs

Our model included the probability of true penicillin allergy, prevalence of GBS and clindamycin resistance, and rates of delivery modes (planned and unplanned cesarean, and vaginal). We searched previously published literature to identify these inputs. We assumed that the true rate of penicillin allergy among pregnant women was similar to what is reported among the general population (2%) [17]. GBS prevalence was estimated using a cohort study of all pregnant women who delivered at a Duke Health affiliated hospital between January 1, 2003 and December 31, 2015. That study found a GBS prevalence of 21.6% [1]. Clindamycin resistance was estimated from a study that collected two hundred random GBS single-patient isolates over a four year time period. That study found that 33% of samples had clindamycin resistance [21]. The rate of planned [17.4%] and unplanned cesarean deliveries (13.1%) were informed by The Consortium on Safe Labor, which used electronic medical records from 19 hospitals across the U.S. to approximate almost 4 million cesarean births [22].

Costs included the cost of antibiotics and testing. We report costs in 2020 U.S. dollars (USD) and we retrieved cost inputs from three major sources: billing data from our standardized cost data warehouse; previously published literature; and the Centers for Medicare and Medicaid Services fee schedule. We pulled the cost of each antibiotic by searching our standardized cost data warehouse for pregnant women giving birth between December 2017 to March 2021, who were prescribed any of the antibiotics of interest. The cost data warehouse is an internal resource that uses widely accepted methods to convert internal costs to standardized costs for publication [23]. To briefly describe the method, we use a hybrid model where Medicare reimbursement rates are applied to professional services. The hospital services are calculated by applying hospital cost-to-charge ratios to the charges. We report the mean and SD of each antibiotic, separately (Table 1). We included costs for all women and note that there was substantial variability due to differences among delivery times, which in turn affected the amount of antibiotics needed. Using those methods, we found that the average cost of a course of penicillin was $65; cefazolin $96; clindamycin $89; gentamycin $56; and vancomycin $68. The cost of a penicillin allergy test was informed by a cost study that included the following components: penicillin allergy evaluation, penicillin skin testing, and a 1-step amoxicillin drug challenge. It found that the average cost of a penicillin allergy test was $220 (2016 USD) [16]. We updated this cost to 2020 USD using the U.S. Bureau of Labor Statistics’ Consumer Price Index for hospital services [24]. The estimated cost of penicillin allergy testing used in this analysis is probably higher than the real world, because many low-risk penicillin “allergic” individuals are currently delabeled by a direct oral amoxicillin challenge without antecedent skin testing. The cost of GBS sensitivity testing was informed by the Centers for Medicare and Medicaid Services fee schedule for procedure code 87186, which was listed as $9 (2020 USD) [25].

Table 1. Model inputs.

Mean SD Distribution
Probabilities
True penicillin allergy [17] 0.06 7x10-6 Beta
Group B Streptococcus [1] 0.216 2x10-3 Beta
Clindamycin resistant [21] 0.33 3x10-2 Beta
Planned cesarean [22] 0.174 2x10-4 Beta
Unplanned cesarean [22] 0.131 2x10-4 Beta
Per person costs, 2020 USD
Cefazolin $ 96 $ 10 Gamma
Clindamycin $ 89 $ 22 Gamma
Gentamycin $ 56 $ 10 Gamma
Penicillin $ 65 $ 19 Gamma
Vancomycin $ 68 $ 10 Gamma
Penicillin allergy test [16] $ 256 $ 144 Gamma
Clindamycin resistant test [25] $ 9 $ 6 Gamma
Open in a new tab

Cost-effectiveness analysis

We conducted our cost-effectiveness analysis in TreeAge Pro v.2019. Our timeline included testing for penicillin allergy and GBS sensitivity, and delivery. Due to the short duration of our model timeline, no discounting of outcomes was needed. Our outcomes of interest included the total cost per patient and the proportion of patients with appropriate antibiotic use, under each strategy. To estimate the incremental difference between the two strategies of interest, we calculated a ratio comprising of the additional cost of the test then treat strategy over usual care, divided by the percent change in appropriate antibiotic prescription of the test then treat strategy over usual care. This incremental ratio may be interpreted as the average additional cost per pregnant woman to ensure that they receive appropriate antibiotics during delivery; and this additional cost would only apply to women with a history of penicillin allergy.

Sensitivity analysis

We conducted one-way and probabilistic sensitivity analysis (PSA). In the one-way sensitivity analysis, we varied each parameter one at a time by one standard deviation of its mean to identify the additional cost of test then treat compared to usual care. Our results are displayed in a tornado diagram (Fig 2A and 2B). For the PSA, we varied all inputs simultaneously in over 10,000 simulations of the model. The inputs were sampled from the corresponding parameter distributions, using the means and standard deviations reported in Table 1. Probabilities were sampled from β (beta) distributions and parameterized using data reported in cited source materials. Cost parameters were sampled from γ (gamma) distributions and parameterized by referring to our internal data when available, or cited source materials when available. We also used tracker variables to identify the proportion of women who would receive appropriate and inappropriate antibiotic care under each treatment strategy. Results from the PSA are displayed as a scatter plot and as counts.

Fig 2.

Fig 2

Open in a new tab

A. 1-way sensitivity analysis on cost parameters, tornado diagram. B. 1-way sensitivity analysis on probability parameters, tornado diagram.

Results

Cost-effectiveness

We found that on average the cost of usual care was $57.55 per person and ensured appropriate antibiotic therapy for 79% of pregnant women with a self-reported penicillin allergy. The test then treat strategy was on average $295.15 per person and ensured that 100% of all pregnant women with a self-reported penicillin allergy would receive appropriate antibiotic therapy. Therefore, the additional cost to ensure appropriate GBS prophylaxis for all women who report a penicillin allergy, using the test then treat strategy, was $1122.38 per person reporting an allergy (Table 2).

Table 2. Cost-effectiveness results.

Cost Δ Cost Effect Δ Effect Cost/Effect N appropriate antibiotic use N in-appropriate antibiotic use
Usual care $ 57.55 0.79 7,843 2,157
Test then treat $ 295.15 $ 237.59 1 0.21 $ 1122.38 10,000 0
Open in a new tab

Sensitivity analysis

The cost parameter that had the greatest impact on our model was the cost of the penicillin allergy test (Fig 2A). From the one-way sensitivity analysis we found that if the cost of the penicillin allergy test were $112 (base-case $256), then the additional cost to ensure appropriate care for all women who report a penicillin allergy was $441.89 per person. Alternately, if the cost of the penicillin allergy test were $400, then the additional cost to ensure appropriate care for all women who report a penicillin allergy was $1802.40 per person. Varying the costs of the antibiotics only increased or decreased the cost of test then treat compared to usual care by $40 at most.

Varying the probabilities of GBS positivity, clindamycin resistance, cesarean birth, and positive penicillin allergy tests did not substantially affect our model results (Fig 2B). When we varied these probabilities, the cost of test then treat compared to usual care increased or decreased by no more than $10.

The incremental cost-effectiveness scatterplot is shown in Fig 3. Each dot represents one simulated scenario while the dark triangle represents our base-case incremental cost-effectiveness ratio (described previously as the cost to ensure appropriate care for all women who report a penicillin allergy, under the test then treat strategy, compared to usual care). The confidence ellipse contains 95% of all simulated scenarios. The upper right quadrant (quadrant I) contains estimates that are potentially cost-effective; however, it should be noted that because we did not specify how much more we would be willing to pay to ensure appropriate antibiotic use, it is not possible to say if test then treat is cost-effective. The lower right quadrant (quadrant IV) contains estimates that are cost-saving, as these estimates represent situations when test then treat is not only less costly than usual care but also increased appropriate antibiotic use. Of the 10,000 simulated scenarios comparing test then treat to usual care, 99% resulted in potential cost-effectiveness (i.e., greater effectiveness at a higher cost) and 1% in cost-savings (i.e., a greater effectiveness at a lower cost). Our tracker variables showed that adopting a test than treat strategy in comparison to usual care would increase the number of appropriate antibiotic use from 7,843 to 10,000 and decrease the number of inappropriate antibiotic use from 2,157 to 0 (Table 2).

Fig 3. Incremental cost-effectiveness scatterplot.

Fig 3

Open in a new tab

Discussion

We evaluated the cost-effectiveness of routine penicillin allergy skin testing in pregnant women who report a penicillin allergy and found the mean additional cost of the test then treat strategy was $1122.38 per person reporting an allergy. This is a potentially cost-effective strategy and future research can explore this by incorporating long-term outcomes, including appropriate antibiotic therapy’s effect on quality of life for mother and baby. It would be less costly to test for a penicillin allergy only among women who are GBS positive, but for timing purposes allergy testing usually occurs prior to GBS testing in clinical practice. Still, our model suggests that systematic prenatal penicillin allergy testing could reduce or eliminate inappropriate antimicrobial therapy for GBS prophylaxis. A retrospective study of pregnant women with reported penicillin allergy found women who received a penicillin allergy skin test had increased use of first-line antibiotics for GBS prophylaxis with intrapartum penicillin (adjusted odds ratio 26.9; 95% CI 6.32 to 114) and with cefazolin for cesarean delivery prophylaxis (adjusted odds ratio 1.94, 95% CI 1.06 to 3.52) compared to women who did not undergo allergy testing [26]. In addition to eliminating the need to use alternatives to penicillin for GBS prophylaxis in pregnancy, penicillin allergy testing may provide long-term benefit for women’s future health care management, including during subsequent pregnancies.

The broader societal impact of de-labeling patients with penicillin allergy could potentially allow for future narrow-spectrum antibiotic selection and slow the growth of antibiotic-resistant bacteria [27]. There are significant public health implications associated with the unnecessary use of vancomycin including the emergence of vancomycin-resistant enterococci and other resistant organisms. In our model, we were able to ensure that 100% of women who had penicillin allergy testing received the appropriate antibiotics during labor and delivery, therefore limiting unnecessary use of alternative antibiotics. We did not account for other indications for antibiotic use during pregnancy, including urinary tract infections, which may benefit from use of a beta lactam.

Strengths of our study include the use of a robust decision tree and modeling based on actual costs of interventions and medications. The sensitivity analysis indicates large cost variation with only one component of the model, the cost of penicillin allergy testing, so the results are likely to hold true across settings. The model included patients undergoing vaginal birth, planned cesarean, and unplanned cesarean birth, thus accounting for every mode of delivery and multiple appropriate antibiotic regimens. Some people may encounter barriers to obtaining testing. The extra time and cost needed for an allergy testing visit may be a deterrent for some pregnant people [17]. In addition, rural patients may not have access to a clinic that offers allergy testing. Undergoing testing may also create anxiety in people who are fearful of an adverse event, and similarly some health systems may not be able to accommodate the increased volume of testing resulting from the ACOG recommendation.

As a limitation, our analysis may under-estimate the cost-effectiveness of the test then treat intervention because it does not include cost estimates for neonatal care or for maternal post-delivery care, and because we considered only the index pregnancy in our model. For neonatal cost considerations, guidance from the Centers for Disease Control and Prevention does not include alternatives to penicillin or cephalosporins as options for achieving adequate neonatal prophylaxis, so despite prophylaxis, infants of GBS positive mothers who received clindamycin or vancomycin are often subjected to further in-hospital testing and observation compared to those with penicillin treatment [28]. Because it will increase the number of neonates receiving adequate prophylaxis during delivery, penicillin allergy testing is likely to decrease the cost of immediate neonatal care. We also anticipate that post-cesarean wound infection frequency and the associated cost of that complication may be higher in those patients who do not receive the most appropriate antibiotics [29]. Results are reported as incremental cost of the intervention rather than in quality-adjusted life years (QALYs) because the model does not include neonatal lives saved or other costs beyond intrapartum antibiotic prophylaxis. Finally, because sensitivity and specificity of pencillin allergy testing have not been definitively determined, our model is limited by assuming 100% sensitivity and not accounting for the potential high cost of false negative testing followed by anaphylactic reaction.

Conclusion

ACOG states penicillin allergy testing is safe during pregnancy and recommends people with a reported penicillin allergy undergo testing, if available. The test then treat strategy is a good-value intervention to ensure appropriate antibiotic use for the current delivery. Additional work is needed to evaluate the cost benefit of testing for additional pregnancies and for future non-pregnancy healthcare of the mother.

Data Availability

Most of the data used in our model is available publicly and we have cited those sources accordingly. The only data not publicly available is the cost of antibiotics data which was pulled from patients admitted to Mayo Clinic in Rochester, MN. The cost data are stored on safe servers at Mayo Clinic, USA and handled confidentially. If access to this data is needed, application for access may be made to the Economic Evaluation Service at the Kern Center for the Science of Health Care Delivery (ees@mayo.edu). Upon approval, applicants would be able to access these data in the same manner as the authors.

Funding Statement

MG and RT received competitive funding from the Mayo Clinic Office of Translation to Practice. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

References

  • 1.Edwards JM, Watson N, Focht C, Wynn C, Todd CA, Walter EB, et al. Group B streptococcus (GBS) colonization and disease among pregnant women: a historical cohort study. Infectious diseases in obstetrics and gynecology. 2019;2019. doi: 10.1155/2019/5430493 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Shenoy ES, Macy E, Rowe T, Blumenthal KG. Evaluation and management of penicillin allergy: a review. Jama. 2019;321(2):188–99. doi: 10.1001/jama.2018.19283 [DOI] [PubMed] [Google Scholar]
  • 3.American Academy of Allergy A, American College of Allergy A, Joint Council of Allergy A, Parameters JTFoP. Drug allergy: an updated practice parameter. Annals of allergy, asthma & immunology: official publication of the American College of Allergy, Asthma, & Immunology. 2010;105(4):259–73. doi: 10.1016/j.anai.2010.08.002 [DOI] [PubMed] [Google Scholar]
  • 4.Penicillin Macy E. and beta-lactam allergy: epidemiology and diagnosis. Current allergy and asthma reports. 2014;14(11):476. [DOI] [PubMed] [Google Scholar]
  • 5.Torda A, Chan V. Antibiotic allergy labels—The impact of taking a clinical history. International journal of clinical practice. 2018;72(3):e13058. doi: 10.1111/ijcp.13058 [DOI] [PubMed] [Google Scholar]
  • 6.Knezevic B, Sprigg D, Seet J, Trevenen M, Trubiano J, Smith W, et al. The revolving door: antibiotic allergy labelling in a tertiary care centre. Internal medicine journal. 2016;46(11):1276–83. doi: 10.1111/imj.13223 [DOI] [PubMed] [Google Scholar]
  • 7.Macy E, Blumenthal KG. Are cephalosporins safe for use in penicillin allergy without prior allergy evaluation? The Journal of Allergy and Clinical Immunology: In Practice. 2018;6(1):82–9. [DOI] [PubMed] [Google Scholar]
  • 8.Puopolo KM, Lynfield R, Cummings JJ, Hand I, Adams-Chapman I, Poindexter B, et al. Management of infants at risk for group B streptococcal disease. Pediatrics. 2019;144(2). [DOI] [PubMed] [Google Scholar]
  • 9.Hansen SM, Uldbjerg N, Kilian M, Sørensen UBS. Dynamics of Streptococcus agalactiae colonization in women during and after pregnancy and in their infants. Journal of clinical microbiology. 2004;42(1):83–9. doi: 10.1128/JCM.42.1.83-89.2004 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Hoogkamp-Korstanje JA, Gerards L, Cats B. Maternal carriage and neonatal acquisition of group B streptococci. Journal of Infectious Diseases. 1982;145(6):800–3. doi: 10.1093/infdis/145.6.800 [DOI] [PubMed] [Google Scholar]
  • 11.Lewin EB, Amstey MS. Natural history of group B streptococcus colonization and its therapy during pregnancy. American journal of obstetrics and gynecology. 1981;139(5):512–5. doi: 10.1016/0002-9378(81)90509-3 [DOI] [PubMed] [Google Scholar]
  • 12.Verani JR, McGee L, Schrag SJ. Prevention of perinatal group B streptococcal disease: revised guidelines from CDC, 2010. 2010. [PubMed] [Google Scholar]
  • 13.American College of Obstetricians and Gynecologists. Committee on Obstetric Practice Prevention of Group B Streptococcal Early-Onset Disease in Newborns: ACOG Committee Opinion, Number 797. Obstet Gynecol. 2020;135:e51–e72. [DOI] [PubMed] [Google Scholar]
  • 14.Johnston C, Godecker A, Shirley D, Antony KM. Documented β-Lactam Allergy and Risk for Cesarean Surgical Site Infection. Infectious Diseases in Obstetrics and Gynecology. 2022;2022. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Sousa-Pinto B, Blumenthal KG, Macy E, Pereira AM, Azevedo LF, Delgado L, et al. Penicillin allergy testing is cost-saving: An economic evaluation study. Clinical Infectious Diseases. 2021;72(6):924–38. doi: 10.1093/cid/ciaa194 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Blumenthal KG, Li Y, Banerji A, Yun BJ, Long AA, Walensky RP. The cost of penicillin allergy evaluation. The Journal of Allergy and Clinical Immunology: In Practice. 2018;6(3):1019–27. e2. doi: 10.1016/j.jaip.2017.08.006 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Desravines N, Waldron J, Venkatesh KK, Kwan M, Boggess KA. Outpatient Penicillin Allergy Testing in Pregnant Women Who Report an Allergy. Obstetrics and Gynecology. 2021;137(1):56. doi: 10.1097/AOG.0000000000004213 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Kuder M, McDonnell J, Weller K, Wang X, Li M, Lang D. The Association Between Risk Stratification Models And The Likelihood of Positive Penicillin Skin Tests. Journal of Allergy and Clinical Immunology. 2021;147(2):AB15. [DOI] [PubMed] [Google Scholar]
  • 19.Macy E. Penicillin skin testing in pregnant women with a history of penicillin allergy and group B streptococcus colonization. Annals of Allergy, Asthma & Immunology. 2006;97(2):164–8. doi: 10.1016/S1081-1206(10)60007-5 [DOI] [PubMed] [Google Scholar]
  • 20.Use of prophylactic antibiotics in labor and delivery. ACOG Practice Bulletin No. 199. American College of Obstetricians and Gynecologists. Obstet Gynecol 2018;132:e103–19. [DOI] [PubMed] [Google Scholar]
  • 21.DiPersio LP, DiPersio JR. High rates of erythromycin and clindamycin resistance among OBGYN isolates of group B Streptococcus. Diagnostic microbiology and infectious disease. 2006;54(1):79–82. doi: 10.1016/j.diagmicrobio.2005.07.003 [DOI] [PubMed] [Google Scholar]
  • 22.Zhang J, Troendle J, Reddy UM, Laughon SK, Branch DW, Burkman R, et al. Contemporary cesarean delivery practice in the United States. American journal of obstetrics and gynecology. 2010;203(4):326. e1-. e10. doi: 10.1016/j.ajog.2010.06.058 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Visscher SL, Naessens JM, Yawn BP, Reinalda MS, Anderson SS, Borah BJ. Developing a standardized healthcare cost data warehouse. BMC health services research. 2017;17(1):1–11. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.2020 U.S. Bureau of Labor Statistics. Consumer Price Index 2021 [Available from: https://data.bls.gov/cgi-bin/surveymost?cu.
  • 25.The Centers for Medicare and Medicaid Services. Physician Fee Schedule 2018 [Available from: https://www.cms.gov/apps/physician-fee-schedule/license-agreement.aspx.
  • 26.Wolfson AR, Mancini CM, Banerji A, Fu X, Bryant AS, Phadke NA, et al. Penicillin allergy assessment in pregnancy: safety and impact on antibiotic use. The Journal of Allergy and Clinical Immunology: In Practice. 2021;9(3):1338–46. doi: 10.1016/j.jaip.2020.10.063 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Mattingly TJ, Heil EL. The Economics of Penicillin Allergy Testing: Still Scratching the Value Surface. Oxford University Press US; 2021. p. 939–41. [DOI] [PubMed] [Google Scholar]
  • 28.Horsley E. CDC Updates Guidelines for the Prevention of Perinatal GBS Disease. American Family Physician. 2011;83(9):1106. [Google Scholar]
  • 29.Williams M, Zantow E, Turrentine M. Cost effectiveness of latest recommendations for group b streptococci screening in the United States. Obstetrics & Gynecology. 2020;135(4):789–98. [DOI] [PubMed] [Google Scholar]

Decision Letter 0

Tefera Chane Mekonnen

Transfer Alert

This paper was transferred from another journal. As a result, its full editorial history (including decision letters, peer reviews and author responses) may not be present.

6 Sep 2022

PONE-D-22-17964EVALUATING THE COST-EFFECTIVENESS OF TESTING PREGNANT WOMEN FOR PENICILLIN ALLERGYPLOS ONE

Dear Dr. Theiler,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

  • The authors should minimize the length of the manuscript, in particular the introduction section and revise as per the journal guideline.

  • The authors also must provide a clear justification about the overestimation of cost-effectiveness of the intervention, as it more deviated from real scenario.  They also critically elaborate  the target population to which the finding is inferred and assumptions used to evaluate their objective.

  • Which guideline was applied to design the research protocol.

  • They have to address  critical comments raised by the reviewers, specially the value addition of the research for clinical practice/inputs and alternative medicines that may affect the current estimated figure.  

Please submit your revised manuscript by Oct 21 2022 11:59PM.  If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

  • A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

  • A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

  • An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Tefera Chane Mekonnen, Master in Public Health(MPH)

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Please include your full ethics statement in the ‘Methods’ section of your manuscript file. In your statement, please include the full name of the IRB or ethics committee who approved or waived your study, as well as whether or not you obtained informed written or verbal consent. If consent was waived for your study, please include this information in your statement as well.

3. Thank you for stating the following in the Acknowledgments Section of your manuscript:

“This work was supported by the Advance the Practice Research Award from the Mayo Clinic Office of Translation to Practice. The content is solely the responsibility of the authors and does not necessarily represent the official views of Mayo Clinic.”

We note that you have provided additional information within the Acknowledgements Section that is not currently declared in your Funding Statement. Please note that funding information should not appear in the Acknowledgments section or other areas of your manuscript. We will only publish funding information present in the Funding Statement section of the online submission form.

Please remove any funding-related text from the manuscript and let us know how you would like to update your Funding Statement. Currently, your Funding Statement reads as follows:

“MG and RT received competitive funding from the Mayo Clinic Office of Translation to Practice. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.”

Please include your amended statements within your cover letter; we will change the online submission form on your behalf.

4. Thank you for stating the following in the Competing Interests section:

“Dr. Theiler has a know-how agreement and research funding from HeraMed and serves on the Medical Advisory Board for Delfina Care. Dr. Borah is a consultant to Exact Sciences and Boehringer-Ingelheim on unrelated (non-Ob/Gyn) projects.”

Please confirm that this does not alter your adherence to all PLOS ONE policies on sharing data and materials, by including the following statement: "This does not alter our adherence to  PLOS ONE policies on sharing data and materials.” (as detailed online in our guide for authors http://journals.plos.org/plosone/s/competing-interests).  If there are restrictions on sharing of data and/or materials, please state these. Please note that we cannot proceed with consideration of your article until this information has been declared.

Please include your updated Competing Interests statement in your cover letter; we will change the online submission form on your behalf.

5. In your Data Availability statement, you have not specified where the minimal data set underlying the results described in your manuscript can be found. PLOS defines a study's minimal data set as the underlying data used to reach the conclusions drawn in the manuscript and any additional data required to replicate the reported study findings in their entirety. All PLOS journals require that the minimal data set be made fully available. For more information about our data policy, please see http://journals.plos.org/plosone/s/data-availability.

Upon re-submitting your revised manuscript, please upload your study’s minimal underlying data set as either Supporting Information files or to a stable, public repository and include the relevant URLs, DOIs, or accession numbers within your revised cover letter. For a list of acceptable repositories, please see http://journals.plos.org/plosone/s/data-availability#loc-recommended-repositories. Any potentially identifying patient information must be fully anonymized.

Important: If there are ethical or legal restrictions to sharing your data publicly, please explain these restrictions in detail. Please see our guidelines for more information on what we consider unacceptable restrictions to publicly sharing data: http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions. Note that it is not acceptable for the authors to be the sole named individuals responsible for ensuring data access.

We will update your Data Availability statement to reflect the information you provide in your cover letter.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: No

Reviewer #2: No

Reviewer #3: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: No

Reviewer #2: N/A

Reviewer #3: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: No

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: The authors present a cost-effectiveness analysis of performing penicillin allergy testing in pregnancy on women that ultimately test positive for GBS.

Introduction

1. The authors need to reduce this section by half. Focus more on the effect of penicillin allergy testing in pregnancy. For example, delete lines 83 to 98.

2. Line 98 to 100 – The objective should be clarified to indicate in pregnant individuals that ultimately test positive for GBS colonization.

Methods

3. Did the authors follow the CHEERS guidelines (Husereau et al. CHEERS 2022 ISPOR Good Research Practices Task Force. Consolidated Health Economic Evaluation Reporting Standards 2022 (CHEERS 2022) Statement: Updated Reporting Guidance for Health Economic Evaluations) for this CEA? If so, please state this.

4. Lines 103 to 105 – It was not clear until you read the Discussion that this CEA was excluding outcomes for neonatal care and/or maternal post-cesarean surgical site infections. This exclusion should be stated up front.

5. Lines 107 to 108 – The authors should better define the population that will be tested. A suggestion would be to use the National Vital Statistics Reports for Births: Final data for 2020. You could then take the 3,613,647 births in 2020 and multiply this number by the estimated rate of individuals who report a penicillin allergy during pregnancy. This would be the cohort to then perform cost estimates on. Further a more robust estimate of the rate of cesarean delivery could be estimated. [Osterman et al. Births: Final data for 2020. National Vital Statistics Reports; vol 70 no 17. Hyattsville, MD: National Center for Health Statistics. 2022].

6. Line 110, Figure 1 - The decision tree is lacking an important outcome option. When a woman reports a penicillin allergy, the recommended antibiotic for intrapartum antibiotic prophylaxis, if she is colonized with GBS, is based on her risk of a severe reaction and the susceptibility of the GBS isolate to clindamycin. For women whose reported penicillin allergy indicates a low risk of anaphylaxis or uncertain severity, a first-generation cephalosporin (ie, cefazolin) is recommended. Many institutions do not perform sensitivity to GBS isolates in this clinical situation and reserve sensitivity testing to clindamycin for individuals that report a high risk of anaphylaxis. In addition, there are estimates available in the medical literature that show the rate of reported low versus high risk of anaphylaxis to penicillin in pregnancy. This treatment option needs to be added to the algorithm.

7. Lines 129 to 131 – A more robust range of GBS maternal colonization in pregnancy in the US (22% to 28%) is available (Russell et al. Maternal colonization with Group B Streptococcus and serotype distribution worldwide: systematic review and meta-analysis. Clin Infect Dis 2017; 65 (Suppl 2): S100-S11).

8. Lines 137 to 150 – The costs associated with the antibiotics are for how many doses? For example, 56% of pregnant individuals will receive two or more doses of intrapartum penicillin for maternal GBS prophylaxis. In addition, the recommended dose and administration interval of vancomycin for maternal GBS prophylaxis changed in 2019. Does this price estimate reflect that?

9. Lines 151 to 153 – The authors cite the Blumenthal article from 2018 (reference 14) that reported a cost for a one-step penicillin allergy testing. Yet the Desravines article from 2021 recommends a 3 step testing (skin prick, intradermal testing, oral challenge) [reference 20]. Should this increased costs have been estimated in the model?

Reviewer #2: General comments:

It has been noted that the greatest reductions in healthcare costs associated with penicillin allergy delabeling are due to less urgent care and outpatient utilization and shorter durations of hospitalizations. It does not appear that these potential effects were incorporated into your modeling.

https://pubmed.ncbi.nlm.nih.gov/28366717/

Please comment on how removing all warnings not to use cephalosporins in the setting of a penicillin allergy would affect your modeling.

https://pubmed.ncbi.nlm.nih.gov/33914051/

The reported rate of unconfirmed penicillin allergy is about 8.0% in pregnant women and up to about 7% are confirmed if all are skin tested prior to an oral amoxicillin challenge. Thus, the true rate of penicillin allergy in pregnant women is only about 0.5%. Women with a penicillin allergy, with or without GBS, also had significantly (about 10%) higher cesarean section rates and spent significantly more (about 0.1) days in the hospital after delivery. Please consider incorporating these data into your modeling.

https://pubmed.ncbi.nlm.nih.gov/33278285/

https://pubmed.ncbi.nlm.nih.gov/28333608/

Specific comments:

Line 153: Please consider commenting here that the estimated cost of penicillin allergy testing used in this analysis is probably higher than the real world, because many low-risk penicillin “allergic” individuals are currently delabeled by a direct oral amoxicillin challenge without antecedent skin testing.

Reviewer #3: This is a straightforward economic evaluation of the cost of introducing testing for penicillin allergy amongst pregnant women and people who believe or have been labelled as being allergic. This is within the context of universal antenatal screening for GBS and use of penicillin intrapartum antibiotic prophylaxis for those testing positive for GBS colonisation.

I could not find the data in Blumthenthal 2018 that suggests those with penicillin allergies are a higher (50%) the risk of surgical site infections. Could this have been the wrong reference?

The model inputs and costs seem appropriately obtained/ derived and used in analysis. I am not familiar with the way in which US healthcare costs are warehoused and how Medicare/Medicaid tariffs are used. The authors state that this information cannot be made publicly available.

Figure 1 model – either strategy, GBS+, resistant to clindamycin: need to explain why for planned/ unplanned C-section the path include clindamycin (alongside gentamycin)? Presumably for maternal prophylaxis for endometritis and not for prevention of early onset neonatal GBS infection.

The outcome is incremental cost per additional women receiving appropriate antibiotics. This is appropriate for short-term modelling – ultimately it the impact on neonatal GBS infection that is most important.

Is it appropriate to assume that the penicillin allergy test is 100% accurate? I appreciate adding in false positive/ negatives expands the model, but there could be substantial costs associated with anaphylaxis from a false negative result.

The absence of a willingness to pay threshold set by policy makers hinders interpretation, as does a utility such as a QALY that can be used to compare against other strategies.

The results pertain only to those women who report or believe they have a penicillin allergy based on past experience. There will be women who have never been exposed to penicillin who will be offered this antibiotic if GBS positive. Could the model be expanded to include women of unknown penicillin allergy status, and also both models restricted to testing only the positive for GBS colonisation. I appreciate that this would need to occur a few days after the GBS test, will probably involve another appointment and may not occur before birth, so have no impact on care. The timepoint at which the allergy test would occur during pregnancy is not stated, and could itself mean another appointment. The authors discuss the societal cost and limitations that an allergy test might bring.

I am not sure of the policy of PLOSOne but generally in the OBGYN literature, we refer to pregnant women or pregnant people, not pregnant patients.

Overall, an interesting and valuable piece of research, albeit limited in generalisability.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: Yes: Eric Macy MD MS FAAAAI

Reviewer #3: Yes: Prof Jane Daniels

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

PLoS One. 2023 Jan 20;18(1):e0280151. doi: 10.1371/journal.pone.0280151.r002

Author response to Decision Letter 0

12 Oct 2022

General comments:

The authors should minimize the length of the manuscript, in particular the introduction section and revise as per the journal guideline.

Thank you for this suggestion. Per reviewer 1’s suggestions, we have omitted lines 90-104 (originally lines 83 to 98).

The authors also must provide a clear justification about the overestimation of cost-effectiveness of the intervention, as it more deviated from real scenario. They also critically elaborate the target population to which the finding is inferred and assumptions used to evaluate their objective.

The work estimates the cost per penicillin allergic patient at the onset of care, which in an ideal scenario approximates the cost per appropriate dose of antibiotic because of the small number of true allergies in this population. Given that uptake of testing will not be 100%, our numbers represent the ideal state and the actual cost in practice will be lower because of the difference in uptake of testing. Additionally, we have modeled the more conservative protocol of skin testing followed by oral challenge, and oral challenge alone would be less costly. We have added a sentence (lines 176-179) reflecting this difference in testing protocols.

Which guideline was applied to design the research protocol.

We followed the guidelines published by the 2nd Panel on Cost-Effectiveness in Health and Medicine (PMID: 27623463 DOI: 10.1001/jama.2016.12195).

They have to address critical comments raised by the reviewers, specially the value addition of the research for clinical practice/inputs and alternative medicines that may affect the current estimated figure.

We have responded below to the reviewer comments.

Editor comments:

Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf

and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

Thank you for these templates, we have updated our manuscript to abide by PLOS One’s style requirements. All changes are tracked in our updated manuscript.

Please include your full ethics statement in the ‘Methods’ section of your manuscript file. In your statement, please include the full name of the IRB or ethics committee who approved or waived your study, as well as whether or not you obtained informed written or verbal consent. If consent was waived for your study, please include this information in your statement as well.

Our study was exempt from IRB review and we have updated our Methods section to reflect this in the following way: In this institutional review board exempt study, we used billing data from a tertiary care center and previously published literature, (lines 115 to 116) in our track changes manuscript.

Thank you for stating the following in the Acknowledgments Section of your manuscript:

“This work was supported by the Advance the Practice Research Award from the Mayo Clinic Office of Translation to Practice. The content is solely the responsibility of the authors and does not necessarily represent the official views of Mayo Clinic.”

We note that you have provided additional information within the Acknowledgements Section that is not currently declared in your Funding Statement. Please note that funding information should not appear in the Acknowledgments section or other areas of your manuscript. We will only publish funding information present in the Funding Statement section of the online submission form.

Please remove any funding-related text from the manuscript and let us know how you would like to update your Funding Statement. Currently, your Funding Statement reads as follows:

“MG and RT received competitive funding from the Mayo Clinic Office of Translation to Practice. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.”

Please include your amended statements within your cover letter; we will change the online submission form on your behalf.

We have removed our funding statement from our manuscript. Please update our funding statement to the following:

MG and RT received competitive funding from the Mayo Clinic Office of Translation to Practice via the Advance the Practice Research Award. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Thank you for stating the following in the Competing Interests section:

“Dr. Theiler has a know-how agreement and research funding from HeraMed and serves on the Medical Advisory Board for Delfina Care. Dr. Borah is a consultant to Exact Sciences and Boehringer-Ingelheim on unrelated (non-Ob/Gyn) projects.”

Please confirm that this does not alter your adherence to all PLOS ONE policies on sharing data and materials, by including the following statement: "This does not alter our adherence to PLOS ONE policies on sharing data and materials.” (as detailed online in our guide for authors http://journals.plos.org/plosone/s/competing-interests). If there are restrictions on sharing of data and/or materials, please state these. Please note that we cannot proceed with consideration of your article until this information has been declared.

Please include your updated Competing Interests statement in your cover letter; we will change the online submission form on your behalf.

We have added the requested financial disclosure statement and disclosed our competing interests in the revised cover letter.

In your Data Availability statement, you have not specified where the minimal data set underlying the results described in your manuscript can be found. PLOS defines a study's minimal data set as the underlying data used to reach the conclusions drawn in the manuscript and any additional data required to replicate the reported study findings in their entirety. All PLOS journals require that the minimal data set be made fully available. For more information about our data policy, please see http://journals.plos.org/plosone/s/data-availability.

Upon re-submitting your revised manuscript, please upload your study’s minimal underlying data set as either Supporting Information files or to a stable, public repository and include the relevant URLs, DOIs, or accession numbers within your revised cover letter. For a list of acceptable repositories, please see http://journals.plos.org/plosone/s/data-availability#loc-recommended-repositories. Any potentially identifying patient information must be fully anonymized. Important: If there are ethical or legal restrictions to sharing your data publicly, please explain these restrictions in detail. Please see our guidelines for more information on what we consider unacceptable restrictions to publicly sharing data:

Thank you. Our decision tree model can be easily replicated by using the figures we listed in Table 1. Most of the data used in our model is available publically and we have cited those sources accordingly. The only data not publically available is the cost of antibiotics data which was pulled from patients admitted to Mayo Clinic in Rochester, MN. The cost data are stored on safe servers at Mayo Clinic, USA and handled confidentially. If access to this data is needed, application for access may be made to the Economic Evaluation Service at the Kern Center for the Science of Health Care Delivery (ees@mayo.edu). Upon approval, applicants would be able to access these data in the same manner as the authors.

http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions. Note that it is not acceptable for the authors to be the sole named individuals responsible for ensuring data access.

We will update your Data Availability statement to reflect the information you provide in your cover letter.

The cost of antibiotics data used to estimate the means and standard deviations in Table 1 is stored by the research group of authors on safe servers at Mayo Clinic. Please refer to our statement about data sharing. All other data has been published and is publically available.

Reviewer 1 comments:

Introduction

The authors need to reduce this section by half. Focus more on the effect of penicillin allergy testing in pregnancy. For example, delete lines 83 to 98.

We appreciate this suggestion and have deleted lines 83 to 98.

Line 98 to 100 – The objective should be clarified to indicate in pregnant individuals that ultimately test positive for GBS colonization.

Thank you. Line 112 has been updated to reflect that we modeled testing of ALL pregnant patients reporting penicillin allergy. Because of the cadence of prenatal care, with GBS testing at the end of the pregnancy, our decision tree reflects testing regardless of subsequent GBS culture results.

Methods

Did the authors follow the CHEERS guidelines (Husereau et al. CHEERS 2022 ISPOR Good Research Practices Task Force. Consolidated Health Economic Evaluation Reporting Standards 2022 (CHEERS 2022) Statement: Updated Reporting Guidance for Health Economic Evaluations) for this CEA? If so, please state this.

Yes, we did follow the CHEERS guideline for reporting and we have added the following statement to our methods section (lines 114 to115): We report our methods using the Consolidated Health Economic Evaluation Reporting Standards 2022 (CHEERS 2022) guidelines.

Lines 103 to 105 – It was not clear until you read the Discussion that this CEA was excluding outcomes for neonatal care and/or maternal post-cesarean surgical site infections. This exclusion should be stated up front.

Thank you for this recommendation. We have added the following to our methods section: “Because of lack of published differential effectiveness outcomes for penicillin vs. alternative antibiotics, our model does not include neonatal or maternal post-cesarean surgical site infection outcomes.” (lines 127-129). We have also acknowledged this limitation in the discussion section: “As a limitation, our analysis may under-estimate the cost-effectiveness of the test then treat intervention because it does not include cost estimates for neonatal care or for maternal post-delivery care, and because we considered only the index pregnancy in our model.”

Lines 107 to 108 – The authors should better define the population that will be tested. A suggestion would be to use the National Vital Statistics Reports for Births: Final data for 2020. You could then take the 3,613,647 births in 2020 and multiply this number by the estimated rate of individuals who report a penicillin allergy during pregnancy. This would be the cohort to then perform cost estimates on. Further a more robust estimate of the rate of cesarean delivery could be estimated. [Osterman et al. Births: Final data for 2020. National Vital Statistics Reports; vol 70 no 17. Hyattsville, MD: National Center for Health Statistics. 2022].

Thank you for this suggestion. The cesarean rate reported by Osterman et al. is the overall cesarean rate; however, for our paper we need the planned and unplanned cesarean rate. For this reason, we have opted to use the rates reported by Zhang et al. Furthermore, there is only a slight difference in total cesarean rates between the Osterman article (31.8%) and the Zhang article (30.5%), and this difference has been captured in our sensitivity analysis.

Line 110, Figure 1 – The decision tree is lacking an important outcome option. When a woman reports a penicillin allergy, the recommended antibiotic for intrapartum antibiotic prophylaxis, if she is colonized with GBS, is based on her risk of a severe reaction and the susceptibility of the GBS isolate to clindamycin. For women whose reported penicillin allergy indicates a low risk of anaphylaxis or uncertain severity, a first-generation cephalosporin (ie, cefazolin) is recommended. Many institutions do not perform sensitivity to GBS isolates in this clinical situation and reserve sensitivity testing to clindamycin for individuals that report a high risk of anaphylaxis. In addition, there are estimates available in the medical literature that show the rate of reported low versus high risk of anaphylaxis to penicillin in pregnancy. This treatment option needs to be added to the algorithm.

Thank you for pointing this out as an alternative. For our decision tree we have used the ACOG-recommended algorithm for GBS sensitivity testing and subsequent treatment.

Lines 129 to 131 – A more robust range of GBS maternal colonization in pregnancy in the US (22% to 28%) is available (. Clin Infect Dis 2017; 65 (Suppl 2): S100-S11).

Thank you for this reference. The above article actually cites an 18% estimate of maternal colonization with GBS. On our review of the literature with an emphasis on U.S. costs, we felt the estimate of 21.6% was most consistent with other U.S. publications and have chosen to base the analysis on that rate as cited. Ultimately, the prevalence of GBS had little impact on our cost outcome because of the sequence of care (GBS testing and sensitivity performed after allergy testing) and we do not think a small change in prevalence will meaningfully impact our primary outcome.

Lines 137 to 150 – The costs associated with the antibiotics are for how many doses? For example, 56% of pregnant individuals will receive two or more doses of intrapartum penicillin for maternal GBS prophylaxis. In addition, the recommended dose and administration interval of vancomycin for maternal GBS prophylaxis changed in 2019. Does this price estimate reflect that?

Thank you for this question. In our methods section we state that there was substantial variability in cost of antibiotics due to differences among labor and delivery times, which in turn affected number of doses of antibiotics needed. Thus, we report the average cost of each antibiotic in our model. The variation in cost and dosing thus reflects the variation in human labor duration and we do not specify a number of doses. The changing of dosage guidelines is reflected in our sensitivity analysis.

Lines 151 to 153 – The authors cite the Blumenthal article from 2018 (reference 14) that reported a cost for a one-step penicillin allergy testing. Yet the Desravines article from 2021 recommends a 3 step testing (skin prick, intradermal testing, oral challenge) [reference 20]. Should this increased costs have been estimated in the model?

Thank you. The Blumenthal article reports on “penicillin skin testing” followed by 1-step oral amoxicillin challenge, which they describe as comprehensive. We assumed this to include skin prick followed by intradermal challenge, which is the standard of care. The “one step” refers to the oral amoxicillin challenge component of testing, and we believe their analysis to be the most comprehensive estimate of actual cost for testing.

Reviewer 2 comments:

It has been noted that the greatest reductions in healthcare costs associated with penicillin allergy delabeling are due to less urgent care and outpatient utilization and shorter durations of hospitalizations. It does not appear that these potential effects were incorporated into your modeling. https://pubmed.ncbi.nlm.nih.gov/28366717/

Thank you for this question. Our model only pertains to women giving birth in an inpatient hospital setting, therefore, patients from urgent care and outpatient settings would not have been included in our model. We do believe considerable cost savings may be achieved over the subsequent life years of these patients and have commented on that in the discussion.

Please comment on how removing all warnings not to use cephalosporins in the setting of a penicillin allergy would affect your modeling. https://pubmed.ncbi.nlm.nih.gov/33914051/

We have incorporated appropriate cephalosporin use according to ACOG clinical guidelines into our decision tree, and previous publications have done the same—including that referenced above as well as our own (https://pubmed.ncbi.nlm.nih.gov/35906008/). The reference cited above does not include a recommendation to remove warnings about cephalosporin use. Removal of all recommendations to avoid cephalosporins in penicillin allergic individuals would likely negate the need for nearly all penicillin allergy testing if we consider cefazolin to be the appropriate substitute in such patients, but we are not aware of such a recommendation for practice change so did not include this discussion in the in the manuscript. We have included a citation of the ACOG practice bulletin with recommendations for prophylactic antibiotics, citation 24 (Use of prophylactic antibiotics in labor and delivery. ACOG Practice Bulletin No. 199. American College of Obstetricians and Gynecologists. Obstet Gynecol 2018;132:e103–19.)

The reported rate of unconfirmed penicillin allergy is about 8.0% in pregnant women and up to about 7% are confirmed if all are skin tested prior to an oral amoxicillin challenge. Thus, the true rate of penicillin allergy in pregnant women is only about 0.5%. Women with a penicillin allergy, with or without GBS, also had significantly (about 10%) higher cesarean section rates and spent significantly more (about 0.1) days in the hospital after delivery. Please consider incorporating these data into your modeling. https://pubmed.ncbi.nlm.nih.gov/33278285/

https://pubmed.ncbi.nlm.nih.gov/28333608/

We agree that the literature suggests higher cesarean rates, and thus longer duration of inpatient stay, in patients with reported PCN allergy. Our decision tree model did not include this differential cesarean rate because we do not know whether patients with confirmed vs. reported allergies will differ in their rates of unplanned cesarean birth, so we felt the most conservative approach was to use overall population data. The above study cites patients with “unverified penicillin allergy”. Larger prospective studies of pregnancy outcomes after incorporation of penicillin allergy testing protocols will be needed prior to using the above data in modeling.

Line 153: Please consider commenting here that the estimated cost of penicillin allergy testing used in this analysis is probably higher than the real world, because many low-risk penicillin “allergic” individuals are currently delabeled by a direct oral amoxicillin challenge without antecedent skin testing.

Thank you for this suggestion. We have included this exact statement in our methods section (lines 175 to178).

Reviewer 3 comments:

This is a straightforward economic evaluation of the cost of introducing testing for penicillin allergy amongst pregnant women and people who believe or have been labelled as being allergic. This is within the context of universal antenatal screening for GBS and use of penicillin intrapartum antibiotic prophylaxis for those testing positive for GBS colonisation.

I could not find the data in Blumthenthal 2018 that suggests those with penicillin allergies are a higher (50%) the risk of surgical site infections. Could this have been the wrong reference?

Thank you for catching the citation error. We have added the correct citation: https://pubmed.ncbi.nlm.nih.gov/35281850/

The model inputs and costs seem appropriately obtained/ derived and used in analysis. I am not familiar with the way in which US healthcare costs are warehoused and how Medicare/Medicaid tariffs are used. The authors state that this information cannot be made publicly available.

The cost of antibiotics data used to estimate the means and standard deviations in Table 1 is stored by the research group of authors on safe servers at Mayo Clinic. Please refer to our statement about data sharing. All other data has been published and is publically available.

Figure 1 model – either strategy, GBS+, resistant to clindamycin: need to explain why for planned/ unplanned C-section the path include clindamycin (alongside gentamycin)? Presumably for maternal prophylaxis for endometritis and not for prevention of early onset neonatal GBS infection.

The model includes clindamycin for cesarean surgical site infection prophylaxis in accordance with ACOG guidelines, reference number 24. We have added this reference (Use of prophylactic antibiotics in labor and delivery. ACOG Practice Bulletin No. 199. American College of Obstetricians and Gynecologists. Obstet Gynecol 2018;132:e103–19.)

The outcome is incremental cost per additional women receiving appropriate antibiotics. This is appropriate for short-term modelling – ultimately it the impact on neonatal GBS infection that is most important.

We agree and have stated this as a limitation of the study. When data become available detailing the relative effectiveness of non-penicillin antibiotics for GBS prophlyaxis, we recommend updating the analysis to incorporate long-term neonatal outcomes.

Is it appropriate to assume that the penicillin allergy test is 100% accurate? I appreciate adding in false positive/ negatives expands the model, but there could be substantial costs associated with anaphylaxis from a false negative result.

We agree that false negative testing in particular may add cost because of rare anaphylactic reactions. However, the exact sensitivity and specificity of penicillin allergy testing have not been well described in the literature to date so were not incorporated into our model. Our model thus assumes 100% sensitivity and may underestimate cost. We have added the following to the discussion, lines 308-311: Finally, because sensitivity and specificity of pencillin allergy testing have not been definitively determined, our model is limited by assuming 100% sensitivity and not accounting for the potential high cost of false negative testing followed by anaphylactic reaction.

The absence of a willingness to pay threshold set by policy makers hinders interpretation, as does a utility such as a QALY that can be used to compare against other strategies.

We agree that this is a limitation of the study as noted in the discussion section. We do believe that this addition to the literature, in combination with other studies, may help to develop the above thresholds in the future.

The results pertain only to those women who report or believe they have a penicillin allergy based on past experience. There will be women who have never been exposed to penicillin who will be offered this antibiotic if GBS positive. Could the model be expanded to include women of unknown penicillin allergy status, and also both models restricted to testing only the positive for GBS colonisation. I appreciate that this would need to occur a few days after the GBS test, will probably involve another appointment and may not occur before birth, so have no impact on care. The timepoint at which the allergy test would occur during pregnancy is not stated, and could itself mean another appointment. The authors discuss the societal cost and limitations that an allergy test might bring.

Thank you for this suggestion. We have discussed the timing of testing prior to GBS culture and that impact on cost, lines 261-263. Because of the rarity of penicillin anaphylaxis in patients without previously described allergy, we did not include that potential cost in the model.

I am not sure of the policy of PLOSOne but generally in the OBGYN literature, we refer to pregnant women or pregnant people, not pregnant patients.

Thank you for this suggestion. We have updated the language throughout our manuscript to remove pregnant ‘patients’.

Attachment

Submitted filename: Response to Reviewers.docx

Click here for additional data file. (29.9KB, docx)

Decision Letter 1

Tefera Chane Mekonnen

12 Dec 2022

PONE-D-22-17964R1EVALUATING THE COST-EFFECTIVENESS OF TESTING PREGNANT WOMEN FOR PENICILLIN ALLERGYPLOS ONE

Dear Dr. Theiler,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

  • Address comments raised by reviewers.

Please submit your revised manuscript by 26/12/22. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

  • A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

  • A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

  • An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Tefera Chane Mekonnen, Master in Public Health(MPH)

Academic Editor

PLOS ONE

Journal Requirements:

Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #2: (No Response)

Reviewer #3: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #2: Yes

Reviewer #3: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #2: Yes

Reviewer #3: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #2: Yes

Reviewer #3: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #2: Yes

Reviewer #3: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #2: General Comments:

Please comment on how removing all warnings not to use cephalosporins in the setting of a penicillin allergy would affect your modeling as had been done at Kaiser Permanente. Please note that removing the warning still did not significantly reduce the increase morbidity associated with an unconfirmed penicillin allergy. It did result in the increased use of cephalosporins with no increased risk of antibiotic-associated hypersensitivity reactions.

https://pubmed.ncbi.nlm.nih.gov/33914051/

Specific Comments:

Lines 19 to 20 and Lines 47 to 49: Please make these lines agree with each other. Consider using the following in both places: “Less than five percent of patients who report a penicillin allergy will have a currently active clinically-significant IgE- or T-cell-mediated hypersensitivity

when appropriately tested.”

Lines 32 to 34: Please consider changing to “…person. Our modeling demonstrated that a test then treat strategy increased the probability of appropriate antibiotic use from 7,843/10,000 to 10,000/10,000 simulations.

Lines 38 to 41: Please consider changing to “Penicillin allergy is the most commonly reported allergy in pregnant women. The reported rate of unconfirmed penicillin allergy of about 8.0% in pregnant women is lower than commonly reported rates in outpatients using healthcare or other hospitalized populations.

Reviewer #3: (No Response)

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #2: Yes: Eric Macy

Reviewer #3: No

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

PLoS One. 2023 Jan 20;18(1):e0280151. doi: 10.1371/journal.pone.0280151.r004

Author response to Decision Letter 1

14 Dec 2022

Reviewer #2: General Comments:

Please comment on how removing all warnings not to use cephalosporins in the setting of a penicillin allergy would affect your modeling as had been done at Kaiser Permanente. Please note that removing the warning still did not significantly reduce the increase morbidity associated with an unconfirmed penicillin allergy. It did result in the increased use of cephalosporins with no increased risk of antibiotic-associated hypersensitivity reactions.

https://pubmed.ncbi.nlm.nih.gov/33914051/

Thank you for bringing this publication to our attention. While we find it to be interesting, we do feel that we adequately accounted for the issue of over-vigilance regarding cephalosporin cross-reactivity in our model and in the text of the manuscript. In accordance with ACOG recommendations, at baseline our decision tree uses only anaphylaxis to penicillins or documented allergy to cephalosporins as a reason not to use cephalosporins in the non-intervention arm of the analysis. Given that, implementing the above warning would not change the outcome of our cost analysis. Additionally, our health system has no warnings in place regarding use of cephalosporins in penicillin allergic patients, so we do not feel the intervention in the above manuscript is broadly applicable regarding GBS prophylaxis. Finally, the removal of such warnings did not affect morbidity in the referenced manuscript and is also unlikely to affect cost in the framework of our decision analysis. For these reasons and in the interest of manuscript lenth we have declined to add the above discussion to this manuscript.

Specific Comments:

Lines 19 to 20 and Lines 47 to 49: Please make these lines agree with each other. Consider using the following in both places: “Less than five percent of patients who report a penicillin allergy will have a currently active clinically-significant IgE- or T-cell-mediated hypersensitivity

when appropriately tested.”

Thank you for that suggestion. Lines 19-20 have been changed to the language above, which matches that in lines 47-49.

Lines 32 to 34: Please consider changing to “…person. Our modeling demonstrated that a test then treat strategy increased the probability of appropriate antibiotic use from 7,843/10,000 to 10,000/10,000 simulations.

This language has been changes as suggested.

Lines 38 to 41: Please consider changing to “Penicillin allergy is the most commonly reported allergy in pregnant women. The reported rate of unconfirmed penicillin allergy of about 8.0% in pregnant women is lower than commonly reported rates in outpatients using healthcare or other hospitalized populations.

This sentence has been changed to use the above language.

Decision Letter 2

Tefera Chane Mekonnen

21 Dec 2022

EVALUATING THE COST-EFFECTIVENESS OF TESTING PREGNANT WOMEN FOR PENICILLIN ALLERGY

PONE-D-22-17964R2

Dear Dr. Theiler,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Tefera Chane Mekonnen, Master in Public Health(MPH)

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Acceptance letter

Tefera Chane Mekonnen

3 Jan 2023

PONE-D-22-17964R2

Evaluating the cost-effectiveness of testing pregnant women for penicillin allergy

Dear Dr. Theiler:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Tefera Chane Mekonnen

Academic Editor

PLOS ONE

Associated Data

    This section collects any data citations, data availability statements, or supplementary materials included in this article.

    Supplementary Materials

    Attachment

    Submitted filename: Response to Reviewers.docx

    Click here for additional data file. (29.9KB, docx)

    Data Availability Statement

    Most of the data used in our model is available publicly and we have cited those sources accordingly. The only data not publicly available is the cost of antibiotics data which was pulled from patients admitted to Mayo Clinic in Rochester, MN. The cost data are stored on safe servers at Mayo Clinic, USA and handled confidentially. If access to this data is needed, application for access may be made to the Economic Evaluation Service at the Kern Center for the Science of Health Care Delivery (ees@mayo.edu). Upon approval, applicants would be able to access these data in the same manner as the authors.

    Articles from PLOS ONE are provided here courtesy of PLOS

    RESOURCES