Abstract
BACKGROUND AND OBJECTIVE:
Inadequate follow-up of positive sexually transmitted infection (STI) test results is a gap in health care quality that contributes to the epidemic of STIs in adolescent women. The goal of this study was to improve our ability to contact adolescent women with positive STI test results after an emergency department visit.
METHODS:
We conducted an interventional quality improvement project at a pediatric emergency department. Phase 1 included plan-do-study-act cycles to test interventions such as provider education and system changes. Phase 2 was a planned experiment studying 2 interventions (study cell phone and patient activation card), using a 2 × 2 factorial design with 1 background variable and 2 replications. Outcomes were: (1) the proportion of women aged 14 to 21 years with STI testing whose confidential telephone number was documented in the electronic medical record; (2) the proportion of STI positive women successfully contacted within 7 days.
RESULTS:
Phase 1 interventions increased the proportion of records with a confidential number from 24% to 58% and the proportion contacted from 45% to 65%, and decreased loss to follow-up from 40% to 24%. In phase 2, the proportion contacted decreased after the electronic medical record system changed and recording of the confidential number decreased. Study interventions (patient activation card and study cell phone) had a synergistic effect on successful contact, especially when confidential numbers were less reliably documented.
CONCLUSIONS:
Feasible and sustainable interventions such as improved documentation of a confidential number worked synergistically to increase our ability to successfully contact adolescent women with their STI test results.
KEY WORDS: adolescent sexual behavior, outcomes research, quality improvement, sexually transmitted infection
Women aged <25 years have higher rates of sexually transmitted infections (STIs) than other population subgroups.1 The prevalence of treatable STIs in 14- to 21-year-old women visiting the emergency department (ED) at Cincinnati Children’s Hospital Medical Center (CCHMC) is extremely high: nearly 40% of adolescent women recruited for research studies had an STI, including Chlamydia (24%), trichomoniasis (19%), and gonorrhea (15%).2,3 Women with clinical herpes and pelvic inflammatory disease are treated and counseled at the ED visit, and the required follow-up for those with syphilis is performed by the local health department. Because STI test results for Chlamydia, gonorrhea, and trichomoniasis may take days, providers must balance the consequences of empirical treatment against the risk of losing an STI-positive patient to follow-up. Thus, empirical therapy was common.
Optimal STI control requires more than testing to disrupt transmission: the Centers for Disease Control and Prevention recommends that infected women be treated with timely antibiotics, and counseled on partner treatment and safer sex practices.4 Implicit in this recommendation is that all infected women be notified of their results. However, clinical practice in our ED was similar to that of others: notify patients with an STI only if they were not treated at their visit.5 We found that adolescent women who believed they had an STI were more likely to abstain from sexual activity and to notify their partners than those who were unaware of their STI. However, women who were treated with antibiotics but did not believe they tested positive for STIs did not change their behavior.6 However, these published results and Centers for Disease Control and Prevention recommendations did not change clinical practice in the ED.
There is no accepted definition or guideline for timeliness of STI follow-up care. Several authors define successful treatment as that given within 7 to 14 days of the visit and define lost to follow-up as those untreated after 30 to 60 days. After an ED visit, some estimate that only 29% of untreated women are treated within 14 days, and 60% are lost to follow-up.5,7,8–10 Sexually transmitted disease clinics report that ∼50% are treated within 14 days, and 20% to 30% are lost to follow-up.8–10 In our adolescent clinic, we contacted 69% of untreated women within 7 days, and <10% were lost to follow-up.11
We identified a gap between current and ideal practice in our ED: <100% of STI-positive women were notified, and the proportion treated within 7 days or lost to follow-up was unknown. The goal of this project was to evaluate interventions to improve our ability to successfully contact adolescent women who have an STI after their ED visit. We aimed to increase the contacted within 7 days rate to 70% and to reduce the loss to follow-up rate to <20%.
Methods
Setting
CCHMC is an urban, pediatric academic medical center. In 2008, CCHMC had 93 753 ED visits, and 10 843 of these visits were made by female patients aged 14 to 21 years. This corresponds to roughly 200 visits by adolescent women each week.
Improvement Team
Our multidisciplinary project team included ED providers and staff, adolescent medicine experts, and a project manager.
Planning the Intervention
Phase 1 used qualitative interviews to understand key drivers affecting our aim. We identified individual design changes (factors) amenable to intervention and serially tested those factors by using plan-do-study-act (PDSA) cycles. Phase 2 was a planned experiment based on knowledge gained in phase 1.
Ethics
Phase I interviews were approved by our institutional review board as no greater than minimal risk human subjects research. The remainder of the project was determined to be quality improvement (QI) work.
Phase 1 Methods: Planning the Intervention
Data were retrieved from the electronic medical record (EMR). All female ED patients between 14 and 21 years of age who were tested for an STI were included. Data included the STI test results, whether empirical treatment was given, and whether a confidential telephone number was documented in the EMR. At baseline, ∼50 young women were tested for STIs every week and of these, ∼25% were positive for at least 1 STI (Chlamydia, gonorrhea, or trichomoniasis). Because our previous work suggested that contacting patients with an STI would change their sexual behaviors, we decided a priori to contact all STI-positive women, regardless of ED treatment.6 When contacted, women could return to the ED or have a prescription called in for treatment. Either treatment was counted as a success.
Key stakeholders (clerical staff, nurses, and physicians in the ED) were interviewed, and pertinent clinical experiences were reviewed. We developed a graphical representation of our theory of the elements influencing follow-up to identify areas amenable to improvement (Fig 1). This QI tool is also known as a key driver diagram, a term adapted from business and used frequently at CCHMC.12 The diagram is developed by identifying our aim and asking: (1) What needs to be in place and working well to achieve this aim? (2) What is the theory/hypothesis for achieving this aim? and (3) What are the key drivers? In our project, we theorized that better follow-up was an achievable goal, shared by ED providers and teenagers alike, which would reduce local STI prevalence and sequelae. Four drivers were identified: systematic approach to follow-up, improving confidential contact, patient/provider information, and empowering/engaging teenagers in their care. The project team brainstormed design changes and interventions that might be implemented. PDSA cycles were used to sequentially test several interventions for each driver.
FIGURE 1.
Key driver diagram: this QI tool starts by identifying our aim (left side of the diagram). We used qualitative interviews and clinical experience to identify the key drivers (center) that affect this aim. We then brainstormed design changes that might be implemented as interventions (right side).
The first driver addressed was a systematic approach to patient notification of positive STI results, including both outgoing and incoming patient calls. We mapped the existing process, which involved numerous steps and providers. We simplified this by assigning a single nurse practitioner (NP) to handle all STI follow-up. She developed a standardized script to disclose STI results and provide recommended counseling and treatment. She developed a “flag” in the EMR that treatment was needed if the patient returned to the ED for a different care episode. She altered time of day that calls were made from 9 am to late afternoon.
Next, to give the patient her test results, we focused on improving the documentation of a confidential telephone number in the patient’s EMR. Others have shown that the lack of a correct telephone number was a significant factor in their ability to contact adolescent women for pelvic inflammatory disease follow-up: of women with no contact, the number listed was disconnected (36%), incorrect (4%), or the parent refused to allow the nurse to speak to the teenager (8%).13 In our adolescent clinic, we learned that adolescent women preferred to receive their confidential STI test results via their personal cell phone. They were more likely to give this number to a clinician than to registration staff. Because it breached confidentiality to leave STI results on voice mail, we refined our goal of “contact” as direct “voice-to-voice” communication between the NP and teenager. Figure 2 outlines some PDSA cycles for this aim. We tried several options for a “callback” number, including the main ED office, the NP’s desk telephone, and a “ghost mailbox” that could be accessed remotely, theoretically by any provider involved in STI care. In cycle 4, we purchased a cell phone so that the NP could make and receive calls even in the midst of a clinical shift, which decreased the number of calls made to each patient and improved satisfaction for both patients and the NP.
FIGURE 2.
Learning from PDSA cycles to increase our ability to make voice-to-voice contact with adolescent women who have a positive STI test result. I, intervention; L, learning.
Another intervention addressed 2 drivers: improving provider communication and engaging teenagers in their care. Patient activation cards were developed based on similar cards developed in our adolescent clinic QI work. We theorized that concrete-thinking teens may not realize that when released from the ED: (1) test results were incomplete; (2) we were requesting cell numbers to give them results; and (3) providing a direct number might empower teenagers to call for results. Each business-sized card read: “Our goal is to keep you healthy! You had tests performed on (date). Your results should be back in about 3-4 days. Tell your doctor or nurse today what number we should call to reach you. We will contact you in a few days if your results are positive. You may also call Rachael at 513-xxx-xxxx to get your test results. Please call between 9 AM and 5 PM” (Fig 3). Nurses assisting with pelvic examinations distributed these cards. Interventions included e-mail and in-services for the nursing staff to promote card distribution.
FIGURE 3.
Patient activation card. Intervention: cards distributed to all adolescent women with STI testing at their ED visit. During the planned experiment, the active version included all text, and the inactive version excluded the italicized text between the asterisks.
Methods of Evaluation
We used feedback from the project team and process measures to evaluate each of the interventions.
Process Measures
The confidential contact process was evaluated by using the NP’s notes and weekly updates on the number of outgoing and incoming calls. The card process was evaluated by counting how many cards were distributed every 2 weeks.
Outcome Measures
Our primary outcome was the percentage of STI-tested women with a confidential telephone number documented in their EMR. Secondary outcomes were the percentage of STI-positive women who were contacted within 7 days or who were lost to follow-up (defined as no voice-to-voice contact or return visit to the ED within 30 days of testing).
Analysis
Shewhart control charts (P charts) were used to evaluate outcomes.14 On each Shewhart P chart, we plotted the percentage of successes (confidential number recorded or contacted within 7 days) on the vertical axis versus time on the horizontal axis. The control limits for P charts were based on the binomial distribution of proportions, as indicated when an outcome is not rare. We used established special cause rules to determine when (where) a change occurred in control charts as follows: (1) a single point outside the control limits; (2) a run of 6 to 8 points in a row above or below the centerline; (3) a trend of 6 consecutive points increasing or decreasing; and (4) 2 of 3 consecutive points in the outer third of a control limit.15 The special cause variation was annotated on each chart. We obtained baseline data from July 7, 2008, through September 21, 2008, and began our series of interventions on September 22, 2008.
Phase I Results
Over the study period, 8800 women age 14 to 21 years made an ED visit; 1343 (15.3%) were tested for STIs and 299 (22.3%) of those had an STI. During the baseline period (usual care), of 120 women tested for STIs, 46% were contacted within 7 days, 40% were lost to follow-up, and most (70%) were empirically treated at the initial visit. For 36 women not treated at the ED visit, 45% were treated within 7 days, and 31% were lost to follow-up.
Between September 9, 2008, and January 31, 2009, we tried a series of 5 interventions to improve documentation of the confidential telephone number. First, we e-mailed reminders to ED clinicians to collect the number whenever STI tests were ordered. Second, we created a field in the EMR to record the confidential number. Third, we distributed patient activation cards to adolescents undergoing STI testing. Fourth, we posted flyers in staff and patient restrooms and provided further education to paramedics and nursing staff about the rationale for collecting a confidential number. Finally, we created an EMR prompt that automatically linked to the “collect a confidential telephone number” field whenever a “pelvic exam set up” was ordered. Trialing the first 4 interventions did not increase documentation of a confidential number. After the EMR prompt, the percentage of records with confidential telephone numbers recorded increased from 24% to 58% (Fig 4). We excluded 1 point (April 21, 2009) as a special cause which occurred during the 2009 H1N1 influenza outbreak when attention was diverted from STI care.
FIGURE 4.
Shewhart P chart: percentage of charts with confidential number documented in the EMR for women with STI testing, over time. n, number of charts per interval. Solid line: mean percentage. Dotted lines: control limits. Initial intervention (box 1): E-mail ED staff reminders to collect confidential number whenever STI tests ordered. Arrows indicate subsequent interventions.
Initially, these interventions did not alter the percentage of women contacted within 7 days from the baseline value of 45%. However, ∼3 weeks after the EMR prompt increased the confidential number, the percentage of women successfully contacted within 7 days rose to 65% (Fig 5). This intervention also decreased the proportion of women lost to follow-up from 40% at baseline to 24% postintervention.
FIGURE 5.
Shewhart P chart: percentage of women with an STI who were successfully contacted within 7 days of their ED visit, over time. n, number of infected women per interval. The vertical axis is the percentage of women contacted. Solid line is the mean, and dotted lines are control limits. Arrows indicate interventions.
Phase 2 Methods: Planned Experiment
Planning the Intervention
We created a 2 × 2 factorial design studying 2 modifiable elements (patient activation card and NP cell phone) with 1 background variable (confidential number), with 2 replications of the experiment over time.14,16 This design can assess if individual interventions work synergistically to improve successful contact for STI-positive is more accurate women.
Because phase 1 demonstrated the importance of obtaining a confidential number, we chose to designate this as a background variable; that is, we did not want to turn this intervention “on/off.” Rather, we wanted the number to be in place for all participants and to then study the effect of additional interventions. Although our goal was 80%, after a year of interventions, we were unable to increase the documentation of a confidential number much above 60%. Because this value had stabilized, we proceeded with our planned experiment.
We then operationalized the other factors, the dichotomous variables “patient activation card” and “NP cell phone.” Patient cards were so well adopted by nursing staff that when we removed the cards, nurses would make replacement copies. Therefore, we had to develop active and inactive versions. The active version included the line “You may also call Rachael at 513-xxx-xxxx to get your test results. Please call between 9 AM and 5 PM” whereas the inactive version did not. The cell phone also had 2 levels: ON (cell phone was used by the NP for all incoming and outgoing STI phone calls) and OFF (outgoing calls were made from a land line and incoming calls went to the main ED line [previous standard of care]).
From our phase 1 data, we estimated seeing 20 STI-positive women per 2-week interval (run). By using our design matrix (Fig 6), we planned 4 runs of 2 weeks each, with a 5-day washout period between runs. An expected 80 STI-positive women in each replication estimated 80% power to detect a change.13 The order of runs was chosen to minimize procedure changes to ED staff and was reversed in the second replication. The first replication occurred immediately after phase 1. The second replication occurred after a 4-month break to accommodate the ED’s transition to a new EMR system.
FIGURE 6.
Design matrix for a 2 × 2 factorial planned experiment denoting the order of runs and replications. The 2 modifiable factors (patient card and NP cell phone) each had 2 levels, resulting in 4 combinations or “runs.” The experiment was replicated, reversing the runs, after a change in the EMR system.
Analyses
We determined the effect of the 4 combinations of factors on the outcome (percent contacted within 7 days) by using graphical displays (response plots). When experiments have >1 factor, response plots can visually display the potential interactions between factors, providing a simple presentation that may provide insight into cause-and-effect relationships.13,15
Phase 2 Results
Fifty-five STI-positive women were identified during the first replication and 42 in the second for a total sample size of 97. Despite our best efforts to stabilize our background variable across the EMR change, we discovered that the percentage of charts with a confidential number recorded dropped from nearly 60% during the first replication to 41% of charts in the second. This unexpected change provided the opportunity to study the effect of our factors under 2 conditions: higher and lower reliability of confidential number documentation. Through ongoing monitoring, we determined that several weeks after the second replication, the percentage of charts with a confidential number documented returned to 60%.
Graphical analyses showed significant differences in the effects of factors between the replications on the outcome “percentage of women contacted within 7 days” (Fig 7). In the first replication, confidential numbers were recorded in 60% of records, and the NP cell phone was effective whether the patient card was active. However, the lowest contact rate occurred when the card was active but the NP cell was off. That the 2 lines are not parallel is evidence that the effect of the card and NP cell were not independent.
FIGURE 7.
Response plots of the outcome (percentage of infected women contacted within 7 days) for the planned experiment according to replication and background variable (confidential telephone number). Each point is the mean result for the 4 conditions of the experiment. Patient card condition is noted on the horizontal axis, and lines represent NP cell condition.
In replication 2, when 41% of records had a confidential number, the interaction between the factors seemed stronger. The patient card significantly increased contact within 7 days and had a synergistic effect with the NP cell phone. In replication 2, 14% were contacted when the patient card was inactive, compared with 60% when the patient card was active. Of note, the run “NP cell on/patient card inactive” contained only 4 subjects with an STI in a 2-week period, and none of these subjects was reached within 7 days. We could not identify a special cause for this anomaly and attributed it to small sample size.
Thus, use of a card activating women to call for results and an NP study cell phone improved the outcome of successful contact within 7 days. This effect was especially pronounced when confidential number documentation was in the low reliability condition (41% of charts) compared with the higher reliability condition (60% of charts).
Discussion
Inadequate follow-up of positive STI results is a gap in health care quality that contributes to the epidemic of STIs in adolescent women, and it can be improved. By using systems level interventions to increase the documentation of a confidential telephone number, we increased the proportion of adolescents who were notified of their positive STI status within 7 days of a visit from 45% to 65% and decreased the lost to follow-up rate from 40% to 24% for a notoriously difficult to reach population (ie, adolescents seen in an ED). Our findings are congruent with those reported by Trent et al,13 who used an interrupted time series approach and showed that multilevel interventions improved care of pelvic inflammatory disease at the time of an outpatient or ED visit. To our knowledge, this is the first publication of a planned experiment studying system changes to improve adolescent STI follow-up care in an ED. We demonstrated that low-cost, sustainable interventions which target provider information/patient empowerment may help when confidential telephone numbers are not routinely recorded. The planned experiment demonstrated the synergistic effect of the NP cell phone and patient card, thus helping to generate institutional support for these efforts. Improved recording of the confidential number was maintained despite transitioning from 1 EMR to another. The annual cost of the patient cards and NP cell phone were <$500. Together, these interventions have the potential to disrupt STI transmission and reduce the burden of STIs in our community.
Our results are important because the inability to provide timely follow-up of STI test results has significant public health consequences. Young women with untreated STIs are at risk for serious sequelae such as pelvic inflammatory disease and infertility.14,17–19 In addition, HIV is more easily acquired when one is co-infected with other STIs.20 Even if women are treated at the ED visit, those who are not notified of their infection cannot be counseled to notify their partner or adopt safer sex practices, and thus are likely to become re-infected.
Others have shown that telephone follow-up after an ED visit improved patient satisfaction and compliance and, furthermore, that >40% of patients required clarification regarding their discharge instructions.21,22 A planned experiment in an ED studied the effect of an intervention given in person, during telephone follow-up, or at both times on compliance with follow-up.23 Although not specifically targeting STI testing, the study included women with vaginitis. Those assigned to the post-visit telephone call were more likely than others to comply with follow-up recommendations. These studies are congruent with our findings of improving treatment success by using post-visit telephone calls.
Our study did have limitations. Although challenging, it is feasible to perform a planned experiment in a busy ED. It can be complicated to “turn on” and “turn off” factors, and patient volume and clinical care can fluctuate for reasons outside of our control. Another approach would have been to continue each run of the planned experiment until we had 20 subjects per run. However, we faced time and logistical constraints. The small number of STI-positive women identified in our final cohort lowered our power to detect differences.
Conclusions
Inadequate follow-up of positive STI results is a gap in health care quality that contributes to the epidemic of STIs in adolescent women. Feasible and sustainable interventions such as improved documentation of a confidential telephone number work synergistically to increase our ability to successfully contact adolescent women with their STI results. Decreasing the interval between testing and notification as well as the lost to follow-up rate may reduce STI prevalence and risks of further sequelae. Rather than lamenting the difficulty of reaching adolescents who have STIs, organizations can make system level changes to achieve this important public health goal.
Glossary
- CCHMC
Cincinnati Children’s Hospital Medical Center
- ED
emergency department
- EMR
electronic medical record
- NP
nurse practitioner
- PDSA
plan-do-study-act
- QI
quality improvement
- STI
sexually transmitted infection
Footnotes
Dr Huppert was responsible for conception and design, acquisition of data, drafting of the manuscript, critical revision, statistical expertise, obtaining funding, and administrative, technical and material support; Dr Reed was responsible for conception and design, acquisition of data, drafting of the manuscript, and critical revision; Ms Munafo handled acquisition of data, analysis and interpretation of data, drafting of the manuscript, and critical revision; Ms Ekstrand conducted acquisition of data and analysis and interpretation of the data; Dr Gillespie performed acquisition of data, analysis and interpretation of data, and critical revision; Dr Holland handled acquisition of data, analysis and interpretation of the data, and drafting of the manuscript; and Dr Britto was responsible for conception and design, analysis and interpretation of the data, statistical support, critical revision, administrative, technical, or material support, and supervision. The manuscript was approved by all authors.
FINANCIAL DISCLOSURE: Dr Huppert reports that she has received kits/reagents from Gen-Probe Incorporated as well as kits/reagents and an honorarium from Genzyme Diagnostics, Inc. She is also a member of the scientific advisory board for Proctor & Gamble, Inc. None of these represent any conflict of interest with this project but are mentioned in the interest of full disclosure. The other authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: Funded by an internal Outcomes Award from Cincinnati Children’s Research Foundation. Dr Reed was supported by Award Number K12HD051953 from the Eunice Kennedy Shriver National Institute of Child Health & Human Development. Dr Huppert was supported by award number 5K23AI063182 from the National Institute of Allergy and Infectious Diseases.
References
- 1.Centers for Disease Control and Prevention. Sexually Transmitted Disease Surveillance, 2008. Atlanta, GA: US Department of Health and Human Services; 2009 [Google Scholar]
- 2.Huppert JS, Biro F, Lan D, Mortensen JE, Reed J, Slap GB. Urinary symptoms in adolescent females: STI or UTI? J Adolesc Health. 2007;40(5):418–424 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Huppert JS, Mortensen JE, Reed JL, et al. Rapid antigen testing compares favorably with transcription-mediated amplification assay for the detection of Trichomonas vaginalis in young women. Clin Infect Dis. 2007;45(2):194–198 [DOI] [PubMed] [Google Scholar]
- 4.Workowski KA, Berman S. Sexually transmitted diseases treatment guidelines, 2010. MMWR Recomm Rep. 2010;59(RR-12):2–8 [PubMed] [Google Scholar]
- 5.Yealy DM, Greene TJ, Hobbs GD. Underrecognition of cervical Neisseria gonorrhoeae and Chlamydia trachomatis infections in the emergency department. Acad Emerg Med. 1997;4(10):962–967 [DOI] [PubMed] [Google Scholar]
- 6.Reed JL, Simendinger L, Griffeth S, Kim HG, Huppert JS. Point-of-care testing for sexually transmitted infections increases awareness and short-term abstinence in adolescent women. J Adolesc Health. 2010;46(3):270–277 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Bachmann LH, Richey CM, Waites K, Schwebke JR, Hook EW, III. Patterns of Chlamydia trachomatis testing and follow-up at a University Hospital Medical Center. Sex Transm Dis. 1999;26(9):496–499 [DOI] [PubMed] [Google Scholar]
- 8.Katz BP, Danos CS, Quinn TS, Caine V, Jones RB. Efficiency and cost-effectiveness of field follow-up for patients with Chlamydia trachomatis infection in a sexually transmitted diseases clinic. Sex Transm Dis. 1988;15(1):11–16 [DOI] [PubMed] [Google Scholar]
- 9.Schwebke JR, Sadler R, Sutton JM, Hook EW, III. Positive screening tests for gonorrhea and chlamydial infection fail to lead consistently to treatment of patients attending a sexually transmitted disease clinic. Sex Transm Dis. 1997;24(4):181–184 [DOI] [PubMed] [Google Scholar]
- 10.Wong D, Berman SM, Furness BW, Gunn RA, Taylor M, Peterman TA. Time to treatment for women with chlamydial or gonococcal infections: a comparative evaluation of sexually transmitted disease clinics in 3 US cities. Sex Transm Dis. 2005;32(3):194–198 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Malik AI, Huppert JS. Interval to treatment of sexually transmitted infections in adolescent females. J Pediatr Adolesc Gynecol. 2007;20(5):275–279 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Langley GJMR, Nolan KM, Nolan TW, Norman CL, Provost LP, eds. The Improvement Guide: A Practical Approach to Enhancing Organizational Performance. 2nd ed. San Francisco, CA: Jossey Bass, a Wiley Imprint; 2009 [Google Scholar]
- 13.Trent M, Judy SL, Ellen JM, Walker A. Use of an institutional intervention to improve quality of care for adolescents treated in pediatric ambulatory settings for pelvic inflammatory disease. J Adolesc Health. 2006;39(1):50–56 [DOI] [PubMed] [Google Scholar]
- 14.Moen RD, Nolan TW, Provost LP. Testing a change. In: Quality Improvement Through Planned Experimentation. 2nd ed. New York, NY: McGraw-Hill; 1999:21–46 [Google Scholar]
- 15.Moen RD, Nolan TW, Provost LP. Appendix A: improvement using control charts. In: Quality Improvement Through Planned Experimentation. 2nd ed. New York, NY: McGraw-Hill; 1999:379–410 [Google Scholar]
- 16.Moen RD, Nolan TW, Provost LP. Experiments with more than one factor. In: Quality Improvement Through Planned Experimentation. 2nd ed. New York, NY: McGraw Hill; 1999:113–164 [Google Scholar]
- 17.Peipert JF, Ness RB, Blume J, et al. Clinical predictors of endometritis in women with symptoms and signs of pelvic inflammatory disease. Am J Obstet Gynecol. 2001;184(5):856–863; discussion 863–864 [DOI] [PubMed]
- 18.Tubal infertility: serologic relationship to past chlamydial and gonococcal infection. World Health Organization Task Force on the Prevention and Management of Infertility . Sex Transm Dis. 1995;22(2):71–77 [PubMed] [Google Scholar]
- 19.Hillis SD, Joesoef R, Marchbanks PA, Wasserheit JN, Cates W, Jr, Westrom L. Delayed care of pelvic inflammatory disease as a risk factor for impaired fertility. Am J Obstet Gynecol. 1993;168(5):1503–1509 [DOI] [PubMed] [Google Scholar]
- 20.Laga M, Manoka A, Kivuvu M, et al. Non-ulcerative sexually transmitted diseases as risk factors for HIV-1 transmission in women: results from a cohort study. AIDS. 1993;7(1):95–102 [DOI] [PubMed] [Google Scholar]
- 21.Jones J, Clark W, Bradford J, Dougherty J. Efficacy of a telephone follow-up system in the emergency department. J Emerg Med. 1988;6(3):249–254 [DOI] [PubMed] [Google Scholar]
- 22.Ritchie PD, Jenkins M, Cameron PA. A telephone call reminder to improve outpatient attendance in patients referred from the emergency department: a randomised controlled trial. Aust N Z J Med. 2000;30(5):585–592 [DOI] [PubMed] [Google Scholar]
- 23.Jones SL, Jones PK, Katz J. Health belief model intervention to increase compliance with emergency department patients. Med Care. 1988;26(12):1172–1184 [DOI] [PubMed] [Google Scholar]