Quality Indicators for Appropriate Outpatient Parenteral Antimicrobial Therapy in Adults: A Systematic Review and RAND-modified Delphi Procedure

Abstract

Background

Our aim in this study was to develop quality indicators (QIs) for outpatient parenteral antimicrobial therapy (OPAT) care that can be used as metrics for quality assessment and improvement.

Methods

A RAND-modified Delphi procedure was used to develop a set of QIs. Recommendations on appropriate OPAT care in adults were retrieved from the literature using a systematic review and translated into potential QIs. These QIs were appraised and prioritized by a multidisciplinary panel of international OPAT experts in 2 questionnaire rounds combined with a meeting between rounds.

Results

The procedure resulted in 33 OPAT-specific recommendations. The following QIs that describe recommended OPAT care were prioritized by the expert panel: the presence of a structured OPAT program, a formal OPAT care team, a policy on patient selection criteria, and a treatment and monitoring plan; assessment for OPAT should be performed by the OPAT team; patients and family should be informed about OPAT; there should be a mechanism in place for urgent discussion and review of emergent clinical problems, and a system in place for rapid communication; laboratory results should be delivered to physicians within 24 hours; and the OPAT team should document clinical response to antimicrobial management, document adverse events, and monitor QIs for OPAT care and make these data available.

Conclusions

We systematically developed a set of 33 QIs for optimal OPAT care, of which 12 were prioritized by the expert panel. These QIs can be used to assess and improve the quality of care provided by OPAT teams.

Thirty-three quality indicators (QIs) for optimal outpatient parenteral antimicrobial therapy (OPAT) care were developed by international experts using a systematic review and a RAND-modified Delphi procedure. These QIs can be used to assess and improve the quality of care provided by OPAT teams.

Outpatient parenteral antimicrobial therapy (OPAT) provides patients with the opportunity to receive parenteral antimicrobials at home as an alternative to inpatient care [1].

OPAT was first described in 1974, and its application has grown rapidly since. Today, OPAT is applied for a variety of infections, for example, bone and joint infections, endocarditis, skin and soft tissue infections, urinary tract infections, and endovascular infections [2]. A growing body of evidence supports its clinical applicability, safety, and cost-effectiveness, leading to expanded use in many countries.

The organization of OPAT care is complex and involves many healthcare professionals and multiple transitions of care. To achieve optimal OPAT care, all professionals involved should provide appropriate OPAT care at all stages of the care pathway, ranging from the organizational phase of OPAT, through the initiation and continuation phase at home, to discontinuation.

Guidelines for OPAT and an OPAT healthcare bundle have been published. These include specific recommendations for patient and drug selection as well as for follow-up, with the goal to establish an effective and safe OPAT program [1–3]. These guidelines and the international literature can be used to systematically develop quality indicators (QIs). QIs are defined as “measurable elements of practice performance for which there is evidence or consensus that they can be used to assess the quality and hence change in the quality of care provided” [4]. QIs are usually divided into 3 categories: structure indicators, reflecting the organization of the healthcare system (eg, the availability of an OPAT team); process indicators, which refer to the care that is actually delivered to patients (eg, infectious diseases (ID) specialist consultation prior to OPAT initiation); and outcome indicators, focusing on the consequences of interventions (eg, hospital readmission and patient satisfaction) [4]. QI performance can help healthcare professionals to set priorities for interventions to improve healthcare and patient outcome. To guide improvement, it is essential that the measurement of the quality of care is reliable and supported by evidence [5]. Ideally, QIs have a clear and direct association with relevant outcomes [6].

Until now, QIs that define appropriate OPAT care in adults have not been systematically developed. Our aim in this study was to develop a set of QIs that can be used to assess the appropriateness of OPAT care.

METHODS

We used a RAND-modified Delphi procedure to develop a set of literature-based QIs for OPAT care [7]. OPAT was defined as “a method for delivering intravenous antimicrobials in the community or outpatient setting, as an alternative to inpatient care” [8]. The procedure ran between November 2017 and June 2018. Because the QIs were developed using the Delphi method, ethics approval was not required under Dutch law.

Selection of OPAT Care Recommendations

Phase 1: Extraction of Previously Identified Quality-of-Care Recommendations

OPAT care recommendations were selected from 2 previously published international guidelines for OPAT and 1 systematic review that focuses on quality of OPAT care [1–3].

Phase 2: Identification of Additional Quality-of-Care Recommendations by a Systematic Review

We developed a comprehensive search strategy (Supplementary Appendix 1) in consultation with an independent research librarian. We searched the electronic databases PubMed, EMBASE, and the Cochrane databases for the period 1 January 2013 to 20 October 2016 for articles in English that described recommendations for OPAT care.

We selected studies published after 1 January 2013, as the previously published guidelines and the systematic review cover the period that preceded this date (phase 1).

Inclusion and Exclusion Criteria

We included articles that described clear recommendations for appropriate adult OPAT care, focusing on any element of practice performance (ie, appropriate structures, processes, and outcomes) and based on published research, guidelines, literature reviews, or consensus procedures [4]. We excluded articles of which a full-text could not be retrieved from our library or through Google Scholar.

Screening Process, Data Collection, and Analysis

M. B. and J. O. independently screened all titles and abstracts to select potentially eligible articles for full text screening. We used the literature review management software DistillerSR (Evidence Partners, Ottawa, Ontario, Canada) for this screening process. Articles were also selected for full text screening when there was no abstract available or when the abstract was insufficiently detailed to allow a proper evaluation of the eligibility criteria.

All selected full-text articles were independently screened by 2 reviewers (M. B. and J. S.) who included relevant articles based on the predefined inclusion and exclusion criteria. The reference lists of all included papers were screened to identify additional papers (so-called snowballing), using the same inclusion and exclusion criteria as described before.

Data extraction was performed by 1 investigator (M. B.) who used a standardized form. Data were extracted on authors, year of publication, country of study origin, study design, and any recommendation on appropriate structures, processes, or outcomes. Since the GRADE classification system [9] to determine the level of evidence was difficult to apply for the types of studies included in this systematic review, we categorized the study design as follows: recommendations based on a research study, recommendations based on a systematic review (review-based indicators), recommendations based on an expert consensus/Delphi round (consensus-based indicators), and recommendations based on guidelines (guideline-based indicators).

All extracted data from the included articles were double-checked by another investigator (J. S.). Any disagreement between these 2 investigators was resolved by discussion, using advice from a third expert when needed (M. H.). After the final list of recommendations was established, 3 reviewers (M. B., J. S., and M. H.) grouped recommendations that addressed similar topics. Next, in consensus, recommendations were merged and duplicates were removed. The selected recommendations were then translated (operationalized) into potential QIs.

Selection of the Expert Panel

Twenty-seven national and international OPAT experts were invited to participate. National experts were selected based on their involvement in national OPAT programs by purposeful sampling with the help of the Dutch Working Party on Antibiotic Policy. International experts were selected based on their involvement in OPAT programs, research, and prior publications. For the international experts, we contacted the executive committee of the European Society of Clinical Microbiology and Infectious Diseases Study Group for Antimicrobial Stewardship to suggest OPAT experts within Europe. In addition, we searched in the literature list of the included articles for the first and last authors of the articles. All experts were invited by email to participate.

All experts were asked to contribute viewpoints, based on their own experiences, rather than research or (inter)national guidelines. No financial incentive was provided for participation.

First Questionnaire Round

The potentially relevant QIs were put into a written questionnaire to be used for a RAND-modified Delphi procedure to achieve expert consensus on these QIs.

The first questionnaire round was performed between December 2017 and February 2018. The questionnaire was sent by email (Limesurvey) to 27 experts. The expert panel was asked to appraise the relevance of the indicators for assessing the quality of OPAT care using a 9-point Likert scale (with 1 denoting “clearly not relevant” and 9 denoting “clearly relevant”), including the option “cannot assess.” Where necessary, QIs were followed by specific questions relevant for operationalizing the recommended care. For example, for the potential QI “There should be a formal OPAT care team,” an additional question queried options that indicate potentially relevant disciplines that should be represented in the OPAT care team. The panel members were able to provide comments for each potential QI, including suggestions for rephrasing. In addition, panel members could add QIs or topics for consideration at the end of the questionnaire.

The results from the first Delphi round were analyzed using standardized consensus methodology. QIs were accepted if the median score was ≥8 and ≥70% of the scores were in the top tertile (scores 7, 8, or 9). QIs were discussed at the consensus meeting if ≥70% of the scores were in the top tertile but the median score was between 7 and 8 or if they had a median score ≥8 but <70% of the scores were in the top tertile [10].

Expert Panel Meeting

National and international experts who completed the first-round questionnaire were invited to the expert panel meeting. Experts from outside the Netherlands could participate through web conferencing. Before the meeting, all experts were sent a personal feedback report that described, per potential indicator, the group score, the individual score, and whether the QI was accepted (green), for up for discussion (yellow), or was not accepted (red). The goal of the meeting was to present the results and comments expressed in the first round and to discuss the QIs labeled “for discussion.” In addition, newly suggested potential QIs were discussed, and accepted QIs with comments from the experts were rephrased in consensus where applicable.

Second Questionnaire Round

After the consensus meeting, the accepted, newly suggested, and rephrased QIs were presented in an extensive summary for final remarks. All experts were asked to rate newly suggested QIs (on a 9-point Likert scale) and to express their agreement on suggested operationalizations (yes/no). Finally, experts were asked to select the top 3 QIs for the following categories: OPAT organization, OPAT initiation, OPAT continuation, and OPAT outcome. A group sum score was calculated to determine the top 3 indicators for each category: 3 points for rank 1, 2 points for rank 2, and 1 point for rank 3 [11].

RESULTS

Literature Search and Selection of Recommendations

Of the 1103 publications published after 1 January 2013 identified, 14 provided recommendations on appropriate OPAT care. Screening of the reference lists of these 14 studies resulted in the inclusion of 3 additional articles. The 2 OPAT guidelines [1, 2], a systematic review on the subject[3], and the 17 papers selected were used to derive 129 OPAT care recommendations. Merging recommendations while removing duplicates resulted in 51 unique recommendations. Figure 1 schematically presents the literature search and selection process. The 20 articles are listed in the Supplementary Table 1.

Figure 1.

Flow chart of the study selection process. Abbreviation: OPAT, outpatient parenteral antimicrobial therapy. *Three articles only described recommendations for pediatric OPAT.

First Questionnaire Round

Nineteen of the 27 invited experts returned the first questionnaire: 14 ID specialists, 1 nurse specialist, 3 clinical hospital pharmacists, and 1 clinical microbiologist (response rate 70%). The expert panel accepted 32 QIs, rejected 12 indicators, and suggested 2 new indicators. Seven indicators were labeled “for discussion.”

Expert Panel Meeting

Eight experts attended the meeting: 6 ID specialists, 1 clinical microbiologist, and 1 clinical hospital pharmacist (42% of first-round responders), including 2 international experts. The group discussed the 7 indicators with disagreement, the 2 newly proposed indicators, and 10 QIs accepted with comments. Comments from the panel members regarding the first questionnaire were used to rephrase 5 accepted indicators and to merge 7 accepted indicators into 4 indicators (Supplementary Table 2). Of the 7 indicators with disagreement, 3 were accepted (2 after rephrasing), 2 were rejected, and 1 was merged with another indicator. The seventh indicator was rephrased and scheduled for the second questionnaire round. One of the newly proposed indicators was rejected, and the other was merged with another indicator and scheduled for the second questionnaire round. No additional potential QIs were added during this meeting.

Second Questionnaire Round, Ranking Procedure

During the second questionnaire round, 2 rephrased QIs were presented for approval, of which 1 was accepted. All 33 accepted QIs were categorized, and experts selected the top 3 for each of the 4 categories. This resulted in a set of 12 core QIs that represented the most important QIs to evaluate the quality of OPAT organization, initiation, continuation, and outcome. Table 1 shows all selected recommendations, including corresponding references. Table 2 shows the 12 prioritized QIs.

Table 1.

The 33 Selected Quality Indicators With Contributing References

Quality Indicator		Corresponding Reference in Supplementary Appendix
1	There should be a structured OPAT program to provide a framework for safe and effective care.	13, 19, 20
2	The OPAT program should be part of an antimicrobial stewardship program.	2,4
3	There should be a formal OPAT care team. The OPAT care team should include an ID specialist or physician knowledgeable about IDs and the use of antimicrobials in OPAT; a nurse who is an expert in intravenous therapy, access devices, and OPAT; and a pharmacist knowledgeable about OPAT.	4, 7, 12, 19
4	The OPAT team should have an identifiable, medically qualified lead clinician who has identified time for OPAT in their job plan.	4
5	There should be a guideline for vascular access systems used, including site care.	4
6	There should be a policy on patient selection criteria for OPAT. The following key aspects of patient selection should be taken into account: patients are willing to comply with the follow-up plan, there is an appropriate home environment/adequate support, there are no clinical contraindications to discharge from the hospital, no intravenous–oral switch is possible, and there is patient and caregiver understanding.	4, 14, 19
7	There should be a policy that outlines the responsibilities of OPAT team members.	4
8	The OPAT ID physician should specify infection-related inclusion and exclusion criteria for OPAT.	4
9	A competent member of the OPAT team should perform the initial assessment.	4, 8
10	An OPAT ID physician consultation should take place prior to intravenous access device placement.	5, 10, 14, 17, 18
11	Patients and caregivers should be given the opportunity to decline or accept this mode (OPAT) of therapy.	4
12	Patients and their families should be informed about OPAT. The information they get should, at a minimum, at least include benefits, side effects, potential complications, vascular access/sterile techniques, responsible physician until patients seen in clinic, instructions for emergencies, antimicrobial use, patient responsibilities, nature of OPAT, contact lists, and use of antibiotics (eg, storage conditions).	4, 14, 19
13	In case of self-administration, both the OPAT nurse specialist and patient/caregiver must be satisfied of the patient’s/caregiver’s competence, and this should be documented.	4
14	There should be a mechanism in place for urgent discussion and review of emergent clinical problems during OPAT according to clinical need.	4, 14
15	There should be a system in place for rapid communication between the patient and team members.	14, 19
16	There should be communication between the OPAT team and other stakeholders. These stakeholders should, at a minimum, at least include a general practitioner, community team (when appropriate), and referring clinician. At a minimum, communication with stakeholders should include notification of acceptance into the OPAT program, notification of completion of therapy, and notification of complications.	4, 14
17	The OPAT plan should be documented in the discharge summary.	14
18	The OPAT treatment plan is the responsibility of the OPAT ID physician, following discussion with the referring clinician.	4
19	Laboratory results should be delivered to physicians within 24 hours after obtaining material for testing.	9, 19
20	The treatment plan of patients who receive in excess of 1 week of antimicrobial therapy should be regularly reviewed by the OPAT specialist nurse and physician (narrow-spectrum antibiotics, intravenous–oral switch) in conjunction/consultation with the referring specialist, as necessary.	4, 14, 18, 19
21	The intravascular access device should be removed at the end of therapy if not needed for another reason.	14
22	The program outcome of patients receiving OPAT should be monitored (eg, therapy completed as planned/therapy not completed as planned because of…).	4, 14, 19
23	Antibiotic use of patients receiving OPAT should be monitored (eg, completed as planned/not completed as planned because of…).	4, 19
24	The survival status of patients who received OPAT should be documented (eg, patient alive, died of infection, died of other causes, lost to follow-up, or status unknown).	19
25	The satisfaction status/experiences of patients receiving OPAT should be monitored.	4, 14, 19
26	The OPAT treatment plan should include the following items: choice, dose, frequency, duration, and follow-up plan.	19
27	The OPAT team should select the drug delivery device in agreement with the home health agency.	4
28	In case of self-administration, patients or caregivers should be trained in the administration of intravenous antibiotics.	4
29	The OPAT team should monitor quality indicators for OPAT care and make these data available.	1, 4, 14, 19
30	Patient educational material should be available in written or in multimedia form.	4
31	There should be an OPAT treatment and monitoring plan. The OPAT treatment and monitoring plan should include, at a minimum, indication, antibiotic name, dose, frequency, duration, type of administration (eg, continuous infusion or bolus infusion), and access device used (eg, peripherally inserted central catheter, tunneled catheter).	3, 4, 14, 15, 19
32	The OPAT team should document adverse events related to devices, antibiotic use, and toxicity.	4, 14, 19
33	The OPAT team should document clinical response to antimicrobial management.	4, 14, 19

Abbreviations: ID, infectious diseases; OPAT, outpatient parenteral antimicrobial therapy.

Table 2.

Core Quality Indicators for Outpatient Parental Antimicrobial Therapy

		Cumulative Points
Organization
1	There should be a structured OPAT program to provide a framework for safe and effective care.	45
2	There should be a formal OPAT care team.	21
3	There should be a policy on patient selection criteria for OPAT.	21
Initiation
1	There should be an OPAT treatment and monitoring plan.	34
2	A competent member of the OPAT team should perform the initial assessment.	28
3	Patients and their families should be informed about OPAT.	13
Continuation
1	There should be a mechanism in place for urgent discussion and review of emergent clinical problems during OPAT according to clinical need.	38
2	There should be a system in place for rapid communication between the patient and team members.	29
3	Laboratory results should be delivered to physicians within 24 hours after obtaining material for testing.	12
Outcome
1	The OPAT team should document clinical response to antimicrobial management.	29
2	The OPAT team should document adverse events related to devices, antibiotic use, and toxicity.	28
3	The OPAT team should monitor quality indicators for OPAT care and make these data available.	17

Abbreviation: OPAT, outpatient parenteral antimicrobial therapy.

Discussion

In this study, we developed a set of QIs that describe optimal OPAT care, based on a systematic review of the literature and a RAND-modified Delphi procedure. The experts selected, in consensus, 33 QIs that describe appropriate care covering the entire OPAT care pathway. This set of QIs provides important metrics for the various steps in OPAT care and can serve as a framework for implementation of an OPAT program. As 12 of these QIs were prioritized by the expert panel, these could serve as the first goals to accomplish when establishing an OPAT program.

While OPAT centers were initially established in the United States in the 1970s, many hospitals and other healthcare facilities across Europe and Australia now provide OPAT services. The number of patients who receive OPAT is ever expanding due to, among other reasons, increasing antimicrobial resistance and, hence, fewer oral treatment options. Systematic assessment of the appropriateness of OPAT care can assess and guide improvement activities in these organizations.

Our QIs describe a clear framework for safe treatment at home. A structured OPAT program and a formal OPAT care team are prerequisites for high-quality OPAT care. As acknowledged by the experts, good communication between the patient and team members is essential; indicators on providing information to patients and on communication between patients and team members received the highest scores. This is in line with earlier studies that addressed patient preferences for OPAT care. Patients indicated that the provision of information and the accessibility of OPAT team members are keystones for high-quality OPAT care, enhancing patients’ feelings of freedom and safety [12–14].

Another important topic in our QI set focused on the availability of a treatment and monitoring plan. Although most OPAT patients are successfully cured, side effects and readmissions are common; readmission rates range from 6% to 26% [15–20]. OPAT care teams should provide a treatment and monitoring plan for every patient in order to prevent side effects and readmissions. Furthermore, outcome indicators should be monitored, including adverse events related to devices, antibiotic use, and toxicity.

During the consensus procedure, we tried to establish uniform advice for monitoring laboratory results. The expert panel concluded that the frequency and content of monitoring should depend on the antibiotic agent used, the patient’s condition and comorbidities, and the duration of OPAT care, which is in line with the recommendations in the recently published Infectious Diseases Society of America (IDSA) guideline on OPAT [21].

During the Delphi procedure, various outpatient treatment strategies were identified. The nurse-administered strategy was widely recognized by the expert panel, while the safety and acceptance of the self-administration model (implicating the patient or a relative) incited much debate. Although the recently published OPAT guideline by the IDSA states that patients (or their caregivers) should be allowed to self-administer OPAT, there is only scarce evidence that self-administered OPAT by patients instead of by professionals is safe [22–25]. The panel suggested that a prerequisite for self-administration be standardized training for patients and/or caregivers and confirmation of their competence. It is clear that this model of OPAT care should be a subject for future studies on effectiveness and safety.

The present set of QIs shows both differences and similarities with the QI set for outpatient antibiotic use that was developed by Le Marechal et al. That study retrieved 32 QIs on both orally administered as well as parenterally administered antibiotics at home; 12 indicators specifically address OPAT [26]. Le Marechal et al defined an outpatient as “a non-hospitalized patient who visits a physician (including a general physician) in an ambulatory care setting.” In our study, we only focused on parenteral antibiotic courses initiated within the hospital but delivered at home. Our set is, therefore, more specific for the OPAT care pathway as defined by Chapman: “a method for delivering intravenous antimicrobials in the community or outpatient setting, as an alternative to inpatient care” [2]. Nevertheless, given the increase in antimicrobial resistance, it is likely that other healthcare providers, such as general practitioners, will increasingly initiate OPAT without intervention of a hospital. This is already the case in some European countries (eg, France).

This study has several strengths. First, the set of QIs was developed using a rigorous literature search strategy and involving a multidisciplinary expert panel in which prominent specialists involved in OPAT care were represented. The variety of specialties involved generated a diversity of opinions, which strengthened our results. Furthermore, 3 of the expert panel members were also involved in the development of the IDSA guideline for OPAT care, which strengthens the validity of this set. All recommendations in the IDSA guideline were covered in our first set of indicators, with the exception of the recommendations for pediatric OPAT and catheter-associated thrombosis. We believe that our set of indicators is complementary to the guideline recommendations [21].

Second, by using the Delphi procedure, we combined scientific evidence with expert opinion. This method is well established and is one of the preferred methods for the development of QIs [27, 28].

This study also has limitations. First, the studies selected by our systematic review were mainly based on retrospective cohort studies, with low quality of evidence. Nevertheless, by systematically combining scientific evidence with the consensus by OPAT experts, we were able to reach agreement on the recommendations based on those studies. Second, only 42% of the first-round respondents attended the consensus meeting, and only 2 of the attendees were international experts. However, the results of the consensus meeting were presented to all first-round respondents for additional comments, and the response rate to the second-round questionnaire was high (95%). We therefore believe that a low attendance rate for the consensus meeting did not undermine the validity of the set of QIs.

Finally, the development of QIs for adult OPAT care is the first crucial step to assess and improve the quality of OPAT care. However, the applicability of these QIs in daily practice has not been determined. This should be an important next step to facilitate acceptance and subsequent implementation of these QIs in everyday practice.

Conclusions

We developed a concise set of 33 QIs for optimal OPAT care using a RAND-modified Delphi procedure, of which 12 were prioritized by an expert panel. These QIs can be used in clinical practice to assess and improve the quality of care provided by OPAT teams throughout the OPAT care pathway.

Supplementary Data

Supplementary materials are available at Clinical Infectious Diseases online. Consisting of data provided by the authors to benefit the reader, the posted materials are not copyedited and are the sole responsibility of the authors, so questions or comments should be addressed to the corresponding author.

Notes

Acknowledgments. The authors are grateful to the following outpatient parenteral antimicrobial therapy experts who participated in the consensus procedure for sharing their knowledge and insights on the subject: Prof, Dr S.E. Geerlings, M.J. Ploeger, PharmD, Dr T. Sprong, Dr M. Tromp, Prof, Dr B.J. Kullberg, Dr J. Stalenhoef, Dr E.M.W. van de Garde, Dr C. van Nieuwkoop, Prof, Dr P.J. van den Broek, Dr K. Verduin, Prof, Dr C. Pulcini, Dr E. Muldoon, Dr N.K. Shrestha, Dr S. Keller, Dr F. Buyle, Dr J. Minton, Dr J. ten Oever, Dr P. Messiaen, and Dr G. Allison. They also thank their research librarian, Elmie Peters, for her help developing the search strategy.

Potential conflicts of interest. The authors: No reported conflicts of interest. All authors have submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Conflicts that the editors consider relevant to the content of the manuscript have been disclosed.