Development of a core outcome set for immunomodulation in pregnancy (COSIMPREG): a protocol for a systematic review and Delphi study

Abstract

Introduction

To establish pregnancy, the maternal immune system must adapt to tolerate the semiallogenic fetus. Less than optimal adaptation of the maternal immune system during (early) pregnancy is implicated in several complications of pregnancy. The development of effective immune modulation interventions as preventive or therapeutic strategies for pregnancy complications holds promise. Several studies sought to evaluate the safety and effectiveness of various approaches. However, a limitation is the high variability in clinical and immune outcomes that are reported. We, therefore, aim to develop a core outcome set for application to studies of immune modulation in pregnancy (COSIMPREG).

Methods and analysis

We will use a stepwise approach to develop a COSIMPREG. First, we will perform a systematic review to identify reported outcomes. For this review, Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines will be followed. Second, we will use the Delphi method to develop a preliminary COSIMPREG. In three rounds, the outcomes of the systematic review will be scored. A panel comprising experts from relevant disciplines and diverse geographical locations will be assembled until a sufficient quality of the panel is reached. We will use predefined decision rules for outcomes. After each round outcomes, including scores, will be returned to the panel for further refinement. The outcomes not excluded after the third round will be taken to a consensus meeting. In this meeting, experts from all relevant disciplines will discuss and finalise the COSIMPREG.

Ethics and dissemination

For this study ethical approval is not required. The systematic review will be published in an appropriate open access reproductive immunology journal. Once the COSIMPREG is finalised, it will be published in an open access reproductive immunology journal, and disseminated at appropriate international meetings, as well as through relevant research and scientific societies. Experts involved in the Delphi study will be asked to give informed consent.

Strengths and limitations of this study.

• The Delphi procedure involves experts from all relevant stakeholder groups including patients.

• The Delphi procedure allows unbiased contributions and is anonymous.

• The systematic review and input of topic experts will assemble and synthesise evidence from a broad, inclusive base.

• This protocol covers a topic which holds enormous potential for future reproductive medicine.

• The intention is to publish the results in open access journals to optimise dissemination.

Introduction

The maternal immune response is instrumental in pregnancy health.¹ Failure of the immune response to adapt and respond correctly to conception and embryo implantation is associated with, and likely plays a causal role in, many complications of pregnancy.² During early pregnancy the maternal immune system must adapt to tolerate the fetus and placenta, both of which express paternal (foreign) as well as maternal histocompatibility antigens. Maladaptation of the maternal immune system is associated with common complications of pregnancy including preterm birth, pre-eclampsia, fetal growth restriction, and recurrent miscarriages.^2–4 Various approaches to immune modulation have been used for several indications in attempts to improve pregnancy outcome.⁵ These approaches include drugs which have effects on the immune system, but also on other pathways. For example, a commonly used therapy is acetylsalicylic acid (aspirin), which is widely used to prevent pre-eclampsia.⁶ In other reproductive disorders, such as recurrent miscarriage, interventions including paternal leucocyte immunisation, progesterone, and steroids have been used, mostly with no demonstrable benefit.⁵ This could be explained by the fact that reproductive disorders, such as recurrent miscarriage, have a multifactorial pathogenesis, and that developing a successful immune modulator depends on selecting appropriate patient groups.

There is a reasonable prospect that given advances in other disease conditions such as oncology⁷ and autoimmune disease,^{8 9} more targeted and effective immune-modulating therapeutic options will emerge for reproduction medicine. Although several preclinical/animal studies show promising results,^10–13 these options must now be tailored to achieve targeted, safe immunotherapy both as prevention and therapy for pregnancy complications. Moreover, since a range of factors, including non-immune related, is implicated in pregnancy complications,¹⁴ selection of the right patients will be essential for the success of therapy.^{5 15}

In the current literature, there is high variability in the reported clinical outcomes and immunological parameters measured.¹⁵ This variability hampers proper comparison across studies and harmonisation of data sets. Therefore, the objective of this study is to develop a core outcome set for studies investigating immune modulation in pregnancy (COSIMPREG).^{16 17} Although immunological studies in pregnancy are usually condition based with associated condition-specific outcomes, the COS developed in the current study will comprise the fundamental outcomes which are considered essential for reporting in all reproductive immunology studies. Specific COSs have now been developed for multiple clinical conditions, with demonstrable benefit for advancing medical care.¹⁸ In cases where immune modulation is studied in a specific clinical condition, then both COS outcomes for the clinical condition and the immune modulation will be collected, and most likely there will be overlap of core outcomes across conditions.

Aim

The aim of this study is to develop a COSIMPREG. We aim to develop COSs for studies both in humans and animals that will be reported separately. We will obtain these COSs by consensus among a group of relevant experts using a Delphi procedure, using a systematic review as the initial input.

Methods and analysis

Overview

To develop a COSIMPREG, a stepwise approach will be used¹⁹:

1. Perform a systematic review to identify reported outcomes for immune modulation already in use.

2. Use a Delphi procedure to develop a preliminary COS with input from the systematic review and experts.

3. Organise a consensus meeting to discuss and finalise the COSIMPREG.

4. Disseminate and promote application of the final COSIMPREG.

This study commenced in December 2017, with an expected completion date of December 2019. The study is registered at the Comet Initiative: http://www.comet-initiative.org/studies/details/1004?result=true.

Patient and public involvement

Patient and public were not involved in the development of this protocol. However, they will be involved and included within the Delphi procedure as expert group. And they will participate in the consensus meeting.

Perform a systematic review to identify reported outcomes for immune modulation already in use

The aim of the systematic review is to identify all outcomes that have been used to date in studies reporting on immune modulation in pregnancy. A secondary aim of this review is to identify potential experts for the Delphi panels. The review will be conducted according to Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines,²⁰ and will be published separately. The review will include all studies, human as well as animal, investigating immune modulation either as therapy or prevention, with the goal of improving pregnancy outcome. A comprehensive search will be conducted using the databases of PubMed, Embase and Cochrane Central Register of Controlled Trials. The search strategy will be different for human and animal studies. We will use free-text words and index terms (Medical Subject Headings (MeSH) for PubMed and Emtree for Embase). See table 1 for the preliminary Medline search strategies for human and animal studies. We will perform the literature search early March 2018. If the selection process extends beyond 6 months, the search will be updated to cover the interim period. No language or date restriction will be applied.

Table 1.

Search strategy

#1	pre-eclampsi*[tiab] OR preeclampsi*[tiab] OR miscarriage*[tiab] OR pregnancy loss*[tiab] OR abort*[tiab] OR pre-term[tiab] OR preterm[tiab] OR growth restrict*[tiab] OR pregnancy fail*[tiab] OR fetal loss* [tiab] OR infertile* [tiab]
#2	‘immunoproteins’[Mesh] OR ‘cytokines’[Mesh] OR ‘immunology’ [Subheading] OR immunomodulation[tiab] OR immune modulation[tiab] OR immunotherapy[tiab] OR ‘immunomodulation’[Mesh]
#3	randomized controlled trial [pt] OR controlled clinical trial [pt] OR randomized [tiab] OR placebo [tiab] OR drug therapy [sh] OR randomly [tiab] OR trial [tiab] OR groups [tiab]))
#4	animal*[tiab] OR mice[tiab] OR mouse[tiab] OR rat[tiab] OR rats[tiab] OR pig[tiab] OR pigs[tiab]OR sheep[tiab] OR goat*[tiab] OR lamb[tiab] OR lambs[tiab]
#5	improve*[Title] OR outcome*[Title] OR loss*[Title] OR treatment*[Title] OR decreas*[Title] OR failure*[Title] OR promot*[Title] OR impair*[Title] OR prevent*[Title] OR induc*[Title] OR restor*[Title] OR rebalanc*[Title]
#6	#1 AND #2 AND #3 NOT #4	Human studies
#7	#1 AND #2 AND #4 AND #5	Animal studies
#8	#6 OR #7	Preliminary search

In order to identify all reported outcomes, we aim to include: (1) randomised clinical trials, open label clinical trials and cohort studies reporting on (2) immune therapy or other interventions targeting the immune response, in (3) pregnant human or animal subjects studying (4) the preventive or therapeutic effect on an adverse reproductive outcome.

Studies will not be included when they do not meet the inclusion criteria, for example: (1) pregnancy outcome reported as secondary outcome; (2) case reports, reviews and expert opinions.

Two reviewers (JRP and FH) will independently screen titles and abstracts of all citations in order to exclude all overtly irrelevant papers. One of the members of the review team (FH) is not involved in obstetric research, and will therefore be unaware of author and journal credentials. Consensus on inclusion will be reached when: (1) both reviewers include a study, (2) agreement is reached after discussion in the case of differing opinions or (3) a third reviewer (SJG) is consulted in the case of persistent disagreement. For potentially relevant papers, the full text will be retrieved and studied in detail, to determine whether the inclusion criteria are met. In case of disagreement, consensus between the reviewers will be reached on discussion, and if necessary through consultation with a third reviewer (SJG). To search for additional studies, reference lists of all included studies and relevant reviews will be checked, conference abstracts will be screened and published protocols without published follow-up data will be identified. If necessary, authors will be contacted.

Two authors (JRP and FH) will independently extract data from the included studies. Data will be extracted on the year of study, study design, study size, study population, human/animal study, reported outcome(s) and authors. The reported outcomes in the included studies will first be summarised into human and animal studies, and thereafter into four categories, namely: maternal clinical outcomes, fetal clinical outcomes, maternal immune parameters and fetal immune parameters. Furthermore, the above categories will be displayed for both preventive and therapeutic immune modulation interventions. The study outcome will have no influence on the extraction of the reported outcomes and parameters. Overlapping outcomes will be collated and reported under a covering term. For each reported outcome the number of times it is reported (absolute and relative) in studies will be shown. This scoring will also be done in the categories mentioned earlier. References will be organised using RefWorks. Data will be collected, entered in a predefined fact sheet and analysed using Microsoft Excel.

Since we aim to include all relevant outcomes and parameters reported to date and we will not discriminate on efficacy of intervention, the included studies will not be assessed regarding their risk of bias, nor will they be graded.

The protocol for the systematic review is not eligible for registration at Prospero as it has no direct health-related outcomes. JRP will be the guarantor of this review. JRP and FH are responsible for selection of studies for inclusion and for data extraction. SJG will be consulted in case of disagreement. For the systematic review, there are no sources of financial support.

The findings of this systematic review will serve three purposes. First, in order to disseminate the results, it will be published in an open access peer-reviewed journal according to PRISMA guidelines.²⁰ Second, the results will be used for the Delphi procedure in order to develop a COSIMPREG. Third, the extracted data regarding authors will help to identify potential experts for the Delphi procedure.

Use a Delphi procedure to develop a preliminary COS with input from the systematic review and experts

To develop a preliminary COS for immune modulation studies in pregnancy, we will use Delphi methodology. In general, the aim of the Delphi method is to obtain consensus on a subject and to develop new knowledge, and this has been applied previously to COS development.^{16 21} In the Delphi process, structured statements are scored by experts on relevance, then these statements are returned to the experts with scores at individual and discipline group level, and this process is repeated until consensus is reached. On average, Delphi procedures are reported to require three iterative rounds.²¹ Since we plan to reach final consensus by adding a consensus meeting at the completion of the Delphi procedure, three rounds of Delphi procedure are expected to be sufficient. All outcomes not excluded through the three Delphi rounds will be taken into the consensus meeting for final approval.

For the Delphi procedure, we will take an inclusive approach and cast a wide net to assemble several panels comprising experts from different professional disciplines, together with a patient/consumer group. To be included on a professional expert panel, members should have worked at least 5 years within their field and/or should have recent publications related to immune modulation in pregnancy, or have a well-known status in a relevant field, and should have adequate English language skills. Experts in obstetrics, paediatrics, laboratory based and clinical immunology, reproduction science, and midwifery will be included on the expert panels. To ensure that all panels have sufficient geographical distribution and to prevent bias, experts will be identified and selected through a range of processes, with a goal to include at least 100 relevant participants. First, potential experts involved in immune-modulating studies will be identified through the systematic review. This will identify potential experts with >5 years of work in this field and with relevant recent publications. Second, we will ask potential panel members to identify other experts and to provide names of other relevant experts (see below). We will ask experts specifically to nominate potential experts in South America, Africa and Asia-Oceania as these are regions that have been under-represented in previous Delphi procedures with an obstetric focus.^{22 23}

Regarding the patient/consumer group selection the procedure is slightly different. We will invite patient and consumer organisations from a range of countries as above to become involved and to nominate appropriate individuals. To ensure geographical diversity in the Delphi procedure, we will include at least 10 experts on each panel (at least 10 paediatricians, at least 10 patients, etc).

As the use of medication or other interventions during pregnancy is dependent on the motivation and understanding of pregnant women, panels will include both healthy pregnant women and women who have experienced adverse outcomes that might reasonably have qualified for prevention or treatment with immune modulation. Women with a history of recurrent miscarriage, preterm birth, fetal growth restriction and/or pre-eclampsia, all complications of pregnancy for which immune modulation is considered as holding promise, will be eligible. To be included within a patient/consumer subpanel, women must have adequate English language skills.

Candidate expert and lay participants will be invited to participate in the Delphi procedure by email in which we will explain the background and goals of the study. Lay participants will be accessed through patient and consumer organisations (eg, the Dutch pre-eclampsia/Hellp foundation/March of Dimes/Perinatal Society of Australia New Zealand) and invitational posters at participating centres distributed around the world.

In the explanatory email, we will use written text supported by a video explaining the need for a set of core outcomes in reproductive immunology, and information on the time commitment and schedule for each Delphi round. The email will also contain a link to accept the invitation, to provide informed consent and to register in the software. Nominated experts will be invited to provide the names of other relevant experts who meet the inclusion criteria, and reasonably should be invited to participate to achieve optimal inclusion. Participants will be asked to not personally contact other potential experts and to not discuss the Delphi procedure, to ensure unbiased input. Responses to the Delphi procedure will be semianonymised, such that participants are aware of their fellow panel members but not of their individual responses. Results returned will include individual expert responses as well as responses on a panel group level.

As not all different panels will include experts in animal studies and since we aim to develop two separate COS documents for animal and human studies, only the reproductive science and immunology panels will be able to contribute to assembling the animal COS.

We anticipate a three-round Delphi procedure to reach consensus on the shortlist of core outcomes. The aim of the Delphi procedure is to eliminate all outcomes that are not fundamental or essential. Experts can only be part of a subsequent round if they complete the former one. In each round, the participants will receive an email with a summary of the response rates and results to date and a link to the next questionnaire. Each round will take approximately 3 weeks. Reminders will be sent to the respondents who have not yet responded, and 2 days before the deadline a final reminder will be sent.

First Delphi round

In the first round, an initial assessment of the relevance of possible outcomes, derived from the systematic review, will be made. Panel members will be asked to score the importance of outcomes on a 9-point Likert scale, following the Core Outcome Measures in Effectiveness Trials (COMET) advice.¹⁹ Items will be ranked and those with a median of at least 7 when a Likert of 9 is used will progress to the next round. To ensure a complete set of outcomes using the input of topic experts and patients/consumers, participants will be asked to ‘rescue’ any outcome missed by the panel’s ranked list of outcomes and which they consider as a core outcome.

Second round

In the second round, the response rate for each panel and the overall response rate will be reported. All outcomes reaching the cut-off threshold from round 1 will be presented again plus outcomes put forward for rescue by at least two participants. For each outcome, the scoring of round 1 will be presented at three levels: (1) at the participants’ individual level; (2) at the level of the expert subpanel and (3) at the level of the other expert panels. These results will be presented graphically in the form of a histogram (as generated by DelphiManager). Panel members’ own individual responses can then be compared against the score of their respective subpanel, and against the score of other subpanels. Participants will be asked to rate the importance of all outcomes again, but now with the knowledge of the scores in round 1. We will again underline in the explanatory text the aim of the study, namely to identify fundamental/essential outcomes to be reported as a minimum set in each study. It will be essential to not be excessively inclusive, in order that a manageable COS is delivered. We will emphasise that for every future reproductive immunology study this will be a minimum outcome set, and that additional outcomes relevant to individual studies can always be added. Furthermore, in this round we poll the panel members for availability to join the consensus meeting as a satellite meeting of another event (see below).

Third round

In round 3, outcomes will not be taken forward from the previous round if more than 70% of the total panel judged the outcome as not essential (scores 1–3 on 9-point Likert scale) and less than 15% of experts regard this same outcome as important (score ≥7 on 9-point Likert scale). All the other outcomes will be presented in round 3.

A preliminary list of outcomes for the consensus meeting will be assembled. To that end, the outcomes retained after round 2 will be presented to the participants. The outcomes will be presented in similar way as in round 2. The reproductive scientists and immunologists will also receive a preliminary list of animal studies core outcomes.

After the third round, all outcomes having a score ≥7 on the 9-point Likert scale in at least 70% of the participants will be taken forward into the consensus meeting as potential COS. Outcomes with more than 70% of the participants judged as less important (score 1–3 on 9-point Likert scale) and less than 15% as important (score ≥7 on 9-point Likert scale) will be excluded. Furthermore, the outcomes not regarded as essential for the COS and also not excluded will be presented at the consensus meeting for further consensus voting.

Organise a consensus meeting to discuss and finalise the COSIMPREG

To finalise the COSIMPREG, we will organise a consensus meeting as a satellite event to an international conference in 2019, most likely to one of the following meetings: Society of Reproductive Investigation, annual meeting of American Society of Reproductive Immunology or International Society for Immunology of Reproduction. This consensus meeting will be divided into a clinical consensus meeting (involving all experts in the human COS) and an animal consensus meeting (involving the reproductive scientists and immunologists only). The Delphi process is expected to take 12 months with final outcome disseminated in 2019. Within this consensus meeting, we aim to have members of each stakeholder group present in person. A full day meeting, with an open and collaborative character, is proposed with an objective facilitator who will actively encourage equal input of all participants and will prevent skewing by strong voices or dominance using nominal group techniques. All outcomes still present after round 3 (of either Likert, so added by rescue) will be presented at the consensus meeting.

To implement the COSIMPREG

After the COSIMPREG is finalised, their uptake and application in studies reporting on immune modulation in pregnancy will be stimulated by publication of both the human and animal COS in an open access, peer-reviewed reproductive immunology journal. Further, dissemination will also be through presentations at appropriate international meetings, through relevant research and scientific societies, and through relevant journals and electronic media channels.

Ethics and dissemination

For the dissemination of the COS, we will use a range of different strategies to maximise awareness and encourage uptake. We will disseminate all possible outcome measures as a systematic review, and publish this in a peer-reviewed reproductive immunology or methodology journal. Then after finalising the COSIMPREG, we will disseminate it through different channels. First, we will publish the COS in a peer-reviewed reproductive immunology journal. Second, we will disseminate the COS at appropriate international meetings, such as reproductive immunology and reproductive sciences meetings. We will furthermore discuss it with patient/consumer organisations with an emphasis on relevance to pregnant women. We will also disseminate the COS through scientific societies and appropriate electronic media.