Application of Agree II Instrument for Appraisal of Postpartum Hemorrhage Clinical Practice Guidelines in Bosnia and Herzegovina

Abstract

Introduction:

Federal Minister of Health and Minister of Health and Social Welfare of the Republika Srpska as a Governmental health authorities in Federation of Bosnia and Herzegovina (FBiH) and the Republika Srpska nominated/appointed health professionals as their representatives to a multidisciplinary Guidelines Adaptation Group (GAG). This group started with its work in September 2015. The main purpose of the guidelines development exercise is to develop guidelines with worldwide recognized methodology for clinical guidelines development and adaptation. At the end of this consultancy, GAG would have develop a clinical practice guideline on Postpartum hemorrhage (PPH) through the adaptation method, starting from published international clinical guidelines and adapting it according to the country specific requirements.

Methodology:

During the process of identifying the best guideline for adaptation, the GAG had to pass several steps. One of the crucial steps was to identify the questions related to clinical practice and health policy for which answers are needed to be addressed by the guideline. These questions included relevant issues regarding the topic area such as diagnosis, prognosis, intervention, service delivery, and training. After that, six guidelines have been researched by the six members of the GAG to see if they answered the identified questions. Evaluating the methodological quality of the selected clinical guidelines was a second essential step before deciding which ones could best fit the needs and interests. AGREE II instrument has been chosen as methods for evaluating clinical guideline quality and appropriateness. Four appraisers conducted the assessment of each of the selected guidelines for PPH. All appraisers passed the training for the AGREE II instrument before conducting appraisals, as recommended by the AGREE collaboration. Each of the four guidelines was rated independently with the AGREE II tool by each appraiser.

Results:

The highest score was obtained by the WHO recommendations for postpartum hemorrhage guidelines and each of four assessors recommended it to be adapted and further implemented.

Discussion:

In spite of several serous attempts and two Agencies for quality improvement in BiH the whole process of guideline adaptation is still in the beginning and therefore the establishment of a strong connection with similar institutions and organizations from our region and the rest of the world is very important. Cooperation and collaboration across institutions and countries and key stakeholders have potential to significantly improve the validity and quality of the adapted guidelines and to positively influence implementation.

1. INTRODUCTION

Evidence based clinical practice guidelines (CPGs) are not routinely used in Bosnia and Herzegovina (1). Although evidence-based medicine is widely accepted in the developed world as a contemporary approach to solving medical problems (although not always without resistance), its significance and implementation in developing countries still needs to be evaluated, and in the majority of these countries is still in its initial phase (2).

Clinical practice guidelines (CPGs are based on the best possible scientific evidence (3), expert opinion and professional consensus and they take into account patients and all key stakeholders (4). In Bosnia and Herzegovina CGPs have advisory character and as such have to be flexible so as to reconcile the most recent scientific and clinical achievements with what can realistically be achieved in the health sector of BiH. Their development must be sustainable process, taking into account the limited resources. CPGs are developed by multidisciplinary teams of health professionals and other stakeholders fostering partnerships, cooperation, active participation and fair representation of all stakeholders.

In 2013, UNFPA Regional Office for Eastern Europe and Central Asia in partnership with the East European Institute of Reproductive Health (EEIRH) and the Royal College of Obstetricians and Gynecologists (RCOG) started a regional initiative on building capacity of local institutions to develop/adapt and implement evidence-based clinical guidelines as an integral part of improving quality of care in sexual and reproductive health (SRH). The purpose of the initiative is to contribute to improving quality of care of SRH services through harmonization and institutionalization of development/adaptation and implementation of guidelines. BiH participated in 2013 with five representatives from various health institutions at the regional training for adaptation/development and implementation, including monitoring and evaluation, of evidence-based clinical guidelines.

UNFPA Country Office for BiH supports the Federal Ministry of Health and Ministry of Health and Social Welfare of the Republika Srpska in developing/adapting guidelines on SRH. Both ministries mentioned above have requested UNFPA and other development partners to provide technical support in developing/adapting new guidelines/protocols. To respond to this request, UNFPA Country Office for BiH has conducted training in July 2014 for developing the capacity of specific institutions to develop, adapt, review and monitor implementation of guidelines and protocols on SRH, taking into account different administrative structures in BiH (two entities: Federation of Bosni and Herzegovina and Republika Srpska and one district: Brcko district). The UNFPA and its implementing partner in BiH Partnership in Public Health plan to support Guidelines Adaptation Group (GAG) to develop clinical guidelines for Postpartum Hemorrhage with the ultimate purpose/aim of reducing the maternal mortality in the country.

Postpartum hemorrhage (PPH) is the most common form of obstetric hemorrhage and is a leading cause of maternal morbidity and mortality. Approximately 30% (in some countries over 50%) of direct maternal deaths worldwide are due to hemorrhage, mostly in the postpartum period. Primary PPH is defined as excessive bleeding in the first 24 hours post birth, a blood loss in excess of 500 ml after vaginal birth and a blood loss in excess of 1000 ml after caesarean section (CS). PPH can be minor (500-1000 ml) or major (more than 1000 ml). Major can be divided to moderate (1000-2000 ml) or severe (more than 2000 ml). Secondary PPH is abnormal or excessive bleeding from the birth canal between 24 hours and 12 weeks postnatal.

There is no single definition for PPH. Diagnosing PPH is an emergent situation most commonly through estimation of volume of blood loss and changes in the hemodynamic state. The World Health Organization`s International Classification of Diseases (ICD-10) defines postpartum hemorrhage as “hemorrhage after delivery of fetus or infant. Most cases of PPH have no identifiable risk factors therefore vigilance and early recognition of hemorrhage is required. Risk factors include suspected or proven placental abruption, known placenta previa, multiple pregnancy, preeclampsia/gestational hypertension; previous PPH, obesity, anemia; retained placenta, operative vaginal delivery, mediolateral episiotomy, prolonged labor, big baby (>4kg), pyrexia in labor, age (>40 years, not multiparous).

Federal Minister of Health and Minister of Health and Social Welfare of the Republika Srpska as a Governmental health authorities in Federation of Bosnia and Herzegovina (FBiH) and the Republika Srpska nominated/appointed health professionals as their representatives to the members of multidisciplinary working group named Guidelines Adaptation Group (GAG). GAG started with its work in September 2015. The main purpose of the guidelines development exercise is to develop guidelines with worldwide recognized methodology for clinical guidelines development and adaptation. At the end of this consultancy, GAG would have develop a clinical guideline on Postpartum hemorrhage through the adaptation method, starting from one or more published international clinical guidelines and adapting it according to the country specific requirements. The Project was planned to be executed through the following phases: Establishment of Steering/Oversight Committee for the guidelines programme, Topic selection, Establishment of the Guideline Adaptation Group (GAG), and Guideline adaptation. The value and usefulness of the guidelines will be evaluated using the AGREE II instrument.

Agency for Quality and Accreditation in Healthcare in Federation of Bosnia and Herzegovina (AKAZ) and Agency for Certification, Accreditation and Health Care Improvement of the Republic of Srpska (ASKVA), are institutions in FBiH and the Republika Srpska for defining, measuring, monitoring and improving standards of clinical practice and management of services in Bosnia and Herzegovina (BiH). AKAZ is also a federal center for the definition, evaluation and dissemination of evidence based clinical guidelines for effective and efficient clinical practice (5).

2. METHODOLOGY

During the process of identifying the best guideline for adaptation, the Guideline Adaptation Group (GAG) has to pass a several steps. One of the crucial steps was to identify the questions related to clinical practice and health policy for which answers are needed to be addressed by the guideline. These questions included relevant issues regarding the topic area such as diagnosis, prognosis, intervention, service delivery, and training. Thus, the GAG members identified five key group of questions related to clinical practice (prevention and treatment) in postpartum hemorrhage consists of 37 questions:

• Definition of the postpartum hemorrhage (minor PPH and major PPH);

• Risk factors for postpartum hemorrhage (PPH) (a) Etiologic category and process and 2. Clinical risk factors) identifications and how they can be minimized;

• PPH prevention (use of uterotonics for the prevention of PPH);

• PPH treatment (Before admission in the hospital and transportation and during the admission in the hospital);

• Organization of the care of postpartum hemorrhage.

After that six guidelines (RCOG GTG 52 Prevention and Management of Postpartum Hemorrhage 2011, FIGO postpartum hemorrhage low settings, 2012, QUEENSLAND Clinical Guidelines-Primary postpartum hemorrhage 2013, WHO recommendations for postpartum hemorrhage, 2012, RANZCOG Management of Postpartum hemorrhage, 2015, Royal College of Physicians of Ireland Postpartum Hemorrhage, 2012) have been researched by the six members of the GAG to see if these guidelines answered the identified questions. The results showed that three guidelines should be evaluated using the AGREE II methodology: RCOG GTG 52 Prevention and Management of Postpartum Hemorrhage (2011), WHO recommendations for postpartum hemorrhage (2012), and Royal College of Physicians of Ireland Postpartum hemorrhage (2012).

Evaluating the methodological quality of clinical practice guidelines is essential step before deciding which ones could fit bests our needs and interests. AGREE II instrument has been chosen as methods for evaluating clinical guideline quality and appropriateness.

The AGREE II tool encompasses 23 items in six domains:

1. Scope and purpose (3 items),

2. Stakeholder involvement (3 items),

3. Rigor of development (8 items),

4. Clarity of presentation (3 items),

5. Applicability (4 items), and

6. Editorial independence (2 items).

The domain scope and purpose assesses whether the guideline describes its overall objective and target population clearly. A guideline should entail a clear definition of the target users, as well as demonstrate that the views and preferences of the target population (e.g. patients, public) have been sought and that the guideline development group includes all relevant professional groups which are assessed in stakeholder improvement.

The next domain assesses the rigor of guideline development, encompassing systematic literature searching methodology, transparency of evidence-gathering process and whether evidence is explicitly linked to the recommendations. It also asks if the guideline has been externally reviewed prior to publication and if an updating strategy has been documented.

The fourth domain examines clarity of guideline presentation. Recommendations must be specific, unambiguous and easily identifiable. Different options for management of the condition or health issue should be clearly presented.

Applicability examines whether guidelines describe facilitators and barriers to application and if they provide advice and/or tools on how the recommendations can be put into practice. It also assesses resource implications for guideline application and monitoring and/or auditing criteria. Transparency of guideline funding bodies and conflicts of interest were examined in editorial independence (6).

Each item is rated from 1 (strongly disagree) to 7 (strongly agree). Detailed criteria for each item are available within the AGREE II tool to assist the appraiser (1)]. The appraisers were asked to provide comments to justify their rating. They also gave an overall assessment of the guideline from 1 (lowest) to 7 (highest) and were asked to state if they would recommend the guideline, recommend it with modifications or not recommend it.

Average appraisal scores were calculated for each appraiser by taking the average rating (1-7) for all items of a single guideline. Percentages for each domain were then calculated, this was done by adding all four appraiser ratings (1-7) of items within a single domain (obtained score) and scaling by maximum and minimum possible domain scores and converting to a percentage.

Four appraisers conducted the assessment of each of the selected guidelines for PPH. All appraisers passed the training for the AGREE II instrument before conducting appraisals, as recommended by the AGREE collaboration. Each of the four guidelines was rated independently with the AGREE II tool by each appraiser. Appraisers did not communicate or confer with each other during the appraisal process. The results were shared anonymously among the authors.

3. RESULTS

Domain 1. Scope and Purpose is concerned with the overall aim of the guideline, the specific health questions, and the target population (items 1-3). Those three items are: The overall objective(s) of the guideline is (are) specifically described, 2. The health question(s) covered by the guideline is (are) specifically described and 3. The population (patients, public, etc.) to whom the guideline is meant to apply is specifically described. Each of four assessors rated each item from 1 to 7 as shown in the Table 1 for all three chosen guidelines.

Table 1.

Quality score for domain 1, WHO recommendations for postpartum hemorrhage

Quality score is calculated for each of the six AGREE II domains. The six domain scores are independent and have not been aggregated into a single quality score. Domain scores are calculated by summing up all the scores of the individual items in a domain and by scaling the total as a percentage of the maximum possible score for that domain according to the following formula: Obtained score - minimum possible score / maximum possible score - minimum possible score x 100.

Results for the domain 1, WHO recommendations for postpartum hemorrhage are the following:

Quality score was calculated for all six domains and all tree guidelines and results of the appraisal are presented in Table 2.

Table 2.

Results of the appraisal

The highest score obtained WHO recommendations for postpartum hemorrhage guidelines and each of four assessors recommended it to be adopted and further implemented.

4. DISCUSSION

Since its original release in 2003, the Appraisal of Guidelines for Research and Evaluation (AGREE) Instrument has advanced the science of clinical practice guidelines (CPG) appraisal and quickly became the international gold standard for PG evaluation and development. The AGREE Instrument has been translated into many languages, endorsed by several organizations e.g., National Institute for Health and Clinical Excellence, and used by many development groups e.g., WHO Advisory Committee on Health Research (7).

Health professionals should choose among many various different CPG which sometimes could be even conflicting. They may offer concise instructions on which diagnostic or screening tests to order, how to provide medical or surgical services, how long patients should stay in hospital, or other details of clinical practice. Flawed clinical guidelines harm practitioners by providing inaccurate scientific information and clinical advice, thereby compromising the quality of care. They may encourage ineffective, harmful, or wasteful interventions. Even when guidelines are correct, clinicians often find them inconvenient and time consuming to use (8). Conflicting guidelines from different professional bodies can also confuse and frustrate practitioners (9).

In Bosnia and Herzegovina health professionals at First BH Cochrane Symposium defined additional challenges in connection to the development and implementation of CPGs:

• Poor knowledge and importance of evidence-based medicine and Cochrane Database of Systematic Reviews;

• Poor knowledge of literature search in order to seek out or look for the evidence;

• Poor knowledge of methods of adaptation;

• Weak organizational culture and the weak professional structures;

• The lack of a consistent structure for the presentation, adaptation/adoption and implementation of guidelines;

• The lack of serious financial support for the development of CPGs;

• Lack of incentives to implement CPGs into practice and quality improvement (10);

• Lack of understanding and resistance to accept whole process of development, adaptation and implementation of CPGs;

Credible system of clinical practice guidelines (CPG) development/adaptation supported by key stakeholders in healthcare system including patient organizations and professionals associations.

Efficient process of development/adaptation includes: Well defined development/adaptation of CPGs; evaluation of CPGs; dissemination and implementation of CPGs; time for updating CPGs and ratification of CPGs from professional societies. Development of adapted evidence-based clinical guidelines in developing countries such as Bosnia and Herzegovina represents a very important way to apply international guidelines in a culturally competent way in diverse, transitional settings as BiH is, in order to promote and improve the quality of health services. In spite of several serous attempts and two Agencies for quality improvement in BiH, AKAZ and ASKVA the whole process of guideline adaptation is still in the beginning and therefore the establishment of a strong connection with similar institutions and organizations from our region and the rest of the world are essential. Cooperation and collaboration across institutions and countries and key stakeholders have potential to significantly improve the validity and quality of the adapted guidelines and to positively influence implementation.