ADOPTION OF THE WHO ASSESSMENT TOOL ON THE QUALITY OF HOSPITAL CARE FOR MOTHERS AND NEWBORNS IN ALBANIA

Abstract

Aim:

The aim of the adoption process of the “Quality of hospital care for mothers and newborns babies, assessment tool” (WHO, 2009) was to provide the Albanian health professionals of maternity hospitals with a tool that may help them assess the quality of perinatal care and identify key areas of pregnancy, childbirth and newborn care that need to be improved.

Methods:

Four maternity hospitals (one university hospital and three regional hospitals) were selected for the assessment using this standard tool covering over 600 items grouped into 13 areas ranging from supportive services to case management. Sources of information consisted of site visits, hospital statistics, medical records, observation of cases and interviews with staff and patients. A score was assigned to each item (range 0-3) and area of care. The assessments were carried out in two rounds: in 2009 and in 2011. These assessments provided semi-quantitative data on the quality of hospital care for mothers and newborns.

Results:

Data collected on the first round established a baseline assessment, whereas the second round monitored the subsequent changes. The findings of the second round revealed improvements encountered in all maternities, notwithstanding differences in the levels of improvement between maternities, not necessarily linked with extra financial inputs.

Conclusions:

The Albanian experience indicates a successful process of the adoption of the WHO tool on the quality of hospital care for mothers and newborn babies. The adopted tool can be used country-wide as a component of a quality improvement strategy in perinatal health care in Albania.

1. INTRODUCTION

Measurement and assessment of health care performance through indicators and standards is an effective approach for the improvement of quality of care. The aim of the adoption of the “Quality of hospital Care for mothers and newborns babies (QoMNC), WHO assessment tool”, (from here Tool) was to provide Albanian Ministry of Health, key stakeholders and partners at national level, hospital managers and health staff at hospital maternity level with a tool that help them carry out assessments of perinatal health care provided at facility level in an homogeneous and valid way, and ultimately aid them to identify key areas of pregnancy, childbirth and newborn care that need to be improved (1).

The relevance of the adoption of this Tool in the Albanian reality consists in the fact that perinatal quality of care is a key issue to improve maternal and neonatal outcomes in Albania where access to institutional births is nearly universal. 97 percent of deliveries in the five years preceding the Albania Demographic and Health Survey 2008-09, took place in a health facility (INSTAT, 2010). Neonatal mortality changed a little over the 15 years preceding the ADHS 2008-09, 15 deaths per 1000 live births in 1994-1998 and 11 deaths per 1000 live births in 2004-2008 (INSTAT 2010). The slow progress in neonatal mortality indicates for quality gaps in perinatal care.

This tool can be used to assess quality of perinatal care at countrywide level, for a sample of facilities or for a single maternity hospital. as a component of the quality improvement strategy in perinatal health care. It can be used to assess actual clinical practice and quality of care before and after interventions introduced to improve practice and quality of care such as WHO and internationally recommended guidelines into clinical practice as well as for incentives and accreditation schemes.

This standard-based Tool provides a semi-quantitative assessment of the quality of care in a variety of key areas grouped in 13 ones ranging from supportive services to case management. The Tool is built on and adds up to the experience of implementing Making Pregnancy Safer Programme and Effective Perinatal Care training package obtained by WHO regional office for Europe and also in the implementation of the pediatric hospital care assessment tool in more than 20 countries. (1, 2, 3).

Adoption of the tool in question is realized by an Albanian Working Group within the framework of the broader project “Support to maternal, newborn and child health reform” 2008-2012 aimed at improving maternal and child health care in Albania, implemented by Ministry of Health of Albania with support from WHO Albania Country office.

Field testing of the Tool was considered as the heart of the adoption process.

2. METHODS

2.1. Institutional context

Improving maternal and children health and reducing maternal and childhood morbidity and mortality are top priorities of Albanian Heath System. The existence of a quality gap is the most likely explanation for slow progress of neonatal mortality in Albania regardless the high access to institutional births. The assessment of the quality of maternal and neonatal care is a component of the implementation of the WHO Making pregnancy safer strategic framework in the European region, (WHO European Strategic approach for making pregnancy safer) which includes technical support to countries to develop or revise policies, laws, norms, regulations and clinical guidelines, to strengthen pre and in service training, to make an appropriate use of technologies, to establish referral system and to introduce maternal and perinatal audits (4, 5).

As per WHO mandate, the adoption process was commenced upon request by Albania Ministry of Health (MoH) through Biannual Collaborative Agreements (BCAs) and carried out in collaboration between MoH and a variety of partners in the framework of the project” Support to maternal and child health reform” 2008-2012. Implementation and scaling up of the process in question has been insured through partnership among MoH, WHO, Spanish Agency for International Development Cooperation.

2.2. Conceptual approach and methods

The adoption process was guided by the aim to adopt a Tool usable and relevant to the epidemiology and mothers and newborn babies health care structures and organization and as well as to local needs and resources in Albania.

Adoption process considered that 1) the tool was designed to assess uniformly/similarly the quality improvement in hospitals of different levels from small district maternity hospitals to tertiary care centres, 2) the national team when planning the assessment, should identify in advance which sections of the tool will be used in hospitals of different levels. So, if one chapter or specific items of a chapter are not applicable in one hospital, it will be sufficient to classify the item/s as not applicable.

The heart of the adoption process is the use of this Tool in assessing the quality of perinatal care in 4 maternity hospitals two times within a period of about 3 years (January, 2009, October 2011).

The two assessments in Albania, among the first ones (6) to use the maternal and newborn care quality assessment Tool, were used to assess and monitor the baseline situation in 2009 and subsequent improvements in 2011. Adoption process of the Tool in Albania is among the first experiences in European region of WHO, as well.

2.3. Hospital sample

The same four hospitals were assessed in 2009 and 2011: three regional maternity hospitals and one tertiary referral maternity hospital with Neonatal Intensive Care Unit. The requirement for this selection was the inclusion of three regional maternity hospitals and one referral/teaching maternity hospital.

2.4. Assessment tool

The Tool (1) through a peer assessment collects data from the main areas which have a great impact on maternal and newborn mortality and serious morbidity, as well as on maternal and neonatal wellbeing in order to identify the areas with poor or substandard care ensuring the involvement of managers, staff and patients at facility level and Ministry of Health in identifying the priority areas and/or actions for interventions. The Tool is a structured indicator and standard based checklist covering 13 main areas: Hospital support system, Care for normal labor, Caesarian section, Maternal complications, Routine neonatal care, Sick newborn care , Advanced newborn care, Emergency Care, Infection control and supportive care, Monitoring and follow up, Guidelines and auditing and Access to hospital care. Each area includes from 10 to over 50 items, totally around 600 items. The majority of items are devoted to case management (6). Semistructured interviews with mothers provides information on the quality of the contact between the mothers and their caregivers and quality of information and counsels provided to them, as well as it explores what can be done to improve their well-being and that of their baby.

Involvement of hospital managers and staff in identifying the key areas/actions for interventions is ensured from semi- structured interviews with them. These interviews explore the knowledge and the use of guidelines, provide information on organizational issues, on critical areas for interventions and on immediate mid and long term improvements that should happen.

The Tool uses four sources of information: hospital statistics, patients files, direct observations of case management, interviews with mothers and staff.

Each item is evaluated with the information gathered by different sources to reach an overall score to each main area of care. For scoring, numbers from 3 to 0 are attributed to each item based on the following criteria, 3 = good or standard care corresponding to international standards (no need for improvement or need for marginal improvement); 2 =substandard care but no significant hazard to health or violation of human rights (need for improvement); 1 = inadequate care with significant health hazards (need for substantial improvement); 0 = very poor care with severe hazards to the health of mothers and/or newborns (need for very substantial improvements of the structure, organization, procedures and case management related to specific items or to the whole area).

The Tool is built on and adds up to the experience of implementing Making Pregnancy Safer Programme and Effective Perinatal Care training package obtained by WHO regional office for Europe, to the previuos experience on criterion base audits of obstetric care (7) and also in the implementation of the pediatric hospital care assessment tool in more than 20 countries (8, 9).

The reference standards for the case management assessment of the Tool are the WHO Integrated Managament of Pregnancy and Childhood (IMPAC) manuals of the global Making Pregnancy Safer Programme (WHOIMPAC Managing Complications in Pregnancy and Childbirth, WHOIMPAC Pregnancy, childbirth, postpartum and newborn care) and the Effective Perinatal Care (EPC) training package developed by the WHO Regional office for Europe and JSI/USAID (WHO Regional Office for Europe Effective Perinatal Care training package) (10, 11).

2.5. Assessment team

The assessment team was composed of international professionals and national ones covering all key disciplinary backgrounds including obstetrics, midwifery and neonatology.

2.6. Assessment methods

First, the Tool was translated into Abanian. The managers of the hospital selected for the assessment was sent the information with the request to fill in the sections of the Tool covering the general information on volume of work, delivery indicators and hospital support systems (availability of drugs, supplies and equipment).

The visit started with an initial briefing to key staff and managers on the objectives and methods of the assessment that included all maternity hospital services from pharmacy to laboratory and clinical units from admission to intensive care.

One assessing visit lasted one and a half day. The assessors met after the visit to discuss the findings, award scores and to prepare the meeting at the end of the assessing visit with the participation of hospital managers, heads of departments/units and support services to present and discuss main findings and recommended actions.

2.7. Analysis

Considering the qualitative nature of the assessment, no statistical analysis was done with exception of calculation of average scores award to each main area of care. A detailed analysis of the findings was performed per each institution. The findings of the first assessment and the second one were aggregated and compared as good evidence in favor of the adaptation process.

Between two assessments, adaptation process applied participatory approach through field visits, direct observations and interviews with health care providers, directors/chiefs and mothers in 4 Maternity Hospitals assessed, including here training of internal assessors carried out during the period February-September 2011.

Three structured workshops with key informants from Ministry of Health, National Institute of Public Health, Medicine Faculty, National Center of Quality, Safety and Accreditation of Health Institutions, Health Insurance Institute, University Obstetric-Gynecologic Hospitals and from all Regional Hospital Maternities were held in the period March-May 2011. Consultancy with the authors (WHO experts) of the original tool evaluated the correctness and validity of the adaptations.

During the adaptation there were compared Tool’s items with the reality of Albanian maternities clinical practice: case management, hospital records, patient’s file, hospital statistics, drugs, equipment and lab test available. Table number 1 shows the main steps of the tool adoption process.

Table 1.

Tool adoption process

3. RESULTS

The assessments covered facilities providing care to on average of 23,5% in 2008 to 26% in 2010, of the total number of deliveries per year respectively 36251 and 33856. The number of deliveries per hospital ranged from 1081 to 4138 in 2008 and from 1198 to 4222 in 2010.

Overall the quality of care was found substandard in all areas in two assessments. Table 2 shows numbers of maternity hospitals with standard care and average scores in the areas covered by the assessment in 2009 and 2011. This tables shows that the overall average score per each main area has increased for 8 out of 12* ones and decreased slightly for 4 areas. Overall comparison of findings per each area between two assessments is presented on Figure 1.

Table 2.

Numbers of maternity hospitals showing standard care and average scores in the areas covered by the assessment in 2009 and 2011

Figure 1.

Overall comparison of findings per each area between two assessments.

The overall average score of case management related areas (Normal labor and delivery, Caesarean section, Maternal complications, Routine neonatal care, Care of sick newborn babies, Infection control and supportive care, Use of guidelines and auditing practices, Monitoring and follow-up and Access to hospital care) in 2009 go from 1 to 2,47 and in 2011 from 1,5 to 2,5.

In 2011 there were no areas with an overall average score under 1.5, infrastructural and procedural related areas of care and case managements ones.

Overall Neonatal care (Routine neonatal care, Sick newborn care and Neonatal Intensive care) in 2011 scored better (>2) in comparison with other areas of care.

Overall Neonatal care including Routine neonatal care, Sick newborn care scored better in 2011 than in 2009 (2,4 versus 2,1 respectively).

Maternal care including Care for normal labor and Maternal Complications, scored as a whole equally in two assessments (1,5 in 2009 and 1,5 in 2011).

In 2011 the neonatal care scored as a whole better than maternal care (2,4 versus 1,5 respectively) than in 2009 (2,1 versus 1,5, respectively) and Advanced newborn care was classified as standard (scored 3).

Overall, the average scores were slightly higher in the referral hospital with respect to other ones in 2011 than in 2009 (respectively 2,23 versus 1,66, and 1.9 versus 1,43.

In 2009 only one hospital (the referral one) had two areas of care classified as standard. In 2011 two Maternity Hospitals (one regional Maternity Hospital and one referral one) had each of them three areas of care classified as standard.

The tool used in 2009 was the Draft of the Tool and assessed 12 areas of care. The tool used in 2011 was the final version, which assesses as a separate area of care namely the “Advanced newborn care” where applicable.

The average scores for each area in all 4 maternity hospitals are shown in Table nr.3 In the Maternity hospital S with exception of one area (Emergency care) the rest of areas of care scored better than in 2009. Three main areas were classified as standard. In this Maternity hospital was found a slight decrease of Caesarean section rate (from 23,6% to 23,2%) in comparison with other maternities where number of SC increased.

Table 3.

Average scores for each area of care in 4 maternity hospitals in 2009 and 2011

Overall, there is a slight improvement in 2011 with respect to 2009. The highest proportion of the overall average score between two assessments at hospital level was for Maternity Hospital S 1,4 (2,25 in 2011 versus 1.6 in 2009).

In most but not in all maternity hospital assessed was reported a slight decrease in perinatal and neonatal mortality, but the quality of data is substandard and contradiction exists with information collected locally.

Main infrastructural and procedural issues identified consistently in 4 maternity hospitals were: lack of national guidelines and local protocols for doctors and midwives/nurses for major conditions, lack of rational policy for antibiotic use for treatment and prophylaxis, in adequate provision and utilization of equipment, Inappropriate indications for caesarean section, lack of continuity of care and flow of information between primary care and hospital, insufficient communication and information to mothers and their families.

Main case management issues identified were inadequate provision and utilization of equipment, inappropriate indications for caesarean section, Lack of national clinical guidelines and local protocols on indications and procedures, Inappropriate use of technology and interventions, lack of continuous medical/nursing education and supervision on neonatal resuscitation and Neonatal Intensive Care Unit knowledge and skills; Referral criteria and systems and communication between levels of care not appropriately in place.

3.1. Beneficiaries’ perceptions

Interviews to 14 women in 4 maternity hospital during the II assessing visits (pregnant women or new mothers or mothers of admitted children) provided information on a number issues including access to hospital, coordination and continuity of care, perceived quality of care and information received at admission during hospital stay and at discharge, physical comfort, emotional support, involvement of mothers and families in decision taking.

The overall sense of improvement in four hospitals was expressed clearly by the mothers gave birth to a previous baby before 2009 (first assessment). The interviews with mothers confirmed the findings of the assessment: the best perceived quality of care from mothers was found in the maternity hospital overall scored higher and in areas of care scored higher and classified as standard (routine neonatal care, sick newborn care, Intensive neonatal care and information to mothers and mother and baby centered care). Comparing the information provided by mothers interviewed in 2009 with that provided in 2011 there is an improvement perceived and experienced by mothers particularly regarding the general environment and friendly care for both mothers and babies (presence of the partner during labor and delivery, mothers were allowed to take different positions, to eat and drink during labor and delivery), general improvement of some basic amenities and services for mothers and babies, more and better informed mothers, adequate information related to breastfeeding. But this reality wasn’t present in four hospitals. The lack of and/or the need for more adequate information, insufficient support during labor and delivery, and inconsistent support for early initiation and continuation of breastfeeding and lack of companion during labor remained the most common complaints mentioned by the interviewed mothers.

3.2. Health professionals’ perceptions

During the assessment 26 health professionals were interviewed: 7 obstetricians (2 chief of hospital maternity), 6 midwives, 6 neonatologists (3 chief of the neonatal ward, one chief of NICU), 7 neonatal nurses (1 Head nurse of neonatal ward, 2 NICU nurses). These interviews provided interesting insights into issues related to quality of care. The information provided from these interviews confirmed the findings of the assessment and took into light a number of issues faced by the staff. Critical issues for quality, were consistently identified, such as: lack of national guidelines and local protocols and the not regularly use of the existing ones; lack of an effective collaboration between different levels of care: primary health care, public and private out-patient services and hospitals; insufficient communication with and information to mothers and their families; lack of staff in certain areas (neonatology, doctors and nurses); insufficient training both on the job and through scheduled Continuous Medical Education activities.

The majority of the interviewees expressed positive attitude regarding the participatory peer to peer approach. They acknowledged the novelty of the method in comparison with inspectional assessment experienced previously. They acknowledged the role of the assessment Tool in identifying areas and specific organizational issues, practices, procedures, and protocols that needed to be improved. Some comments emphasised that implementing the recommendations of the first assessment “The way of working has changed: team work, communication and attitude to the patients; demedicalization of labour, more safety during labour”,

Useful suggestions for further improvements were expressed clearly:“We need national guidelines officially approved by Ministry of Health to be used as a guider in the local adaptation process”

“Mothers should be informed better on delivery before she comes to the maternity hospital from public and private primary health care structures offering care for pregnant women: what is offered in the maternity hospital, about breastfeeding and contraception”

“Public and private primary health care structures offering care for pregnant women should be also part of such exercises aiming at improving maternal and newborn health care”.

Some comments emphasised that the Tool has been not only an assessment tool, but also a guide to practice effective perinatal care principles and standards on day by day basis.

Some professionals emphasized the new, non judgemental supportive approach and the fact that the use of the Tool to assess areas that require the collaboration of various teams and services was promoting a more collaborative approach among professionals.

3.3. Maternity Hospitals’ responses to Tool Monitoring

Along with quantitative data, reassessment revealed qualitative information on actions taken in response to monitoring with the Tool. The follow-up reassessment (2011) of these actions confirmed improvements in both patient management and outcome (for example, a slight decrease of perinatal and neonatal mortality in four Hospitals). The responsiveness of the hospital is the best measure of the value of an indicator (13) in our case of the Tool. Overall, the actions induced by the tool monitoring are grouped as follows (14) in the Table 4 Maternity hospitals’ responses to tool monitoring.

Table 4.

Maternity hospitals’ responses to tool monitoring. *Babies A Newborn Health Management Information System: www.care.org

The assessment process led to the identification of key quality gaps and priority actions at Maternity hospital level as well as national level according to four main health system functions: stewardship and governance, resource generation, financing and service delivery (WHO, World health report 2000), (WHO European ministerial Conference on health Systems, 2008) (15, 16).

4. DISCUSSION

The assessment carried out in a sample of 4 maternity hospitals in Albania in 2011, overall showed that the quality of care for mothers and newborn babies was substandard in all the areas explored, but comparing with the findings of the assessment in 2009 there are some improvements happened in all Maternity Hospitals (4) assessed, in the interval (less than 2 years) between two assessments.

Overall, there has been a generalized improvement in the environment and friendly care for both mothers and babies. There was a general improvement in some basic amenities and services: availability of cold and warm water, toilets and basic supplies such as soap and antiseptics; there have been improvements in the availability of essential drugs and laboratory support, rooming-in almost a rule in all maternity wards.

Findings of the interviews with mothers confirm an overall positive direction of change regarding the communication and information to them and their families as a consequence of a better recognition of such rights among managers and health professionals and higher awareness of their implications for health and wellbeing of both mothers and babies. The findings showed that improvement are not the same for all Maternity Hospitals but in some cases substantial improvements had taken place in key aspects of case management (care during normal labour and delivery, and care for sick newborn babies), with improvements of appropriate procedures and abandonment of inappropriate ones. For example, percentage of episiotomy decreased, enema is not provided routinely, choice of position during labour and delivery is allowed.

In most, but not all centres, a slight decrease was reported in perinatal and neonatal mortality, but quality of data is substandard and some contradictions exists with information that is collected locally.

It was noticed that there was a slight decrease in the number of caesarean sections in one Maternity Hospital S (23,6% to 23,2%), in comparison with the rest where the number of caesarean sections was increased.

Areas (such as essential neonatal care, case management of sick newborn care and neonatal intensive care and) related to newborn baby care, still obtained better scores, compared to other areas: all over scored 2 and more.

In two maternities the poorest performances were observed in key aspects such as the management of normal labour and delivery, CS, maternal complications, infection control, availability and use of appropriate guidelines, monitoring and follow-up, audit systems.

Infrastructure, staffing, equipment, availability of essential drugs and supplies and provision, distribution and utilization of equipment were often found substandard and sometimes inadequate.

However, as previously reported in CEE/CIS countries and elsewhere (8, 16), these deficiencies did not seem to represent the main limiting factor to ensure safety, effectiveness, and patient responsiveness, i.e. the main dimensions of quality care (17).

The findings of the assessments revealed widespread and important weaknesses in cross-cutting components of care e.g. availability and the use of updated national guidelines and local protocols, case reviews and audits, which do not require sophisticated infrastructure or equipment.

Many of the identified deficiencies can be effectively addressed at local level, others, such as those regarding perinatal referral system, standards and norms, national clinical guidelines, equipment, pre-service and intraining, require primarily action at MoH and/or at government level.

The quality gap in the care for mothers and newborn babies is increasingly recognized as a priority (18, 19). Many systems for quality assessment and improvement, certification and accreditation systems included, have been proposed, implemented and evaluated, in Europe and elsewhere (19, 20). The Table 5 shows the differences between these systems and participatory approaches based on peer review that the Tool applies.

Table 5.

Differences between systems for quality assessment and the tool

The table shows that the use of the Tool seems to be more promising because the financial implications are less, sources of information are daily clinical set of files and the driving forces are the motivation of professionals through their involvement and developing their professionalism as a solid base for continuous quality improvement (6).

The limitations of the approach are related to the facts that the comparability of the findings among health facilities cannot be completely guaranteed first, nevertheless the scoring system is based on clear criteria and the continuity of the evaluation team was ensured; second, the sample of hospitals was based on Ministry of Health definitions, with possible bias towards the better performing institutions. However, the main purpose of the assessment was not to guarantee the maximum of accuracy and reproducibility, but rather to prompt quality improvement cycles through a participatory identification of key deficiencies and relevant actions to address them.

5. CONCLUSIONS

Official endorsement of the Tool in Albania was based on “live” experience of its use. There were experienced into Albanian reality of maternity hospitals the appropriateness and relevance of the Tool: case management direct observation, hospital records, patient’s file, hospital statistics, drugs, equipment and lab tests available.

The use of the Tool in two Assessments, improvements inspired by the measuring process was the best supporting evidence of the validity of the use of this Tool in hospital care for mothers and newborn babies in Albania.

The findings revealed the fact that the improvements after first Assessment were encountered in 4 Maternity Hospitals but the level of improvement is different amongst them. These differences are likely to be linked with different organizational ways of working, with the variable level of responsibility and readiness of maternity/hospital leadership to internalize the quality continuous improvement requirements and to inspire and maintain a supportive environment for the working group spirit and for recognizing and creating the due space to each member of the team providing care for mothers and newborn babies. The assessing process revealed that improvement of clinical management of normal childbirth and complicated cases is just starting and the pace of improvement is slow because it involves changing practices and attitudes of staff, which usually requires a longer time to be implemented.

The experience and the positive trend towards the quality improvement of hospital care for mothers and newborn babies, assessed using the Tool, showed that this approach is a feasible and effective way to monitor quality improvements, build awareness on quality issues and promoting change at facility level but at the same time at system level.

During the adoption of the Tool and/or assessment/ reassessment of the quality of 4 Maternity Hospitals (over a two- year period), national capacities have been strengthened which is an added value of this process for scaling up the approach to all maternity hospitals.

The experience of the use of the Tool showed that it’s crucial a) to adopt a multiprofessional approach, which necessarily should include midwifes and nurses, b) to involve hospital directors and staff in a peer-to-peer supportive approach and c) to support the development and implementation of action plans at facility level.

Official endorsment of the Tool, created the institutional conditions to link the use of this approach with the certification/accreditation and/ or continuous improvement quality process.

The supportive and participatory approach applied by the Tool, was particularly welcome and confirmed by professionals who had previously experienced only bureaucratic controls and sanctions.

Improvements identified and measured, “live” experience of 4 Maternity Hospitals in identifying key areas of pregnancy, childbirth and newborn care that need to be improved and the experienced validity of the use in the quality continuous improvement of the Tool in question, are the main reasons that the adopting process was successful.

Conflict of interest

None declared.