Experiences from the field: maternal, reproductive and child health data collection in humanitarian and emergency situations

Abstract

Background

Humanitarian emergencies can disproportionately affect women of reproductive age, and children. Good data on reproductive maternal, newborn and child health (RMNCH) are vital to plan and deliver programmes to address RMNCH needs. There is currently a lack of information regarding the availability, use and applicability of data collection tools.

Methods

Key informant interviews (KII) were conducted with participants with experience of data collection in humanitarian settings, identified from relevant publications. Data were analysed using the thematic framework approach.

Results

All participants reported challenges, especially in the acute phase of an emergency and when there is insufficient security. Four common themes were identified: the importance of a mixed methods approach, language both with regard to development of data collection tools and data collection, the need to modify existing tools and build local capacity for data collection. Qualitative data collection was noted to be time consuming but considered to be important to understand the local context. Both those who have experienced trauma (including sexual violence) and data collectors require debriefing after documenting these experiences.

Conclusions

There were numerous challenges associated with data collection assessing the health status of, and services available, to women and children in humanitarian settings, and researchers should be well prepared.

Introduction

The collection of data in humanitarian and emergency settings can present special challenges due to the settings being logistically difficult to access, limited availability of funding and the scale of the emergency.¹ Infrastructure such as roads may have been destroyed or travelling in areas with unstable security may require aid workers to obtain armed escorts to reach affected populations. Conducting research and/or data collection during an emergency or humanitarian crisis has also been considered a distraction, taking attention or shifting the focus away from medical and other priorities.¹

There is no clear definition of a ‘humanitarian emergency’ but this term is generally used to include crisis situations involving a threat to the health, safety, security or well-being of large numbers of people and this may include war, conflicts, epidemics, famine and natural disasters.²

The United Nations High Commissioner for Refugees (UNHCR) recently highlighted that in 2013, 51 million people were forcibly displaced globally as a result of conflict or persecution, more than have ever before been recorded.³ Such humanitarian emergencies have been described by WHO as having a disproportionate effect on women and children.⁴ Of the 10 countries globally with the highest maternal mortality ratio, eight are currently undergoing armed conflict or are in a phase of post-conflict reconstruction.⁵ It is estimated that more than half (approximately 38 of 51 million) of all refugees or internally displaced people (IDP) are women of reproductive age or children under the age of 18 years.³ For these populations, humanitarian emergencies add an additional layer of complexity to what may already be a hazardous stage of life.

Collecting data relating to sexual and reproductive health (SRH) or maternal, newborn and child health (MNCH), can present specific challenges due to the sensitive nature of the information being gathered, In addition, some questions may lead to recall of past trauma. For example, there is an estimated 45% life time risk of partner and non-intimate partner physical or sexual violence in Africa.⁶ In addition, with high rates of under-five mortality in low income countries (approximately 1 in 12), many women are likely to have experienced the loss of one or more children.⁷

We previously conducted a systematic review which identified 100 publications (28 reports and 72 peer-reviewed papers) summarising the types of SRH and MNCH data that have been collected in humanitarian emergencies and the tools used to collect such data.⁸ The majority of these data have been collected to monitor or evaluate the impact of existing humanitarian activities and health services (for example assessments of availability of health services), with less emphasis on maternal and child health per se.⁹

The review identified up to 16 different toolkits that are currently available and can be used to collect data in emergency or humanitarian settings but only a few studies provided information regarding the applicability and usefulness of these tools. The lack of information regarding the practical aspects of conducting data collection in humanitarian and emergency settings was identified as an important gap in current knowledge. Information relating to the use of existing tools, potential problems and lessons learnt during this type of data collection may assist other workers in the future. Therefore, we conducted a study to explore the methods used and to document the challenges faced by those collecting data on sexual and reproductive, maternal, newborn or child health in humanitarian and emergency settings, particularly with regard to data collection in the field, development of tools and methodology for collection of data.

Methods

Sampling and study participants

Participants were purposively sampled based upon their involvement with a published research study or report on data collection relating to SRH or MNCH in a humanitarian or emergency situation. The participants included representatives from a range of large international non-governmental organisations who had experience of data collection in the field as well as individuals conducting smaller scale research studies.

Prior to conducting the interview, all participants were sent a detailed information sheet (by email) which included the contact details for the Centre for Maternal and Newborn Health (CMNH) and investigators, a description of the study, assurance that confidentiality and anonymity would be maintained and an explanation that participants had the right to discontinue with the interview at any time during the interview if they wished to do so.

Data collection

Interviews were carried out with participants via Skype or telephone. Notes were also taken to aid with interpretation.

A topic guide was developed and used as a basis to guide the interview. Topics explored included: the respondent's involvement in the development or modification of data collection tools, their experience of data collection, factors that influenced their choice of data collection tool and method, the setting in which data were collected, if the data obtained had met their needs and the practical aspects of collecting data in humanitarian and emergency situations (feasibility, acceptability, opportunities and challenges).

Data management and analysis

All recordings were stored on password protected computers and transcribed by an independent co-worker. The transcripts were then coded using NVivo 9 software (QSR International, Daresbury, Cheshire, UK) and analysed using the thematic framework approach.¹⁰ Data were examined using a deductive approach, themes were developed from the coded text and a narrative summary of findings provided.

Ethical approval

Informed consent was obtained from each individual who agreed to be interviewed. At the start of the interview, all participants were explicitly asked to confirm: receipt of the full information sheet, their willingness to participate in the interview and their understanding of the premise under which it was being conducted. All interviews were recorded with participant agreement.

Results

Of the 19 participants invited, nine were interviewed (a response rate of 47%). A variety of different organisations were represented including universities, government departments and international non-governmental organisations (Table 1). Participants had collected data in numerous countries including across Africa, Asia, the Middle East, Europe, North and South America. The settings and populations for which data had been collected included internally displaced person (IDP) or refugee camps in conflict areas, urban refugee populations, the aftermath of a hurricane, terrorist attacks and following a fire in a day care centre.

Table 1.

Participants, type of affiliation and setting from which experience repeated, methods and tools used

Participant	Affiliation	Settings	Methods used	Tools used
A	Government agency	West Africa	Survey	Reproductive health assessment toolkit for conflict affected women11 - modified
B	University	Africa, Asia	Qualitative	Various
C	Government agency	Kosovo, Serbia	Clinical trial	Study specific
D	University	High resource settings	Pre-disaster prediction of impact factors	None
E	Data collection company	Tanzania, Malawi, India, Sri Lanka	Electronic Data Collection	Study specific
F	INGO	Africa, Asia, Middle East	Mixed methods	Reproductive health assessment toolkit for conflict affected women11 (family planning only)
G	Independent	DR Congo	Ethnography	Study specific
H	University	Africa, Asia	Mixed methods	Reproductive health assessment toolkit for conflict affected women11 Refugee reproductive health needs assessment field tools12 Field-friendly Guide to Integrate Emergency Obstetric Care in Humanitarian Programs13
I	INGO	Africa, Asia, Middle East	MISP data	Refugee reproductive health needs assessment field tools12 MISP assessment tool14

INGO: international non-governmental organisation; MISP: Minimum Initial Service Package.

Four major thematic areas were identified: methodology used to obtain information; the importance of language in the development and use of data collection tools; the variety of and need for adaptation of existing data collection tools; and capacity of staff for data collection in emergency and humanitarian settings.

Thematic area 1: Methodology used to obtain information

Qualitative and quantitative methods

Types of data collection methodology varied widely and included both qualitative and quantitative techniques. Some participants used an ethnographic approach with in-depth interviews, a number used focus group discussions (FGDs), others deployed more quantitative methods (for example analysis of birth weight and stillbirth rates following major terrorist attacks).

Participants reported that qualitative methods (interviews and FGDs) were useful in eliciting the opinions of beneficiaries of relief and development aid, as well as the opinions of healthcare providers working in healthcare facilities. Qualitative data were reported to be particularly useful in three main areas: obtaining the beneficiary perspectives, e.g., a study looking at what interviewees considered to be the factors that impacted on their health; providing information in support of quantitative data, e.g., to assess why particular emergency obstetric care signal functions¹⁵ were not available in healthcare facilities; and as a complementary method to clinical testing situations where qualitative methods can provide additional information, e.g., for documentation of possible causes of malnutrition and lead poisoning in refugee camps.

In general, respondents reported that quantitative data were easier to collect in the earliest stages of a disaster or emergency situation as such data were generally easier to obtain, often being collected centrally at health facilities instead of from the dispersed population directly. However, those with experience of qualitative methods suggested these should be used more often and at the earliest possible opportunity to garner the opinions of those most directly involved, providing beneficiary perspectives on humanitarian aid efforts. There was clear consensus among respondents that the immediate priority of saving lives took precedence in the aftermath of a disaster, but data collection was considered necessary as soon as possible afterwards to understand the needs of the target population. Participant B stated that it is necessary in order ‘to get more and more beneficiary perspectives earlier in the mix of priorities… as part of a priority setting negotiated process’.

Feasibility of data collection

A number of challenges in terms of data collection methods were reported, mostly centred around security and/or whether there was sufficient time to obtain information and data. Some challenges were general in nature but others were considered specific to particular populations and settings. One example provided was the difficulty in enumerating children under the age of five years in a refugee camp. Older children can often be identified through the education system but as not all births within a camp setting are registered it is often difficult to accurately assess the number of pre-school children.

Time scales and security of setting

A number of participants reported that in the acute phase of an emergency lack of time was a problem, because of security problems and the need to wait for military escorts or for roads to be cleared or where the situation changed so quickly that data collection plans were no longer viable. Availability of time also impacted on who could undertake certain parts of the data collection. Training local people to carry out FGDs or key informant interviews (KII) was reported to take up to 10 days and this may not be possible in the time available. Facility assessments could take several hours to complete and in settings with security issues, data collection might have to be curtailed:

‘I think it would be difficult to collect data in a place where security, there are problems, if you can only get to spend an hour in a facility, you would definitely have to cut [the data collection tool] down to be able to use it there.’ (Participant I)

The situations in which participants had worked were often reported to change rapidly, e.g., when carrying out baseline data collection:

‘it was still at the time of insecurity but a lot of people were in camps so that made it somewhat easier to get to people’, but when carrying out follow-up… ‘it was much safer, there weren't really any security concerns but people had left the camps and had moved back home.’ (Participant H).

The descriptions of the security situation varied considerably. Participant G described a setting as ‘really, really dangerous’. A village where data were being collected was attacked and was too dangerous for the United Nations to land. The need to have security protocols in place and to negotiate permission in-country in order to visit certain areas and camps ahead of data collection was mentioned by several participants. Some participants cited security as a deciding factor which impacted upon whether or not they were able to collect data in particular countries. However, others commented that ‘I never felt in danger, unwanted or scared’, and that they had ‘been in neighbourhoods in the United States that were much more frightening than the camps’ (Participant C).

Thematic area 2: The importance of language in the development and use of data collection tools

Most of the data collection tools were available either in English or French which often necessitated translation into local languages for the benefit of both the interviewee and local interviewer and to ensure consistency. A method of translating the tools then back-translating was recommended as the best method to ensure accuracy and authenticity. In some cases, it was necessary to translate the tool into a number of languages to include as many of the diverse nationalities present within a refugee camp. Mistranslation was reported to occur relatively frequently and, although this was sometimes noticed early enough to be corrected, mistranslation was noted to have the potential to impact results.

The added difficulty of language variation within a country was reported by a number of respondents. This might vary by region and/or social class, with translators from urban areas or different population groups (than the affected population) sometimes having a demeaning attitude towards those from rural areas or those affected by the humanitarian emergency. Interpreters would in some instances ‘try to protect the reputation of their country by mistranslating if somebody gives a “wrong” answer’ (Participant B). It could be difficult when working in a setting where a national language was used which was not spoken in all areas of the country or by all population groups or not spoken by the population affected by the emergency. In one programme, the two study populations refused to speak the common national language, sometimes resulting in the necessity of impromptu trilateral interpretation. In cases where no professional interpreter was available, ‘somebody who has some education and speaks a little English’ was employed (Participant C).

The use of medical terminology presented problems to some participants, both in ensuring it was understood by those who were being interviewed and by the interviewers. Understanding and using local terms (as used by the interviewees) was considered to be very important. One participant discussed a setting where the in-country health professionals available had very limited medical knowledge and the need to train them in basic physiology before they could be involved in clinical interventions in a refugee camp. Contextual modification of language was also reported to be necessary in some circumstances such as undertaking household surveys or for religious and political reasons, e.g., when discussing contraception in a Middle Eastern setting.

Thematic area 3: The variety of and need for adaptation of existing data collection tools

Four currently available toolkits were specifically described as having been used the most frequently by participants. These were generally described in terms of their development, modification and use and included ‘Reproductive health (RH) assessment toolkit for conflict affected women’: a comprehensive tool to assess the reproductive health status of women affected by conflict¹¹; ‘Refugee RH needs assessment field toolkit’: a set of five assessment tools to gather information from refugees and IDP's regarding their needs and attitudes towards reproductive health practices¹²; ‘Field-friendly Guide to Integrate Emergency Obstetric Care in Humanitarian Programs’: a combination of tools to assess women's emergency obstetric needs and facility provision of emergency obstetric care in refugee and IDP situations¹³; and ‘Minimum Initial Service Package assessment toolkit’: a mixture of quantitative and qualitative tools to assess the provision of the Minimum Initial Service Package in humanitarian situations.¹⁴

Development and modification of tools

Respondents were clear that the main tools and toolkits that were publicly available were often not static and isolated but had been developed over time and had built on previous work and were continuing to be developed and modified: ‘we've adapted, revised, hopefully improved, before each assessment, a sort of continuous improvement effort’ (Participant I). Participants also recommended that tools be thoroughly tested before being made publicly available to ensure that they were robust. It was suggested that a disadvantage of the comprehensive published toolkits was that those using the tools were tempted to not carry out the background work necessary before using the tool such as conducting site visits and exploring better ways of collecting data.

According to respondents, modifications were sometimes necessary to ensure that the tools were suitable for specific contexts, for example, one facility assessment tool aimed at emergency obstetric care was ‘adapted to cover other reproductive health technical areas’ (Participant I), but in other circumstances respondents were careful to try to avoid modifications to ensure that the data collected were comparable across different settings: ‘we were trying to keep the same tool across all of the programmes … trying to make sure that if we made changes in country X, that the word got to the teams in country Y’ (Participant I).

Utilisation

All of the tools mentioned in the interviews were intended for collecting data but some were also used as a means of advocacy such as raising awareness of the Minimum Initial Service Package and highlighting the difference this can make in the field.¹⁴ Facility assessments tools had been used in urban settings with mixed refugee and non-refugee populations and to allow visiting NGO staff to monitor improvements of quality of care and compliance to guidelines over a period of time. Qualitative tools could be used to ‘facilitate communication’ (Participant B) enabling the relief agencies (and the wider world) to ‘hear’ the beneficiary communities. On a larger scale, the facility assessment tool,¹⁵ was used as part of a global evaluation of reproductive health, repeated over a number of years.

Thematic area 4: Capacity of staff for data collection in emergency and humanitarian settings

Participants explained that the amount of training and support available to data collectors varied markedly depending upon the local security situation and previous experience. One programme employed Public Health Masters students as supervisors, who were trained for a week and then deployed to different countries to train local data collectors. In one situation, training of local staff was limited to one day because of the humanitarian crisis, instead of the required two or three days. One participant, who had carried out facility assessments, advocated the need to ‘spend some time in the classroom going through the tool and it's very important to go with the teams to do a practice run in at least one facility’ (Participant H).

When asked about the support provided for data collectors, most participants commented on financial remuneration. During one study examining the reproductive health needs of conflict affected women who had suffered violence and trauma, it was noted that data collectors had themselves needed psychological support following their work to document and analyse this type of data. The situation on the ground was described as ‘tragic, horrifying, disturbing, upsetting’ explaining that it was ‘really, really difficult even for the enumerators, and even though the women had lived through the trauma, they were re-traumatised during the data collection’ (Participant A). To try to address this supervisors had debriefed enumerators daily about what they were experiencing in addition to putting support services in place for the affected women.

Discussion

Main findings

To the best of our knowledge, this is the first study to document the practical experiences of those involved in data collection to assess SRH and MNCH needs and services in humanitarian and emergency settings.^8,16 By focussing on a range of researchers from both small and large international organisations involved in this work, we have highlighted the personal experiences and challenges faced by those collecting data in humanitarian and emergency settings, information which is not usually included in research reports and papers.

Although this study focused on SRH and MNCH in humanitarian and emergency situations, in ‘real life’ this includes a wide range of topics and can be as diverse as gender based violence in conflict situations and/or lead poisoning amongst children in refugee camps. The settings in which data were needed and collected were also diverse both in geography (spread over four continents) and type of emergency situation (terrorist attack, hurricane, armed conflict). In almost all cases, the lack of security was reported to be a determining factor impacting on the ability to collect data as well as the quality of data obtained. In less stable situations, often only shorter quantitative data collection could be carried out, although the need for more qualitative, ‘beneficiary perspective’ studies was recognised and recommended by experienced field researchers.

The available data collection tools frequently required modification by field workers to ensure these were suitable for use in the different circumstances and settings. In addition to the type of data to be collected and the anticipated purpose for which the data were considered to be needed (planning, description of burden of disease, advocacy), the type of humanitarian emergency was also noted to play a large role with regard to the selection of data collection tools and/or methods.

The importance of language in the development and use of data collection tools was highlighted by the majority of those who had experience of collecting data in humanitarian and emergency situations. The need to build capacity for data collection both with regard to the number of people who understand the data collection tools and basics of research methodology and the need to support these people during and after the process were recognised.

Strengths and weaknesses

This study sought to obtain practical information relating to SRH and MNCH data collection directly from experienced researchers and field workers and to provide potentially unique insights into some of the challenges relating to this type of data collection. In 2011, the US Centres for Disease Control and Prevention published an evaluation of a single RH toolkit for conflict affected women¹⁷ which identified similar findings. Our study echoed these findings covering a variety of humanitarian and emergency situations and types of data collected and thereby broadening the scope of evidence.

The main limitation of this study was the relatively small number of participants. This was largely due to out-of-date contact details in the public domain (paper, report, website) and respondents being unable to take part in an interview as they were engaged in field work. In addition, our primary focus was on SRH and MNCH and there are a comparatively small number of people with experience of humanitarian data collection in this area.

Implications for research and practice

It is recognised that the general actions needed in the immediate aftermath of an emergency are often well known and documented.¹⁸ However, the wishes and specific priorities as perceived by those affected, need to be garnered and acknowledged more frequently and more specifically. It was recognised that this may only be possible after the immediate emergency or conflict situation has been addressed.

Certain aspects of data collection can be challenging in any situation, such as language barriers and the need to provide appropriate translation and interpretation. However, where resources are severely limited and people may be struggling to meet their basic daily needs in post-conflict settings or the aftermath of a hurricane, these challenges may be far harder to resolve. When planning data collection in such settings, researchers may need to allow for extra time as well as additional human and financial resources. This needs to include identifying appropriate psychological support for those traumatised and sometimes re-traumatised through having to recall events, as well as those documenting this information. Those undertaking research need to adopt a flexible approach in order to take into account extenuating circumstances such as hostilities within a refugee camp or changes in the political control of an area from government to rebel forces. Future research investigating the practicalities of data collection in humanitarian emergencies might benefit from longer time frames for conducting interviews and wider parameters from which to select respondents.

Conclusions

To the best of our knowledge, this is the first study exploring perspectives of experienced field researchers who are involved with the collection of SRH and MNCH data in humanitarian and emergency settings. Many of them commented that due to the often complex and varied nature of such situations, there are numerous and diverse ‘real life’ challenges associated with collection of such data. Whilst some of these may be common to research in any setting, a number are recognised to be specific to humanitarian and emergency situations including the challenge of working in unsafe or non-secure settings, the lack of time and on-the-ground capacity to collect data as well as the fact that data being provided or collected pertain to traumatic experiences and loss. There is a great need for better and more information relating to the SRH and MNCH needs of people in humanitarian and emergency settings but it is also essential that those carrying out the data collection are well prepared for the challenging environments in which they may find themselves, and have compassion and respect for those from whom they are collecting data and who are survivors of adverse circumstances beyond their control.