Lessons from rapid field implementation of an HIV population-based survey in Nigeria, 2018

Abstract

Background:

The need for accurate HIV annual program planning data motivated the compressed timeline for the 2018 Nigerian HIV/AIDS Indicator and Impact Survey (NAIIS). The survey team used stakeholder cooperation and responsive design, using survey process and paradata to refine survey implementation, to quickly collect high-quality data. We describe processes that led to generation of data for program and funding decisions, ensuring HIV services were funded in 2019.

Setting:

Nigeria is the most populous country in Africa, with approximately 195 million people in 37 states. Challenges include multiple security threats, poor infrastructure, seasonal-rains, and varied health system capacity.

Methods:

Stakeholders worked together to plan and implement NAIIS. Methods from other population-based HIV impact assessments were modified to meet challenges and the compressed timeline. Data collection was conducted in 6 webs. Responsive design included reviewing survey monitoring paradata and laboratory performance. Costs required to correct data errors, e.g., staff time and transportation, were tracked.

Results:

NAIIS data collection was completed in 23 weeks, ahead of the originally scheduled 24 weeks. Responsive design identified and resolved approximately 68,000 interview errors, affecting approximately 62,000 households, saving about US$4.4 million in costs. Biweekly field laboratory test quality control improved from 50% to 100% throughout NAIIS.

Conclusions:

Cooperation across stakeholders and responsive design ensured timely release of NAIIS results and informed planning for HIV epidemic control in Nigeria. Based on NAIIS results, funds were provided to place an additional 500,000 HIV-positive Nigerians on antiretroviral therapy by the end of 2020, pushing Nigeria toward epidemic control.

Introduction

Describing the HIV epidemic in Nigeria is critical to ensuring all people living with HIV (PLHIV) will know their status, access sustained antiretroviral therapy (ART), and have viral suppression, which are key to ending the epidemic. National HIV prevalence models using data from antenatal clinic surveys, like the HIV Sentinel Survey (HSS), and population-based surveys, like the National HIV/AIDS and Reproductive Health Survey (NARHS), was much higher than expected compared to programmatic data [1]. Updated, accurate HIV data were urgently needed to describe the HIV epidemic in Nigeria by early 2019, when the U.S. President’s Emergency Plan for AIDS Relief (PEPFAR) and The Global Fund to Fight AIDS, Tuberculosis, and Malaria (Global Fund) set funding priorities to ensure services were aligned to fight the HIV epidemic. This urgent need for high-quality data drove the creation and implementation of the Nigerian HIV/AIDS Indicator and Impact Survey (NAIIS).

The Federal Republic of Nigeria comprises 36 states and the Federal Capital Territory (FCT), with a population of about 195 million people (Nigeria National Population Commission, unpublished data) and 19 institutional languages. As a federal republic, states, not the central government, administer health programs. Primary languages and infrastructure, including transportation, power supply, and health systems, vary across and within states, many of which have the size and population of other African countries. Security risks, including terrorism, kidnapping, bandits, and other internal conflicts, are present in almost all states. Despite these challenges, NAIIS urgently needed to collect high-quality population-based HIV data in all 36 states and FCT to guide program planning for the next round of funding applications.

NAIIS leveraged the existing population-based HIV impact assessment (PHIA) survey methods [2] by using a stratified cluster sample design of enumeration areas (EAs) and households, modified to address local challenges and the compressed timeline. Stakeholders were to support survey implementation. NAIIS used survey responsive design to intensively manage survey progress, allowing changes in operation to ensure collection of valid HIV impact and indicator data within the compressed timeline [3]. Responsive survey design is a strategy for dealing with survey uncertainty through identification of potential risks related to costs or errors, develop indicators for tracking these risks, and then plan design changes for controlling these costs or errors while the survey is still being completed [4]. We describe the processes that led to generation of NAIIS data in time to guide program and funding decisions, ensuring HIV services were available for PLHIV in 2019.

Methods

Stakeholder cooperation

A key strategy of NAIIS was cooperation across stakeholders. The Government of Nigeria (GoN), represented by the Federal Ministry of Health (FMoH) and the National Agency for the Control of AIDS (NACA), led the implementation of the NAIIS survey. The Nigeria Population Commission (NPoPC) and the National Bureau of Statistics (NBS) were crucial GoN partners. Funding was provided by PEPFAR, through the Centers for Disease Control and Prevention (CDC), and the Global Fund. CDC provided technical assistance, from the CDC Nigeria office and headquarters. The University of Maryland, Baltimore (UMB) led the NAIIS Consortium, a group of institutions and organizations brought together to implement the survey (naiis.ng). GoN led the NAIIS Steering Committee (SC), including leaders from FMoH, NACA, NPoPC, NBS, Joint United Nations Programme on HIV/AIDS (UNAIDS), World Health Organization (WHO), United States Agency for International Development (USAID), and CDC Nigeria and academics from selected Nigerian universities, and the NAIIS Technical Committee (TC), including staff from SC organizations; national security staff from the police, military, and civil defense corps; and other Nigerian stakeholders and members of civil society. The NAIIS SC and TC met at least monthly to review and provide input on operations. During survey implementation, the NAIIS Survey Implementation Team (SIT), including senior field staff members, met daily, and a working group, including SIT and technical advisors, met at least once weekly.

Most participating organizations had fieldwork experience in Nigeria. Organizations shared existing information on primary languages spoken, access, and security information. Nigeria has six geopolitical zones (North East, North Central, North West, South East, South South, and South West), each including five to seven states (Figure 1). Hausa is the primary language spoken in most of Northern Nigeria. Igbo is spoken in the South East. Yoruba is spoken in the South West. Hard to reach areas are found in all The North East is most affected by the Boko Haram insurgency, North West by kidnapping, although kidnapping can be present in other zones, and North Central by herdsmen attack (Figure 1) where communal clashes are a major security threat. During the survey, security information was updated with input from a security contractor and the national security staff to ensure the safety of staff and participants.

Figure 1.

Map of Nigeria showing primary languages spoken, access, and security risk information for the Nigerian HIV/AIDS Indicator and Impact Survey (NAIIS) 2018

Survey structure and responsive design:

Similar to the PHIAs, NAIIS used a two-stage cluster-sampling design (EA followed by households) [5]. The NAIIS target population was people aged 0–64 years. The estimated sample size was 172,604 participants from 100,055 households across 4,035 EAs, of which 364 were deemed to have security risks. The overall expected blood draw response rate was 77% (Figure 2).

Figure 2:

Nigerian HIV/AIDS Indicator and Impact Survey (NAIIS) 2018 - Key Performance Indicators.

To maximize efficiency, NAIIS data collection was completed in survey groupings, called webs, based on the geography of the country. Separate survey teams were deployed to collect data simultaneously in each zone during a web, usually in a single state (Table 1). Using a shared protocol, each of the zonal teams within a web operated independently, with oversight, support, and monitoring staff dedicated to each zone.

Table 1:

Nigerian HIV/AIDS Indicator and Impact Survey (NAIIS) 2018 Field Data Collection Timeline by Web, Geopolitical Zone, and State

NAIIS responsive design included monitoring key performance indicators to inform changes in survey methods, keeping the survey on schedule. Indicators included survey progress by web, including enumeration areas stratified by security risk, household questionnaire completion rate, and response rates, including blood draw response rates (calculated as the ratio of the total weighted number of individuals interviewed and tested to the total weighted number of individuals eligible for HIV testing), and field laboratory quality results. These paradata were displayed on a dashboard, updated throughout the survey (Figure 2) and changes were reflected in the timeline (Table 1).

Staff were recruited to support the level of effort required to operate multiple data collection teams in each web. Six zonal teams, comprising a coordinator, two to three sub-zonal coordinators, two information technology staff, a zonal laboratory coordinator, two sub-zonal laboratory coordinators, one logistician, and one administrative aide, provided oversight and local coordination to data collection teams deployed in their zone. Other members of the zonal team included community mobilization staff, who were familiar with the local culture and could facilitate access to and cooperation of local leaders and communities with the NAIIS data collection activities, and security personnel. Experienced mappers from NPoPC, familiar with each community, were responsible for identifying households before NAIIS activities. Each data collection team had two vehicles to transport staff, equipment, supplies, and laboratory samples.

Data collection teams, each comprised of a team lead, two interviewers, two HIV-testing counselors, two field laboratorians, two drivers, and a tracker, collected all household and laboratory data in the field. The tracker for the team identified and mapped the households assigned to the team. Additional buffer staff were recruited in each job category to ensure staff could be replaced in cases of attrition and or incapacitation. Staff recruitment and deployment considerations included previous experience with surveys, socio-cultural affiliations, and language abilities. To improve acceptability of field staff by communities, staff were recruited and deployed within their states as much as possible. Field teams included both men and women to accommodate cultural needs to have interactions segregated by sex. Field staff were required to be proficient in English and at least one local language where they would be deployed. State Ministries of Health and State Agencies for Control of HIV and AIDS played integral roles in selecting qualified field staff.

All data collection staff participated in survey-specific trainings, including human subject research ethics, for 3 weeks. Training included verbal translation skills and techniques to enable data collection staff to administer questionnaires when one was not available in the preferred language. A one-week pilot was conducted to assess survey procedures and staff competencies and to identify potential problems. Buffer staff joined field teams periodically to maintain their proficiency throughout the survey.

Laboratory staff, including field (n=415), satellite (n=105, six to eight staff per satellite laboratory), and central (n=10) laboratorians, managed all laboratory processes (specimen collection, transport, processing, testing, and storage). Several laboratory trainings varied depending on the level of the laboratorian. Field laboratorians underwent a 2 week of training, separated into two groups due to high number of trainees (150 per group). Training included all household procedures, e.g., blood collection, biomarker testing, and specimen management. Satellite laboratorians underwent 2 trainings. The first included all household procedures, enabling the satellite laboratorians to be trainers for the field laboratorians. The second included 2 weeks of training focused on sample reception and processing, DBS preparation, quality assurance testing, sample storage, and sample transport. Central laboratorians underwent 2 weeks of training.

Satellite laboratories were assessed to meet geographic spread and quality standards such as availability of running tap water, connection to national electricity grid, availability of adequate workbenches (clean and dirty areas), space to accommodate additional refrigerators, quality management system, supply chain management system, biohazard waste management and fire safety. Those that did not meet quality standards were upgraded to meet NAIIS standards. Each state had two to three satellite laboratories (12–18 laboratories throughout the country) that were active during each web. EAs were mapped to laboratories to ensure that travel time did not exceed 3 hours. Two mobile laboratories were periodically deployed in hard-to-reach locations where travel time was more than 3 hours and in EAs surrounded by high-risk areas. Laboratory data quality was monitored throughout the survey, including biweekly quality control (QC) (positive and negative controls), retesting and confirmatory testing at the satellite lab, and proficiency testing using dried tube specimens.

A central team, including the leads for the field survey implementation, central data monitoring center, laboratory, community mobilization, logistics, and support staff, were responsible for overall oversight and coordination of NAIIS activities across the six zones and six webs. The central data monitoring center lead and 30 data monitors were responsible for ensuring data were available to assess responsive design key indicators. Data systems were designed to allow real-time reporting, using a dashboard to track survey progress to provide the SIT and working group up-to-date information to guide survey implementation.

To support real-time data reporting, the electronic data entry system used Census and Survey Processing System (CSPro) application (version 7.1) using the Computer Assisted Interview (CAPI) system for Android tablets. Staff directly responsible for data collection, i.e., team leads, interviewers, counsellors, and field laboratorians, had a tablet programmed with the questionnaire. All devices used by a field team were linked to each other through Bluetooth technology, enabling rapid communication. Team leads assigned households and survey participants to interviewers through Bluetooth. Data were transmitted from the interviewer to the counsellor and the field laboratorian. After data collection, team leads checked all interviews and blood collection to ensure all assigned interviews, data, and specimens were complete. Team leads were required to connect to the 3G mobile phone network daily, whenever possible. Data were transmitted to the central team daily via an internet connection from the 3G mobile phone network. In EAs not covered by a mobile phone network, data were transmitted at the earliest opportunity, usually not exceeding 24 hours.

The central team was responsible for daily editing and confirming completeness of data. The team of 30 central survey data monitors reviewed the daily data uploads. Each central survey monitor was assigned specific field teams. Data monitors communicated with data collection team leads via messaging and Voice over Internet Protocol software for any necessary clarifications. Data collection teams also provided qualitative impressions about survey progress to the central data monitors. Activity Information Management System (AIMS), an integrated survey management tool, provided a real-time data monitoring interface. As the data arrived on the central server, raw data were automatically imported into the database. AIMS was updated continuously to inform data monitors of errors and discrepancies. A Laboratory Data Management System (LDMS) was used to manage all laboratory data from the satellite and central laboratories. The interview and LDMS data were merged and synchronized in the central server. The data were displayed on dashboards in the situation room data monitoring center and could be viewed from remote locations using a secure link.

Before departing an EA, the team lead was required to confirm the completion of all data collection with the central team. The central survey monitors notified team leads of errors and discrepancies daily before the field team left an EA so that they could easily contact the participant if clarification was needed. Data errors and discrepancies were tracked. Costs required to correct errors using real-time data reporting and traditional data reporting, e.g., number of EAs and households’ revisits required (in hours), security escorts, number of additional vehicles and transportation time, accommodations, and additional compensation for the field staff and monitors, were estimated.

Protection of human subjects:

NAIS was conducted under a protocol reviewed and approved by the National Health Research Ethics Committee (NHREC), Nigeria, and the UMB and CDC Institutional Review Boards (IRBs). All incidents were reported to the zonal coordinator, submitted to the central team, including the Principal Investigators, within 48 hours. Reportable incidents such as adverse events and protocol deviation were submitted within the required timeline to NHREC and the UMB and CDC IRBs.

Results

NAIIS was formally launched by the President of Nigeria on June 29, 2018. Data collection started the following month, on July 17, 2018). NAIIS deployed over 190 teams across all six geopolitical zones in Nigeria (25–35 teams per zone). These teams used 380 fleet vehicles to transport staff, equipment, supplies, and laboratory specimens. Teams completed interviews for an average of four to six households per day. A total of 98 satellite laboratories and one central laboratory were upgraded. Each satellite laboratory processed a maximum of 200 samples per day. In all, 276 survey implementation teams, 28 working groups, 7 technical committees, and 4 steering committee meetings supported the NAIIS deployment.

NAIIS data collection was completed in 23 consecutive weeks, ending December 22, 2018 (Figure 2, Table 1). The final NAIIS sample size was 225,169 participants from 97,250 households across 3,848 EAs. Household size was larger than originally estimated with as many as five persons per household in some northern states. The blood draw response rate was 93%.

Monitoring real-time data reporting identified and resolved approximately 68,000 discrepancies and errors associated with age and sex discrepancies, duplicates households and individual listing, affecting approximately 62,000 households, while teams were still in the EAs. The data collection team corrected the data discrepancies and errors as soon as they were notified, with input from the households, if necessary. The most frequent errors were sex and age discrepancies (62,267 [91.6%]) followed by data related to participants’ availability, including discrepancies in refusals and consents (3,982 [5.9%]). Internal consistency was high with 15 out of 20 indicators of interest having <1.0%, four indicators having <10.0% and only 1 indicator with 17.0% inconcistency. Similarly, frequency checks among survey variables revealed highlevel of concurrance between expected and actual values, median difference = 0.02% (range = 0.001% - 0.137%). We observed progressive improvement in structural check (decline in structural edits) from web 1 to web 6 (figure 3). Responsive design using real-time data monitoring saved an estimated 700 hours of fieldwork by correcting errors in the field vs. requiring teams to return to EAs, saving an estimated US$4.4 million, primarily in saving on security escorts, vehicles, and compensation for field staff and monitors.

Figure 3:

Trends of Certain Structural Edits Across the Webs, NAIIS 2018

The initial biweekly field laboratory test QC pass rates was an average of 50% across the six zones, in part because QC materials were not effectively delivered. Biweekly QC pass rates increased after web 1 and were at 100% across all zones by the end of NAIIS (Figure 4). Similarly, highlevel of proficiency was observed among field and laboratory testers with 512 (98.0%) pass in two cycles. We observed significant drop in PIMA error rate for CD4 pointcare testing from 11.6% in web 1 to 8.0% in web 3 and 4.4% in web 6. Highlevel of specimen potency was attained with only 114(0.05%) of the 207,304 specimens collected hemolyzed.

Figure 4:

Quality control rate (%) of filed testers by geopolitical zones, NAIIS 2018

Additional driver training was provided before each web in response to an accident with a fatality during the first web. A truck capsized in a flash flood in Bauchi State in October 2018. Members of the data collection team were able to preserve the tablets and specimens, resulting in no loss of data. The teams were immediately pulled out of the EAs affected by heavy rains and flooding for their safety. Data collection resumed in December when the risk of floods was low.

There was one notable security incident in a northern state. In October 2018, approximately 30 individuals, armed with automatic weapons and riding motorcycles surrounded staff who were in two clearly marked NAIIS vehicles. Village leaders had already met with the community mobilization staff and were aware of the survey. The Village Chief met the armed individuals and convinced them to allow the NAIIS staff and vehicles to pass safely. After this event, the zero risk approach was enforced to minimize security risks for the survey teams. Teams were restricted from entering EAs designated as “red” (highest risk) and “yellow” (risky). In “green” (safe) EAs surrounded by red or yellow EAs, a “hit and run” approach was used, in which more data collection teams were deployed per EA to complete data collection within 1 to 2 days to minimize the exposure time in security risk areas. In 71 of the 334 EAs with security risks, data collection was not completed due to the risk to survey staff, and in one EA, data were not collected due to flooding (Bayelsa State). Most of the security risk EAs without data collection were in two northern states (Borno and Zamfara States). (Figure 2).

In March 2019, within 3 months of the end of data collection, the preliminary HIV prevalence and viral load suppression results were announced by the President of Nigeria, were used in calculations of UNAIDS estimates, and were included in PEPFAR program planning (FGN, 2019).

Discussion

NAIIS was completed on a compressed timeline without compromising quality, which required cooperation across stakeholders and preparation to address the unexpected. While the methods and lessons from previous PHIAs were essential to complete NAIIS, the scope of NAIIS was substantially more complex than previous surveys to date, covering 36 states and a central capital, conducted in multiple languages, and under difficult security challenges. The centralized coordination, using responsive design principles to track progress across the country, allowed the team to rapidly respond to incoming paradata, ensuring NAIIS completion in 23 consecutive weeks, faster than the projected 28 weeks.

Stakeholders worked to their strengths to support NAIIS. GoN and USG leadership, including the President of Nigeria, Honorable Minister of the Federal Ministry of Health, Director General of the NACA, US Ambassador to Nigeria, and CDC Nigeria Country Director actively participated in survey implementation and received at least weekly progress updates from NAIIS. Importantly, leadership worked together to raise awareness of NAIIS. The formal launch of the survey by His Excellency, the Nigeria President [6] showed high level political will and strengthened NAIIS staff commitment throughout the survey.

NAIIS methods were improved by data and paradata as the survey progressed. The essence of responsive design is using data to prepare for the unexpected [4]. Data collection teams provided qualitative feedback to the zonal and central teams when participants expressed concern about providing a blood specimen for testing. The overall data collection schedule was modified, placing community mobilization teams in the field immediately before web data collection started to ensure local leaders and populations were aware of the survey immediately before the field teams arrived to begin data collection helping to increase blood draw response rates. Additionally, real-time data reporting and central data monitoring allowed implementation of responsive design, which ensured data were collected on schedule with high internal consistency, low discordance between expected and actual frequencies of collected variables and progressive improvement in number of structural edits from web 1 to web 6. Structural edits are generally a review of whether all data expected to be collected were collected, and whether any data not expected remained in the data files. An effective system like we did in NAIIS is required to enable decline in structural edits. Ongoing feedback from data collection teams helped improve community mobilization, increase response rates, and ensure the cooperation of local leaders. Responsive design led to revising laboratory distribution, providing supplies laboratory staff and ensuring quality controls was implemented throughout the survey. Daily communication allowed zonal and central staff to adapt plans, including suspending survey activities in flooded areas, ensuring staff safety in high-risk EAs, and revising the schedule to allow mop-up activities in the final web to maximize data availability. Additionally, the system in place to protect human subjects helped ensure any concerns noted in the field were reported quickly and addressed. The NHREC and the UMB and CDC IRBs determined that all follow-up actions sufficiently addressed each incident reported. No incident that occurred during NAIIS changed the IRB-determination of level of risk.

Sharing data was a critical success factor, especially for ensuring the safety of survey teams in EAs with security risks. Organizations, including the Nigeria Army, Nigeria Department of State Security Service, Nigeria Police, Civil Defense Corp, a contracted security firm, local vigilante groups, traditional rulers, and donors working in high-risk areas shared up-to-date security information, which was used to adjust survey methods. This security information helped survey sampling experts prioritize survey locations and omit the highest risk EAs from the survey with minimal effect on results. Increasing the staff in EAs surrounded by high-risk areas allowed data collection to take place in the shortest amount of time, minimizing exposure to potential violence. No NAIIS staff member was harmed by violence during NAIIS.

This report is subject to several limitations. First, we could not directly measure the effect of stakeholder cooperation on the success of NAIIS implementation. However, the consensus among the staff is that NAIIS would not have been completed as efficiently or effectively without stakeholder cooperation. Second, some of the data used in decision-making included anecdotal reports, e.g., community mobilization could improve response rates. The NAIIS team decided to address any issue that could affect survey quality, even when quantitative data were not available. Third, multiple interventions may have been implemented to correct a problem, e.g., staff number and composition may have been changed at the same time as additional training was provided, so the effects of individual interventions could not be determined.

NAIIS not only provided data for program planning but also reset the understanding of what Nigeria can do to address the HIV epidemic. The timely completion of the survey and release of the preliminary results enabled Nigeria to prioritize investments to ensure control of the HIV epidemic [6, 7]. The 2019 funding decisions made by PEPFAR include a program surge to identify an additional 500,000 HIV-positive Nigerians, providing them have access to ART, by the end of 2020 [8]. The best practices of NAIIS, cooperation across partners and using quantitative and qualitative data, are guiding this Nigeria ART Surge, showing the way forward to halt the HIV epidemic.