A low-cost emergency and communication transportation system used 3-wheeled motorcycles driven by trained community volunteers. Delivery referrals were redirected from health centers to hospitals capable of advanced services including cesarean deliveries, which was associated with reduced facility-based maternal mortality.
A low-cost emergency and communication transportation system used 3-wheeled motorcycles driven by trained community volunteers. Delivery referrals were redirected from health centers to hospitals capable of advanced services including cesarean deliveries, which was associated with reduced facility-based maternal mortality.
ABSTRACT
Although Ghana has a well-organized primary health care system, it lacks policies and guidelines for developing or providing emergency referral services. In 2012, an emergency referral pilot—the Sustainable Emergency Referral Care (SERC) initiative—was launched by the Ghana Health Service in collaboration with community stakeholders and health workers in one subdistrict of the Upper East Region where approximately 20,000 people reside. The pilot program was scaled up in 2013 to a 3-district (12-subdistrict) plausibility trial that served a population of approximately 184,000 over 2 years from 2013 to 2015. The SERC initiative was fielded as a component of a 6-year health systems strengthening and capacity-building project known as the Ghana Essential Health Intervention Program. Implementation research using mixed methods, including quantitative analysis of key process and health indicators over time in the 12 intervention subdistricts compared with comparison districts, a survey of health workers, and qualitative systems appraisal with community members, provided data on effectiveness of the system as well as operational challenges and potential solutions. Monitoring data show that community exposure to SERC was associated with an increased volume of emergency referrals, diminished reliance on primary care facilities not staffed or equipped to provide surgical care, and increased caseloads at facilities capable of providing appropriate acute care (i.e., district hospitals). Community members strongly endorsed the program and expressed appreciation for the service. Low rates of adherence to some care protocols were noted: referring facilities often failed to alert receiving facilities of incoming patients, not all patients transported were accompanied by a health worker, and receiving facilities commonly failed to provide patient outcome feedback to the referring facility. Yet in areas where SERC worked to bypass substandard points of care, overall facility-based maternal mortality as well as accident-related deaths decreased relative to levels observed in facilities located in comparison areas.
INTRODUCTION
African nations achieved considerable progress in child health during the Millennium Development Goal era. Despite this progress, maternal and perinatal mortality remain among the leading causes of death throughout Africa. According to the World Health Organization, approximately 800 women die from pregnancy or childbirth-related complications every day.1 Nearly 99% of these deaths occur in developing countries and over half occur in sub-Saharan Africa, where only 7% of the global population resides. Most maternal deaths could be prevented if women received timely care when emergencies arise from associated causes, such as hemorrhaging, unsafe abortions, obstructed labor, infection, or eclampsia.2
Most maternal deaths could be prevented if women received timely care during medical emergencies.
Nearly all maternal deaths are accompanied by associated neonatal deaths. Although most neonatal deaths are preventable if skilled attendants assist during deliveries,3,4 rates remain high even where child health and survival are otherwise improving. Yet evidence repeatedly shows that facility delivery and appropriate support for newborn care can reduce neonatal mortality if referral services are functioning and attendants are skilled in recognizing problems and immediately providing post-delivery interventions such as “Kangaroo Mother care,” asphyxia management, care for febrile illness, and tetanus prevention.5–8
Public health systems in Africa are therefore making the development of emergency care systems a priority.6,7 The World Health Organization defines 3 core components of emergency care: care provided in the community, during transportation, and at the health facility.8,9 Each component incurs corresponding sources of risk that elevate death and disability: delays in (1) seeking care,10 (2) reaching care,11,12 and (3) receiving care upon arrival at the referral facility.13 In rural Ghana, and elsewhere in Africa, these delays are driven, respectively, by (1) lack of awareness of the importance of emergency care,14 lack of family resources to cover referral costs,15,16 and concerns about the quality of care17; (2) poor road conditions,18 a scarcity of vehicles,19,20 and limited means of communication21; and (3) inaccessibility of competent providers of essential acute care.20–22
Public health systems in Africa are making development of emergency care systems a priority.
While Ghana has a well-organized, decentralized primary health care system, the country has yet to develop clear emergency referral service guidelines. The Upper East Region is one of Ghana's most impoverished and remote localities: The 13 districts of the region are characterized by a scarcity of vehicles, poor road networks, impassible terrain, and geographic barriers to reaching health services.23,24 Patients in urgent need of acute care reach health facilities by walking or riding bicycles, donkey carts, or motorbikes. In all districts of the Upper East Region, ambulances are typically absent, in disrepair, or located so remotely from communities that they fail to address emergency needs. Even where equipment is available, there is no organized emergency communication system to link one level of care to another and ensure that referrals are successfully executed. Cultural norms can also constrain timely care seeking behavior. Moreover, since Ghana's National Health Insurance Scheme does not cover costs associated with emergency transportation, referral can be prohibitively expensive, with costs further increasing people's hesitation to seek acute care.
In Ghana's Upper East Region, ambulances are absent, in disrepair, or located so remotely that they fail to address emergency needs.
The Ghana Essential Health Intervention Programme (GEHIP) is a systems-strengthening initiative that was designed to increase universal access to health care.25 GEHIP's aims are to expand coverage of the national primary health care system with the Community-Based Health Planning and Services (CHPS) initiative at the community level,26 and to identify gaps in care for newborns, children, and pregnant women at multiple levels of the health system. In addition to addressing issues that had hindered CHPS scale-up, GEHIP has trained midwives in neonatal resuscitation, provided frontline CHPS community nurses with skills in emergency delivery, and trained CHPS community nurses and community volunteers in community-based newborn care.25
GEHIP also documented the urgent need for emergency referral services, including emergency obstetric care, in the Upper East Region. In response, the Ghana Health Service (GHS) pilot tested an emergency referral program, the Sustainable Emergency Referral Care (SERC) initiative, for all types of medical emergencies. This article provides a summary of the initiative components and evaluates the effectiveness of the program using results from mixed-methods implementation research.
THE SERC INITIATIVE
The SERC initiative aimed to develop a community- and subdistrict-level emergency referral system that would improve survival in impoverished rural Ghanaian communities. To address common access, organizational, and knowledge barriers to emergency care services, SERC was designed as a low-cost emergency transportation and communication system together with community education activities. The program aimed to facilitate rapid transport of patients from their community locations or subdistrict health center to higher levels of care.
SERC was designed to address barriers to emergency care services through a low-cost system of transportation, communication, and education.
GEHIP used the tools and methods of participatory planning27–29 to design and implement SERC in collaboration with community members as well as community, subdistrict, district, and regional health system officials. GEHIP research staff held meetings, focus group discussions, and in-depth interviews with community members, frontline workers, and supervisors throughout the planning process to solicit stakeholder advice. Project research assistants were recent graduates of local universities who were hired by GEHIP and assigned to each District Health Management Team to support SERC implementation activities and liaise across levels of the health system.
SERC was conducted initially as a 5-month pilot program with community stakeholders and health workers in a subdistrict of Bongo District in 2012. In July 2013, the program added 12 subdistricts of the Upper East's Bongo, Builsa North, and Builsa South districts for a trial that served a population of approximately 184,000. Remaining districts of the Upper East Region, where social, economic, and ecological conditions are comparable to the SERC coverage areas, served as comparison areas for evaluating the program. The SERC interventions included a referral strategy informed by an assessment of population needs and health systems capabilities; adequately resourced referral centers; active collaboration between referral levels and across sectors; formalized communication and transportation arrangements, with specific protocols specified for referrer and receiver and mechanisms for ensuring supervision and accountability; affordable service costs; capacity to monitor effectiveness; and policy support.
SERC was developed as a component of the health systems development program GEHIP, and SERC scale-up was led by GEHIP staff based at the Upper East Regional Health Directorate (RHD).
Transportation
For the expanded pilot study that began in 2013, GHS procured a fleet of 3-wheeled motorcycles known as Motorkings to serve as emergency transport vehicles. The 24 SERC Motorkings were distributed among 12 subdistricts of Bongo, Builsa North, and Builsa South districts in the Upper East Region. Based on driver advice from SERC's pilot phase, structural modifications were made to the Motorkings to enhance patient safety and comfort. These adjustments involved installing a welded frame and tarpaulin to provide privacy and protection for patients during transport, extended rearview mirrors for maximum visibility, a mattress and safety belt for patients, a seat for an accompanying health worker, and a hook for intravenous drips. To identify the Motorking vehicles as ambulances, each was marked with the GHS logo and a red cross. Each vehicle was equipped with a first aid kit, a spare tire and jack, and protective rain gear for drivers. Recognizing the importance of vehicle maintenance, vehicles were routinely serviced by staff mechanics from the Upper East RHD. Spare parts were procured and kept in stock at the RHD to ensure timely repair in the event of breakdowns.
GHS purchased and modified 24 three-wheeled Motorking motorcycles for use as ambulances.
Geographic information systems (GIS) data were used to estimate the optimal placement of ambulance stations and configuration of catchment areas to ensure community access to an ambulance.28 The SERC ambulances were deployed to 9 subdistrict health centers, 12 community health posts, and 3 communities that lacked facilities or community resident nurses. In Ghana, community health posts function as the first point of care, but only half of the planned locations for these facilities are functional. In the 3 locations that lacked health facilities, community leaders were engaged to determine an appropriate location for the community-based ambulance station. In each of these 3 villages, the community chose an assemblyman's or subchief's home as the station due to its centrality, relative security, and social acceptability for this responsibility. The number of Motorkings was based on an appraisal of the equipment required to effectively cover all communities of the 12 subdistricts. The size of the study area, in turn, was determined by the volume of referrals that would be required to provide statistical power for evaluation.
The community selected 48 volunteers (2 per ambulance), who were trained to serve as drivers. Drivers varied in age, but were typically literate young adult men. A collaboration of the RHD's Transport Unit, the Motorking vendor, the Driver and Vehicle Licensing Authority, and the Ghana Red Cross provided training to all drivers in basic first aid, infection prevention, defensive driving, basic maintenance, transport policies, communications protocols, and recordkeeping.
Communication
Before SERC, no integrated emergency communication system had been established to link patients to emergency care services at the community and subdistrict levels. Therefore, the RHD procured communication equipment: dual-SIM mobile phones were distributed to health facilities, health workers, and volunteer drivers. Emergency phones were assigned to nurses called community health officers who were based at community facilities, in subdistrict health centers, or in district hospitals' outpatient departments. In communities that lacked a resident nurse, a volunteer was provided an emergency phone and cell phone time for calls to emergency numbers. This collaboration with a cell phone vendor ensured that every community had access to a mobile phone for eliciting rapid referral. At the tertiary referral point, the regional hospital designated a phone line in each ward for receiving incoming calls about impending cases.
The regional hospital dedicated phone lines to receiving calls about incoming emergencies.
Community Engagement
Ghanaian cultural groups have well-defined systems of social organization and community consensus building that rely upon durbars, which are open forums for discussing matters of collective importance to the community. In concert with these traditions, the SERC program convened durbars in all ambulance catchment areas to explain the intended use of the ambulance, introduce the local SERC health workers and volunteers and review their roles, and discuss the importance of seeking care during emergencies. An emergency phone number was provided to each community and placed on posters at the nearby health facilities and community gathering points. Participating health workers also liaised closely with traditional chiefs and elders, whose support was essential.
GEHIP equipped SERC staff and volunteers with emergency phones and trained them in mobile phone use, criteria for ambulance use, protocols, and recordkeeping. GEHIP staff held quarterly refresher training sessions to ensure that knowledge and skills were retained, and trained district- and subdistrict-level supervisors to oversee SERC activities and provide routine monitoring and supervision. Monthly review meetings were held across worker tiers to discuss challenges that arose and system improvements needed.
Protocols specified various emergency scenarios in the community and facility setting. Key guidelines included verification of emergency by a health worker and alerts to facilities to prepare for incoming patients and minimize delays. Frontline workers at community health posts were trained in basic triage procedures. All patients being transported were to be accompanied by a health worker. Facilities that received an emergency case were required to provide feedback to the referring facility upon discharge to facilitate follow-up scheduling. The program design included routine monitoring of resources and supplies to assess availability of human resources, equipment, medication, and forms.
GHS supported the operating costs of the SERC emergency referral system. Pregnant women and children under 5 years of age were provided free emergency transport. To encourage facility-based delivery, normal labor cases were transported free of charge. Other ambulance users were charged a nominal cost recovery fee (US$2.50–$5.00) that was determined by each District Health Management Team. In one district, the district assembly covered maternal and child referral fuel costs.
Pregnant women and children under 5 years of age were provided free emergency transport.
Health worker feedback was solicited on SERC to continually inform strategies for educating communities about emergencies. Qualitative appraisal methods were used to determine what community members needed to learn regarding emergencies and to identify strategies for developing a culturally appropriate community education program. The aim was to increase capacity in the community to recognize signs and symptoms of emergencies, encourage prompt decision making to seek care, and increase use of SERC. Opinion leaders and community members contributed to the development of educational materials, which were translated into local languages. These materials included educational flip charts (for use by health workers) and informative songs played on local radio stations and on speaker systems in outpatient hospital wards. Dramas depicting emergency scenarios were developed, filmed, and shown at evening durbars, and posters displayed in health facilities and meeting points depicted actions to be taken in emergency situations.14 Discussions of the possible harm to SERC that could arise if equipment was misused were integrated into community education sessions.
METHODS
An iterative systems development approach was employed to continuously refine the SERC initiative in response to community reactions and administrative realities. GEHIP staff and consultants conducted implementation research to identify operational challenges and potential solutions. Methods included a quantitative analysis of key process and health indicators over time, a survey of health workers, and continuous qualitative systems appraisal with frontline workers and community members.
Time Series Analysis of Key Indicators
Volunteers, health workers, and district supervisors completed monthly SERC monitoring records and submitted them to GEHIP staff at the RHD, who created visualizations of results to help supervisors assess both referral volume by location and the types of cases associated with referral operations. Monitoring included station-specific information on distances traveled, transit times, adherence to protocols, types of emergency, and patient outcomes. The monitoring used technology designed to integrate SERC monitoring into the routine GHS data system operations known as the District Health Information Management System (DHIMS). Educational aids and training sessions were developed to help regional and district-level managers use the DHIMS database for practical decision making.
Monitoring information included distances traveled, travel times, types of emergency, and patient outcomes.
DHIMS data are aggregated at the facility level and provide indicators of the monthly care caseload by indicator and by type of facility (e.g., CHPS, subdistrict health center, or hospital). Monthly DHIMS data are routinely available for all primary health care service points in Ghana, including the community health posts and community nurses involved in SERC, and we used these aggregated data to support the program evaluation. For the purpose of this analysis, the 12 SERC-exposed “treatment” subdistricts provided a basis for assessing the effect of the program. District facilities in the Upper West Region and the remaining subdistricts of the Upper East Region served as comparison areas. The dependent variable was monthly case volume of each relevant indicator; exposure versus nonexposure to SERC in the facility catchment area was the key independent variable.
For the SERC evaluation, we compared DHIMS time series data from SERC facilities with data from facilities located in unexposed districts of the Upper East and Upper West regions. The comparison applied generalized linear mixed models with an exchangeable covariance structure to account for repeated observations.30 This basis for inference ensures simultaneous adjustment for autoregressive error in time series models31 and hierarchical adjustment formultilevel clustering.32 For each dependent variable of interest, a model of monthly time series data takes the form:
where
yij is a DHIMS-reported value of outcome y from facility j at time i,
xij is a dummy variable defining whether facility j is in the SERC area or in a control area,
tij is a dummy variable defining whether time i is before or after the start of the SERC intervention,
uj is a random intercept for facility j, and
εij is a random error term for facility j at time i.
The parameters β0, β1, β2, and β3 are estimated by maximum likelihood, with β3 estimating the difference-in-difference association of SERC exposure with the number of events of interest recorded by 14 hospital facilities over 70 months of observation. This approach to evaluation represents a regression extension of the Heckman procedure for estimating the impact of interventions in nonexperimental designs.33,34
The sign of the 3 coefficient in the equation indicates the direction of the net change in expected monthly case volume between treatment and control: A positive sign implies a positive SERC effect on case volume (i.e., an increase in the treatment facility volume relative to the control), and a negative sign implies a negative SERC effect on case volume (i.e., a decrease in the treatment facility volume relative to the control). For example, a value of +6 for β3 in the equation for an analysis of facility-based delivery would indicate that the expected mean number of monthly deliveries in the treatment facilities increased by 6 deliveries relative to the mean volume of deliveries in the comparison area between the pre- and post-intervention periods. Statistical tests for this coefficient assess whether this relative change is significant. We employed a similar difference-in-difference approach to evaluate the effect of SERC on maternal mortality; however, we substituted a generalized linear Poisson model to properly estimate the maternal mortality ratio. Repeated observations within a facility were adjusted by assuming an exchangeable correlation structure.35,36 Table 1 reports robust standard errors obtained via the sandwich operator.37 Although the time series models in this analysis have employed conventional adjustments and statistical safeguards, all such models incur an element of instability. Results therefore merit further investigation and validation.38
TABLE 1.
(1) | (2) | (3) | (4) | (5) | (6) | (7) | (8) | (9) | |
---|---|---|---|---|---|---|---|---|---|
Deliveries | Cesarean Delivery Rate | Referrals In | Referrals Out | Pneumonia | Other Upper Respiratory Tract | Septicemia | Accidents | Diarrheal Diseases | |
Treatment area | −52.93* | −0.00651 | −12.08** | −6.499 | −4.537 | −100.5** | 2.555 | −15.35 | 20.96 |
(26.18) | (0.0113) | (4.345) | (3.735) | (8.015) | (37.8) | (25.98) | (8.088) | (36.05) | |
Time period | 32.55*** | 0.025** | −3.05* | −6.80*** | 0.687 | 35.57 | 23.64* | −3.24 | 18.12 |
(9.55) | (0.00964) | (1.435) | (1.87) | (12.75) | (47.67) | (11.9) | (3.209) | (19.13) | |
SERCa | −4.88 | 0.0035 | 12.27* | 1.60 | 10.99 | 22.57 | 35.09 | 20.52* | 11.71 |
(12.76) | (0.015) | (5.18) | (3.52) | (12.92) | (49.82) | (41.91) | (9.90) | (33.54) | |
Constant | 89.73*** | 0.12*** | 17.45*** | 12.18*** | 46.96*** | 237.40*** | 28.38 | 28.17*** | 72.04*** |
(23.81) | (0.02) | (4.17) | (3.04) | (14.13) | (58.81) | (30.88) | (6.84) | (14.96) | |
Observations | 861 | 795 | 361 | 500 | 787 | 748 | 237 | 796 | 804 |
Number of hospitals | 14 | 13 | 13 | 14 | 14 | 14 | 10 | 14 | 14 |
Abbreviation: SERC, Sustainable Emergency Referral Care.
Note: Estimates are from multilevel linear regressions of outcomes from monthly hospital records in the Upper East and Upper West Regions of Ghana. Regressions include random facility intercepts to account for clustering at the facility level. Standard errors are calculated assuming an exchangeable correlation structure and are reported in parentheses.
* P<.05; ** P<.01; *** P<.001.
a The SERC effect (difference-in-difference) is given by the interaction of treatment area with time period.
Survey of Health Workers
A survey was administered to health workers by trained professional interviewers in December 2013 to assess their perspectives on SERC components and challenges. The questionnaire was pretested and then revised based on feedback provided. The sampling frame comprised all staff based at subdistrict- and community-level health facilities that were involved in the SERC program, including staff affiliated with ambulance stations, responsible for referral operations at ambulance stations, or charged with receiving SERC referrals. This yielded a list of 124 health workers and a response rate of 89% (N = 110), as 14 potential respondents were on annual leave during the week the survey was administered. During that 1-week period, the 110 respondents completed the survey instruments as self-administered questionnaires. Respondents were encouraged to provide candid feedback, and were provided with de-identified forms and blank envelopes to preserve anonymity. These procedures assured respondents of confidentiality so that they could answer questions without risk that critical comments would incur supervisory concern or reprisal.
Qualitative Systems Appraisal
Trained professional facilitators conducted a qualitative systems appraisal in March 2014 to assess community stakeholder, patient, and volunteer experiences with SERC, employing qualitative research procedures that have been applied to CHPS assessment in the past.28,29 An analysis of focus group discussions (FGDs) and in-depth interviews (IDIs) with community stakeholders assessed the acceptability of the intervention at the community level. IDIs were also used to examine patient experiences with SERC, satisfaction with care, and suggestions for improvement. A total of 16 FGDs were conducted with men, women, drivers, and volunteers from the subdistricts implementing the SERC program. Twenty-three IDIs were conducted with chiefs, emergency referral users, and volunteers equipped with emergency phones. To enhance representativeness, each focus group category (i.e., women, men, community health volunteers) was sampled in a different community, and each FGD was held within the community to promote discussion. Respondents provided written consent; all IDIs and FGDs were conducted in local languages Buili and Guruni; and all interviews were tape-recorded, transcribed, and analyzed using the NVivo 9 software package.39
RESULTS
Findings From the Time Series Analysis of Key Indicators
From July 2013 through June 2015, 1,290 patients used SERC transport services. The average trip time and distance traveled were 56.6 minutes and 9.92 kilometers, respectively. Most referrals were to higher levels of care at subdistrict health centers and district hospitals, with a high concentration of care occurring at 2 facilities that are well staffed and equipped to manage emergencies (Figure 1). The next most common reasons for referral included malaria, anemia, diarrhea, acute respiratory illnesses, and injury. Most patients (98%) were treated and discharged successfully, while 2% of all emergency referrals resulted in death.
The referral profile changed over time as SERC progressed (Figure 2). However, regardless of time period, obstetric cases were the predominant type of referral. Consequently, nearly three quarters of patients were women. Although inappropriate use of the referral system could not be monitored directly, care for minor situations that were not emergencies tended to be labeled as “other” types of referrals. As Figure 2 shows, the proportion of such referrals declined with time, suggesting that the high initial frequency of inappropriate SERC referrals may have diminished as operations progressed.
Table 1 presents the association of SERC exposure with facility output indicators. At baseline, before the introduction of SERC services, there were approximately 53 fewer deliveries per month in hospitals in the SERC intervention area. These facilities also received fewer referrals and reported fewer upper respiratory tract infections at baseline than the facilities serving the comparison area. Cesarean delivery rates were no different between facilities in the SERC and comparison areas at baseline. SERC had no statistically significant effect on the number of deliveries; the cesarean delivery rate; the number of referrals “out” from sub-district clinics to district hospitals or the number of pneumonia cases, other respiratory tract infections, septicemia cases, or diarrheal disease cases. However, SERC did increase the number of referrals into district hospitals from CHPS workers and clinics by more than 12 patients per month and the number of accidents treated by almost 21 per month.
There was a shift in the location of delivery care within districts where SERC was introduced. In the SERC area, more deliveries occurred at facilities capable of acute care (i.e., district hospitals), displacing delivery care at health centers and clinics where surgical procedures are not performed (Figure 3). Hospitals staffed and equipped to provide acute care also received more referrals where SERC was operative than elsewhere (Table 1, column 3). This relocation of care was associated with a reduction in facility-based maternal mortality (incidence rate difference, −352; 95% confidence interval, −639 to −65; P<.05) (Table 2), although there was no significant effect on the cesarean delivery rate (Table 1, column 2). There were several specific indicators of volume of acute care episodes, and only the volume of care for accidents and for maternal emergencies appear to have been affected (Table 1, columns 3 and 8, respectively). The impact of SERC on acute care for accident victims is important, not only for the evidence shown in Table 1 but also because evidence now suggests that modest economic gains in the region have led to dramatic increases in the purchase of motorbikes, with accident-related morbidity and mortality rapidly expanding as a consequence.18 Time series regression results can be unstable owing to autoregressive error.38 Nevertheless, the relationships demonstrated in Table 1 suggest that SERC has had effects on mortality.
TABLE 2.
MMR (95% CI) |
Differences |
|||
---|---|---|---|---|
Pre-SERC | Post-SERC | IRD (95% CI) | IRR (95% CI) | |
Comparison district hospitals (n = 12) | 326 (272, 380) | 261 (194, 328) | −65 (−140, 10) | 0.80† (0.61, 1.05) |
Intervention district hospitals (n = 2) | 618 (392, 844) | 201 (22, 381) | −417** (−693, −140) | 0.33* (0.13, 0.83) |
Difference-in-differences (SERC effect) | −352* (−639, −65) | 0.41† (0.15, 1.07) |
Abbreviations: CI, confidence interval; IRD, incidence rate difference (deaths per 100,000 live births); IRR, incidence rate ratio; MMR, maternal mortality ratio; SERC, Sustainable Emergency Referral Care.
Note: Estimates are from multilevel Poisson regressions of monthly hospital records of births and maternal deaths at 14 facilities in the Upper East and Upper West Regions of Ghana from 2009 to 2015. The hospital MMR is calculated as the number of facility-based deaths per 100,000 live births. The 95% confidence intervals were calculated using robust standard errors accounting for clustering at the facility level.
* P<.05; **P<.01; ***P<.001; ‡ P<.10.
Hospitals staffed and equipped to provide acute care received more referrals where SERC was operative than elsewhere.
The information monitored included process indicators such as staff compliance with protocols. Contrary to guidelines, less than half (49%) of the patients transported were accompanied by referring health workers, and receiving facilities were alerted to incoming patients in only 46% of the monitored referral episodes.
Findings From the Health Worker Survey
Of the 110 survey respondents, over half were community health officers (56%); the remainder were clinic-based nurses (25%), midwives (13%), or physician assistants (6%). Places of work included community health posts (69%), subdistrict health centers (27%), and hospitals (4%). Nearly three-quarters (74%) of the respondents had personally referred patients using SERC's transport service since the program launched.
Perceived Effectiveness and Safety of Motorking Ambulances
The majority of health workers (66%) considered SERC to be “very effective” in improving the community- and subdistrict-level emergency referral system, and 33% considered Motorkings to be “somewhat effective.” Most health workers perceived the Motorking to be safe, with 26% categorizing the vehicle as “very safe” and 61% considering it to be “somewhat safe.”
Two-thirds of health workers considered SERC to be “very effective” in improving the emergency referral system.
Driver Dedication and Availability
Respondents generally perceived drivers to be dedicated to their roles (56% reported finding them “very dedicated”; 41% found them “somewhat dedicated,” and only 3% found them “not at all dedicated”). However, there were instances when health workers were unable to promptly contact the volunteer drivers. When asked whether health workers at ambulance stations should be trained to drive Motorkings in such instances, the majority of respondents (75%) said yes.
Protocol Adherence
Protocol noncompliance was evident for some aspects of care. For instance, referring facilities often failed to call in advance to alert receiving facilities of an incoming patient. Moreover, many patients transferred were unaccompanied by a health worker despite the protocol requiring it. Although surveyed health workers nearly universally affirmed the importance of these procedures, 67% of the referred patients who were interviewed reported that they had been transferred without accompaniment. When workers were asked why they were unable to accompany the patient, the most commonly cited reasons (respondents could choose more than one) were that another health worker accompanied the patient (37%); the respondent was the only staff member at the facility and could not leave the post (35%); or the respondent was attending another patient (32%). Some respondents reported that they did not accompany the patient because they did not feel comfortable riding in the ambulance (16%), while 4% thought the patient would not benefit from riding with a health worker. Protocols also obligate receiving facilities to provide patient outcome feedback to the referring facilities for every case, but this requirement was typically ignored.
Referring facilities often failed to alert receiving facilities of an incoming patient, and many patients were unaccompanied by a health worker during transit.
Frontline Worker Perspectives
Surveyed health workers were asked to identify the primary challenges to effective emergency referral services (the health workers could choose more than one challenge). Poor road conditions (95%) was the most commonly reported challenge, followed by lack of driver motivation (59%); cultural practices that delay care seeking and lack of knowledge of the importance of seeking care (40%); poor communications networks (32%); and adverse weather conditions (29%). Less frequently reported responses included patient inability to take time away from work or family obligations (20%); the cost or unavailability of fuel (19%); poor communication between health facilities (18%); lack of readily available transport options (13%); or lack of Motorking acceptability (12%).
Findings From the Qualitative Systems Appraisal
The qualitative systems appraisal shed light on the acceptability of services and on community perspectives on potential areas for improvement. Four main categories of themes emerged during data analysis: community endorsement of SERC; logistical challenges; communication challenges; and interpersonal relationships.
Community Endorsement of SERC
Overall, community members strongly endorsed SERC and expressed appreciation for the service. SERC was generally perceived as reliable and reactive, with a committed staff that supported the system. For instance, a woman who had used SERC said this:
It sent me to the clinic to deliver and I did that safely without any bad thing happening to me. I delivered safely. That is the beauty of it.
Several users reported that they would recommend SERC services to anyone in need of emergency care. The removal of fees for pregnant women and children under 5 was seen as a key contributor to high SERC uptake. Although some participants preferred 4-wheeled ambulances, respondents generally believed that the 3-wheeled ambulance was better than the available alternatives, such as walking, bicycles, donkey carts, or motorbikes:
The removal of fees for transporting pregnant women and children under 5 was seen as a key contributor to high uptake.
It has been so beneficial to the pregnant women and the children under 5 because they do not pay when the vehicle is transporting them. In the past, we used to transport pregnant women in donkey carts and on bicycles but today there is ready and reliable means of transport for them in emergencies. —Community volunteer participating in an FGD
Some participants acknowledged that 3-wheeled vehicles such as the Motorking can traverse narrow passages that are inaccessible to 4-wheeled vehicles:
If not for the Motorking, women, especially pregnant women and children, would have been suffering a lot. … It is able to go to the interior [of communities] to carry cases like the one I told you about with the woman who was in labor and nearly died if not for the sake of the Motorking ambulance. —Community subchief, in an IDI
Perceptions of reduced delays and increased numbers of facility-based births as a result of SERC were mentioned by several participants, along with the impression that SERC services were helpful, safe, quick, and lifesaving:
When a woman is in labor and is not quickly sent to the health facility, she might deliver. She might also lose either the baby or even herself. Kids like this, once they are weak, they can easily pass on. So the impact I see is that the emergency referral saves lives. —SERC driver participating in an FGD
Community and household consensus endorsing SERC was uniformly evident in each FGD and IDI and was a key determinant of the sustainability of the system.
Logistical Challenges
While communities were receptive to SERC services, several logistical challenges were identified. Some intervention areas remained inaccessible due to harsh weather and terrain, especially during the rainy season. (These challenges were perceived to affect all vehicle types, not just the Motorkings.) Some communities conveyed interest in overcoming logistical or geographical barriers through collective action or political advocacy. As 2 FGD respondents suggested:
Some communities conveyed interest in overcoming logistical or geographical barriers through collective action or political advocacy.
I am of the view that the community members can contribute something, however little, and seek assistance from the authorities to work on our routes or roads for us.
Our youth, if they could help us to repair our roads small, small [bit by bit], and when the motors come, they can be running without problems.
Although concerns about roads did not constrain SERC use, some participants noted that communities that were remote from an ambulance station anticipated delay and often sought alternative means of emergency care. Indeed, this observation is consistent with GIS data analyses showing that communities located far from ambulance stations had lower use rates than proximate communities.
Communication Challenges
Communication problems introduced further complications. Poor phone networks, which are common in rural Ghana, exacerbated service delays. Although this did not compromise care seeking resolve, solutions to communication gaps sometimes involved walking great distances to alert a health worker or volunteer.
Poor phone networks did not compromise decisions to seek care, but solutions sometimes involved walking great distances to alert a health worker or volunteer.
Patient perspectives on comfort during transport varied: Some patients described the vehicle as being unstable and uncomfortable, while others described feeling very safe, with minimal discomfort. This problem was associated with poor road quality. Any discomfort, however, did not appear to be severe enough to deter people from using SERC in the event of emergencies:
There are issues like discomfort, safety, and others when you are being transported, but as a sick person you do not have those issues in mind when there is an emergency. Anything that can hurriedly get you to the place on time is what you will be looking for. All vehicles have the tendency of falling when transporting people so it will not be fair relating safety issues to the Motorking alone. —Man participating in an FGD
Everyone has his problem, and when the vehicle picked me [up] the driver knew that it was a painful thing being in labor, so they also became careful with the way they were driving and we got there safely. Now I will not be able to speak for another person, but for my experience it was comfortable. —Woman who had used SERC, in an IDI
Community members also expressed support for improving the work conditions for drivers. There were concerns over drivers being exposed to unfavorable weather and the risk of robbery during late-night service. Although no such incidents were reported, a few drivers worried about driving at night:
There are beasts at night and also ghosts. From where I come, there are so many spirits that it is not advisable to move out at night. The people sit protected in the vehicle while you are left alone in front. In addition to that you are not supposed to speed the vehicle, and you can imagine how exposed you are if someone intends to harm you. —SERC driver participating in an FGD
Drivers advised SERC to develop roadside repair protocols for addressing unanticipated mechanical problems. Drivers also noted that personal transportation was a challenge, as many drivers had no means of personal transportation home following late-night referrals. Drivers were provided with 2 bars of soap monthly as a token of appreciation, and this was universally perceived as being insufficient. Cash incentives were preferred by all drivers who were interviewed. Staff participating in an FGD considered cash payments as being critical to sustaining driver motivation in the future. Some community members recommended that SERC choose drivers from the ranks of existing community health volunteers, given prevailing volunteer commitment to community health.
Community Trust and Expectations
Some participants noted instances of mistrust between health workers and drivers. For example, a driver mentioned an episode where the network was down but a health worker accused him of having turned off his phone. In another example, a man explained during an FGD how at times users might misconstrue basic triage practices as health worker neglect:
Some of the pregnant women will be complaining that they came and they are thrown away, they don't care about them. Because there is no understanding between the pregnant women and the midwife when she tells them it's not time for them to deliver and they should wait. Because of that, the women say the workers are not serious, but for me, the way I know about the work, I know they are serious.
Although some patients experienced negative interactions with health workers, many described satisfaction with their performance during emergencies:
We think that the child was saved by the nurses because of the timeliness of our arrival. We were happy when we got into the hands of the nurses. —Woman who had used SERC, in an IDI
Drivers expressed concern that the community lacked respect for their contribution. Some community members believed that drivers were paid employees rather than volunteers, and some drivers reported frustration over receiving dismissive and ungrateful comments. Although most community members interviewed indicated gratitude for drivers' services, some complained that drivers operated Motorkings at unsafe speeds.
DISCUSSION
Mixed-methods implementation research enriched learning about the scalability, acceptability, and potential impact of implementing a community-based emergency referral system in a severely resource-constrained setting. The findings suggest that the strategies used for the emergency referral system can be adapted to the needs of impoverished, remote communities elsewhere in Ghana.
Overall, the SERC system was well received by communities and health workers alike as an effective means of reducing acute care risks. A key lesson learned was the importance of people-centered planning for obtaining and sustaining community endorsement and use of services. Without the engagement of community leaders from the very beginning, acceptance of the program would have been limited. Focused outreach targeting heads of household and familial gatekeepers is also crucial to ensuring continued support and understanding of services. Moreover, the collaborative role of transportation authorities and vehicle manufacturers in the planning, training, and implementation processes proved vital to program success.
While SERC aimed to use process evaluation results to improve system functioning, the pursuit of such improvements was constrained by resource limitations, poor communication network infrastructure, and impassible roads. Nearly all the health workers consulted in this appraisal expressed willingness to use emergency radios to offset poor mobile phone coverage. However, given limited funds for equipment purchases and lack of locally available communication equipment, use of radio devices could not be implemented. Instead, workers were obligated to develop improvised solutions when networks were not functioning.
The Motorking was locally available, affordable, and suitable for traversing rough terrain. Nonetheless, Motorking ambulances received mixed reviews for comfort and safety, indicating a need to explore additional equipment options. A costing analysis that compares 3-wheeled motorcycle ambulances with enhanced Motorkings or higher-quality vehicles is warranted. Similarly, strategies should be investigated for determining an appropriate and sustainable incentive and recruitment system for drivers in order to minimize turnover, improve motivation, and optimize efficiency.
Quite apart from equipment considerations, the quality of emergency care services will be limited by the poor state of infrastructure more generally. Several of the community members who were interviewed expressed concern about the status of the development of primary health care facilities and the slow pace of CHPS implementation, highlighting the fact that effective referral care requires a fully functioning primary health care system.
Effective referral care requires a fully functioning primary health care system.
The SERC experience attests to the value of routine monitoring and evidence-based supervision, in conjunction with refresher training for health workers and volunteers. Lack of accountability mechanisms, supervision, and training can lead to poor adherence to protocols.
Feedback mechanisms are needed to foster timely implementation of systems improvements. For instance, after it was discovered that 30% of the trip report forms were incomplete, the forms were simplified, the format of review meetings was revised, and GIS-based vehicle tracking procedures were instituted to facilitate practical use of data for decision making. Similarly, adhering to a routine vehicle maintenance protocol that ensures prompt repairs was found to be crucial for preventing breakdowns and minimizing service disruptions.
Training for quality assurance is important. Although most patients reported positive experiences with staff involved in facilitating referrals, some patients experienced negative or insensitive comments. While clinical skills are crucial to operations, it is equally essential to foster health workers' patience and understanding of patients' perspectives on the quality of emergency care operations.
For quality emergency care, clinical skills, patience, and understanding of patients' perspectives are equally essential.
Some volunteer drivers perceived community members to be unappreciative of their services. Although FGD participants may have been reluctant to criticize drivers, the general discussion suggests that drivers were, in fact, appreciated, and participants generally agreed that the incentives provided to drivers should be increased. The exchanges among FGD participants nonetheless suggested a need for durbars and other means of community engagement that would promote awareness of the lifesaving service and dedication of volunteers.
Focus group discussions suggested a need for community engagement to promote awareness of the service and dedication of volunteers.
Remote communities sometimes preferred to find their own means of transport to offset ambulance delays. This fact attests to the need for implementation research that investigates the mechanisms such communities use for emergency referral and transport. Community-based solutions to referral problems may be relevant to operations more generally.
SERC has made its impact by transporting emergency cases, at considerable cost, to distant hospitals where physicians are available to provide essential acute care. However, bypassing subdistrict clinics and relocating care to hospitals is less sustainable than developing service capability at the subdistrict level. While bypassing for delivery is a logical and common strategy in Ghana40,41 and elsewhere in Africa,42,43 the implication of this finding is clear: A new round of implementation research is needed to explore implementing SERC in concert with a program that trains midlevel providers to manage emergencies directly.
CONCLUSION
While facility-based emergency health care is important to reducing mortality, facility-focused approaches can fail to achieve their full lifesaving potential in the absence of effective referral. Moreover, if receiving facilities are poorly equipped, inadequately staffed, and unable to respond to clinical emergency needs, effective referral will be little more than a program for relocating mortality. The SERC time series research presented in this article attests to the lifesaving potential of redirecting referral to facilities where emergencies can be competently managed. In the areas where SERC rechanneled acute care to specified facilities, we found decreased facility-based maternal mortality and accident-related deaths relative to comparison facilities. In the future, SERC could expand its intervention regimen with training and capacity building to enable more frontline care providers in smaller facilities to more effectively manage emergencies that arise. This strategy would offset the existing strategies for bypassing substandard care facilities.
In the future, SERC could train more frontline care providers to more effectively manage emergencies that arise.
Just as SERC's success was dependent on implementation research and evidence, effective scale-up of these results will require effective systems approaches to replication trials in other regions of Ghana, with contrasting social, economic, and ecological conditions. The need for reform of referral systems persists throughout the country, but care should be taken to develop solutions that are informed by SERC, yet tailored to local contexts in the central and southern regions. The transition from pilot to trial clarified training and engagement requirements; replication of SERC elsewhere in Ghana could clarify the practical milestones in establishing a large-scale system of referral care.
The Ghana Health Service has adopted the SERC model and has included it in the national CHPS implementation guidelines. Donors including the Japan International Cooperation Agency, the Korean International Cooperation Agency, and the U.S. Agency for International Development have funded the purchase of Motorkings, which are being used in 4 of Ghana's 10 regions, with plans to use them in 2 additional regions.
Acknowledgments
The SERC initiative was funded by grants to the University of Ghana School of Public Health and the Ghana Health Service by the British charity Comic Relief and by grants to the Columbia University Mailman School of Public Health by the Doris Duke Charitable Foundation.
Competing Interests: None declared.
Peer Reviewed
Cite this article as: Patel S, Awoonor-Williams JK, Asuru R, Boyer CB, Tiah JAY, Sheff MC, Schmitt ML, Alirigia RM, Jackson EF, Phillips JF. Benefits and limitations of a community-engaged emergency referral system in a remote, impoverished setting of Northern Ghana. Glob Health Sci Pract. 2016;4(4):552–567. http://dx.doi.org/10.9745/GHSP-D-16-00253
REFERENCES
- 1.World Health Organization (WHO). Maternal mortality: to improve maternal health, barriers that limit access to quality maternal health services must be identified and addresses at all levels of the health system: fact sheet. Geneva: WHO; 2014. Available from: http://apps.who.int/iris/handle/10665/112318 [Google Scholar]
- 2. Bhutta ZA, Ali S, Cousens S, Ali TM, Haider BA, Rizvi A, et al. Interventions to address maternal, newborn, and child survival: what difference can integrated primary health care strategies make? Lancet. 2008;372(9642):972–989. 10.1016/S0140-6736(08)61407-5. [DOI] [PubMed] [Google Scholar]
- 3. Kinney MV, Kerber KJ, Black RE, Cohen B, Nkrumah F, Coovadia H, et al. Science in Action: Saving the lives of Africa’s Mothers, Newborns, and Children working group. Sub-Saharan Africa’s mothers, newborns, and children: where and why do they die? PLoS Med. 2010;7(6):e1000294. 10.1371/journal.pmed.1000294. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Liu L, Johnson HL, Cousens S, Perin J, Scott S, Lawn JE, et al. Child Health Epidemiology Reference Group of WHO and UNICEF. Global, regional, and national causes of child mortality: an updated systematic analysis for 2010 with time trends since 2000. Lancet. 2012;379(9832):2151–2161. 10.1016/S0140-6736(12)60560-1. [DOI] [PubMed] [Google Scholar]
- 5. Hsia RY, Mbembati NA, Macfarlane S, Kruk ME. Access to emergency and surgical care in sub-Saharan Africa: the infrastructure gap. Health Policy Plan. 2012;27(3):234–244. 10.1093/heapol/czr023. [DOI] [PubMed] [Google Scholar]
- 6. Hsia R, Razzak J, Tsai AC, Hirshon JM. Placing emergency care on the global agenda. Ann Emerg Med. 2010;56(2):142–149. 10.1016/j.annemergmed.2010.01.013. [DOI] [PubMed] [Google Scholar]
- 7. Razzak JA, Kellermann AL. Emergency medical care in developing countries: is it worthwhile? Bull World Health Organ. 2002;80(11):900–905. Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2567674/pdf/12481213.pdf. [PMC free article] [PubMed] [Google Scholar]
- 8. Murray SF, Pearson SC. Maternity referral systems in developing countries: Current knowledge and future research needs. Soc Sci Med. 2006;62(9):2205–2215. 10.1016/j.socscimed.2005.10.025. [DOI] [PubMed] [Google Scholar]
- 9.World Health Organization (WHO); United Nations Population Fund (UNFPA); United Nations Children’s Fund (UNICEF); World Bank. Pregnancy, childbirth, postpartum and newborn care: a guide for essential practice. Geneva: WHO; 2003. Available from: http://www.afro.who.int/index.php?option=com_docman&task=doc_download&gid=2011 [PubMed] [Google Scholar]
- 10. Campbell OMR, Graham WJ. Lancet Maternal Survival Series steering group. Strategies for reducing maternal mortality: getting on with what works. Lancet. 2006;368(9543):1284–1299. 10.1016/S0140-6736(06)69381-1. [DOI] [PubMed] [Google Scholar]
- 11. Ganga-Limando M, Moleki M, Modiba L. Potential barriers to utilization of maternal health services in public health facilities in rural and remote communities: a qualitative study. Life Sci J. 2014;11(10):973–979. Available from: http://www.lifesciencesite.com/lsj/life1110/150_26757life111014_973_979.pdf [Google Scholar]
- 12. Kowalewski M, Jahn A, Kimatta SS. Why do at-risk mothers fail to reach referral level? Barriers beyond distance and cost. Afr J Reprod Health. 2000;4(1):100–109. [PubMed] [Google Scholar]
- 13. Lule G, Tugumisirize J, Ndekha M. Quality of care and its effects on utilisation of maternity services at health centre level. East Afr Med J. 2000;77(5):250–255. [DOI] [PubMed] [Google Scholar]
- 14. Olokunde TL, Awoonor-Williams JK, Tiah JA, Alirigia R, Asuru R, Patel S, et al. Qualitative assessment of community education needs: a guide for an educational program that promotes emergency referral service utilization in Ghana. J Community Med Health Educ. 2015;5(4):363 10.4172/2161-0711.1000363 [DOI] [Google Scholar]
- 15.Ghana Statistical Service. Poverty profile in Ghana (2005–2013): Ghana living standards survey round 6. Accra (Ghana): Ghana Statistical Services; 2014. [Google Scholar]
- 16. Macintyre K, Hotchkiss DR. Referral revisited: community financing schemes and emergency transport in rural Africa. Soc Sci Med. 1999;49(11):1473–1487. 10.1016/S0277-9536(99)00201-4. [DOI] [PubMed] [Google Scholar]
- 17. Thaddeus S, Maine D. Too far to walk: maternal mortality in context. Soc Sci Med. 1994;38(8):1091–1110. [DOI] [PubMed] [Google Scholar]
- 18. Bawah AA, Welaga P, Azongo DK, Wak G, Phillips JF, Oduro A. Road traffic fatalities–a neglected epidemic in rural northern Ghana: evidence from the Navrongo Demographic Surveillance System. Inj Epidemiol. 2014;1(1):22. 10.1186/s40621-014-0022-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19. Adamtey R, Frimpong J, Dinye RD, Čater T. An analysis of emergency healthcare delivery in Ghana: lessons from ambulance and emergency services in Bibiani Anhwiaso Bekwai District. Ghana J Develop Stud. 2015;12(1–2):71–87. 10.4314/gjds.v12i1-2.5 [DOI] [Google Scholar]
- 20. Atuoye KN, Dixon J, Rishworth A, Galaa SZ, Boamah SA, Luginaah I. Can she make it? Transportation barriers to accessing maternal and child health care services in rural Ghana. BMC Health Serv Res. 2015;15:333. 10.1186/s12913-015-1005-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21. Krasovec K. Auxiliary technologies related to transport and communication for obstetric emergencies. Int J Gynaecol Obstet. 2004;85 Suppl 1:S14–S23. 10.1016/j.ijgo.2004.02.007. [DOI] [PubMed] [Google Scholar]
- 22. Macintyre K, Lochigan M, Letipila F. Understanding referral from primary care clinics in rural Kenya: using health information systems to prioritize health services. Int J Health Plann Manage. 2003;18(1):23–39. 10.1002/hpm.689. [DOI] [PubMed] [Google Scholar]
- 23.Ghana Health Service. Annual report of the Upper East Region (UER), Bolgatanga, Upper East Region. Accra (Ghana): Ghana Health Service; 2012. [Google Scholar]
- 24.Ghana Statistical Service. Ghana living standards survey report of the fifth round (GLSS 5), 2008. Accra (Ghana): Ghana Statistical Service; 2008. Available from: http://www.statsghana.gov.gh/docfiles/glss5_report.pdf. [Google Scholar]
- 25. Awoonor-Williams JK, Bawah AA, Nyonator FK, Asuru R, Oduro A, Ofosu A, et al. The Ghana Essential Health Interventions Program: a plausibility trial of the impact of health systems strengthening on maternal & child survival. BMC Health Serv Res. 2013;13 Suppl 2:S3. 10.1186/1472-6963-13-S2-S3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Ghana Health Service. Community-based health planning and services (CHPS): the operational policy. Policy Document No. 20. Accra (Ghana): Ghana Health Service; 2005. Available from: http://www.moh.gov.gh/wp-content/uploads/2016/02/CHPS-Operational-Policy-2005.pdf. [Google Scholar]
- 27. Cornwall A, Jewkes R. What is participatory research? Soc Sci Med. 1995;41(12):1667–1676. 10.1016/0277-9536(95)00127-S. [DOI] [PubMed] [Google Scholar]
- 28. Nazzar A, Adongo PB, Binka FN, Phillips JF, Debpuur C. Developing a culturally appropriate family planning program for the Navrongo experiment. Stud Fam Plann. 1995;26(6):307–324. 10.2307/2138097. [DOI] [PubMed] [Google Scholar]
- 29. Nyonator F, Jones TC, Miller RA, Phillips JF, Awoonor-Williams JK. Guiding the Ghana community-based health planning and services approach to scaling up with qualitative systems appraisal. Int Q Community Health Educ. 2003;23(3):189–213. Available from: http://journals.sagepub.com/doi/pdf/10.2190/NGM3-FYDT-5827-ML1P. [Google Scholar]
- 30. Leiva R, Roy A. Linear discrimination for multi-level multivariate data with separable means and jointly equicorrelated covariance structure. J Stat Plan Inference. 2011;141(5):1910–1924. 10.1016/j.jspi.2010.12.001 [DOI] [Google Scholar]
- 31. McCulloch CE. Maximum likelihood algorithms for generalized linear mixed models. J Am Stat Assoc. 1997;92(437):162–170. 10.1080/01621459.1997.10473613 [DOI] [Google Scholar]
- 32. Burgess JF, Christiansen CL, Michalak SE, Morris CN. Medical profiling: improving standards and risk adjustments using hierarchical models. J Health Econ. 2000;19(3):291–309. 10.1016/S0167-6296(99)00034-X. [DOI] [PubMed] [Google Scholar]
- 33. Heckman JJ. Effects of child-care programs on women’s work effort. J Polit Econ. 1974;82(2, Part 2):S136–S163. 10.1086/260297 [DOI] [Google Scholar]
- 34. Dimick JB, Ryan AM. Methods for evaluating changes in health care policy: the difference-in-differences approach. JAMA. 2014;312(22):2401–2402. 10.1001/jama.2014.16153. [DOI] [PubMed] [Google Scholar]
- 35. Zeger S, Liang K. Longitudinal data analysis for discrete and continuous outcomes. Biometrics. 1986;42(1):121–130. [PubMed] [Google Scholar]
- 36. Zeger S, Liang K, Albert PS. Models for longitudinal data: a generalized estimating equation approach. Biometrics. 1988;44(4):1049–1060. [PubMed] [Google Scholar]
- 37. Moulton LH, Dibley MJ. Multivariate time-to-event models for studies of recurrent childhood diseases. Int J Epidemiol. 1997; 26(6):1334–1339. 10.1093/ije/26.6.1334. [DOI] [PubMed] [Google Scholar]
- 38. Parker SC. Opening a can of worms: the pitfalls of time-series regression analyses of income inequality. Appl Econ. 2000;32(2):221–230. 10.1080/000368400322903 [DOI] [Google Scholar]
- 39. Berg BL. Qualitative research methods in the social sciences. Fifth edition Boston: Pearson; 2004. [Google Scholar]
- 40. Yaffee AQ, Whiteside LK, Oteng RA, Carter PM, Donkor P, et al. Bypassing proximal health care facilities for acute care: a survey of patients in a Ghanaian accident and emergency centre. Trop Med Int Health. 2012;17(6):775–781. 10.1111/j.1365-3156.2012.02984.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41. Saleh K. World Bank study: a health sector in transition to universal coverage in Ghana. Washington (DC): World Bank; 2012. Available from: https://openknowledge.worldbank.org/handle/10986/2728 [Google Scholar]
- 42. Kahabuka C, Kvåle G, Moland KM, Hinderaker SG. Why caretakers bypass primary health care facilities for child care–a case from rural Tanzania. BMC Health Serv Res. 2011;11(315):315. 10.1186/1472-6963-11-315. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43. Kanté AM, Exavery A, Phillips JF, Jackson EF. Why women bypass front-line health facility services in pursuit of obstetric care provided elsewhere: a case study in three rural districts of Tanzania. Trop Med Int Health. 2016;21(4):504–514. 10.1111/tmi.12672. [DOI] [PubMed] [Google Scholar]