Assessing the quality of primary care in Haiti

Abstract

Objective

To develop a composite measure of primary care quality and apply it to Haiti’s primary care system.

Methods

Using the Primary Health Care Performance Initiative’s framework, we defined four domains of primary care service delivery: (i) accessible care; (ii) effective service delivery; (iii) management and organization; and (iv) primary care functions. We gave each primary care facility in Haiti a quality score for each domain and overall, with poor, fair and good quality indicated by scores of 0.00–0.49, 0.50–0.74 and 0.75–1.00, respectively. We quantified access and effective access to primary care as the proportions of the population within 5 km of any primary care facility and a good facility, respectively.

Findings

Of the 786 primary care facilities in Haiti in 2013, only 332 (43%) facilities were classified as good for accessible care. Fewer facilities were classified as good in the domains of effective service delivery (30; 4%), management and organization (91; 12%) and primary care functions (43; 5%). Although about 91% of the population lived within 5 km of a primary care facility, only an estimated 23% of the entire population – including just 5% of the rural population – had access to primary care of good quality.

Conclusion

Despite an extensive network of health facilities, a minority of Haitians had access to a primary care facility of good quality. Such facilities were especially scarce in rural areas. Similar systematic analyses of the quality of primary care could inform national efforts to strengthen health systems.

Introduction

Thirty years after the Declaration of Alma-Ata, the 2008 World Health Report declared that primary health care was a global priority “now more than ever”.¹ Primary care forms the cornerstone of a functional health system. High-quality primary care systems can improve health outcomes, increase equity in health care and optimize efficient use of resources.^2–4 In low- and middle-income countries, however, primary care is often poor, with a general lack of provider effort, high rates of misdiagnosis and incorrect treatment, and long wait times.^5–8

Research on the quality of primary care includes investigations of provider behaviour and knowledge,⁵ programme evaluations⁹ and small-scale case studies.¹⁰ Broader assessments of primary care systems, particularly in the wake of conflict¹¹ or natural disaster,¹² have included the development of balanced scorecards. These scorecards have focused on infrastructure inputs and community perspectives and given relatively little attention to the processes of care. One limitation of the research in this field is the lack of a comprehensive definition of primary care quality that is applicable across contexts and countries.

In an effort to guide quality measurement and improvement in the field of primary care, the Primary Health Care Performance Initiative reviewed over 40 different conceptual frameworks of primary care and consolidated them into a single framework.¹³ This framework, which is still evolving, unifies previous work into five key areas: system, inputs, service delivery, outputs and outcomes. An important contribution of this framework is the delineation of the service delivery area, a critical but understudied element of primary care quality, into five interconnected domains. These are population health management, e.g. community engagement; facility management and organization; access to care that is timely and affordable; the availability of effective services; and high-quality primary health care. The final domain follows from the others and encompasses Starfield’s formulation of primary care’s roles and functions: coordination, comprehensiveness, continuity and first-contact access.¹⁴

The development of new metrics based on this framework is a critical next step in assessing the quality of the delivery of primary health care. Metrics that align with updated theoretical frameworks and shed light on the quality of care provided to patients are needed to understand primary care performance more fully. Such metrics can help health ministries identify shortfalls in the provision of quality primary care and prioritize appropriate action.

Given its poor population health outcomes and its recent attempts to build a strong primary care system, Haiti presents a compelling case study of primary care quality. Life expectancy at birth is 65 years, and mortality among children younger than five years is more than double that in the neighbouring Dominican Republic.¹⁵ There is only one doctor or nurse per 3000 population and public sector health spending is among the lowest in the world. An earthquake in January 2010 placed further strain on the health system and caused tremendous loss of life and immense physical damage, destroying 50 health facilities.^16,17 Despite natural disasters, poverty and underinvestment in health, Haiti has achieved some notable health gains in recent decades, including a steady decline in mortality among children younger than five years.¹⁵

In 2008, Haiti’s primary care system was classified as selective, with targeted application of high-impact interventions in facilities that, in general, struggled with the provision of routine care.¹⁸ In 2007, the Haitian Ministry of Health’s National Quality Committee launched HIVQual, a system for data collection, based on electronic medical records, designed to measure and improve the quality of services for people living with the human immunodeficiency virus (HIV).¹⁹ In 2012, this system was expanded to cover some non-HIV services’ care and to reach a larger number of facilities.¹⁹ As global health policy pivots towards universal health coverage and to tackling the broad array of health challenges outlined in the sustainable development goals,^20,21 it is an opportune moment to test a methodology for assessing coverage of comprehensive, high-quality primary care.

Below, we describe the development of a theoretically grounded metric of primary care quality, based on existing survey and geospatial data, and the metric’s application in measuring the quality of Haiti’s primary care system. We drew on a census of Haiti’s health facilities to evaluate the performance of the country’s primary care system in 2013, describe geographical access to quality care and assess the disparities in such access. To highlight the challenges and opportunities of measurement in this understudied area, we focused on the service delivery component of the Primary Health Care Performance Initiative’s framework.

Methods

Study sample

We used data from the Service Provision Assessment, which is a census survey of health facilities conducted in Haiti in 2013 by the Demographic and Health Survey Program. The census included a facility assessment, a questionnaire for health-care providers, observations of sick child, antenatal care and family planning visits, and exit interviews with observed clients. We limited our analysis to the data collected on outpatient primary care facilities, i.e. dispensaries and health centres with or without beds.²²

We also used WorldPop maps to obtain estimates of the 2015 population density of Haiti, at a resolution of 100 m².²³

Measuring primary care quality

We developed metrics of service delivery quality following the Primary Health Care Performance Initiative’s framework. Several modifications were required to adapt the framework for health facility assessment (Fig. 1). We excluded the domain “population health management”, because of a lack of relevant facility-related data. For clarity, we also altered the labels for two of the domains, using “effective service delivery” for the availability of effective services and “primary care functions” for high-quality primary health care.¹⁴

Fig. 1.

Conceptual framework of quality in primary health care

Source: Adapted from the Primary Health Care Performance Initiative’s framework,¹³ for use in Haiti.

We reviewed the data available in the survey and selected 28 indicators that most appropriately matched each of the quality subdomains included in our analysis. For this selection, we were guided by the Primary Health Care Performance Initiative’s method note.¹³ Each indicator is a proportion or an index that ranges from 0 to 1. For example, the indicator “sick child did not first visit traditional healer” measures first-contact access to a facility as the proportion of sick children who came to the facility for care without first visiting a traditional healer. All selected indicator definitions are available from the corresponding author. Within the survey data, we were unable to find relevant indicators for two of the subdomains that we wished to investigate: geographical access and the organization of team-based care. As people need to be able to access health facilities to benefit from quality care, we used the WorldPop maps to determine geographical access to facilities.

For each primary care facility, we calculated a score for each of four service delivery domains: (i) accessible care; (ii) effective service delivery; (iii) management and organization; and (iv) primary care functions. Each of these scores, which could range from 0 to 1, was the mean of all the indicators under the domain. As we considered the four domains to be equally important elements of quality primary care, we took the mean of the four scores calculated for each facility as the overall measurement of the quality of the facility’s service delivery for primary care.

Although the census covered all but two of the health facilities in Haiti in 2013, two of the survey tools, i.e. clinical observations and patient interviews, were applied only in a selected subset of facilities. For each indicator included in our analysis, we used multiple imputation to generate five versions of a completed data set for all quality indicators. We based the imputation on observed covariates, e.g. management type and urban, and the non-missing indicators.

Finally, we assessed the distribution of indicators across facilities and sought valid groupings of better and worse quality. Given the lack of universally defined minimum quality thresholds and the rudimentary nature of many of the indicators included in our analysis, we divided the facility scores into three categories of quality. Scores of less than 0.50, 0.50–0.74 and at least 0.75 were considered indicative of poor, fair and good quality, respectively.

Covariates

We defined each 100 m² block of population as an urban or rural population using the census’ urban or rural classification of the facility nearest to the centre of the block. As a sensitivity check, we also defined an urban population as one in which there were at least five people per 100 m² block.

Analysis

We calculated descriptive statistics of the primary care facilities with non-response weights. We summarized mean values and uncertainty intervals for each indicator, domain and overall quality score for service delivery. As the data we analysed provided a census of the primary care facilities in Haiti in 2013, the uncertainty intervals that we calculated indicate the measurement error attributable to missing data.²⁴ Using inverse distance-weighted interpolation, we mapped, across Haiti, the quality of the primary care available to a nearby population. In the resultant map, the colour of each 100 m² block indicates whether the quality of the nearest primary care facility was poor, fair or good. We used the global Moran’s I statistic, which tests for the presence of spatial autocorrelation,²⁵ to investigate whether facilities of good or poor quality, in terms of each of the four domains of interest, were clustered geographically. Moran’s I can range from −1 to 1. In our analyses, positive I values would indicate that primary care facilities of similar quality were clustered together. We defined proximity using an inverse-distance weight matrix.²⁶ In keeping with prior research on physical access to care in Haiti,²⁷ we calculated the percentages of the entire Haitian population, rural population and urban population living within 5 km of any facility and within the same distance of a facility with a good overall score. Finally, we mapped the areas that lay within 5 km of any facility and a facility with a good overall care score.

Multiple imputation was conducted in R 3.2 (R Core Team, Vienna, Austria). All other analyses were conducted in Stata version 14.0 (StataCorp, LP, College Station, United States of America). We used QGIS version 2.12²⁸ to map the data.

Ethical approval

The Harvard University Human Research Protection Program categorized this secondary analysis of data as exempt from human subjects review.

Results

The survey obtained detailed data from 905 (99.8%) of the 907 health facilities in Haiti in 2013, 786 of which were primary care facilities and included in the analysis (Table 1). Most primary care facilities were classified as rural, although there was a high concentration of primary care facilities in and around Port-au-Prince. Fig. 2 summarizes the performance of the primary care facilities across the four domains of primary care service delivery. At the average facility, 86% and 94% of clients, respectively, stated that they did not find wait times or the costs of care to be a problem, even though about half of all primary care services required payment and over half of the primary care facilities had mean wait times in excess of one hour. Large gaps in quality were evident in the metrics for the availability of effective services. The indicators for provider motivation and safety were found to be especially low. Basic elements of clinical care were not universally followed. For example, at the average facility only 57% of the providers asked about maternal age at a first visit for antenatal care. Low quality scores for primary care functions were partially attributable to poor provider communication. Under management and organization, only 2% (18) of the primary care facilities had a system for gathering feedback from their clients and nearly three-quarters (577) did not have routine quality assurance processes. For their overall quality of service delivery, the primary care facilities in Haiti achieved a mean score of 0.59.

Table 1. Characteristics of primary care facilities, Haiti, 2013.

Characteristic	No. of facilitiesa (%) n = 786
Setting
Rural	526 (67)
Urban	260 (33)
Facility type
Health centre with beds	129 (16)
Health centre without beds	297 (38)
Dispensary	358 (46)
Management type
Public	292 (37)
Private, not for profit	142 (18)
Private, for profit	180 (23)
Faith-based	170 (22)
With inpatient or maternity beds	355 (46)
Offers pharmacy services	766 (98)
Offers laboratory services	544 (69)

^a Each facility was staffed by a mean of 1.15 (standard deviation, SD: 3.26) generalist doctors, generalist surgeons and/or specialist doctors and a mean of 3.87 (SD: 4.71) auxiliary nurses, midwives, nurse/midwives and/or nurses.

Fig. 2.

Indicators of the quality of the delivery of primary care services, Haiti, 2013

ANC: antenatal care; CI: confidence interval; HMIS: health management information system: NCDs: noncommunicable diseases.

Note: Error bars illustrate the between-imputation variance in the proportions. Error bars not visible when variance <1%.

Most facilities (84%; 660/786) had fair overall quality of care and only 15 had good overall quality (Fig. 3). Nearly half of the 786 primary care facilities (43%; 332) offered good accessible care but only 4% (30) and 6% (42) ranked as good in terms of effective service delivery and primary care functions, respectively. Tests of spatial autocorrelation provided evidence of geographical clustering in terms of overall quality and the quality of accessible care, effective service delivery and management and organization. However, the magnitude of every such association was small (available from the corresponding author), indicating that considerable geographical heterogeneity in the quality of primary health care existed.

Fig. 3.

Population coverage of primary care of poor, fair or good quality, Haiti, 2013

Note: The maps show the populations for which the nearest primary care facility provided care that was scored as poor, fair or good by domain and overall. The administrative boundaries shown are based on data from the GADM spatial database.²⁹

Source: Authors, using data from the GADM spatial database and Service Provision Assessment.²⁹

Fig. 4 illustrates geographical access of Haiti’s entire, rural and urban populations to primary care. At the time of the census, an estimated 90.6% of Haiti’s population lived within 5 km of a primary care facility. Although almost 8 million people, that is, 72% of the national population, lived within 5 km of a primary care facility providing good accessible care, smaller numbers lived as close to facilities providing good management and organization (51%), good primary care functions (31%) or good effective service delivery (30%). Compared with rural dwellers, urban residents had higher access to good quality care along all of the domains. For example, only 8% of rural dwellers but 57% of urban residents had access to effective service delivery of good quality. Similar trends were observed when, in a sensitivity check, we defined an urban population as one in which there were at least five people per 100 m² block (available from the corresponding author). At the time of the census, only an estimated 2.5 million people (of 10.65 million total) in Haiti lived within 5 km of a facility with good overall quality of care (Fig. 5).

Fig. 4.

Geographical access to any primary care facility and facilities providing primary care of good quality, Haiti, 2013

CI: confidence interval.

Note: A person was considered to have geographical access to a facility if they lived no more than 5 km, measured linearly, from the facility. Error bars illustrate the between-imputation variance in the proportions. Error bars not visible when variance <1%.

Fig. 5.

Map of the geographical access to any primary care facility and facilities providing primary care of good quality, Haiti, 2013

Note: A person was considered to have geographical access to a facility if they lived no more than 5 km, measured linearly, from that facility. The magnified box presents the capital, Port-au-Prince. The administrative boundaries shown are based on data from the GADM spatial database.²⁹

Source: Authors, using data from the GADM spatial database, Service Provision Assessment survey and WorldPop population density.²⁹

Discussion

This study applies a novel approach to measuring the quality of primary care services in Haiti in terms of four quality domains and using existing data sources. We found that the Haitian population’s access to primary care of good quality in 2013 was very limited: while 91% of the population lived within 5 km of a primary care facility, only 23% lived within 5 km of a facility with service delivery of good quality. The mean overall score for the quality of service delivery was only 0.59, indicating there are many gaps in the provision of high quality primary care. In general, the primary care facilities performed reasonably well on access indicators but poorly in terms of effective service delivery and management and organization, with particular deficits in provider motivation and quality improvement. As the quality indicators based on clients’ responses tended to be more positive than those that had been more objectively assessed, it appears that clients may have had low expectations when seeking care. We found limited evidence of geographical clustering of quality. The quality of service delivery varied substantially from facility to facility within both rural and urban areas, although it was, in general, relatively poor in rural areas.

Most previous studies of the quality of primary care in Haiti have used service utilization, for example, by the numbers of antenatal care visits or vaccination rates, as an indicator of quality.^30,31 Our study, which incorporated indicators for preventive services and curative services for communicable and noncommunicable diseases, moved beyond utilization to consider the service environment and the whole process of care during a primary care visit. Like a recent review of the literature on the service delivery experience of patients using primary care,⁵ our study indicates profound gaps in the provision and receipt of primary care of good quality. In Haiti, as elsewhere, robust quality measurement is a crucial input to the ongoing efforts to improve the quality of primary care. A recent assessment of the impacts of a Haitian programme to improve HIV services found that the programme had triggered a broad and beneficial change in the culture of quality improvement.³²

Our application of the Primary Health Care Performance Initiative’s framework for the delivery of primary care services yielded several insights. The framework currently represents the most comprehensive and well-defined approach to primary care quality – with a particular emphasis on the often undermeasured area of service delivery. However, we feel that the framework’s current domain labels could be made more intuitive and, more substantively, that the framework leaves patient safety as a poorly defined construct in the context of primary care delivery. There is also a lack of comprehensive data covering all of the relevant domains and subdomains. We identified several priorities for improving the measurement of primary care quality (Box 1). The framework, the tools used in surveys of health facilities and the links between the results of facility surveys and national policy processes all need critical review. Additionally, while the measurement of primary care quality is intrinsically important, future work should also link the quality of care to population health outcomes.

Box 1. Priorities in improving the measurement of primary care quality.

• Better definitions, e.g. in patient safety;

• Better data, e.g. in first-contact accessibility, continuous care, coordinated care, population health management and the team-based management of care;

• More efficient measures, e.g. identification of the smallest set of indicators needed to measure quality effectively and definition of the roles of routine information systems and special studies in providing the data required to monitor the quality of primary care;

• More meaningful measures, e.g. determination of appropriate minimum thresholds for good quality and the best methods to compare such thresholds across different settings;

• Investigation of the population served by a primary care facility, particularly the difference between planned catchments and actual utilization and the identification of the data needed, from the population, to conduct equity analyses on the quality of care;

• Investigation of the best ways to translate the results of quality measurement into effective methods of quality improvement.

Our study had several other limitations. Although we performed multiple imputation to account for missing data, such data could still have introduced bias into the analysis – i.e. if the facilities without observations and patient interviews differed systematically, in ways not captured by the covariates used in imputation, from the other facilities. In addition, our use of linear distance to estimate geographical access may have been misleading. Especially in rural and mountainous regions, the distance that an individual has to travel between two points may be much greater than the linear distance between those points. Detailed data on road networks and quality and on transportation costs would strengthen our observations. Finally, as Service Provision Assessment data are only available for a small set of countries, the approach that we followed may not be applicable in many other settings, although indicators similar to the ones that we investigated are available from other health facility assessments.^33,34

Our results have several implications for primary health care in Haiti. As an immediate next step, they can be used, by funders, planners, policy-makers and practitioners, to compare performance within administrative areas and to identify the best- and worst-performing facilities within each area. This should allow improvement interventions to be better targeted at particular facilities and at known weaknesses. Despite Haiti’s challenging topography, primary care of good quality has been achieved, and should be more widely achievable, in all areas. Most primary care facilities of poor quality in Haiti are close to, and could learn from, a facility of good quality. Elements of service delivery quality that were found to be absent from almost all of Haiti’s primary care facilities, e.g. the gathering of client feedback and good provider communication, should be targeted for improvement and regular measurement. Strategies to address these gaps could include provider training on patient-centred approaches,³⁵ the strengthening of data feedback loops for providers³⁶ and the enhancement of managerial supervision.³⁷

More broadly, our study complements the efforts of Haiti’s National Quality Committee over the past decade to monitor and improve the quality of health services, particularly via the development of broader health services based on HIV care.³² The comprehensive definition of quality that we employed provides an opportunity to spur the development of standardized systems to monitor and improve the quality of primary care services and to complement Service Provision Assessments or similar periodic evaluations. The Haitian Ministry of Health could build on its experience with programmes to improve the quality of HIV services to adapt and apply strategies, systems and tools for the routine monitoring of indicators across all dimensions of primary care quality. Work is already underway to compare the utilization and patient-outcome indicators from other quality measurement approaches, such as HIVQual,¹⁹ with the indicators that we investigated. Only with the routine measurement of quality and evidence-based quality improvement can the quality of services offered at primary care level improve health outcomes and meet the legitimate expectations of Haitians.