Abstract
Pharmaceutical products are indispensable for improving health outcomes. An extensive body of work on access to and use of medicines has resulted in an assortment of tools measuring various elements of pharmaceutical systems. Until now however, there has been little attempt to conceptualize a pharmaceutical system as an entity and define its strengthening in a way that allows for measuring systems strengthening. The narrow focus of available tools limits their value in ascertaining which interventions result in stronger, more resilient systems. We sought to address this shortcoming by revisiting the current definitions, frameworks and assessment tools related to pharmaceutical systems. We conducted a comprehensive literature review and consulted with select pharmaceutical experts. On the basis of our review, we propose that a pharmaceutical system consists of all structures, people, resources, processes, and their interactions within the broader health system that aim to ensure equitable and timely access to safe, effective, quality pharmaceutical products and related services that promote their appropriate and cost-effective use to improve health outcomes. We further propose that pharmaceutical systems strengthening is the process of identifying and implementing strategies and actions that achieve coordinated and sustainable improvements in the critical components of a pharmaceutical system to make it more responsive and resilient and to enhance its performance for achieving better health outcomes. Finally, we established that, in addition to system performance and resilience, seven components of the pharmaceutical system are critical for measuring pharmaceutical systems strengthening: pharmaceutical products and related services; policy, laws and governance; regulatory systems; innovation, research and development, manufacturing, and trade; financing; human resources; and information. This work adds clarity to the concept of pharmaceutical systems and their strengthening by proposing holistic definitions on the basis of systems thinking. It provides a practical starting point for measuring the progress of pharmaceutical systems strengthening.
Keywords: Access and utilization, monitoring, pharmaceutical system, pharmaceutical products, systems strengthening
• Clear definitions and reliable measures of pharmaceutical systems strengthening are needed to guide interventions to improve the performance and resilience of pharmaceutical systems.
Introduction
The World Health Organization (WHO) ‘building blocks’ framework defines health systems as ‘all organizations, people, and actions whose primary intent is to promote, restore or maintain health’ (WHO 2007). It identifies six essential building blocks, or key functions, of health systems: service delivery; health workforce; information; medical products, vaccines, and technologies; financing; and leadership and governance. It further defines health systems strengthening as ‘improving these six health system building blocks and managing their interactions in ways that achieve more equitable and sustained improvements across health services and health outcomes’ (WHO 2007).
The interactions between the building blocks define health systems better than the blocks themselves. Recognizing the importance of these interactions, scholars have adopted a systems thinking approach to study health systems (de Savigny and Adam 2009, Atun et al. 2010, Atun 2012, Paina and Peters 2012, Swanson et al. 2012, van Olmen et al. 2012). This approach brings into focus two fundamental ideas: All health interventions tend to have a system-level effect, and health system processes are dynamic and nonlinear. The nature of relationships between system components and the environment influences system adaptations and outcomes (Eidelson 1997). In health systems, the complexity of reactions and interactions can render the overall system policy-resistant, causing well-intentioned interventions to produce unintended consequences (Sterman 2006, de Savigny and Adam 2009, Atun 2012).
Ensuring equitable access to essential medicines, vaccines and technologies, and their appropriate use is a core function of the health system: ‘A well-functioning health system ensures equitable access to essential medical products, vaccines and technologies of assured quality, safety, efficacy and cost-effectiveness, and their scientifically sound and cost-effective use’ (WHO 2007). All components involved in this function may be conceptualized as a subset of the health system, that is, a pharmaceutical system. Terms such as pharmaceutical system, pharmaceutical management/supply system, and pharmaceutical sector have been used interchangeably (Roberts and Reich 2011, Yadav et al., 2011), and the distinction between them is somewhat unclear. In addition, there is no explicit consensus on what constitutes a pharmaceutical system, and no clearly defined framework or agreed approach to measure progress toward stronger, more resilient pharmaceutical systems. This is despite an extensive body of work on access to medicines, various components of pharmaceutical systems and measurements of their performance (Rational Pharmaceutical Management Project et al. 1995, Brudon et al. 1999, Holloway et al. 2013, WHO and HAI 2008, Cameron et al. 2009, Seiter 2010, WHO 2011, Windisch et al. 2011, MSH 2012, Bigdeli et al. 2013, Xiao et al. 2013, Zaidi et al. 2013).
In the absence of clear definitions and generally agreed-upon reliable measures, countries and donors lack information to guide investments in pharmaceutical systems strengthening (PSS) and evaluate PSS interventions. This is a considerable shortcoming, given that, in low- and middle-income countries, medicines on average account for 25% of total health expenditure and can be as high as 67% (Lu et al. 2011). Further, access to and appropriate use of affordable medicines are requisites for achieving universal health coverage (UHC), and many countries are currently implementing health reforms in a push toward UHC (WHO 2010a, Boerma et al. 2014, Bigdeli et al. 2015). There are growing concerns about the resilience of health systems and their ability to respond to the challenges of UHC (Wagner et al. 2014); health emergencies such as Ebola (Kieny et al. 2014, Kieny and Dovlo 2015); economic crises (Hou et al, 2013, Thomas et al. 2013, European Commission 2014, Pisu 2014); and climate change (Hess et al. 2012, Mayhew and Hanefeld 2014, Wulff et al. 2015). Efforts to strengthen the resilience of health systems to cope with these challenges will have to include strengthening pharmaceutical systems.
This paper aims to advance the current thinking about pharmaceutical systems by building upon existing approaches to understand and strengthen health systems. The objective of our study was to articulate explicit definitions of the pharmaceutical system, PSS and the system components to guide measurement of PSS. We limited our focus to the pharmaceutical system for pragmatic reasons, fully recognizing that it is embedded in and influenced by the broader health system. We conducted a comprehensive literature review in conjunction with expert consultations to assemble and analyse the existing knowledge about pharmaceutical systems and PSS. This paper reports our findings and discusses the holistic definitions that resulted from our research.
Methods
We employed a two-fold strategy to conduct our research, combining a comprehensive review of the literature with a series of expert consultations. In the literature review, we first used the institutional knowledge in consultation with senior experts at Management Sciences for Health (MSH) to create an initial list of search terms, key actors, and agencies conducting research on strengthening pharmaceutical systems (Table 1). Our search focused on English language sources, with no publication time limit. An iterative process was used in which the results and bibliographies of relevant documents were used to guide subsequent searches. We deemed the search had reached saturation when subsequent searches failed to provide any noticeably new publication or organization. This led us to identify over 200 publications. The title and abstract of each one was quickly screened for relevance. We included all reports or studies that focused on definitions of a pharmaceutical system, pharmaceutical management system, or PSS; description of a framework aligned with one of these definitions; identification of one or more components of a pharmaceutical (management) system; description of a performance indicator or metric of such a system; description of an intervention to improve such a system; and review or discussion of the conceptual or theoretical basis for such a system or one of its components. Also included were documents addressing definitions of health systems strengthening and related frameworks. We excluded national assessment reports but included the assessment tools or framework on which they were based. Articles about pharmaceutical innovation and industry performance and pharmacology-related topics were excluded; 106 materials were retained after the screening. We sorted them into three virtual bins: definitions and frameworks for pharmaceutical systems and PSS (n = 26); definitions and frameworks for health systems and strengthening (n = 27); and assessment tools and indicator sets (n = 53). All selected articles were carefully read. Those meeting our selection criteria (n = 61) were included in the final review and analysis used to draft the background discussion paper that served as the basis for expert consultations (Hafner and Walkowiak 2014).
Table 1.
Databases and Websites | Search terms |
---|---|
Published Articles
|
|
Websites and Gray Literature
|
Experts were consulted during a meeting with MSH staff implementing the US Agency for International Development (USAID)-funded Systems for Improved Access to Pharmaceuticals and Services (SIAPS) Program and program partners. The meeting, held on September 11-12, 2014, brought together 30 participants from SIAPS’ core and resource partners, including the Accreditation Council for Pharmacy Education, Ecumenical Pharmaceutical Network, Harvard Pilgrim Health Care Institute, Harvard University School of Public Health, Imperial Health Sciences, Logistics Management Institute, Results for Development, University of Washington, VillageReach, and William Davidson Institute. Participants also included experts from the Boston University School of Public Health, Pan American Health Organization (PAHO) (representing WHO), and USAID. Key objectives of the meeting were to reach agreement on proposed definitions of a pharmaceutical system and PSS and to identify key components of a pharmaceutical system deemed necessary to measure PSS. During the meeting, concepts were discussed and the background paper was critiqued; draft definitions of pharmaceutical system and PSS were reviewed, improved, and agreed upon; and essential pharmaceutical system components identified. After the meeting, a targeted literature search was conducted to finalize the definitions and key elements of the pharmaceutical system and PSS that are discussed in this paper.
Results
We found few conceptualizations of pharmaceutical systems and their strengthening in the literature. Our review yielded three explicit definitions of a pharmaceutical system and two definitions related to pharmaceutical management. We also identified seven frameworks that provide insight into understanding the goals and scope of a pharmaceutical system. We did not find any explicit or implicit definition of PSS, but reviewed three definitions of health systems strengthening and 47 assessment tools related to pharmaceutical systems.
Existing definitions of pharmaceutical system and pharmaceutical management
The three definitions of pharmaceutical system are specific in their purpose and origin, and therefore limited in scope. Roberts and Reich (2011) use the terms system and sector interchangeably in the context of implementing pharmaceutical sector reforms, with an emphasis on the life cycle of pharmaceutical products from a producer/supplier or provider perspective. They define the pharmaceutical system/sector as a linear progression of eight functions/subsystems: research and development, clinical trials, registration, manufacturing and packaging, procurement and importing, supply chain, dispensing, and sales/use. On the other hand, the WHO transparency assessment instrument distinguishes between pharmaceutical system and sector. It defines the system as ‘the relationship/interactions between the various actors of the pharmaceutical sector and the way decisions are made in particular in the government’ (WHO 2009). It defines the sector as the various actors (e.g., government, private for-profit organizations, private not-for-profit organizations) engaged in the ‘medicine chain’, which includes research and development; clinical trials; filing patents; manufacturing; registration; selection, procurement and distribution of essential medicines; inspection of manufacturers and distributors; prescribing; dispensing; pharmacovigilance; and the control of promotion (WHO 2009). A related paper on the need for good governance in pharmaceutical systems makes the same distinction between pharmaceutical sector and system as WHO and offers a similar definition of the system: ‘The actions of public and private stakeholders as they move drugs through the supply chain from purchasing to providing to patients’ (Kohler et al. 2014).
Two pharmaceutical management definitions are worth considering for additional insight into the goals and scope of pharmaceutical systems. The first one defines the pharmaceutical supply system as the procedures and methods used to accomplish the four key pharmaceutical management functions—selection, procurement, distribution, and use (RPM Plus 2005; Miralles 2010). The second defines the management of medical products, vaccines, and technologies as ‘the whole set of activities aimed at ensuring the timely availability and appropriate use of safe, effective, quality medicines and related products and services in any health care setting’ (Health Systems 20/20 2012).
Definitions of health systems strengthening and the concept of resilience
In the absence of explicit definitions of PSS, we turned to existing definitions of health systems strengthening. The first definitions were published in 2007. One describes health systems strengthening as ‘any array of initiatives and strategies that improves one or more of the functions of the health system and that leads to better health through improvements in access, coverage, quality, or efficiency’ (Islam 2007). The other, issued by WHO, made implicit reference to the concepts of system resilience and sustainability: ‘Improving [the] six health system building blocks and managing their interactions in ways that achieve more equitable and sustained improvements across health services and health outcomes’ (WHO 2007). The WHO definition was updated in 2014: ‘The process of identifying and implementing the changes in policy and practice in a country’s health system, so that the country can respond better to its health and health system challenges’ (WHO 2014).
A sustainable health system has to cope with constant change (de Savigny and Adam 2009). Resilience is the system’s capacity to handle change and unexpected disturbances. It has three dimensions: absorptive capacity, the ability to cope with disturbances; adaptive capacity, the ability to learn and adjust to changing internal and external factors; and transformative capacity, the capacity to reorganize into a fundamentally new system when economic and social conditions make the existing system no longer feasible (Holling 2001, Walker et al. 2004, Béné et al. 2014). Resilience emerges from the synergies and trade-offs between the three dimensions, and the system’s response to a disturbance depends on the intensity of the shock (Janssen and Osnas 2005, Béné 2013). As such, resilience is not just about the ability to maintain or return to a previous state, but also about adapting and learning to live with the changes and uncertainty (Bowen et al. 2011, Bahadur et al. 2013, Béné et al. 2014, Blanchet 2015). Kruk et al. (2015) define health systems resilience as the ‘capacity of health actors, institutions, and populations to prepare for and effectively respond to crises; maintain core functions when a crisis hits; and, informed by lessons learned during the crisis, reorganize if conditions require it’ (p. 1910). In addition to improving health system performance, strengthening efforts should also improve system resilience (Balabanova et al. 2013, Blanchet 2013, Rhodes 2013, Kruk et al. 2015). Without understanding how interventions in one part of the system affect the entire system, investments in health systems are unlikely to produce more resilient systems and sustainable improvements (Adam and de Savigny 2012, Agyepong et al. 2012, Sturmberg et al. 2012, Swanson et al. 2012).
Existing frameworks
Our literature review identified seven frameworks relevant to pharmaceutical systems: the MSH pharmaceutical management framework (Management Sciences for Health 1997); the USAID-funded Rational Pharmaceutical Management Plus (RPM Plus) Program pharmaceutical management system framework (RPM Plus 2005); the WHO ‘building blocks’ framework (WHO 2007); the International Health Partnership and related initiatives (IHP+) monitoring and evaluation of health systems strengthening framework (WHO et al. 2009, WHO 2010b); the ‘control knobs’ framework (Roberts and Reich 2011); the access to medicines from a health system perspective framework (Bigdeli et al. 2013); and the SIAPS PSS framework (SIAPS 2013). Supplementary Appendix A provides illustrations of the frameworks. Each of these was systematically characterized by their source, focus, key elements, goals, and overarching principles and qualifiers (Table 2). We then identified all the domains of interest to consider in our definitions of pharmaceutical system and PSS: goals, product, product characteristics, outcome characteristics, overarching principles, stakeholders, functions, system components, and context. We extracted from each framework all the attributes related to each of these domains. Table 3 displays a list of all the attributes this analysis was able to identify and their sources, by domain.
Table 2.
Framework | Source | Focus | Key elements of the framework | Goal | Overarching principles and qualifiers |
---|---|---|---|---|---|
Pharmaceutical management system framework | RPM Plus 2005, Miralles 2010 | Relationship between health system and pharmaceutical subsystem |
|
|
|
Pharmaceutical management framework | MSH 1997, MSH 2012 | Functions and elements of pharmaceutical management |
|
|
|
Medical products building block, WHO health systems framework | WHO 2007, WHO 2010b | Core functions (building blocks) of a health system |
|
|
|
Control knobs framework | Roberts and Reich 2011 | Means for effecting adjustments in the pharmaceutical system |
|
|
|
Conceptual framework of access to medicines from a health systems perspective | Bigdeli et al. 2013 | Health systems perspective to address demand- and supply-side barriers to access to medicines |
|
|
|
PSS framework | SIAPS 2013 | Approach to strengthening pharmaceutical systems |
|
|
|
Note: The IHP+ monitoring and evaluation of health systems strengthening framework is excluded from this overview because it is based on the WHO health systems framework building blocks framework.
Table 3.
Domains | Pharmaceutical management system framework | Pharmaceutical management framework | Medical products building block, WHO health systems framework | Control knobs framework | Conceptual framework of access to medicines from a health systems perspective | PSS framework | |
---|---|---|---|---|---|---|---|
RPM Plus 2005, Miralles 2010 | MSH 2012 | WHO 2007, WHO 2010b | Roberts and Reich 2011 | Bigdeli et al. 2013 | SIAPS 2013 | ||
Goals | Access (accessibility, availability, acceptability, affordability, quality) | ▪ | ▪ | ▪ | ▪ | ▪ | ▪ |
Use (appropriate, rational) | ▪ | ▪ | ▪ | ▪ | ▪ | ||
Contribute to health outcomes/status | ▪ | ▪ | ▪ | ▪ | ▪ | ||
Coverage | ▪ | ▪ | |||||
Efficiency | ▪ | ▪ | ▪ | ||||
Social and Financial protection | ▪ | ▪ | |||||
Responsiveness | ▪ | ||||||
Satisfaction | ▪ | ||||||
Products accessed and used | Medicines | ▪ | ▪ | ▪ | ▪ | ▪ | ▪ |
Pharmaceutical products/pharmaceuticals | ▪ | ▪ | ▪ | ▪ | |||
Medical products | ▪ | ▪ | |||||
Vaccines | ▪ | ||||||
Health technologies | ▪ | ||||||
Pharmaceutical services | ▪ | ▪ | ▪ | ▪ | |||
Characteristics of products accessed and used | Quality | ▪ | ▪ | ▪ | ▪ | ▪ | ▪ |
Essential | ▪ | ||||||
Safe/safety | ▪ | ▪ | ▪ | ▪ | |||
Effective (efficacy) | ▪ | ▪ | ▪ | ▪ | |||
Cost-effective | ▪ | ▪ | |||||
Qualities associated with access and use | Quality | ▪ | ▪ | ▪ | ▪ | ▪ | |
Safe/safety | ▪ | ||||||
Effective | ▪ | ▪ | ▪ | ||||
Cost-effective | ▪ | ||||||
Scientifically sound | ▪ | ||||||
Overarching principles | Equity | ▪ | ▪ | ▪ | ▪ | ▪ | |
Timeliness | ▪ | ||||||
Human rights | ▪ | ||||||
Stakeholders | Structures/institutions/organizations | ▪ | ▪ | ||||
Individuals/people | ▪ | ▪ | |||||
Government | ▪ | ||||||
Providers | ▪ | ||||||
Communities and households | ▪ | ▪ | ▪ | ||||
Public sector | ▪ | ||||||
Private sector | ▪ | ||||||
International, national, subnational, and local | ▪ | ||||||
Functions (subsystems) | Selection | ▪ | ▪ | ||||
Procurement (procurement and importing) | ▪ | ▪ | ▪ | ||||
Distribution (supply chain) | ▪ | ▪ | ▪ | ||||
Use (dispensing; sales) | ▪ | ▪ | ▪ | ||||
Research and development (including clinical trials) | ▪ | ||||||
Regulation (including registration and licensing of individuals and facilities) | ▪ | ||||||
Manufacturing and packaging | ▪ | ||||||
System Components | Service delivery | ▪ | ▪ | ▪ | ▪ | ||
(Leadership and) governance | ▪ | ▪ | ▪ | ▪ | ▪ | ||
Policies, law, and regulation (supported by good governance) | ▪ | ▪ | ▪ | ||||
Resources (management support systems/inputs): | |||||||
Medical products, medicines, vaccines, health technologies | ▪ | ▪ | ▪ | ▪ | |||
Human resources/health workforce | ▪ | ▪ | ▪ | ▪ | ▪ | ||
Information | ▪ | ▪ | ▪ | ▪ | ▪ | ||
Financing (pricing; price setting/negotiation) | ▪ | ▪ | ▪ | ▪ | ▪ | ||
Infrastructure | ▪ | ||||||
Organization | ▪ | ||||||
Context | Market forces | ▪ | |||||
Innovation | ▪ | ||||||
Transparency | ▪ | ||||||
Donor’s agenda and funding | ▪ |
The MSH pharmaceutical management framework identifies the four pharmaceutical management functions: selection, procurement, distribution and use (MSH 1997). These functions are supported by a core of management support systems: organization, financing and sustainability, information management, and human resources management. The core and support functions are enabled (and constrained) by policies, laws, and regulations and supported by good governance principles and practices that establish and sustain the public commitment to the essential medicines supply (MSH 2012). The RPM Plus pharmaceutical management system framework conceptualizes the pharmaceutical system as a subsystem of the health system that includes all the institutions and stakeholders in both the public and private sectors that are involved in the procedures and methods used to accomplish the four pharmaceutical management functions (RPM Plus 2005, Miralles 2010). Pharmaceutical management aims to ensure the timely and equitable access to and appropriate use of safe, effective, quality medicine and related products and services (Miralles 2010).
The WHO health systems building blocks framework (WHO 2007) does not refer to a pharmaceutical system, but rather to the provision of medical products as a core function of the health system. Five requirements identified for achieving access and use are national policies, standards, guidelines and regulations that support policy; information on prices, international trade agreements, and capacity to set and negotiate prices; reliable manufacturing practices and quality assessment of priority products; procurement, supply, storage, and distribution systems that minimize leakage and other waste; and support for rational use of essential medicines, commodities, and equipment through guidelines and strategies to ensure adherence, reduce resistance, maximize patient safety, and training. By implication, the pharmaceutical system is a subunit of the health system that aims to achieve access and appropriate use of medicines. The framework developed by IHP+ is for monitoring and evaluating health systems strengthening and is based on the building blocks framework (WHO et al. 2009, WHO 2010b).
Roberts and Reich (2011) adapted the ‘control knobs’ framework, which identifies five control knobs for reforming health systems to achieve system goals (Roberts et al. 2008) and applied it to pharmaceutical systems. The adapted framework focuses on the role of government in influencing pharmaceutical sector performance and identifies five control knobs—financing, payment decisions, organization of activities, regulation, and persuasion efforts—as structural components of the pharmaceutical system, which can be adjusted to improve system performance. It divides the system goals into intermediate and ultimate performance goals. The intermediate performance goals—efficiency, quality, and access—are characteristics of the functioning of the system and the means to the ultimate performance goals—health status, financial protection, and citizen satisfaction. The control knobs are the adjustable, independent variables that influence the ultimate goals of the system. Implicitly, the health system is an external factor that can influence the pharmaceutical system, and the various components and functions of the pharmaceutical system overlap with those of the general health system (Roberts and Reich 2011).
Bigdeli et al. (2013) adapted the building blocks framework and three existing access-to-medicines frameworks (CPM 2003, Frost and Reich 2008, WHO 2004) to develop a systems approach to access to medicines. The authors do not attempt to define a pharmaceutical system, but in a subsequent publication, they describe the framework as showing the ‘medicine subsystem’ nested in the broader health system (Bigdeli et al. 2014). It is the only conceptual framework that identifies and gives full consideration to the demand- and supply-side barriers to access and their interactions with the building blocks throughout the various levels of the health system.
The PSS framework is the most recent framework and the only one to explicitly focus on the strengthening of pharmaceutical systems (Figure 1). Designed by the SIAPS Program, this framework is grounded in systems thinking and builds on the WHO health systems framework to identify the key linkages and interactions between the health system and its pharmaceutical subsystem. It places the ‘product’ function at the centre of a set of interacting elements derived from the WHO building blocks. It integrates key stakeholders, and also presents expected PSS outcomes as contributing to the broader outcomes of efficient health systems.
Existing assessment tools
Much of the thinking and knowledge about pharmaceutical systems and their performance has been incorporated over time into the development and improvement of a variety of assessment tools and indicators. Assessment tools are considered in this study to the extent that they helped us identify which components of the pharmaceutical system are critical to assess its strength. We reviewed 47 of the 53 screened tools, the majority of which focus on some aspect of service delivery or supply chain management. Table 4 summarizes our classification of the tools by the number of tools found for each component or function measured. (See supplementary Appendix B for a detailed description, classification, and analysis of assessment tools.)
Table 4.
Component/Function | No. of tools |
---|---|
Access | 15 |
Access and use | 2 |
Financing | 8 |
Governance | 4 |
Health/pharmaceutical services/laboratory services | 6 |
Human resources | 7 |
Information systems | 3 |
Manufacturing, industry, and trade | 5 |
Miscellaneous indicator categories | 13 |
Organization and management support | 2 |
Policies, legislation, and regulation | 20 |
Quality/quality assurance/pharmacovigilance | 15 |
Service Delivery | |
Distribution | 6 |
Procurement | 8 |
Procurement and distribution | 2 |
Selection | 3 |
Selection and procurement | 2 |
Selection and use | 1 |
Selection and registration | 1 |
Services and logistics | 1 |
Supply chain/supply chain management/logistics | 7 |
Transport | 1 |
Use | 21 |
Note: A full list of the assessment tools reviewed and the categories of indicators on which these counts of components/functions are based is provided in Supplementary Appendix B.
Proposed definitions
The following definitions emerged from our literature research and expert consultations:
A pharmaceutical system consists of all structures, people, resources, processes, and their interactions within the broader health system that aim to ensure equitable and timely access to safe, effective, quality pharmaceutical products and related services that promote their appropriate and cost-effective use to improve health outcomes.
Pharmaceutical systems strengthening is the process of identifying and implementing strategies and actions that achieve coordinated and sustainable improvements in the critical components of a pharmaceutical system to make it more responsive and resilient and to enhance its performance for achieving better health outcomes.
Seven pharmaceutical system components are critical for guiding the measurement of PSS: pharmaceutical products and related services; policy, laws and governance; regulatory systems; innovation, research and development, manufacturing, and trade; financing; human resources; and information.
Discussion
Our findings underline the complexity of the pharmaceutical system, alternatively described in terms of structures or organizations (e.g., regulatory agencies, procurement agencies); individuals/people; resources (human, financial, information); processes; or some combination thereof that determines the actions and interactions between various actors in the pharmaceutical sector. In the next paragraphs, we will discuss how our analysis addresses this complexity and how our proposed definitions progress the concept of a pharmaceutical system toward a common approach for measuring its strengthening.
Defining the pharmaceutical system
Our new definition encompasses all key elements of existing definitions and provides a strong foundation for measuring the performance of pharmaceutical systems. It incorporates the recurring theme that the pharmaceutical system should be treated as a subsystem within the broader health system taking into account key stakeholders, and further defines goals for the pharmaceutical system as discussed below.
System stakeholders
It is important to consider the key stakeholders who play a role in the pharmaceutical system. In that regard, the Bigdeli et al. (2013) framework offers an important contribution to identifying the various stakeholders in the system and their roles with respect to access to medicines. The framework assigns five levels to the health system. At the first level are individuals, households, and communities. Individual preferences, household economics, and social and cultural factors in the community influence health-seeking behaviour and trigger demand in the system. Individuals and communities are not passive end-users of services but act as stewards of the system by demanding quality service and better accountability and expressing their (dis)satisfaction with products and services (WHO 2007, Roberts and Reich 2011).
Levels 2 through 5 represent the supply side. Level 2 consists of the health service delivery: wholesalers, manufacturers, and various service providers such as hospitals, pharmacies, clinics, and medicine shops, whether public or private, formal or informal. None of the analysed frameworks explicitly considers the role of payers. However, payers are also key stakeholders at this level particularly with respect to financial risk protection strategies as countries try to achieve universal health coverage. These health service delivery stakeholders perform their pharmaceutical management activities in the policy and regulatory environment of the health sector, level 3 of the health system. Levels 4 and 5 refer to the national and international contexts. Cross-cutting policies related to market forces, innovation and transparency and other national priorities that impact the health system also affect the pharmaceutical system (Roberts and Reich 2011, Bigdeli et al. 2013). At the international level, the agenda of donor agencies and global health initiatives and trade issues can also have supply-side effects (Marchal et al. 2009, Roberts and Reich 2011, Bigdeli et al. 2013).
System goals
We found consensus that the purpose of pharmaceutical systems is to ensure access and positively influence use. The term access is most commonly understood as availability, affordability, (geographical) accessibility, and (cultural) acceptability of quality products and services (Penchansky and Thomas 1981, CPM 2003). In keeping with the existing definitions and frameworks, our definition qualifies access as timely and equitable. We found various terms for what is accessed: medicines, pharmaceuticals, vaccines, pharmaceutical products, medical products, health technologies, and/or commodities. These terms are used with various qualifiers including essential, quality, safe, and effective (WHO 2007, WHO 2010b, Health Systems 20/20 2012). We argue that pharmaceutical products, medical products, and health technologies are broad but not interchangeable terms. For example, health technologies ‘may be used to promote health, to prevent, diagnose or treat acute or chronic disease, or for rehabilitation. [They] include pharmaceuticals, devices, procedures, and organizational systems used in health care’ (HTA Glossary 2014). Hence, health technologies may include interventions such as surgical methods, while the term medicines may exclude other pharmaceutical products such as vaccines. Taking these into consideration, we think the term pharmaceutical product is the most appropriate because it is sufficiently inclusive and delineates unambiguous boundaries for the goals of a pharmaceutical system. In agreement with the literature, we qualify pharmaceutical products as being ‘safe, effective, and quality’. We acknowledge that health systems need to prioritize access to essential medicines. However, we exclude essential as a qualifier of pharmaceutical products because the system is not limited solely to products categorized as essential. Our definition includes ‘related services’ because a pharmaceutical product is unlikely to be beneficial and effective without related services such as prescribing, dispensing, and medication counselling.
Use is generally understood as prescribing, dispensing or sale, and consumption or end use by the patient. Its qualifiers in the literature include rational, appropriate, cost-effective, timely, and equitable. Given that use can be rational from the provider or user’s perspective but actually inappropriate (Bigdeli et al. 2014), we propose that use should be qualified as appropriate. However, ‘rational medicines use’ usually denotes safe, effective and cost-effective use, so we suggest that cost-effective be added to the qualifiers for use (Holloway and van Dijk 2011). The term is particularly relevant, given the growing potential for excessive expenditures on pharmaceutical products at the household and national levels (Lu et al. 2011, Wagner et al. 2011).
Beyond ensuring access to and positively influencing use of medicines, there are other intermediate and ultimate system goals. According to Roberts and Reich (2011), the intermediate system performance goals—efficiency, quality, and access—are the means to improving health status, financial protection, and citizen satisfaction in the target population. This conceptualization is similar to the health system goals where ensuring access to and coverage for quality and safe services is the intermediate goal and the means for achieving the ultimate goals—improved health, system responsiveness, social and financial risk protection, and improved efficiency (WHO 2007). We postulate that the affordability dimension of access to pharmaceutical products includes costs at both the user and system levels and accounts for the financial risk protection goals of the health system. Further, system performance includes the efficiency with which the system allocates products and services among the population and at what cost; the quality of pharmaceutical products and related services; and the responsiveness of the pharmaceutical system to the health needs of the population. We argue that the ultimate goal of a pharmaceutical system is to improve health outcomes, even if the multiple determinants of health make it impossible to directly attribute positive health outcomes to the pharmaceutical system.
Defining PSS
The health systems strengthening literature offer two important perspectives: Strengthening includes building resilience and the need to distinguish systems strengthening from interventions that rely on continued external inputs to sustain improved performance. The pharmaceutical system often faces disruptive changes and must respond appropriately to anticipated and unanticipated disturbances targeting its components. Therefore, we see resilience as a key characteristic of a well-functioning system. The concept of resilience is embedded in our definition of PSS to underline that sustainable progress is achieved when the system is more resilient.
Another relevant observation from the health system literature is that system strengthening is different from the mere support that only addresses current constraints. Strengthening targets system performance drivers and aims to change the system so that it can address future constraints (Chee et al. 2013). Importantly, strengthening and system support are connected, and a stringent distinction between the two may be artificial (Huff-Rousselle 2013). Chee et al. (2013) propose four criteria for assessing whether an intervention strengthens the health system: it has cross-cutting benefits beyond a single disease; addresses policy and organizational constraints or strengthens relationships between the building blocks; produces permanent systemic impact beyond the term of the project; and is tailored to country-specific constraints and opportunities, with clearly defined roles for country institutions. We postulate that these criteria are also useful to recognize PSS interventions.
Critical components and system outcomes for measuring PSS
Our identification of critical components draws upon input from the SIAPS consultative meeting. Meeting participants used the results of our analysis of frameworks and assessment tools to guide their discussions and reach consensus on which components of the pharmaceutical system are essential for the purpose of developing PSS measurement tools. The following paragraphs highlight the main arguments for selecting each component.
The pharmaceutical products and related services component is at the centre of the system and encompasses the functions of selection, procurement, and distribution of pharmaceutical products. In a properly functioning system, selection is informed by the health needs of the population and guides the procurement and distribution of essential pharmaceutical products (FHI 360 2012, Health System 20/20 2012, MSH 2012). A suite of patient-centred service delivery strategies and strengthening interventions targets appropriate and cost-effective prescribing, dispensing, retail practices, as well as correct use of pharmaceutical products by end-users.
Policy, laws and governance is the hub of coordination for the entire system and directly interacts with all system components. It provides the framework, structures, and systems for organizing, financing, and regulating the system, and coordinating the activities of the various institutions and stakeholders to achieve the system objectives. It also determines the mechanisms and procedures needed to facilitate participation, transparency, and accountability, and prevent corruption and unethical practices, thereby reducing system inefficiencies and inequities. A related component, regulatory systems, focuses on ensuring the safety, efficacy and quality of pharmaceutical products and related services. It includes product registration, licensing of pharmaceutical establishments and personnel, inspection and enforcement, surveillance of product quality and safety, regulation and oversight of clinical trials, and control of pharmaceutical marketing practices.
Innovation, research and development, manufacturing and trade is the entry point for pharmaceutical products into the system. Innovation includes the development of both products and new delivery systems. Intellectual property protections in national legislation and international trade agreements shape innovation and trade, and affect the availability and affordability of pharmaceutical products. This component also refers to domestic manufacturing capacity-building, where economically viable, to produce quality assured and competitively priced generics and to the research and development of new pharmaceutical products.
The financing component refers to the management of resources to ensure the adequate and sustainable financing of pharmaceutical product purchases, related services, human resources and other costs associated with system functioning. It also involves financial risk protection strategies and monitoring and controlling costs and prices to reduce financial barriers to access pharmaceutical products and related services and promote affordability at the system and individual level. Human resources component ensures the availability of adequate numbers of appropriately trained staff for managing the supply and delivery of pharmaceutical products and related services. It includes policy and strategy, and personnel management and development. The information component refers to the generation and dissemination of timely and reliable information to support decision-making. Information is essential to all components of pharmaceutical systems.
PSS measurement tools will also need to account for the primary system outcomes: access and use. Access refers to the affordability, availability, accessibility, acceptability of products and related services. Use refers to the prescribing, dispensing or sale, and consumption or end use of pharmaceutical products. Performance and resilience are two system attributes that are important for measuring PSS. Performance includes the efficiency with which the system allocates products and services among the population and at what cost; the quality and safety of pharmaceutical products and related services; and the responsiveness of the pharmaceutical system to the health needs of the population. Resilience is the capacity of the system to prepare for and effectively respond to crises thereby maintaining core functions, adapting to changing circumstances as needed and, transforming when social and economic conditions make the existing system no longer viable.
Limitations
Our work has several limitations, some related to our methods. As with any literature review, there is a concern that important publications may have been omitted. Our literature search relied heavily on online databases and we may have missed relevant sources that are unavailable online, such as conceptual or background papers written for the purpose of developing some of the assessment tools we reviewed. Also, the literature on health system resilience is only emerging, and we may need to refine our understanding of this concept and its measurement as more empirical evidence becomes available. In addition, the series of consultations that took place involved high-level experts representing a wide range of subspecialties, opinions, organizations, institutions, and countries. At the end of an intense two-day meeting, 30 experts were able to reach consensus. However, the meeting outcomes only embody the points of view of meeting participants, especially with regard to identifying key components of the pharmaceutical system.
Another limitation is related to the inevitable trade-off between complexity and pragmatism. Addressing the complexity of the pharmaceutical system and its connections with the broader health system requires an integration of multidisciplinary perspectives that cannot be reduced to a simple list of system components. However, aiming for a highly complex and exhaustive model of the system would be self-defeating for the purpose of developing practical measurement tools. Our work did not attempt to build a definitive pharmaceutical system model within the health system framework. We recognize that a seven-component system does not capture important informal interactions, such as those taking place at the provider–patient level for example, which may drive demand and use. Still, even if our definitions and proposed system components do not entirely reflect the complexity of the pharmaceutical system, they uncover two key measurable attributes: performance and resilience.
Our ultimate objective is to develop and deploy actual PSS measurement tools. This work is only the starting point of a larger project. The next steps will be to identify the most important elements associated with each system component that reflect performance and resilience. This will allow the selection of a number of suitable indicators related to each component. These indicators will then need to be refined through repeated testing. They will ultimately be deployed as a measurement tool to assess PSS, which can help guide future interventions to ensure that they result in stronger, more resilient pharmaceutical systems.
Conclusions
Our research sought to provide conceptual clarity to the fragmented thinking about pharmaceutical systems and their strengthening. Our definitions of pharmaceutical system and PSS emerged from a comprehensive review of the literature, which provided background information to the experts who participated in the consultations. This cohesive effort to conceptualize the pharmaceutical system as an entity may serve as an important reference to academics and practitioners working in the field. Despite its limitations, our work underlines how critical the pharmaceutical system is in achieving the health and risk protection goals of UHC. It completes a necessary step toward our ultimate objective, which is developing and deploying measurement tools to assess progress toward stronger and more resilient pharmaceutical systems within health systems. It provides a practical starting point for evaluating investments in PSS and measuring the progress of PSS interventions.
Supplementary Material
Acknowledgments
The authors are grateful to the participants of the SIAPS Partners’ Consultative Meeting for their input. They are also grateful to Michael Cohen, Richard Laing, Ruth Musila, Patricia Paredes Jodrey, Susan Putter, Dennis Ross-Degnan, Evans Sagwa, Maura Soucy Brown, Catherine Vialle-Valentin, and Veronika Wirtz for their comments on earlier versions of this manuscript.
Funding
This work was supported by the Systems for Improved Access to Pharmaceuticals and Services (SIAPS) Program, which is funded by the US Agency for International Development (USAID), under the terms of cooperative agreement number AID-OAA-A-11-00021. The contents are the responsibility of the authors and do not necessarily reflect the views of USAID or the United States Government.
Conflict of interest statement. None declared.
References
- Adam T, de Savigny D.. 2012. Systems thinking for strengthening health systems in LMICs: need for a paradigm shift. Health Policy and Planning 27(suppl 4): iv1–3. [DOI] [PubMed] [Google Scholar]
- Agyepong IA, Kodua A, Adjei S. et al. 2012. When ‘solutions of yesterday become problems of today’: crisis-ridden decision making in a complex adaptive system (CAS)—the Additional Duty Hours Allowance in Ghana. Health Policy and Planning 27(suppl 4): iv20–31. [DOI] [PubMed] [Google Scholar]
- Atun R. 2012. Health systems, systems thinking and innovation. Health Policy and Planning 27(suppl 4): iv4–8. [DOI] [PubMed] [Google Scholar]
- Atun R, de Jongh T, Secci F. et al. 2010. Integration of targeted health interventions into health systems: a conceptual framework for analysis. Health Policy and Planning 25: 104–11. [DOI] [PubMed] [Google Scholar]
- Bahadur AV, Ibrahim M, Tanner T.. 2013. Characterising resilience: unpacking the concept for tackling climate change and development. Climate and Development 5: 55–65. [Google Scholar]
- Balabanova D, Mills A, Conteh L. et al. 2013. Good Health at Low Cost 25 years on: lessons for the future of health systems strengthening. The Lancet 381: 2118–33. [DOI] [PubMed] [Google Scholar]
- Béné C. 2013. Towards a quantifiable measure of resilience. IDS Working Papers 2013: 1–27. Retrieved April 27, 2015 from http://bit.ly/1Jgh4sQ [Google Scholar]
- Béné C, Newsham A, Davies M. et al. 2014. Review article: Resilience, poverty and development. Journal of International Development 26: 598–623. [Google Scholar]
- Bigdeli M, Garabedian LF, Javadi D. et al. 2014. Why a health systems approach? In Bigdeli M, Peters DH, Wagner AK (eds). Medicines in Health Systems: Advancing Access, Affordability and Appropriate Use. Geneva: World Health Organization, 18–26. [Google Scholar]
- Bigdeli M, Jacobs B, Tomson G. et al. 2013. Access to medicines from a health system perspective. Health Policy & Planning 28: 692–704. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bigdeli M, Laing R, Tomson G. et al. 2015. Medicines and universal health coverage: challenges and opportunities. Journal of Pharmaceutical Policy and Practice 8: 8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Blanchet K. 2013. Governance of health systems; Comment on "“A network based theory of health systems and cycles of well-being"”. International Journal of Health Policy and Management 1: 177.. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Blanchet K. 2015. Thinking shift on health systems: from blueprint health programmes towards resilience of health systems; comment on "“Constraints to applying systems thinking concepts in health systems: A regional perspective from surveying stakeholders in Eastern Mediterranean countries"”. International Journal of Health Policy and Management 4: 307–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Boerma T, Eozenou P, Evans D. et al. 2014. Monitoring progress towards universal health coverage at country and global Levels. PLoS Medicine 11: e1001731. doi:10.1371/journal.pmed.1001731. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bowen KJ, Friel S, Ebi K. et al. 2011. Governing for a healthy population: towards an understanding of how decision-making will determine our global health in a changing climate. International Journal of Environmental Research and Public Health 9: 55–72. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Brudon P, Rainhorn JD, Reich MR.. 1999. Indicators for Monitoring National Drug Policies: A Practical Manual. Geneva: World Health Organization; Retrieved from http://apps.who.int/medicinedocs/pdf/whozip14e/whozip14e.pdf [Google Scholar]
- Cameron A, Ewen M, Ross-Degnan D. et al. 2009. Medicine prices, availability, and affordability in 36 developing and middle-income countries: a secondary analysis. The Lancet 373: 240–9. [DOI] [PubMed] [Google Scholar]
- Chee G, Pielemeier N, Lion A. et al. 2013. Why differentiating between health system support and health system strengthening is needed. The International Journal of Health Planning and Management 28: 85–94. [DOI] [PMC free article] [PubMed] [Google Scholar]
- CPM (Center for Pharmaceutical Management). 2003. Defining and Measuring Access to Essential Drugs, Vaccines, and Health Commodities: Report of the WHO-MSH Consultative Meeting, Ferney-Voltaire, France, December 11–13, 2000. Prepared for the Strategies for Enhancing Access to Medicines Program. Arlington, Va.: Management Sciences for Health. Retrieved from http://projects.msh.org/seam/reports/measuring_access_Dec2000.pdf
- de Savigny D, Adam T (eds). 2009. Systems Thinking for Health Systems Strengthening. Geneva: World Health Organization; Retrieved from http://whqlibdoc.who.int/publications/2009/9789241563895_eng.pdf [Google Scholar]
- Eidelson RJ. 1997. Complex adaptive systems in the behavioral and social sciences. Review of General Psychology 1: 42. [Google Scholar]
- European Commission. 2014. Communication from the Commission on Effective, Accessible and Resilient Health Systems. Brussels: European Commission; Retrieved from http://ec.europa.eu/health/healthcare/docs/com2014_215_final_en.pdf. [Google Scholar]
- FHI 360. 2012. Health System Rapid Diagnostic Tool. Framework, Operational Guide, and Metrics to Measure the Strength of Priority Health System Functions. Durham NC: FHI 360; Retrieved from http://www.fhi360.org/resource/health-system-rapid-diagnostic-tool [Google Scholar]
- Frost LJ, Reich MR.. 2008. Access: How do Good Health Technologies Get to Poor People in Poor Countries. Boston, MA: Harvard Center for Population and Development Studies [Google Scholar]
- Hafner T, Walkowiak H.. 2014. Defining and Measuring Pharmaceutical Systems Strengthening: Report of the SIAPS Partner Consultative Meeting. September 11-12, 2014. Submitted to the US Agency for International Development by the Systems for Improved Access to Pharmaceuticals and Services (SIAPS) Program. Arlington, VA: Management Sciences for Health.
- Health Systems 20/20. 2012. The Health System Assessment Approach: A How-To Manual. Version 2.0. https://www.hfgproject.org/wp-content/uploads/2015/02/HSAA_Manual_Version_2_Sept_20121.pdf
- Hess JJ, McDowel JZ, Luber G.. 2012. Integrating climate change adaptation into public health practice: using adaptive management to increase adaptive capacity and build resilience. Environmental Health Perspectives 120: 171.. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Holling CS. 2001. Understanding the complexity of economic, ecological, and social systems. Ecosystems 4: 390–405. [Google Scholar]
- Holloway KA, Ivanovska V, Wagner AK. et al. 2013. Have we improved use of medicines in developing and transitional countries and do we know how to? Two decades of evidence. Tropical Medicine & International Health 18: 656–64. [DOI] [PubMed] [Google Scholar]
- Holloway K, van Dijk L.. 2011. Rational use of medicines In: WHO. The World Medicines Situation, 3rd edition Geneva: World Health Organization; Retrieved from http://who.int/medicines/areas/policy/world_medicines_situation/en/index.html [Google Scholar]
- Hou X, Velényi EV, Yazbeck AS. et al. 2013. Learning from Economic Downturns: How to Better Assess, Track, and Mitigate the Impact on the Health Sector. Washington DC: World Bank Publications. [Google Scholar]
- HTA Glossary. July 1, 2014. Health technology. Retrieved from http://htaglossary.net/health+technology.
- Huff-Rousselle M. 2013. Reflections on the frameworks we use to capture complex and dynamic health sector issues. The International Journal of Health Planning and Management 28: 95–101. [DOI] [PubMed] [Google Scholar]
- Islam M. (ed). 2007. Health Systems Assessment Approach: A How-To Manual. Submitted to the US Agency for International Development in collaboration with Health Systems 20/20, Partners for Health Reformplus, Quality Assurance Project, and Rational Pharmaceutical Management Plus. Arlington, VA: Management Sciences for Health.
- Janssen MA, Osnas EE.. 2005. Adaptive capacity of social-ecological systems: lessons from immune systems. EcoHealth 2: 93–101. [Google Scholar]
- Kieny MP, Dovlo D.. 2015. Beyond Ebola: a new agenda for resilient health systems. The Lancet 385: 91–2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kieny MP, Evans DB, Schmetsa G. et al. 2014. Health-system resilience: reflections on the Ebola crisis in western Africa. Bulletin of the World Health Organization 92: 850-850. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kohler JC, Mackey TK, Ovtcharenko N.. 2014. Why the MDGs need good governance in pharmaceutical systems to promote global health. BMC Public Health 14: 63.. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kruk ME, Myers M, Varpilah ST. et al. 2015. What is a resilient health system? Lessons from Ebola. The Lancet 385: 1910–2. [DOI] [PubMed] [Google Scholar]
- Lu Y, Hernandez P, Abegunde D. et al. 2011. Medicine expenditures In: WHO. The World Medicines Situation, 3rd edition Geneva: World Health Organization; Retrieved from http://apps.who.int/medicinedocs/documents/s18767en/s18767en.pdf. [Google Scholar]
- Management Sciences for Health. 1997. Managing Drug Supply: The Selection, Procurement, Distribution, and Use of Pharmaceuticals. Arlington, VA: Management Sciences for Health. [Google Scholar]
- Management Sciences for Health. 2012. MDS-3: Managing Access to Medicines and Health Technologies. Arlington, VA: Management Sciences for Health. [Google Scholar]
- Marchal B, Cavalli A, Kegels G.. 2009. Global health actors claim to support health system strengthening—is this reality or rhetoric? PLoS Medicine 6: e1000059.. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Mayhew S, Hanefeld J.. 2014. Planning adaptive health systems: the climate challenge. The Lancet Global Health 2: e625–6. [DOI] [PubMed] [Google Scholar]
- Miralles MA, 2010. Strengthening health systems to improve access to antimicrobials and the containment of resistance In de J. Sosa A, Byarugaba DK, Amabile C. et al. (eds). Antimicrobial Resistance in Developing Countries. New York: Springer, 385-401. [Google Scholar]
- Paina L, Peters DH.. 2012. Understanding pathways for scaling up health services through the lens of complex adaptive systems. Health Policy and Planning 27: 365–73. [DOI] [PubMed] [Google Scholar]
- Penchansky R, Thomas JW.. 1981. The concept of access: definition and relationship to consumer satisfaction. Medical Care 19: 127–40. [DOI] [PubMed] [Google Scholar]
- Pisu M. 2014. Overcoming Vulnerabilities of Health Care Systems. OECD Economics Department Working Papers, No. 1132. OECD Publishing; Retrieved April 28, 2015 from http://dx.doi.org/10.1787/5jz159228n6j-en [Google Scholar]
- Rational Pharmaceutical Management Plus Program. 2005. Pharmaceutical System Performance within the Context of Health Sector Reform. Arlington, VA: Management Sciences for Health. [Google Scholar]
- Rational Pharmaceutical Management Project, Latin America and Caribbean Health and Nutrition Sustainability Project, Regional Program on Essential Drugs Pan American Health Organization. 1995. Rapid Pharmaceutical Management Assessment: An Indicator-based Approach. Arlington, VA: Management Sciences for Health. [Google Scholar]
- Rhodes MG. 2013. A network based theory of health systems and cycles of well-being. International Journal of Health Policy and Management 1: 7–15. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Roberts MJ, Reich MR.. 2011. Pharmaceutical Reform. A Guide to Improving Performance and Equity. Washington, DC: The World Bank. [Google Scholar]
- Roberts MJ, Hsiao W, Berman P. et al. 2008. Getting Health Reform Right. A Guide to Improving Performance and Equity. New York: Oxford University Press; http://www.oxfordscholarship.com/view/10.1093/acprof:oso/9780195371505.001.0001/acprof-9780195371505 [Google Scholar]
- Seiter A. 2010. A Practical Approach to Pharmaceutical Policy. Washington DC: World Bank Publications. [Google Scholar]
- Sterman JD. 2006. Learning from evidence in a complex world. American Journal of Public Health 96: 505–14. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Sturmberg JP, O'Halloran DM, Martin CM.. 2012. Understanding health system reform–a complex adaptive systems perspective. Journal of Evaluation in Clinical Practice 18: 202–8. [DOI] [PubMed] [Google Scholar]
- Swanson RC, Cattaneo A, Bradley E. et al. 2012. Rethinking health systems strengthening: key systems thinking tools and strategies for transformational change. Health Policy and Planning 27(suppl 4): iv54–61. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Systems for Improved Access to Pharmaceuticals and Services (SIAPS). 2013. SIAPS Fact Sheet Retrieved from http://siapsprogram.org/wp-content/uploads/2013/09/SIAPS-Fact-Sheet_2013.pdf.
- Thomas S, Keegan C, Barry S. et al. 2013. A framework for assessing health system resilience in an economic crisis: Ireland as a test case. BMC Health Services Research 13: 450. [DOI] [PMC free article] [PubMed] [Google Scholar]
- van Olmen J, Marchal B, Van Damme W. et al. 2012. Health systems frameworks in their political context: framing divergent agendas. BMC Public Health 12: 774. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wagner AK, Quick JD, Ross-Degnan D.. 2014. Quality use of medicines within universal health coverage: challenges and opportunities. BMC Health Services Research 14: 357.. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wagner AK, Graves AJ, Reiss SK. et al. 2011. Access to care and medicines, burden of health care expenditures, and risk protection: results from the World Health Survey. Health Policy 100: 151–8. [DOI] [PubMed] [Google Scholar]
- Walker B, Holling CS, Carpenter SR. et al. 2004. Resilience, adaptability and transformability in social–ecological systems. Ecology and Society 9: 5. [Google Scholar]
- WHO. 2004. Equitable access to essential medicines: a framework for collective action. Policy Perspectives on Medicines 8: 1–6. Retrieved from http://apps.who.int/medicinedocs/en/d/Js4962e/ [Google Scholar]
- WHO. 2007. Everybody's Business–Strengthening Health Systems to Improve Health Outcomes: WHO's Framework for Action. Geneva: World Health Organization. [Google Scholar]
- WHO. 2009. Measuring Transparency in the Public Pharmaceutical Sector. Assessment Instrument. Geneva: World Health Organization. [Google Scholar]
- WHO. 2010a. The World Health Report – Health Systems Financing: The Path to Universal Coverage. Geneva: World Health Organization; Available: http://www.who.int/whr/2010/en/ [DOI] [PMC free article] [PubMed] [Google Scholar]
- WHO. 2010b. Monitoring the Building Blocks of Health Systems: A Handbook of Indicators and Their Measurement Strategies. Geneva: World Health Organization. [Google Scholar]
- WHO. 2011. Pharmaceutical Human Resources Assessment Tools. Geneva: World Health Organization; Retrieved from http://apps.who.int/medicinedocs/en/d/Js18717en/ [Google Scholar]
- WHO. 2014, June 16. Health Systems Strengthening Glossary Retrieved from http://www.who.int/healthsystems/hss_glossary/en/index5.html.
- WHO, HAI. 2008. Measuring Medicine Prices, Availability, Affordability and Price Components, 2nd ed.Geneva: World Health Organization and Health Action International; Retrieved from http://www.haiweb.org/medicineprices/manual/documents.html [Google Scholar]
- WHO, World Bank, Global Alliance on Vaccines Initiative (GAVI), Global Fund to Fight AIDS, Tuberculosis and Malaria (GFATM). 2009. Monitoring and Evaluation of Health Systems Strengthening. An Operational Framework. Geneva: WHO; Retrieved from http://www.who.int/healthinfo/HSS_MandE_framework_Nov_2009.pdf. [Google Scholar]
- Windisch R, Waiswa P, Neuhann F. et al. 2011. Scaling up antiretroviral therapy in Uganda: using supply chain management to appraise health systems strengthening. Global Health 7: 25. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wulff K, Donato D, Lurie N.. 2015. What is health resilience and how can we build it?. Annual Review of Public Health 36: 361–74. [DOI] [PubMed] [Google Scholar]
- Xiao Y, Zhao K, Bishai DM. et al. 2013. Essential drugs policy in three rural counties in China: what does a complexity lens add? Social Science & Medicine 93: 220–8. [DOI] [PubMed] [Google Scholar]
- Yadav P, Smith RD, Hanson K.. 2011. Pharmaceuticals and the health sector In Smith RD, Hanson K (eds). Health Systems in Low-and Middle-Income Countries: An Economic and Policy Perspective. Oxford: Oxford University Press, 147. [Google Scholar]
- Zaidi S, Bigdeli M, Aleem N. et al. 2013. Access to essential medicines in Pakistan: policy and health systems research concerns. PloS One 8: e63515. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.