A Results-Based Logic Model for Primary Healthcare: A Conceptual Foundation for Population-Based Information Systems

Abstract

A conceptual framework for population-based information systems is needed if these data are to be created and used to generate information to support healthcare policy, management and practice communities that seek to improve quality and account for progress in primary healthcare (PHC) renewal. This paper describes work conducted in British Columbia since 2003 to (1) create a Results-Based Logic Model for PHC using the approach of the Treasury Board of Canada in designing management and accountability frameworks, together with a literature review, policy analysis and broad consultation with approximately 650 people, (2) identify priorities for information within that logic model, (3) use the logic model and priorities within it to implement performance measurement and research and (4) identify how information systems need to be structured to assess the impact of variation or change in PHC inputs, activities and outputs on patient, population and healthcare system outcomes. The resulting logic model distinguishes among outcomes for which the PHC sector should be held more or less accountable.

Over the past decade, First Ministers have made a number of commitments to renew primary healthcare (PHC) in Canada and to enhance accountability and transparency regarding those investments (Government of Canada 2000, 2003, 2004, 2006). All federal, provincial and territorial governments now support renewal initiatives designed to improve the organization, funding and delivery of these services. While it is possible to use case studies, project evaluations and anecdotal evidence to learn about the impact of renewal initiatives and highlight progress, population-based information and reporting systems are needed as healthcare policy makers and managers seek to monitor the performance of this sector, identify areas requiring attention and action, assess the relative impact of different strategies to catalyze renewal and account to citizens on progress.

If PHC information systems are to generate relevant information, then these systems must be designed to (a) monitor and report on indicators of performance important to stakeholders, (b) support identification of factors associated with improvements or declines in service quality and (c) measure whether objectives of reform are being achieved. In essence, they must be structured using robust methods of organizing and analyzing data to support performance management and research. The purposes of this paper are to describe work conducted in British Columbia since 2003 to (a) create a Results-Based Logic Model for PHC that reflects the approach of the Treasury Board of Canada in designing management and accountability frameworks, (b) identify priorities for information within that logic model, (c) use the logic model and priority within it to implement performance measurement strategies and (d) describe how information systems need to be structured to assess the impact of variation or change in PHC inputs, activities and outputs on patient, population and healthcare system outcomes.

The Treasury Board of Canada focuses on implementing performance measurement to guide quality improvement and public reporting. The first step in this approach is the creation of a results-based logic model that linearly links resource inputs to activities performed, services delivered and outcomes achieved based on the goals and objectives defined for a policy, program or initiative. In this process, a results-based logic model can be used to focus and unify evaluative efforts by enabling diverse stakeholders to work from a shared conceptual foundation and lexicon of inputs, activities, outputs and outcomes. The model establishes a common theory about the logic links among these different dimensions and a shared set of assumptions about these dependencies. It illuminates the array of activities that are potential levers for change, recognizes the unique and distinguishing features of PHC and outcomes attributable to this sector and identifies the way in which PHC and other health sectors converge to affect health system–level performance and the health of the population. As such, the model defines the areas in which information, evaluation and evidence are needed for policy, administrative and practice communities to plan, implement and report on PHC renewal. The second step in the process is the identification of indicators of performance. Subsequent steps include monitoring, research and reporting in order to identify and communicate opportunities to improve the speed and direction of progress in strengthening primary healthcare.

What Did We Do and What Did We Learn?

A conceptual framework: Performance measurement and accountability in PHC

We chose the approach of the Treasury Board of Canada in designing a results-based management and accountability agenda because outcome-oriented frameworks have underpinned health system transformations that have produced rapid and impressive improvements in quality of healthcare (Asch et al. 2004; Kizer 1999). The Treasury Board's “modern management agenda” is outlined in Results for Canadians: A Management Framework for the Government of Canada (Treasury Board of Canada 2000). It recognizes that public services exist to serve Canadians and that a citizen focus must be built into all government-funded activities, programs and services. The objective is achievement of “results” for Canadians. This approach focuses on measuring and reporting on outcomes throughout the life cycle of a policy, program or initiative, and on integrating strategy, people, processes and measurement to improve decision-making and drive change (Treasury Board of Canada 2001).

To create the Results-Based Logic Model for PHC we conducted a policy analysis, reviewed the literature and consulted with stakeholders. To identify the goals and objectives relevant to PHC renewal in Canada, we conducted an analysis to identify prominent and recurring themes in relevant policy statements and documents such as the final reports of the Commission on the Future of Health Care in Canada (Romanow 2002) and the Standing Senate Committee on Social Affairs, Science and Technology (Kirby 2002) as well as the objectives of the Primary Health Care Transition Fund (Government of Canada 2000, 2001). The logic model aligns with First Ministers' objectives for PHC renewal as articulated in recent policy documents (Government of Canada 2003, 2004).

We also reviewed existing conceptual models of health/illness, healthcare and PHC that have been informed by (or inform) health services and policy research (Aday 2001; Andersen and Newman 1973; Donabedian 1966; Lamarche et al. 2003; Starfield 1992). This review was done to identify important dimensions of PHC relevant to access, services and outcomes. Existing performance measurement and accountability frameworks were reviewed with particular focus on PHC frameworks used in other countries (e.g., Sibthorpe and Gardner 2007; Department of Health 1999). Finally, a literature review was conducted to identify PHC inputs, activities, outputs and outcomes as well as to substantiate the relationships among them as determined through research.

A multi-stage iterative feedback and revision process was used for stakeholder consultations. These consultations were undertaken for a period of nine months, and the model was continually revised in response to feedback. The following stakeholder groups received formal requests for comment, participated in small focus groups or attended presentations to solicit feedback on draft versions of the model: PHC practitioners from various health regions responsible for delivering PHC and implementing or evaluating renewal in British Columbia (∼200 people in small focus groups); individuals from university departments or professional associations/colleges representing medicine, family medicine, nursing and pharmacy (∼40 people via interviews); senior academic researchers and consultants across Canada who specialize in PHC research and evaluation (∼10 people sent formal request for written feedback); PHC leaders and evaluation specialists who work for provincial and territorial ministries of health (∼50 people in small focus groups); and participants who attended a session hosted at a national conference for PHC (∼350 people via an open request for written feedback).

PHC was defined as products or services designed to address acute and episodic health conditions and to manage chronic health conditions. It is also where health promotion and education efforts are undertaken, where patients receive first contact care and where those in need of more specialized services are connected with other parts of the healthcare system. We focused principally on services directed towards individuals, rather than PHC services directed towards communities and contexts that influence health. The Results-Based Logic Model for PHC is summarized here (Figure 1) and described in more detail elsewhere (Watson et al. 2004).

FIGURE 1.

Results-Based Logic Model for PHC

In the Results-Based Logic Model we identify inputs as the resources used to carry out activities, produce outputs and accomplish results. Inputs include fiscal (e.g., amount of funding), material (e.g., information technology; office size and layout) and human resources (e.g., number and types of providers). PHC activities are the work processes intended to enable, support or produce specific outputs or products and services and are the primary link in the chain through which outcomes are achieved. PHC activities are differentiated from outputs in that the former are undertaken “in preparation” for delivery of PHC products and services. These activities can be categorized into three types: policy- and governance-level activities and decisions (e.g., financing and regulation), healthcare management-level activities and decisions (e.g., hours of operation; use of teams) and clinical-level activities and decisions that support outputs (e.g., training in quality improvement; the degree to which clinicians elect to specialize in specific types of clientele such as older adults). Together, these elements form the structure and foundation of a PHC system.

PHC outputs are direct products or services delivered as a result of PHC activities and represent the interface between this sector and individuals. PHC services and products (e.g., visits; information brochures) can be described by volume, distribution, type and qualities. PHC outputs can be further described in terms of their distribution: who gets how much of what types of services and products. Types of output include health promotion and disease prevention as well as curative, rehabilitative, supportive and palliative services to targeted individuals or populations. As gatekeepers, PHC providers also offer referrals to specialty providers. Finally, PHC services and products can be described and measured in terms of distinguishing qualities, for example, first-contact accessibility, comprehensiveness of services, continuity, cultural sensitivity, interpersonal communication, respectfulness, technical quality of clinical care and so on. While terms to describe PHC outputs are abundant, the Results-Based Logic Model for PHC relies on a limited set of important attributes and operational definitions achieved through consensus among Canadian experts (Haggerty et al. 2007).

PHC outcomes represent “results” for Canadians. PHC outcomes can be immediate, intermediate or final, depending on the degree – as identified by stakeholders – to which the PHC sector should be held more accountable (immediate outcomes) or less accountable (intermediate and final outcomes). Immediate outcomes are those most directly attributable to outputs and for which the PHC workforce of policy makers, managers and practitioners can reasonably assume control, responsibility and accountability. Indeed, we opted to develop a Results-Based Logic Model to distinguish among immediate, intermediate and final outcomes because no other conceptual framework differentiates outcomes for which PHC should be held more or less accountable.

Three immediate outcomes are, for the most part, under direct control of the PHC sector: increased knowledge about health and healthcare among the population; reduced risk, duration and effects of acute and episodic conditions; and reduced risk and effects of continuing health conditions. A fourth immediate outcome is the maintenance or improvement of the work life of the PHC workforce.

Intermediate outcomes include areas in which PHC stakeholders have a lesser degree of control, but for which PHC services are still expected to have an impact. These outcomes include appropriateness of provider and place (e.g., minimize unnecessary use of emergency room services); healthcare system efficiency (e.g., avoidable hospitalizations such as admissions for asthma); acceptability or satisfaction; and healthcare system equity (e.g., individuals who receive healthcare directly from PHC or via its gatekeeper function are those most in need; individuals with similar need receive similar levels of care). Final outcomes are the long-term, ultimate objectives of PHC for individuals, the population and the healthcare system. They include a sustainable and accountable healthcare system; improvement and maintenance of function, resilience and health for individuals; and improved population-level health and wellness.

The Results-Based Logic Model for PHC (Figure 1) includes shaded boxes and arrows that represent contexts and expected linkages among inputs, activities, outputs and outcomes. Care has been taken to highlight special circumstances. For example, shaded boxes and arrows have been included to highlight that:

• external forces (social, cultural, legal/regulatory, physical and economic contexts, as well as population characteristics and participation in PHC) influence inputs, activities, outputs and outcomes;

• health human resources influence and are influenced by policy, governance and management activities and decisions as well as fiscal inputs;

• policy/governance, management and clinical activities and decisions are influenced by population characteristics, public participation and contexts;

• PHC products and services are influenced by policy/governance, management and clinical activities as well as population characteristics and contexts;

• immediate outcomes that relate to clients/patients influence one another and, collectively, influence intermediate outcomes, and also have a direct effect on one final outcome – improvement and maintenance of functioning, resilience and health for individuals who receive PHC;

• the immediate outcome that relates to maintaining or improving the work life of the PHC workforce has a direct influence on sustainability of the healthcare system.

In accordance with the Treasury Board approach, the efficiency of the PHC system is seen as a function of inputs, activities and outputs. For example, efficiency can be assessed by comparing outputs per unit of inputs. By comparison, effectiveness of the PHC system is a function of outputs and outcomes. Because external factors affect immediate, intermediate and final outcomes, statistical analytic methods are required to attribute PHC inputs, activities and outputs to relevant health and health system outcomes. This type of analysis can be used to test the association and direction of linkages among concepts reflected in the logic model. Equity or the relative distribution of resources can be measured from multiple perspectives: at the population or subgroup level and across inputs, outputs and outcomes.

Priorities for information: Accountability and performance improvement in PHC

The Results-Based Logic Model for PHC describes this sector from macro-, meso- and micro-levels and, essentially, includes the universe of relevant constructs. The next challenge is to define priorities for information among healthcare policy makers, managers, providers and the public to design data systems strategically to meet these needs. The following projects represent initiatives carried out in British Columbia and elsewhere using this logic model to identify priorities for information relevant to PHC in Canada.

Priorities for Quality Improvement. In 2005, in order to establish priorities for quality improvement and performance measurement from the public's perspective, we conducted 11 focus groups with adults in British Columbia (n=75) to identify the dimensions of PHC that they think are important and mention in discussions regarding areas that could be improved. A content analysis was conducted using a coding scheme based on the Results-Based Logic Model for PHC. These analyses revealed six dimensions in the following rank order of priority:

1. first-contact accessibility;

2. continuity;

3. responsiveness in terms of waiting in the office and the amount of time spent with the healthcare provider;

4. interpersonal communication;

5. technical quality of clinical care; and

6. whole-person care.

The dimensions most frequently associated with acceptability and satisfaction were interpersonal communication and continuity of care (Wong et al. 2008). This information can now be used to target initiatives to improve quality in ways that align with priorities among the public; it could also be used in priority-setting exercises regarding performance measurement to support public reporting. Recent work by others should also inform this agenda (Berta et al. 2008).

Policy Synthesis and Evaluation. In 2005, in order to identify a core set of evaluation questions deemed a priority among policy makers, managers, providers and academics in Canada, Haggerty and Martin (2005) synthesized national and international policy documents. Then they conducted key-informant interviews, devised a national consensus process to ensure face validity and used the Results-Based Logic Model for PHC to ensure content validity. This work was commissioned by Health Canada as part of the federal, provincial and territorial government-sponsored National Evaluation Strategy. The result was a set of 39 evaluation questions and seven policy objectives that laid the foundation for the identification of performance indicators by the Canadian Institute for Health Information (CIHI 2006a).

Performance Indicators. In 2005, CIHI used scientific evidence and national consultations to identify 105 performance indicators (30 of them high-priority) relevant to these evaluation questions (CIHI 2006a, 2006b). In order to assist readers who seek to categorize these indicators into contexts, inputs, activities, outputs and outcomes as highlighted in our Results-Based Logic Model for PHC, we developed a document that is available on our website (www.chspr.ubc.ca).

Importantly, priority indicators represent dimensions relevant to performance monitoring (i.e., boxes in the logic model), but any assessment of evaluation questions requires statistical analyses to assess the association and direction of linkages between these dimensions (i.e., arrows in the logic model). For example, one of the above-mentioned seven policy objectives is “to increase the emphasis on health promotion, disease and injury prevention and chronic disease management” (CIHI 2006a). Performance indicators can be used to measure and monitor the degree to which this shift in emphasis occurs over time. One of the 39 priority evaluation questions associated with this policy objective is “Do self-management strategies for patients with chronic conditions significantly improve the quality of life, reduce the number of visits to specialists and reduce hospital admissions (number and length of stay)?” (CIHI 2006a). Analyses are required to assess the degree to which this type of chronic disease management strategy (i.e., self-management as a PHC output) improves quality of life (i.e., final outcome) and reduces use of specialists and hospitals (i.e., intermediate outcome). As such analysis involves testing the strength of association or links between outputs and outcomes, PHC information systems must be organized to support this type of evaluative work.

Developing and using priority information for accountability and performance improvement

Since 2005, the Results-Based Logic Model for PHC has been used or customized for use, for example:

• to structure a systematic review conducted by a team in Australia to summarize expected outcomes of alternative PHC service delivery models (McDonald et al. 2006);

• to guide the development and use of a population-based information system at the Centre for Health Services and Policy Research at the University of British Columbia;

• to design a performance measurement and accountability strategy for a foundation that supports PHC services targeted to special populations (Ontario Neurotrauma Foundation 2006);

• to evaluate PHC renewal in the Yukon and community health centres in Central America, China and Ontario;

• to develop the Canadian Survey of Experiences with Primary Health Care (CSE-PHC) which was conducted by Statistics Canada in 2007 at a national level with sponsorship from the Health Council of Canada and in 2008 at the national and provincial levels with sponsorship from the Health Council of Canada and the Canadian Institute for Health Information; and

• to guide analyses of the CSE-PHC survey data regarding, for example, the simultaneous impact of PHC activities (interdisciplinary teams) on output type (emphasis on health promotion and disease prevention), output qualities (experiences with accessibility; comprehensiveness of services) and intermediate outcomes (acceptability; use of emergency departments and hospitalization; confidence in the healthcare system) (Khan et al. 2008; Watson et al. forthcoming in Healthcare Policy).

Because the Results-Based Logic Model has been used extensively across jurisdictions, it is now available in English, Chinese, French and Spanish (www.chspr.ubc.ca).

What Are the Implications?

Now that work has been done to identify a framework, priorities for information, relevant evaluation questions and performance indicators, a PHC information system needs to be structured to measure these dimensions and support attribution analyses at provincial and national levels. Such analysis requires that data systems measure shifts over time in priority dimensions, as well as the factors that influence temporal, geographic and organizational variation in patient, population and healthcare system outcomes.

Because data relevant to measuring these dimensions often reside in different databases, linkages between them are often required to support person-specific analyses (i.e., provider or patient) and nested data structures (e.g., providers reside or are nested within organizations, and patients reside or are nested within health regions) (see Watson 2009; page 105 of this special issue of Healthcare Policy).

Individual-level linkage of person-specific data offers the opportunity to aggregate anonymized information across databases about (a) the population perspective, to create a fuller understanding of peoples' use (or non-use) of health services, or (b) the provider perspective, to create a richer picture of professionals' practice patterns. Individual-level linkages between person-specific and organizational- or area-level information create nested data structures that offer the opportunity to test hypotheses regarding, for example, the impact of different organizational characteristics or community contexts on supply, distribution, delivery and use of PHC services.

Illuminating the impact of government commitments to renew PHC and assessing the impact of initiatives to catalyze changes require that information systems be designed to offer a temporal perspective on system performance. While it is also important to monitor and report on supply-side changes in the organization and delivery of healthcare from the provider perspective, it is the results of these renewal initiatives on temporal shifts in patients' experiences that are most important to quality improvement and to Canadians. Thus, the juxtaposition of population- and provider-based perspectives across time offers the most meaningful information to those charged with improving PHC or reporting to Canadians.

To track the effects of efforts to renew PHC, information systems must be designed to offer insights at the geographic and organizational levels. Information often needs to be aggregated, measured and reported on a regional basis, because the governance and accountability structure of healthcare in Canada – as established through legislative authority – is assigned to geographic regions. Thus, in most jurisdictions, governments, regional authorities and community health centres are responsible for community-based healthcare services delivered to all people who reside in a catchment area. In some jurisdictions, however, information needs to be aggregated, measured and reported at the level of an organization or network of organizations, because these entities are responsible, through contractual relationships, for services delivered to defined patient populations. In these instances (e.g., Primary Health Care Organizations or Integrated Health Networks in British Columbia, Primary Care Networks in Alberta, Family Health Teams in Ontario) there will be a greater need to measure and report on PHC inputs, activities, outputs and outcomes at the organizational level. But support for geographic-based analyses will continue to be important to governments and Canadians, because emerging evidence suggests that the strength of the PHC sector relates to the composition of primary, secondary and tertiary provider organizations in a region. Thus, initiatives to strengthen the PHC system in a region will always require geographic-based analyses.

Conclusion

In order to understand and account for the impact of targeted investments in PHC renewal and identify areas requiring improvement, many individuals and groups have invested energy in establishing indicators of PHC system performance. These efforts represent important contributions to the development of new standards of accountability. However, the lack of connections and conceptual links among disparate performance measures has created an administrative burden for providers, and is a significant barrier to moving the quality agenda forward (Institute of Medicine 2006). This lack of explicit connections and conceptual links among indicators is evident across Canada and internationally. It is compounded by the dearth of PHC information systems to support performance measurement. This situation has a high potential for burdening administrators and providers, who may then simply not follow through with the necessary work. As a result, the collective good efforts of many will fail to be translated into a cohesive and feasible strategy to support performance measurement, quality improvement and accountability.

Although a number of conceptual frameworks are used to profile healthcare systems and guide the design of information systems, none differentiates the unique features of the PHC sector or the outcomes attributable to it. The Results-Based Logic Model for PHC identifies health and healthcare outcomes for which the PHC sector should be held more or less accountable. The work outlined in this paper to create a cohesive, outcome-oriented framework, and to identify work completed to identify priorities for information, is intended to streamline efforts to support performance measurement and accountability.