Evaluating integrated health care: a model for measurement

Abstract

Purpose

In the development of integrated care, there is an increasing need for knowledge about the actual degree of integration between different providers of health services. The purpose of this article is to describe the conceptualisation and validation of a practical model for measurement, which can be used by managers to implement and sustain integrated care.

Theory

The model is based on a continuum of integration, extending from full segregation through intermediate forms of linkage, coordination and cooperation to full integration.

Methods

The continuum was operationalised into a ratio scale of functional clinical integration. This scale was used in an explorative study of a local health authority in Sweden. Data on integration were collected in self-assessment forms together with estimated ranks of optimum integration between the different units of the health authority. The data were processed with statistical methods and the results were discussed with the managers concerned.

Results

Judging from this explorative study, it seems that the model of measurement collects reliable and valid data of functional clinical integration in local health care. The model was also regarded as a useful instrument for managers of integrated care.

Discussion

One of the main advantages with the model is that it includes optimum ranks of integration beside actual ranks. The optimum integration rank between two units is depending on the needs of both differentiation and integration.

Introduction

In the Swedish health care system the county councils are responsible for all health care except care of the elderly and disabled, which is a responsibility of the municipalities. According to the National Board of Health and Welfare [1], two out of three Swedish county councils have recently restructured their health care delivery systems and implemented a system of “Local Health Care” in collaboration with the municipalities. This is an upgraded family- and community-oriented primary care, supported by flexible hospital services. Many politicians and policy makers are convinced that such an integrated system will assure both high quality and a cost-effective health care.

Although a lot of efforts have been put into this development, there is no common definition of Local Health Care. Most county councils have had the ambition to create a system that is tailor-made for the needs of the local population. This means that the content of Local Health Care may differ from one county council to another. Nevertheless, some common characteristics can be distinguished. Local Health Care is expected to offer accessible and comprehensive health services with a continuity of care and a strong patient focus. It is supposed to be the foundation of the whole health care system with an orientation towards diseases of frequent occurrence and needs among major population groups, such as families with children, older people, and patients with chronic diseases [1].

In spite of the ambitions to create an integrated delivery system on the local level, it is unusual to find a high degree of organisational cohesiveness in Local Health Care. Instead, it is common that the system is linked together through chains of care or other types of collaborative networks, many of them with a focus on older patients with multiple diseases. There is, however, not so much knowledge about the degree of integration that is actually taking place within these networks, although it seems that such knowledge would be essential to guide the further implementation of Local Health Care.

Following Lasker and colleagues [2], the fundamental conditions of integrated health care can be referred to as functional integration, which includes clinical integration as well as integration of information systems and financial arrangements. This functional integration may lead to integration synergy, which in turn may influence the integration effectiveness. According to Provan and Milward [3], integration effectiveness in the public sector should be evaluated at community, network and organisation or participant levels. While the different levels are related, each has its own set of effectiveness standards that should be considered. Alaszewski and colleagues [4] have also pointed out the importance of adapting the standards to the specific kind of integration to be evaluated.

The purpose of this article is to conceptualise and validate a model of measurement that can be used to evaluate the degree of integration in Local Health Care and similar arrangements of integrated care. Such an evaluation is important for researchers as well as practitioners, but it is particularly important for the managers who are responsible for implementing and sustaining integrated care. This model of measurement will be limited to the functional aspects of clinical integration. It will not measure integration synergy or integration effectiveness, although a well functioning integration is likely to influence the ability to achieve high degrees of synergy and effectiveness [2, 5].

Conceptual framework

Different forms of integration are often described in terms of a continuum. According to Ovretveit [6], such a continuum should include both the subjective feeling of “closeness of working” and objective organisational features. At one end is a loosely knit network based upon voluntary membership. At the other extreme is a closely integrated organisation, and in between team members link with each other in different ways. Leutz [7] has developed a similar continuum of integration based on the degree or intensity of connections between services or organisations: linkage, coordination in networks and full integration. Following Nies [8], these forms of integration can be described and exemplified as follows:

• Linkage takes place between existing organisational units. It aims at an adequate referral of patients to the right unit at the right time and good communication between the professionals involved in order to promote continuity of care. The different units and professionals understand who is responsible for each type of service, and there is no cost shifting between them. Clinical guidelines describing what shall be done by whom and when, are examples of mechanisms used in this form of integration.

• Coordination in networks is a more structured type of integration, but it still operates largely through existing organisational units. The aim is to coordinate different health services, to share clinical information, and to manage the transition of patients between different units. Chains of care and other health care networks are included in this form of integration, but there are usually no network managers appointed. The existence of such managers would entail a higher degree of integration.

• Full integration implies that resources of different organisational units are pooled in order to create a new organisation. The aim is to develop comprehensive services attuned to the needs of specific patient groups. The comprehensive services are managed through the new organisation, which includes close cooperation between different professional groups. The most pressing issues in connection with this form of integration are to define target groups, to assemble the necessary services, and to allocate appropriate resources.

To this continuum of integration may be added a zero point, namely full segregation, which is the absence of any form of integration between services or units. In addition, as indicated above, there may also be an intermediate form of integration between coordination in networks and full integration. This is a form of coordination where network managers are appointed to improve the contacts between the organisational units involved, but these units are still quite independent. This form of integration can be described as cooperation [9]. The enlarged continuum of integration is illustrated in Figure 1.

Figure 1.

Continuum of integration.

This continuum can be used to analyse integration both within organisations (intra-organisational integration) and between organisations (inter-organisational integration). Moreover, the different forms of integration may involve both vertical integration by a common organisational hierarchy, and horizontal integration between more or less independent organisational units on the same hierarchical level [10, 11].

It is important to point out that this continuum of integration does not say anything about the optimum form of integration between different providers of health care. The highest degree of integration, that is full integration, could be worth aiming at for some providers, while others may be content with lower degrees of integration. According to Lawrence and Lorsch [12], the degree of integration should be related to degree of differentiation of services and units, which, in turn, is related to the objectives and the environment of the organisations involved. A high degree of differentiation creates a need for a high degree of integration. Thus, the different forms of integration in health care should be related to the needs, objectives and circumstances of integrated care.

A model of measurement

There are some models for measuring different aspects of integrated care in the literature of integrated care. Most of these models are measuring structural aspects of integration [13–16]. However, integrated care is also to a large extent shaped by and based on professional behaviour and attitudes. These aspects are measured in other models [17, 18]. Both of these approaches are relevant, since there is always an interaction between structural and behavioural aspects of integrated care [19].

Thus, it is important to develop a model of measurement that includes structural as well as behavioural aspects. For practical reasons, it is also important that the model should enable evaluators to calculate sums and mean ranks of integration. In this way, the measurement results will be easier to communicate and therefore more useful for managers of integrated care compared with models based on qualitative data or discontinuous variables. At the same time, however, it is important to avoid the normative implications of a ratio scale that higher ranks of integration are always “better” than lower ranks.

Browne and her associates [18] have developed a useful model measuring human service integration. In this model, different types of integration are measured with a scale developed in a Canadian programme [20]. This model is interesting as it focuses on functional service integration, but it has some practical disadvantages of calculation and communication using an ordinal scale instead of a ratio scale.

In order to construct a practical model of measurement, the enlarged continuum of integration described above has been operationalised into a ratio scale of different categories with an equidistant graduation of rank beginning with full segregation (rank=0) and ending in full integration (rank=100) as illustrated in Figure 2. This instrument was designed to take advantage of the benefits of the ratio scale with an algorithm that makes it possible to derive any rank between the beginning and the ending point of the scale.

Figure 2.

Scale of functional clinical integration.

The derivation of an integration rank between two health care units starts with an identification of the highest existing degree of integration. For example, if chains of care are developed, but no network managers have been appointed, then the integration rank is derived from the chain of care category. The greyish squares in the figure illustrate with growing darkness an increasing relative number of cases in a specific category. When the category has been identified, the process continues with selection of the most appropriate square and, at the same time, also an integration rank.

This procedure of deriving ranks of integration does not consider variations between different patient groups, since it was designed primarily for dealing with Local Health Care and similar arrangements of integrated care. As mentioned before, Local Health Care is orientated towards diseases of frequent occurrence, which means that they are dealing with large but relatively few patient groups. However, it is possible to consider variations in integration between different patient groups and to weigh them into an overall integration rank.

The derivation of integration ranks requires a consensus between different professional groups within the health care units. In order to reach such a consensus, there should be a detailed guideline with a terminology adapted to local settings. When a conclusion is reached, the derived integration rank can be registered in a self-assessment form.

The aggregated data will give the participating health care units not only the self-reported ranks of integration with other units, but also the integration ranks reported by those units. With all of these data, it is possible to calculate integration ranks and the scope of integration (the percentage of units with an integration rank>0) for both inter- and intra-organisational integration, and for both vertical and horizontal integration.

Beside the actual ranks of integration, the health care units should be requested to specify what they think is the optimum rank of integration with each other unit, using the same scale of integration. This is a way to counteract the normative implications of the ratio scale. In this connection, they should also be asked about factors hindering or facilitating a closer correspondence between the actual and the optimal ranks of integration. Finally, they should be asked about how they experience the measurement instrument, its reliability and validity.

An explorative study

The model of measurement was tested in an explorative study at Kungsbacka Local Health Care (KLHC) in the south-west part of Sweden. The catchment area is Kungsbacka municipality with about 67,700 inhabitants. KLHC has its own political board and there is a Local Health Care manager responsible for the organisation. KLHC consists of 25 units providing primary care (180,000 patient contacts and 79,000 outpatient visits per year), rehabilitation (59,000 patient contacts per year), seven county level specialities (29,000 outpatient visits and 3900 bed-days per year) and dental service (7700 treatments of children per year).

The scale of functional clinical integration was adapted and tailored to the integration terminology used in KLHC. After that, the managers of the different units got instructions about the study in a management committee meeting. A self-assessment form was distributed to the unit managers together with a detailed guideline for the data gathering including explanations of terms used in the scale (see Appendix 1).

The managers had six weeks to arrange internal multiprofessional meetings to derive their integration ranks and to answer the validation questions. After that, the data were transferred from the self-assessment forms to an Excel-file prepared for statistical analysis. The results were finally presented and discussed in a management committee meeting of the KLHC.

Evaluation of integration

The results of the study are summarised in Table 1. The mean integration rank declared by the different units of the KLHC was calculated to 15. The highest mean integration ranks were among some of the primary care units (I–M in the table). These relatively high ranks were achieved in spite of the fact that the primary care units have zero in integration rank between each other due to their parallel and identical assignments. Two dental care units (S–T in the table) differed from the other units with relatively low mean integration ranks.

Table 1.

Declared actual integration

Declared actual integration		Health Care units																				Mean
		A	B	C	D	E	F	G	H	I	J	K	L	M	N	O	P	Q	R	S	T
Scope (%)	Vertical	67	67	53	73	73	73	47	60	100	93	87	73	71	53	13	67	67	47	7	20	61
	Horizontal	83	75	67	83	50	67	67	75	50	58	42	42	42	75	75	100	75	83	25	8	62
	Intra organisational	80	75	60	80	80	85	55	70	70	75	55	70	70	70	45	90	65	70	15	20	65
	Inter organisational	57	57	57	57	14	29	57	43	100	86	71	43	33	43	29	57	86	57	0	0	49
	Mean	78	70	59	74	63	70	56	67	78	78	67	59	58	63	41	81	70	70	15	15	62
Rank (0–100)	Vertical	17	16	15	11	16	24	14	13	32	26	21	18	17	11	1	11	17	7	1	1	14
	Horizontal	23	12	20	13	8	14	16	17	17	14	17	10	7	24	30	21	23	6	3	3	15
	Intra organisational	21	15	17	13	17	25	17	17	25	21	17	16	14	21	21	20	21	6	3	3	16
	Inter organisational	16	11	18	9	2	4	7	9	21	18	23	7	6	9	1	7	17	7	0	0	10
	Mean	19	14	17	12	13	20	15	15	24	20	19	14	12	18	16	16	20	6	2	2	12

The standard deviation for the integration ranks was 16, indicating a high variation between the different health care units. The mean vertical and horizontal integration ranks were 14 and 15, respectively, but there was a considerable difference between the mean intra-organisational and inter-organisational integration ranks. The mean intra-organisational integration rank was 16, while the mean inter-organisational integration rank was only 10. This implies that the mean intra-organisational rank was 60% higher than the mean inter-organisational rank. In the same way, the mean scope of integration was 62%, but there was a considerable difference in the mean scope of intra- and inter-organisational integration, 65 and 49%, respectively.

Nineteen of the 20 responding units declared a mean optimum rank of integration higher than the actual rank, which is illustrated in Figure 3. In many cases, there was a desire for a future upgrading of existing clinical guidelines to chains of care. Furthermore, some of the units had recognised the advantages of full integration, for example by pooling the budgets between two units. The main differences between actual and optimum ranks were in vertical and intra-organisational integration with a mean of 10 rank points.

Figure 3.

Actual versus optimum integration.

In their specifications of factors hindering a closer correspondence between the actual and the optimal rank of integration, the units mentioned lack of time and resources, an inadequate organisational structure, and also an inability and unwillingness to recognise the advantages of integration. Facilitating factors were the determination and ability to cooperate around common patient groups, the formation of KLHC as a legitimate form of integration, and the advantages of a relatively small organisation such as flexibility, communication, etc.

Reliability and validity

The reliability of a data collection procedure is usually described as the extent to which it will generate the same results when repeated [21]. The goal is to minimise errors and biases. In this study, the derived integration ranks in the multiprofessional groups were based on consensus. This means that biases, like for example one profession dominating a derived outcome, were diminished. The derivation process was comprehensive and, according to all the units, the results would most likely have been the same if the evaluation had been repeated under identical conditions. Hence, the model of measurement seems to have a good intra-respondent reliability.

Furthermore, a Pearson correlation co-efficient of declared and received integration rank was calculated to 0.821, which indicates a strong positive correlation between the two variables (two-tailed p-value=0.000). Thus, the model of measurement seems to have also a good inter-respondent reliability, which means that two responding units tend to come to the same results concerning the integration of these units.

The reliability of a data collection procedure is a prerequisite for validity but not a sufficient condition for producing valid data. Validity is usually explained as the extent to which the collected data measure what they are supposed to measure [22]. In this study, the scale has been restricted to functional clinical integration and there have been no attempts to measure integration synergy, effectiveness or other aspects of integration. Consequently, the scale of functional clinical integration can be regarded as simple and comprehensible, and in this way as fostering construct validity, which is a central issue in organisational research [23, 24].

Eight out of ten participant units regarded the instructions to derive integration ranks as sufficient for making an accurate classification. The remaining units expressed some difficulties in deriving the integration ranks, although their ranks can be considered as accurate compared with known facts about the degree of functional integration and also in comparison with the integration ranks of other units. Thus, the model of measurement seems to have also a good face validity [22].

When the results of the study were presented to the management committee of the KLHC, the feedback from the committee supported the conclusions concerning the reliability and the validity of the model of measurement.

Discussion

In the initial contacts with representatives of KLHC there was a need for explanation of the term integration and the different forms of integration. In some of the participating units there was an understanding of integration as meaning full integration. Moreover, this form of integration was perceived as a threat, or as Leutz [7] has put it in one of his “laws” of integration: “your integration is my fragmentation”.

It is therefore important to explain that the optimal degree of integration varies between different health care units and depends mainly on the need for integration to deal with common patient groups. The challenge is to meet the demands for both a high differentiation of services and a necessary integration of work [12]. In some cases it is necessary to pool budgets, while in other cases integration can be limited to patient referrals. This shows the importance of using terms like “degree of integration” or “integration rank” in the development of integrated health care, instead of a general term like “integration”.

Twenty percent of the units participating in this study were critical of ambiguities in the instructions and the information given. The latter problem is easy to take care of, for example by making sure that everyone gets first-hand information. It is worth noting that half of the units that were critical had received second-hand information, which turned out to be partly incorrect. Even though the scale and the instructions were adapted to terms used in KLHC, the results demonstrate the importance of putting sufficient efforts into introduction meetings and also to provide the participants with unambiguous written instructions.

The results of the study show that the highest mean integration ranks were among some of the primary care units. This indicates that the role of primary care in Local Health Care is like the spider in the web. Another result was that the inter-organisational integration is considerably weaker than the intra-organisational integration. This may be due to the fact that KLHC is a newly established organisation and as such in the beginning of developing the integration between the different health care units. Considering this, it is not surprising that the results show a potential of increased inter-organisational integration among several units.

The importance of goal setting and attainment of goals in integrated care is addressed by several researchers [25, 26]. The results of this study show that the declared optimum integration ranks of the participating units were in some cases quite different. This indicates that it would be important to reach a consensus about integration targets before starting to improve the integration. In addition, the agreed integration targets must in one way or another be put into a strategic framework and approved by the management of the Local Health Care organisation. In this way, sub-optimisations can be reduced and there can be a focus on overall organisational effectiveness and the extent to which common integration goals are fulfilled.

Past attempts to change working practices by chains of care have proved to be difficult [27]. Therefore, in order to reach integration targets in Local Health Care, change management should have high priority and managers should be aided in their work by evaluations of integration. Moreover, the driving force of change in the measurement of integration shall not be under-estimated. A negative result can instinctively start self-correcting processes. Furthermore, measuring integration can in itself be perceived as a normative signal from the management that reaching integration targets have a high priority.

Finally, there are some important limitations that should be pointed out. First of all, the model of measurement is focusing on the actual integration among health care units and not the perceived integration among health care personnel. These perceptions may also be important to consider in the development of integrated care. Moreover, the model is restricted to measure functional clinical integration, and a high degree outcome must therefore not be interpreted as high quality integration outcome. It is important to avoid normative implications of these results. Another limitation is the adaptation of the model to Local Health Care. The scale needs to be weighted when used in settings with larger numbers of common patient groups.

Concluding remarks

The explorative study carried out in KLHC shows that the presented model of measurement generates reliable and valid data on functional clinical integration, which indicates that it may have a general applicability in Local Health Care or similar arrangements of integrated care. This is in accordance with the need to develop measurable variables on integration, as expressed in a recent European research agenda on integrated care [28].

By repeating such studies several times, managers will get longitudinal data on integration, which they can use in follow-ups and in supporting organisational units in their efforts to achieve target ranks of integration. In this sense, the model of measurement has an advantage as it in-corporates optimum ranks of integration beside actual ranks. All units do not need to be fully integrated. The optimum integration rank between two units is depending on the needs of both differentiation and integration.

The results of this study show that primary care has an important role in Local Health Care, but more work is required to improve inter-organisational integration. Chains of care may be an important step to develop integrated care in Local Health Care organisations, but there are a number of formal and informal obstacles to overcome. The set of values held by health care personnel has proven to be a key factor in the development of integrated care [29, 30]. A concordance between the staff values and the integration objectives give opportunities to develop a well-functioning integration. But this is not sufficient, since integration effectiveness also requires that the integration activities are positively directed towards the patients and achieves results like improved health, good service accessibility etc.

Appendix 1: Instructions for derivation of integration levels in Kungsbacka Local Health Care

We are aiming for high validity and reliability in the data-gathering. Hence, we ask you to summon a group of colleagues who represent different professions in your unit (if not your management group fulfils this condition). Representatives of professions not included in the management group can possibly be called in to participate. This work should be done as comprehensive as possible and not by a single person or profession.

The integration measurement work shall be accomplished at latest 21-01-2005.

The data gathering concerns the level of clinical integration, i.e. the collaboration, coordination or cooperation with other health care units in clinical work for specific patient groups. Thus, this study does not consider integration effects, such as changes in health status among patients, health care costs etc. We also exclude the integration of information systems, financial arrangements and other conditions facilitating the daily communication between the units. A continuous scale is used for measurement of the integration rank, see the figure below. You shall derivate one integration rank with each one of the health care units mentioned on the next page. Register your results in the column “actual” in the registration form.

The following text explains the terms presented in figure 4:

• Patient referrals are referrals from one specialist (remittent) to another (referral receiver). The referral flow often occurs between two levels of care, i.e. between primary care and secondary care.

• A clinical guideline is a clinical action program (what shall be done and by whom) for a specific patient group that is more elaborate than local health care agreements. There are different kinds of clinical guidelines. In this survey we place national, regional and local guidelines in the same category.

• A chain of care is based on a clinical guideline. Beside the clinical agreement on treatment procedures for a specific patient group, work routines and work organisations are coordinated between the health care units concerned.

• A network manager has the authority and resources to do continuous improvement work of a chain of care, but no staff or financial responsibilities for the chain of care.

• Pooled resources means that all earmarked resources for a specific group of patients are put together so that all the health care activities for this patient group are fully integrated.

Figure 4.

The derivation of integration rank starts with identification of the highest existing form of integration. For example, if chains of care are developed together with a specific unit, but no network managers have been appointed, then the integration rank is derived from the chain of care category. The greyish squares in the figure illustrate with growing darkness the increasing relative number of cases in the different categories.

The scale consists of some overlapping parts where two categories are equal, for example a high proportion of local health care agreements and relatively few clinical guidelines. Please observe that the scale is continuous, which means that you can use any number between 0 and 100.

Furthermore, we ask you to register the levels of integration that you consider to be the optimum levels for developing a well functioning Local Health Care. This shall be done for all the participating units. Please use the same integration scale, derive a number between 0 and 100, and register your results in the column “optimum” in the registration form.

The Kungsbacka study consists of the health care units presented in Table 2, for whom we ask you to derive the two kinds of integration ranks.

Table 2.

Pediatrics	PHC – Onsala	Dental district service
Ear, nose and throat	PHC – Särö	Orthodontics
Ophthalmology	PHCe – Åsa	Adult psychiatry
Internal medicine	PHC – Kungsbacka (2)	Child psychiatry
General surgery and ortopedic surgery	Child & mother health service psychologists	Rehabilitation of disabled children
Women's diseases	Psychological consultation	Eyesight consultation
Allergology	Physiotherapy	Audiology
Radiology	Occupational therapy	Home care
PHC – Fjärås	Youth welfare	School health service
PHC – Kungsbacka (1)

The form is concluded with some questions about factors hindering or facilitating a higher degree correspondence between actual and optimum ranks. We are also grateful for answers on the questions about the validity of the measurement.

The registration form is distributed as an ICT-document, which can be used to for the registration of your integration ranks.

Reviewers

Gina Browne, Founder and Director, System-Linked Research Unit Director, CLEAR Unit, Professor, Nursing and Clinical Epidemiology & Biostatistics McMaster University, Faculty of Health Sciences at Frid, Hamilton, Ontario, Canada.

Jon Ovetvreit, Director of Research, The Karolinska Institute Medical Management Centre, Stockholm, and Professor of Health Policy and Management, Bergen University Faculty of Medicine, Norway, Stockholm, Sweden.

Nick Gould, Research fellow Welsh Institute for Health and Social Care (WIHSC), University of Glamorgan, United Kingdom.

Morton Warner, Professor and Director Welsh Institute for Health and Social Care (WIHSC), University of Glamorgan, United Kingdom.