Public strategies for improving eHealth integration and long‐term sustainability in public health care systems: Findings from an Italian case study

Summary

eHealth is expected to contribute in tackling challenges for health care systems. However, it also imposes challenges. Financing strategies adopted at national as well regional levels widely affect eHealth long‐term sustainability. In a public health care system, the public actor is among the main “buyers” eHealth. However, public interventions have been increasingly focused on cost containment. How to match these 2 aspects? This article explores some central issues, mainly related to financial aspects, in the development of effective and valuable eHealth strategies in a public health care system: How can the public health care system (as a “buyer”) improve long‐term success and sustainability of eHealth solutions? What levers are available to match in the long period different interests of different stakeholders in the eHealth field? A case study was performed in the Region of Tuscany, Italy. According to our results, win‐win strategies should be followed. Investments should take into account the need to long‐term finance solutions, for sustaining changes in health care organizations for obtaining benefits. To solve the interoperability issues, the concept of the “platform approach” emerged, based on collaboration within and between organizations. Private sector as well as beneficiaries and final users of the eHealth solutions should participate in their design, provision, and monitoring. For creating value for all, the evidence gap and the financial needs could be addressed with a pull mechanism of funding, aimed at paying according to the outcomes produced by the eHealth solution, on the base of an ongoing monitoring, measurement, and evaluation of the outcomes.

1. INTRODUCTION

Public and universal health care systems aim to ensure quality of care and equitable access to maintain the financial sustainability of the system. To accomplish these goals, public proposals increasingly focus on containing costs and avoiding “wasteful spending” in health care, privileging “value for money” interventions that can demonstrate improvements in quality for each euro spent.1 For these reasons, coverage, procurement, reimbursement policies, and other public interventions aimed at regulating and funding innovative solutions are recognized as interventions that can play a crucial role in achieving this strategy in health care and in addressing some current challenges.

In the wide field of eHealth (EH), interventions based on information and communication technology (ICT) can be considered innovative if they enable change. The disruptive innovativeness of EH is mainly related to its ability to support organizational or process changes, produce new models of care, reorganize health care services, and improve patient engagement.2, 3, 4 In fact, in sociotechnical approaches to ICT in health care, the introduction of a technological innovation is considered a social process that involves a large network of internal and external stakeholders and can deeply affect the health care organization.5, 6, 7, 8

However, various investments in EH have been considered “wasted” resources because they were unable to add value to patient outcomes.9, 10, 11 In fact, there is mixed evidence on the cost effectiveness of EH interventions and widespread skepticism based on the known challenges that EH initiatives impose on the health care sector.12 Some of these challenges are related to the (i) low diffusion rate; (ii) low replicability; (iii) difficulty in sustaining them in the long term; (iv) barriers related to financing on both demand and supply sides; and (v) high dependence on the local context and institutions.13, 14 A few EH projects have overcome the “pilot phase” and become established as routine practice, changing how health care services are delivered.11, 15, 16, 17, 18 In fact, EH works given the right approach, technology use, context, and implementation process.12 To achieve their full potential and to create (not waste) value, the integration of EH solutions in routine services is a necessary condition.2, 19

Several factors influence the integration, pervasiveness, and success of EH solutions. In resource‐constrained environments, financing models widely affect the sustainability of these solutions.3, 14 Indeed, in general, all EH solutions require long‐term financing needs, which are more relevant in the initial phases, and development and planning, and then slowly decrease over the long term.2 In contrast, EH financing models generally follow a short‐term approach without the need for a long‐term commitment.19 Moreover, EH investments are often related only to the use or procurement of ICT and underestimate or ignore the cost and organizational efforts needed to manage the cultural and managerial changes resulting from the introduction of EH.3

In several European countries, the financial dynamics and models are described as barriers to EH at various levels: a lack of resources for the introduction of EH projects and nonrecurring expenditure for their continuation; excessively long processes for procurement and prefinancing procedures; and problems related to licensing and accreditation, reimbursement, and intellectual property rights.20, 21, 22

The public actor may be among the most important “buyers” of innovative solutions, particularly in public health care systems. The “public payer” relies on public acquisition processes, such as public procurement. These functions can create opportunities for demanding innovative products and services from the private sector, such as EH,23, 24 and for serving socially and economically valued goals, such as health and innovation.23, 25

Given these premises, the focus of this research work is on the levers that may affect the process of integration of an EH solution in the routine delivery of health care services at the strategy, policy, and health care provider levels. This work focuses, in particular, on the financial aspects of EH, and the context of reference is the public decentralized health care system.

2. FRAMEWORK

Several articles and reports in the literature and gray literature have addressed the funding and financial aspects of EH, considering several levels of interventions.

At a policy level, the issue of investment and funding of EH has been widely tackled, including national and supranational policies that have encouraged public‐private partnership (PPP), as well as innovative forms of public procurement.26

The private sector is considered an essential partner in realizing the potential of EH.27 However, the results of PPP in Europe are mixed.28 Another important trend is the innovative use or the use of innovative forms of public procurement. In fact, there are various types of innovation‐friendly public procurement, mainly characterized by a common approach based on the identification and involvement of a wide variety of stakeholders with the aim to better understand their needs and define solutions. In this changing context, public agencies should be considered not only important buyers of innovations but also part of a crucial demand system for innovation, which requires the holistic involvement of public procurers, managers and political leaders, and private firms and organizations on the supply side.4

These new aspects are relevant at both the policy and the health care provider levels. In fact, according to the literature,26, 29 various factors related to the process of acquisition at the jurisdictional and provider levels, which are still cited as characteristics of innovative forms of acquisition and partnerships, are considered to enhance the capabilities of health care organizations for the successful implementation of EH. These aspects can be summarized as follows: engagement of stakeholders; processes of change management; and procurement typologies and rules, including the use of evidence in the selection of EH projects to be financed, and the criteria for ongoing monitoring and evaluation of EH interventions.

Several reports describe case studies or concrete examples of innovative forms of procurement at local, regional, and national levels.30, 31, 32, 33, 34, 35 However, to the best of our knowledge, there is still little evidence on this topic.36, 37, 38

The literature review revealed a difference in the uptake of the different types of demand‐side innovation policies or actions between national and regional levels. At a regional level (in this case, a region means part of a country), the emphasis of demand‐side innovation policy is on promoting the application of already existing technological innovations in the public and private sectors.39 At this more “applicative” level, 2 elements of demand‐side innovation policies appear to be important: the opportunity to better articulate needs and closer interaction with stakeholders, particularly end‐users or beneficiaries. Indeed, the main characteristics of systemic policies at a regional level are learning by interaction and the ability to involve end‐users.

Figure 1 synthesizes the framework built on the literature analysis.

Figure 1.

Schematic of the framework drawn from the literature analysis

3. AIMS

This article explores some issues that are central to the successful implementation and integration of EH solutions. In particular, this work is based on a case study designed to explore key stakeholders' expectations and opinions of EH and to analyze the gap between what is possible and what is actually done. The main research questions this work aims to answer are the following: Which strategies and financial mechanisms can a public health care system adopt for the successful long‐term implementation of EH interventions? How can innovation‐oriented public policies facilitate the financial sustainability and integration of innovations in health care? What financial levers are available to match the interests of different stakeholders to ensure long‐term benefits from EH, while also considering the various levels of intervention in a decentralized public health care system?

4. METHODOLOGY

This paper presents the results of a qualitative case study carried out in the region of Tuscany (Italy) between September 2014 and March 2015. The adoption of an inductive approach, in particular the analysis of face‐to‐face interviews with key informants, was considered appropriate to explore the influencing factors, capture and interpret informants' perceptions,40, 41 and allow an in‐depth and fine‐grained analysis of a new phenomenon in a complex and evolving system.42

According to the theoretical framework, a questionnaire was designed for the study and piloted before use. The backbone of questions that guided the interviews is reported in Appendix A.

Semistructured in‐depth interviews were conducted, lasting between 60 and 90 minutes. We identified a multilevel and multistakeholder sample by using a purposive sampling method43 with a snowballing approach. The sample included informants from key stakeholder groups, as detailed in Appendix B. We referred to published studies to compile the list of key stakeholder groups, which included 5 categories: physicians and other caregivers, patients, health care organizations, innovator companies, and regulatory agencies.2, 44 The first category encompassed both general practitioners and specialist professionals, including academic clinicians. For the second category, we involved patients who had participated or refused to participate in an EH project. For the third category, we included both medical and nonmedical managers of health care organizations. Among the innovator companies, 3 firms that developed EH solutions and participated in EH projects were involved: 2 small and medium enterprises (SMEs) and a larger firm. Finally, the sample included several informants from the regional public institutions with various different responsibilities for regional policies and strategies, including regional laws; for procurement in the health care sector; and for the implementation of ICT infrastructures. Moreover, to improve the representation of the whole region, we selected informants for each group from all of the local health authorities (LHAs) in Tuscany, hospitals and teaching hospitals with previous experience of EH, and regional public institutions of Tuscany. Several informants among the clinicians and chief executive officers also had EH experience at a national and European level. In relation to the dimension of the sample, we did not define an “a priori” target number of informants. According to the sampling method in the grounded theory, no further informants were interviewed when no additional information emerged and the “saturation point” was reached.45 We contacted 45 people, of whom 34 agreed to be interviewed (77%) and 33 were effectively interviewed (73% of the initial sample; Table 1). All informants agreed to be interviewed. The interviews were conducted in the local language and tape‐recorded. Verbatim transcripts were translated into English. Observational field notes were kept. The interview data were analyzed by using ATLAS.Ti 7 software to obtain quantitative information on the occurrence of words in the interview transcripts and to obtain an in‐depth description of informants' opinions by analyzing the determinant factors in the transcripts together with fieldworks notes. To enhance the robustness of the analysis, both of the authors analyzed and discussed the results.

Table 1.

Informants by stakeholder category and occupational background. Appendix B presents the characteristics of each informant in more detail

Stakeholder Category	Occupational Background	Number of Respondents
Physicians and other caregivers	Clinicians and clinical academics	6
	General practitioners	2
Health care organization	Medical manager	3
	Nonmedical manager	6
Regulatory agencies	Chief executive officer	2
	Technicians	7
Patients	Patients and deputies of patients' associations	4
Innovator companies	Deputies of innovative firms in the field of EH	3
Total		33

4.1. Country context of the case study

Italy has a public and universal health care system. It is financed by general taxation and is largely free of charge at the point of delivery. The health care system is regionally managed. In all Italian regions, LHAs manage public health services and are composed of health care districts (HDs).

In Italy, there was a substantial growth in the development of EH projects.46, 47 Most of these projects were pilot studies or experiments with a “life expectancy” of 3 years; the others met with rapid failure.47 Information and communication technology spending in the Italian health care sector was 1340 million Euros in 2015,48 equivalent to 1.2% of the Italian GDP (less than the European average, which was between 2% and 3% of national GDP).49 eHealth solutions were also large beneficiaries of European funds.50, 51

A new Italian law on public acquisitions was recently approved (D.Lgs. n. 50/2016), integrating the previous Legislative Decree 163/2006 (“the Public Procurement Code”) with elements of the EU Directives on public procurement (2014/24/UE). However, the informants in the present study referred to the previous code that, according to the threshold in monetary value, allowed for direct purchasing or negotiated and simplified, restricted procedures.52 The tendering procedure, in most cases, remains the responsibility of the individual procuring authorities at regional and local levels. Italy has launched a national program aimed at encouraging regional governments to use EU Structural Funds for Pre‐Commercial Procurement (PCP) for research and innovation. It is based on a co‐financing mechanism in which research and product development is paid for by the regional ministry and the procurement process is financed by the contracting authority. It is not yet possible to draw conclusions on the success of this new approach.39, 53 A regional PCP initiative was developed in the region of Lombardy, based on the Italian framework for PCP. Its application areas included ICT‐based R&D and innovation projects in the health care sector.24 Although regional innovative public procurement measures can be found in Italy, no demand‐side policies are present at a regional level.39, 54

The context of the present study is the region of Tuscany, which has a population of more than 3.7 million and a global health care budget of 6700 million Euros in 2015. In this region, the Regional Entity for Techno‐Administrative Support (ESTAR) is in charge of regional public procurement, coordinating, and supporting specific activities, such as ICT in health care. It was constituted in 2014 by aggregating 3 entities (Entity for Techno‐Administrative Support of “Area Vasta”) and was operative from 2015. The previous entities were also responsible for supporting LHAs in the acquisition and procurement of services and products. The region is organized into 3 LHAs, which were constituted in 2015 by grouping the previous 12 LHAs. Tuscany shows characteristics of EH diffusion and implementation and R&D expenditure in line with the national context.47, 54, 55

5. RESULTS

The words used by informants were grouped into concepts on the basis of their meaning. The main concepts are listed in Table 2. Figure 2 represents the main concepts expressed by informants in decreasing order of frequency, while Table 2 reports some descriptive statistics in alphabetic order.

Table 2.

Results from interviews: Frequency of occurrence of concepts

Concepts	Frequency	Mean	Confidence Interval (Error 0.05)
Academia	18	0.6	±0.51723
Benefit	19	0.6	±0.55765
European	85	2.7	±1.58198
Environment	14	0.4	±0.31629
Evidence	38	1.2	±0.97149
Industry	21	0.7	±0.45175
Institutional	8	0.3	±0.31899
Integration	21	0.7	±0.61980
International	10	0.3	±0.23548
Interoperability	1	0	±0.07561
Level	164	5.1	±0.87293
LHA	49	1.5	±1.12449
Local	9	0.3	±0.32077
Long(‐term)	35	1.1	±0.70441
National	38	1.2	±0.65390
Partnership	22	0.7	±2.00423
Private	108	3.4	±2.00324
Procurement	32	1	±0,87292
Public	74	2.3	±1.44368
Regional	180	5.6	±2.21466
Reimbursement	2	0.1	±0.10422
Risk	95	3	±1.86460
Sharing	60	1.9	±1.32970
Short(‐term)	6	0.2	±0.19424
SME	40	1.3	±1.34147
Strategy	35	1.1	±0.67836
Value	50	1.6	±0.81607
Vision	49	1.5	±0.98202

Figure 2.

Occurrence of concepts in interviews in order of decreasing frequency

In general, when asked about determinant factors, the informants mainly reported impeding factors. The interview findings were mainly concerned with the health care service provider (the local level) and the external environment, both in the national health care system and the national as well as supranational economic‐political contexts.

5.1. The actors and the levels

The first theme to emerge was the informants' attention to the level of intervention: “level” and “regional” represent the most repeated concepts. In particular, the word “region*” was used more by chief executives and representatives from firms than the other informants. The interviewed people from firms pronounced more words related to “Europe*” (Figure 3).

Figure 3.

Connections among concepts related to the concept of “level”

The presence of innovation‐oriented interventions at a regional level was reported as a potential positive factor because the context was perceived and described as lacking in both national and regional programs of policies for innovation funding and support. The presence of funding opportunities from European programs was seen as a stimulus for knowledge creation and the early introduction of innovation in health care (ie, in planning, development, piloting, and experimenting). However, informants did not consider them as facilitators of the integration, routinization, and potential scaling‐up of EH solutions because their focus is on R&D initiatives. Moreover, they described the main mechanism of European actions as mainly pushing because they reward projects independently from their long‐term results. This implies a high risk for European entities, co‐financers, and health care organizations involved in the projects and a greater focus on the medium‐term financial aspects. Public policies at regional and national levels were described as a great opportunity, particularly for providing a strategic direction for EH investments, for facilitating the translation of new knowledge into concrete and feasible projects, and for financially sustaining innovations after the experimental period. The regional level appeared to be more important for coordinating EH efforts and contributing to the reduction of EH fragmentation caused by patchy local interventions.

However, from the interviews, it emerged that decisions are more often taken at an individual or local level than at a regional level. The informants reported a lack of and need for both a strategic direction for regional investment and policies and coordination of local or individual efforts.

“We need central coordination among national, regional and local projects” (chief executive officer).

The informants also reported the lack of effective national incentive programs in Italy as a negative factor. They were referring to incentives as financial‐economic stimuli for health care service providers: for organizations to increase the initial adoption of EH solutions and for health workers in general to incentivize their long‐term use in the routine delivery of care. The informants also referred to other types of incentives to be introduced by the national or regional governments to make this market more attractive and accessible for SMEs and innovative firms.

“We have few resources to invest in e‐health” (nonmedical manager).

5.2. Risks and public procurement

The words “risk,” “private,” and “public” were mentioned several times during the interviews (Figure 4). usually by nonmedical managers from ESTAR and in relation to PPP. The latter was described as a risky form of collaboration for both public health care organizations and private firms, in particular smarter entrepreneurial realities. The concomitant presence of an unclear regulatory framework for PPP and restrictive norms on linked issues (eg, “antimafia” laws) was described as a barrier against positive direct collaboration between the public and private sectors. Differences in goals, time horizons, and interests were also seen as problematic issues. Informants from public entities saw private profit as a competing mandate in respect to the public health and societal goals of a public health care organization. From the private sector's perspective, this negative perception of profit was reported as a negative influence on their willingness to partner with public entities.

Figure 4.

Connections among concepts related to the concept of “risk”

The words “private,” “public,” and “procurement” were also linked to the word “risk” (Figure 4). Informants considered the standard public procurement process too bureaucratic and rigid. Models were described as limited in scope, more adapted for ready‐made solutions from big suppliers, not open to stakeholders' engagement, and not facilitating small innovative firms or innovations that can really enable change. A condition for the successful uptake of EH projects was indicated in the firms' and entrepreneurs' capacity to enter this market. Informants considered the bureaucracy of the standard public procurement process inappropriate for introducing or supporting EH innovations, particularly those from SMEs.

“Our procurement is good for mature systems, not for emergent fields” (chief executive officer).

“We pretend to act like Anglo‐Saxons with Medieval rules” (technician).

Representatives of innovative firms widely talked about the risky conditions for them and the rules of public procurement, which do not facilitate innovation in the public health care sector. In the words of a firm's representative, “the lack of administrative and bureaucratic tools that allow a pilot project to be easily copied and adapted to another context” is an important impeding factor.

Again, the lack of an innovative, open, and clear regulatory framework for new models of procurement and partnership in Italy was reported as a negative factor in relation to the models for reimbursement of EH interventions, contracts, privacy, and refunds. Some informants regarded PCP with interest, as a facilitator for SMEs and more innovative firms in providing EH solutions for public health care services. Two interesting aspects emerged: this kind of process seems to facilitate the co‐production of solutions between the institution‐in‐need and the firm, and it includes a payment or reward for each phase of the process in which firms participate.

5.3. Strategy and vision

Informants recognized, in the lack of long‐term strategy and vision at regional and national levels, important impending factors for the routinization of EH solutions. The issue was strictly related to the absence of a national, or at least regional, office responsible for EH coordination and guidance. The responsibility for this specific field of innovation in health care was considered necessary not only for coordinating interventions but also for responding to the need for transparency and accountability.

The words “strategy” and “vision” were repeated together 84 times. The concept of “strategy” was mostly present in the interviews with firms' representatives, whereas the word “vision” was mainly used by clinicians from LHAs and technicians from ESTAR. These words were primarily used in association with the concepts “long‐term,” “integration,” “environment,” and “interoperability” (Figure 5).

Figure 5.

Connections among concepts related to the concept of “vision”

The lack of long‐term vision was also considered “normal” at the health care service provider level. Informants described the financial approach of health care organizations as too focused on the initial investment in EH solutions. Short‐term investments were defined as “useless,” while long‐term management of and investment in the solution (which consider the whole life cycle of the solution) were identified as facilitators. A patient described the financial approach of health care organizations as too focused on the initial investment in EH solutions. In the opinion of informants, a strategic approach to EH should be translated into interoperable and interconnected investments, which would enable data sharing among health care workers and organizations and replicability and deployment of EH experience at a system level.

5.4. Regulatory framework and responsibility

The lack of a specific and clear regulatory framework for EH in Italy was reported as a negative factor. In contrast, new models of financing, refunding, and reimbursement were indicated as potential positive factors for the long‐term sustainability of EH projects and for producing long‐term benefits for health care organizations and EH final users. In the opinion of informants, this gap may impede the development of EH solutions outside the restricted arena of fixed term projects financed by European financing sources, in particular DGRP. These kinds of European programs were described as facilitating easier access to initial financial resources and for creating large networks among health care organizations and between private and public organizations. The lack of regulation affects different fields: reimbursement and refund models, the openness of health workers' contracts, privacy issues, and interoperability standards.

5.5. The “platform approach”: Interoperability and engagement

Among the selected concepts of interest, those that were mentioned the least were “interoperability,” “reimbursement,” “short‐term,” “institutional,” “local,” and “international” (Table 2).

However, while “interoperability” is a strange and difficult word to pronounce in Italian, the concept behind this word appeared to be very clear and important during the interviews. The need for interoperable solutions was widely expressed in the capacity of a health care system (or at least organization) to offer an integrated set of digital and technological solutions. In fact, informants also referred to organizational myopia regarding the design of solutions; they mentioned, for example, projects' target populations that were too small and low “integrability” with existing practices, values, and other solutions. Among the reasons for this short‐sightedness was the lack of a platform approach with a more general and shared vision of the wide applicability of EH solutions. Indeed, the lack of integration of EH into a strategic approach was described in infrastructural and interoperability problems.

The issue of integration referred to both the characteristics of solutions, which should be interoperable, and to the characteristics of tasks or practices and their fit with the EH solution. The fit applies to both the fit of the technology to the task and of the task to the technology. In the first case, it emerged from the interviews that the acquisition of already developed EH solutions from the market or unsolicited bids is a very negative factor for EH routinization. Nevertheless, EH innovations are often pushed or driven by firms in the public health care sector, mainly at the provider level. The involvement of internal stakeholders in the early stages of the innovation process was seen as an important, but lacking, facilitator. In the second case, the practical integration of tasks into the organizational models and processes of the provider was considered to be affected by the organization's openness to opportunities for change. In fact, EH solutions were not considered innovative in technologies, which are generally mature, but in the organizational and cultural changes resulting from their integration.

“Tele‐health is not innovative anymore. Its integration in the organizational models and processes is the real innovation” (medical manager).

Several informants referred to a transversal open ecosystem as a potential facilitator of innovations in health care. It was described as an environment in which public entities at regional and local levels, clinicians, academia, firms, and patients can meet, bring up problems and needs, share existing good practices, and co‐produce solutions. Informants saw the actual processes of decision making and innovation as negative factors because they were not based on multilevel and multistakeholder participation. As anticipated, the decision‐making process was described as a local process, widely emphasizing the crucial role of the single individual (the champion). However, often, the consequent EH initiatives were not even internally “visible” within the same local health care organization. In the opinion of the informants, this factor may impede communication and the sharing of good practices and project results, the deployment of solutions, and the capture of new financing sources for the continuation of the project.

5.6. Long‐term investments

About economic, human, and time resources, the informants reported several determinant factors at provider level, such as (i) health care organizations' budget constraints; (ii) the difficulties of justifying investments with an uncertain cost‐utility; (iii) the difficulties of finding new financial sources after the introduction; and (iv) an overemphasis on the cost of technologies rather than on change management and long‐term project management and assistance.

“Investments are useful only in the presence of infrastructures, time and people” (medical manager).

The informants recognized that the risk adversity of health care management at both national and local levels represents an important barrier. In contrast, EH solutions can be introduced only with a long‐term financial plan and initial investment. In this regard, considering the current economic crisis and budget constraints, investments did not appear to be sustainable without disinvestments in other areas of the health care organization or of the health care system itself. In the informants' opinion, a smart reallocation of resources could facilitate EH implementation and deployment. This implies ongoing monitoring of performance in relation to all crucial areas of activity in the health care organization, including innovative interventions such as EH. Systems for performance measurement and evaluation may represent a powerful tool for transparent monitoring of innovative interventions.

Informants from firms mentioned various barriers in relation to the time horizon among (i) the innovation process and the urgency of responding to the needs of the health care service provider; (ii) the annual evaluation of health care managers and long‐term investments that EH solutions need; (iii) the first investments for developing the EH solution and the resolution of payments by PAs, which seems particularly risky for SMEs; (iv) the life cycle of an EH solution and the time horizon of public investments, which often focus on covering the initial costs and not on the recurring costs of the project's continuation; and (v) very time‐consuming public procurement processes. A firm's representative indicated that this type of scenario was the reason for the “vine‐effect”—the transition from a short‐term financing source (mainly granted with a short‐term approach) to another one short‐term financing source, with the aim to sustain the EH solution.

5.7. Generalizable evidence and local‐specific evidence

Regarding the difficulty of justifying investments in EH, the informants described access to evidence as problematic. The lack of a unique national, or at least regional, health technology assessment process for nonpharmaceutical innovations was reported as a potential barrier. The standard procedures of an HTA were also described as potential barriers because the informants considered the contribution of an evaluation that produced generalizable evidence without external validity for a specific implementation context (ie, feasibility and adaptability) to be insufficient.

“Not only Cochrane, but also evaluations along the whole life‐cycle in the context of real implementation” (medical manager).

Some informants called it “local evidence” and considered it a facilitating factor together with a “second learning” process of understanding and using evidence. The word “evidence” was mentioned 38 times by the informants, in particular by medical managers, hospital clinicians, and firms. Most of those interviewed believed that more evidence is needed to allow evidence‐based decision‐making at economic and financial levels. Furthermore, transparent use of this evidence could allow participation and control by all stakeholders within the regional health care system or organization.

6. DISCUSSION

Some novel aspects emerged from the case study in respect to the framework provided by the literature (Figure 6).

Figure 6.

The framework from the literature (black text and lines), integrated with the results from the case study (gray italic text and dashed gray lines)

Public procurement was generally considered a valuable instrument for innovation‐oriented policies,23, 25 but only without a “paternalistic approach” or a governmental “stewardship,” which represent push mechanisms for the informants. On the contrary, they considered the use of pull mechanisms crucial for facilitating the long‐term sustainability of EH projects. The informants' expectation of pull mechanisms explains their interest in PCP processes and PPP. The first provides rewards for participating firms at each stage of the process. Public‐private partnership was considered helpful in sharing the financial burden and risk of EH investments, dealing with long‐term recurring costs, allowing ongoing monitoring evaluations, and improving direct collaboration and cross‐fertilization between private and public sectors.3, 56, 57

The second key finding resulting from this work is the importance of engagement, which is in line with the previous literature. It was linked to the “ecosystem” concept. The participative procedures of PCP processes and the direct collaboration resulting from PPP are considered crucial determining factors. eHealth financing decisions were also considered to be dependent on the health care value chain and the whole system of stakeholders.2, 58 Multilevel and systematic participatory processes including health care workers at different levels, academia, end‐users, and entrepreneurs contribute to the success of innovative paths from introduction to deployment.39, 57 A wider collaboration may provide more possibilities to learn and gain value56 by considering the real (societal and organizational) needs and interests of all potential internal and external stakeholders. This kind of engagement includes participation in the measurement, evaluation, and discussion of EH results to improve the accountability of interventions, stimulate feedback from health care managers and workers (who are the basic operators of change), and link the financial burden of EH to achieved goals.59 The emerged concept of a transversal ecosystem deals with “real” engagement and collaboration, rather than merely consultation. This latter was considered a more feasible form of participation at a national level. In contrast, a wider “real” engagement imposes multilevel participation and collaboration within and among organizations and systems, according to the sociotechnical approaches to ICT in health care.5, 6, 7, 8 Moreover, the sharing attitude that characterizes this “ecosystem” also involves interaction among health care organizations. The successful implementation and use of EH solutions imply effort in coordination, integrated work, and a platform approach among organizations to reduce fragmentation at several levels.3, 58, 60, 61, 62

The third central element concerns knowledge, evaluation, and evidence. The application of knowledge and research in practice is preferable in a regional or local context, rather than R&D‐oriented procurement, according to the results of this work and to the literature.39, 63 Therefore, a focus on practical aspects (ie, actionable research, management aspects, and scalability of projects) is required when gathering evidence and evaluating which projects to finance. In addition, greater attention to financial aspects is appropriate in regional and local contexts: in the business case or model evaluation of the project; in the “evaluative implementation,” which encompasses the ongoing monitoring of implemented EH solutions; and in the final assessment of a solution's impact, with which refund mechanisms can be associated. These procedures are likely to produce new knowledge (“local evidence”), which may facilitate the adaptive evolution of the EH solutions to the context and the organizational changes. The literature already emphasizes the need for “evidence‐based health informatics,”64 in some cases asking for empirical demonstration of the benefits and best use of technological applications.65 Moreover, this kind of evaluation allows the identification of areas that can be disinvested and areas that can be invested in to produce more value, consequently informing decisions on resource reallocation. The need for “local evidence” implies the ongoing monitoring of EH projects to produce and measure intermediate results that may allow the adaptation of the EH solution. Moreover, continuous monitoring and evaluation of EH projects help the health care provider to “control” the results of EH and may grant the result‐related payment of private firms. This approach facilitates financial risk sharing, for example, in PCP processes, by connecting the financial conditions with the achievement of clinical and organizational goals. Furthermore, public disclosure of the evaluation results may also allow for public social “control” of the health care organization's or system's performance in the EH field. According to the literature, the public disclosure of data can indirectly affect the achievement of goals by local and regional health care systems, acting, for instance, through reputational pressure. The integration of an evaluation system with governance mechanisms (ie, by linking health care managers' goals and incentives to their performance evaluation) can be another valid incentive to achieve results.59 These mechanisms may contribute to the wider sharing of goals and responsibilities to improve (and not waste) the value created by EH.

According to the literature analysis, systemic policies and policies with an indirect impact on innovation demand are considered to be more appropriate at a national level; for instance, (i) reimbursement for health care services provided through EH3; (ii) R&D policies that also use PCP processes; (iii) efforts toward standardization; (iv) more innovation‐friendly PPP regulations; (v) nonfinancial support for R&D and dissemination of knowledge (ie, awareness and consultancy); (vi) education and training (of managers and procurers as well as the workforce); and (vii) national HTA on “soft” technology or organizational innovations.

The reimbursement of health care services provided by EH is uncommon. In Italy, there are some initial regional examples of EH‐based health care services being reimbursed as the corresponding “usual” service. However, the informants expected this type of policy to be implemented at a national level, due to its wide power to open the public health care sector to organizational innovations. Similarly, interoperability appears to be a central topic at local, regional, and national levels. Although EH is mostly based on mature technology, there is a lack of standards that are essential for a “platform approach.” Standard setting is considered to be the responsibility of industrial and nonprofit organizations, while central government should maintain and facilitate the diffusion of standards and support public demand for interoperable solutions and private compliance to standards.66

The results of this research show that several dimensions should be taken into account in the definition of innovative policies and action on EH. The importance of sociotechnical approaches in the implementation of ICT in health care is widely recognized in the literature.5, 67 Win‐win processes may solve several issues. Participative processes can allow a customization of solutions (or policies) to the specific health care organization (or context) and can give space and consideration to the needs, interests, goals, and experiences of all key stakeholders. The difference in interests (ie, societal needs or public service versus profit) may be solved by sharing the goals, responsibilities, and financial risks of the EH investment. Goal and financial risk‐sharing may enable the longer term sustainability of the EH project. It is reasonable to hypothesize that promising pull mechanisms, such as the project financing or health impact fund models, may hold useful lessons for EH. Some examples are (i) initial payment only for the technological cost of solutions (or initial costs of development and adoption) but recurring finance only for successful innovations according to predefined conditions; (ii) co‐financing with funds from the national government, regions, LHAs, hospitals, patients' associations, telecom firms, and nonprofit organizations; and (iii) defining selection and evaluation criteria through participative processes. In a public health care sector, an innovative reimbursement policy or centralized PCP process ending with some sort of accreditation or certification for innovative firms can be considered a goal and financial risk‐sharing model.

7. CONCLUSIONS

A challenging multilevel policy mix emerged as a solution to the initial problem. There is no better financing model that can be adopted to ensure the long‐term sustainability and integration of EH innovations in health care. There is no better policy that can be used for creating value through EH. On the contrary, various aspects need to be considered in the definition of strategy. Public levers to implement valuable EH in the long‐term appear unrelated to financial aspects. To support EH sustainability in the long term and facilitate its integration, routinization, and scaling up, public strategies should integrate sociotechnical aspects in their financing models for EH by including processes of engagement and interaction with different stakeholders. These processes also embrace monitoring, measurement, and evaluation, based on publicly disclosed data.

The main level available, which may allow the integration of the aspects resulting from this research work, is the consideration of goal and financial risk‐sharing models of EH from design to evaluation, based on an ongoing publicly disclosed process of monitoring, measurement, and assessment of the EH interventions. This model is grounded on an approach that privileges the vertical and horizontal integration of EH, in interoperability (“platform approach”), engagement (“transversal ecosystem”), and evidence (generalizable and context‐specific or “local,” linked to reward and control systems). A win‐win approach needs agreement between public and private sectors, as well as between public organizations/systems and the whole population: the public disclosure of ongoing monitoring and evaluation results represents a precondition for implementing public policies and strategies in innovative fields, assuring the quality of care and supporting the financial sustainability of the system. Long‐term strategies in this field need accountable mechanisms that are able to lever the reputation of health care organizations and to improve cooperation and trust.

CONFLICT OF INTEREST

The authors declare that they have no competing interests.

ETHICS STATEMENT

Ethical approval was not required for this case study. However, ethical issues were considered in the overall design and conduct of the research. Both authors declare that the study followed the research integrity rules in the country in which it was performed.

APPENDIX A.

Themes and questions used as base for the interviews

Introductory Themes

What is the definition of EH?

What is your opinion about technologies and innovations as solution to challenges for public health care system?

What kind of technologies may contribute in improving quality and reducing costs?

Should the public health care system or the health care service provider invest in EH? Why?

How should you describe the diffusion of EH in the national and regional system? And the integration?

What are the main determinant factors of the actual situation of EH routinization? (barriers and levers)

Vision, Strategy, and Policies in Support of EH

What are the priorities for the EH investments?

What are the goals related to EH investments?

Is there a vision on the EH?

(if yes) What is the national vision? And the local vision?

Is there a strategy on EH?

Are there any programs, policies, or actions on EH at any levels?

(if yes) What interventions?

Is the vision common or shared at each level of the health care system? And of the organization?

Is there a (national, regional, and local) coordinator for EH?

Does the environment matter?

Impacts of EH, Evidence, and Evaluation

What kind of impacts is more likely to be produced by EH?

What are the main organizational and cultural changes that EH brings? (if yes) Do they have a cost/value?

Can EH impact on health outcomes? (if yes) What kind of impact?

Do we need more evidence on EH?

Could standard indicators for evaluation be useful? How? To whom?

Financial Issues

How are the determinant factors of EH sustainability?

How should you describe the actual models of EH financing? And the policies of EH support/incentive?

What are actually the reimbursement models for EH‐based services?

Are there any positive models in EH financing? And reimbursement?

What is the role of the public actor at national level? And at provider level?

What tools do you think can be used by the public actor for supporting EH in the long period? (at each level)

Is the public procurement a lever or a barrier to EH innovation introduction? And deployment?

Stakeholder Engagement

What is usually the role of stakeholders? (final users, industry, and public sector)

Ideal Process

What are the better processes for EH deployment? (bottom up or top down)

What is the ideal process of EH development, introduction, and integration in health care?

Specific Projects

Are there any EH projects (ongoing or closed) in your organization?

(if not) Do you remember a notable experience of EH in another organization?

What kind of projects? (description, pilot project, and integrated projects)

What source of financing did they have? And what model of financing?

What kind of business model did/do they present?

Were/are the projects object of evaluation? (if yes) What kind of evaluation?

(for pilot‐ projects) If the project remained at pilot stage, what are the reasons? What could enable the routinization and integration of the project in the routine?

What kind of organizational changes did the EH innovation bring in your experience?

In what health outcomes did it result? (quality of life and health condition improvement)

What kind of project should you develop?

Studies on Topics Related to eHealth Financial or Sustainability Issues

Author	Facilitating Factors
(Khoja et al, 2012)	Investment policies aimed at encouraging partnerships between public and private institutions or within the same sector. Jurisdictional policies for identifying and including stakeholders from a different user and supporting groups in the planning of eHealth programs. Policy issues at the institutional level for defining the processes for change management, ensuring training and support to all users defining the rules for procurement of equipment, and evaluating new technologies in local environments before implementation. Policy goal setting at jurisdictional level for covering the costs of equipment and time needed for health care providers to bring eHealth services into broad acceptance and developing governance and management structures. Institutional policy issues for ensuring universal standards of care. Evaluation and research policies for guiding the process of evaluation and research to generate evidence for the adoption of eHealth. A combination of policies at different levels (countries and organizations) was recommended when developing eHealth policies.
(Torbica e Cappellaro, 2010)	Without comprehensive and robust data on cost effectiveness, it is difficult for regulators to take informed decisions. Creating economic incentives for health care providers to use these innovative technologies can lead to a sharp rise in expenditures, if not accompanied by supplemental measures to govern innovation. Procurement policy also has a significant effect on innovation, and changes to reimbursement policy will normally influence procurement practice. There is a trend toward centralized procurement. The criteria used in tenders vary extensively and in relation to specific technologies. However, in most cases, competition is reduced to price competition: Implications for R&D are negative. In the long run, this could be a barrier to innovation.
(Rolfstam, 2012)	Public agencies should also be considered as demand systems for innovation, requiring a holistic involvement of not only public procurers, managers, and the political leadership but also firms and other organizations on the supply side. There is a variety of innovation‐friendly public procurement.
(Wintjes, 2012)	There are differences between the national and regional level concerning the uptake of the different types of demand‐side innovation policies. At regional level: The emphasis of demand‐side innovation policy is on promoting the application of existing technological inventions in the public and private sectors; innovation policy programs or strategies often consist of a mix of policy instruments. Two important elements of demand‐side innovation policies are better articulation of needs and interaction with intermediate and end‐users. Characteristic for systemic policies at regional level is learning by interaction and the ability to involve end‐users. Regulation and standardization policies are mostly at the national and European levels and not at the regional level. Current successful demand‐side innovation policies at regional level mainly include systemic policies and stimulating private demand for innovations. Concerning public procurement, there is an underused potential at regional level. Regions would especially benefit from promoting innovative demand from public procurers in their daily procurement activities. Promoting innovative procurement is relevant for all regions. Promoting the procurement of R&D with dedicated regional precommercial procurement programs may not be relevant for each and every region.
(Friedman et al, 2009)	PPP: The success of collaboration between the private and public sectors has varied according to the country. However, businesses are indispensable partners, and it is certainly in the interest of public bodies and authorities for them to share the same vision of the future health care system.
(Shortell et al, 2002)	Importance of 2 elements of community health improvement: an explicit vision of what is to be accomplished and a management model that recognizes the inherent complexity of interorganizational alliances formed largely to achieve communitywide benefits rather than individual organizational member benefits. The perceived benefits and costs were strongly associated with overall management capabilities.
(Barlow, Roehrich, and Wright, 2013)	European experience with public‐private partnerships (in particular on hospital setting) has been mixed. Early models of these partnerships—for example, in which a private firm builds a hospital and carries out building maintenance, which we term an “accommodation‐only” model—arguably have not met expectations for achieving greater efficiencies at lower costs. Newer models offer greater opportunities for efficiency gains but are administratively harder to set up and manage. Public‐private partnerships in health care are only peripherally about perceived private‐sector efficiencies, easier financing, or the removal of expenditure from national balance sheets. They are much more about ensuring that risks arising from the development and operation of health care infrastructure are optimally allocated between public and private partners, thereby reducing the risk premium. Bundling activities and using the payment mechanism to create incentives for high performance by the different contractual parties is one theoretical way of achieving this result.

Initiatives for Sustaining and Financing eHealth in 4 European Countries (UK, Spain, Denmark, and Sweden)

Country	Author	Initiatives	Processes	Mechanisms	Expected Impact	Evidence
UK	(Dobrev et al, 2008; Clark and Goodwin, 2010; Beale et al, 2010; The Scottish Government, 2012)	Governmental procurement framework agreement for telecare, telehealth, and telecoaching	(centralized) Public procurement	Centralization Stakeholders engagement Different types of firms involved Accreditation/certification process	Less costs Less time More interoperability	Simplification and easier processes
Spain	(Torres and Pina, 2010)	Strategy for research and innovation and Spanish center for research and innovation (CDTI) Local agreement	PPP PCP‐PPI Grants	Introduction of innovation/PPP‐friendly regulations Financial incentives for PCP and PPI Co‐financing Separate funds for ICT acquisition and solution maintenance	Less problems in direct collaborations More PCP More PPI Cover of all financial needs (from initial to recurring)
Denmark	(OECD, 2010; Authority, 2014; Kierkegaard, 2015)	PWT Foundation‐Investments in Public Welfare Technology Danish Market Development Fund	PCP‐PPI Grants	Co‐financing Criteria of evaluation based on national deployment feasibility Monitoring of projects Financial incentives for PCP and PPI Support to public procurer Business model evaluation Involvement of final users	Less costs Less time Increase of demand More cooperation More PCP More PPI More collaboration Faster to market	Too demanding and stringent standards required; too bureaucracy (OECD, 2010) Relative positive impact on new job creation (Authority, 2014)
Sweden	(Laage‐Hellman, Mckelvey, and Johansson, 2009; Elg and Håkansson, 2012)	Vinnvård VINNOVA	PCP‐PPI Grants	Financing of actionable research Cooperation within public sector and between public and private sectors Financial incentives for PCP and PPI Support to public procurer	More research transfer into practice More PCP More PPI	More knowledge sharing; more collaboration; general impact rather than effects on specific projects (Dobrev et al, 2010) Increased R&D; low impact on projects commercialization/spread out (Elg and Håkansson, 2012)

APPENDIX B.

Informants' Description

Stakeholder Category	Occupational Background	ID	Level	Organization of Reference	Sector of Activity or Study	eHealth Project(s) (a Selection of)
Physicians and other caregivers	Clinicians and clinical academics	1	LHA/hospital	Azienda Sanitaria di Firenze	Geriatrics	inChianti FARseeing ProFouND
		2	LHA/hospital	USL 11 Empoli	Neurophysiopathology	CLEAR ProFouND
		3	National‐international	Univ of Florence‐Italian Telemedicine Society (SIT)	Cardiology, internal medicine, and telemedicine	SIT
		4	LHA	Amici del cuore ONG Livorno	Nephrology	///
		5	LHA	Amici del cuore ONG Livorno	Cardiology	///
		6	LHA	USL 5 Pisa	Rehabilitation	Virtual reality in rehabilitation
	General practitioners	7	LHA	ANSPI‐FIMMG‐SIT‐USL 5 Pisa	Primary care	SIT
		8	LHA	Promed‐SIT‐USL 5 Pisa	Primary care	SIT
Health care organization	Medical manager	9	Regional	Tuscany Region‐Regional HTA Centre	Cardiovascular surgery	HTA
		10	LHA	USL 11 Empoli	Hygiene and preventive care	///
		11	LHA	USL 2 Lucca	Hygiene and preventive care‐community care	///
	Nonmedical manager	12	Regional	Tuscany Region	ICT, innovation, and research in health care	Tuscany imaging diagnostic informatization
		13	LHA	USL 8 Arezzo	Human resources and administrative management	Telemonitoring project
		14	ESTAR‐LHA	ESTAR‐USL 8 Arezzo	ICT	Server cloud
		15	ESTAR	ESTAR	Economics and administrative management	///
		16	ESTAV‐LHA	USL 11 Empoli‐Local HTA	Management and political sciences‐procurement	Local experimentation using mobile devices
		17	LHA	USL 12 Viareggio	Administrative management and political sciences	Informatization projects (ie, RIS‐PACS)
Regulatory agencies	Chief executive officer	18	Regional	Tuscany Region	ICT	Regional ICT system and APP
		19	Regional	Tuscany Region	Innovation	Various eHealth projects and strategies
	Technicians	20	Hospital	Azienda Ospedaliera Universitaria Careggi‐Firenze	Innovation	ENRICH AMON Cal 118 Various telehealth care projects
		21	LHA‐ESTAV	USL 7 Siena‐ESTAV Centro‐Local HTA	ICT and innovation	HTA
		22	ESTAR‐LHA	ESTAR	ICT	///
		23	Regional	Tuscany Region	ICT	///
		24	Regional	Toscana Life Sciences (TLS)	Innovation	Various projects of TLS
		25	Regional	Tuscany Region	Innovation	i‐CAR Various projects of digitalization
		26	ESTAR	ESTAR	ICT	Decipher (PCP)
Patients	Patients and deputies of patients' associations	27	LHA	AGDAL ONG Livorno	Endocrinology	///
		28	///	///	///	CLEAR
		29	///	///	///	CLEAR
		30	///	///	///	RICHARD
Innovator companies	Deputies of innovative firms in the field of EH	31	International	i+	Telemedicine and eHealth	DG‐home Domus HEXPERIENCE
		32	National‐international	Signo Motus	Telemedicine and eHealth	CLEAR RICHARD Habilis
		33	Regional‐national	Biocare provider	Telemedicine and eHealth	drDrin ClinicaMobile

The italic in the column eHealth projects indicated that there is not listed a specific project but a category (ie, telemonitoring projects), or a generic description due to the lack of a specific name of the project (ie, virtual reality in rehabilitation), or an entity/activity (ie, SIT and HTA). The 3 slashes (///) indicate that there is no information for that informant.

De Rosis S, Nuti S. Public strategies for improving eHealth integration and long‐term sustainability in public health care systems: Findings from an Italian case study. Int J Health Plann Mgmt. 2018;33:e131–e152. https://doi.org/10.1002/hpm.2443