Implementation factors affecting the large-scale deployment of digital health and well-being technologies: A qualitative study of the initial phases of the ‘Living-It-Up’ programme

Abstract

Little is known about the factors which facilitate or impede the large-scale deployment of health and well-being consumer technologies. The Living-It-Up project is a large-scale digital intervention led by NHS 24, aiming to transform health and well-being services delivery throughout Scotland. We conducted a qualitative study of the factors affecting the implementation and deployment of the Living-It-Up services. We collected a range of data during the initial phase of deployment, including semi-structured interviews (N = 6); participant observation sessions (N = 5) and meetings with key stakeholders (N = 3). We used the Normalisation Process Theory as an explanatory framework to interpret the social processes at play during the initial phases of deployment.

Initial findings illustrate that it is clear − and perhaps not surprising − that the size and diversity of the Living-It-Up consortium made implementation processes more complex within a ‘multi-stakeholder’ environment. To overcome these barriers, there is a need to clearly define roles, tasks and responsibilities among the consortium partners. Furthermore, varying levels of expectations and requirements, as well as diverse cultures and ways of working, must be effectively managed. Factors which facilitated implementation included extensive stakeholder engagement, such as co-design activities, which can contribute to an increased ‘buy-in’ from users in the long term. An important lesson from the Living-It-Up initiative is that attempting to co-design innovative digital services, but at the same time, recruiting large numbers of users is likely to generate conflicting implementation priorities which hinder − or at least substantially slow down − the effective rollout of services at scale.

The deployment of Living-It-Up services is ongoing, but our results to date suggest that − in order to be successful − the roll-out of digital health and well-being technologies at scale requires a delicate and pragmatic trade-off between co-design activities, the development of innovative services and the efforts allocated to widespread marketing and recruitment initiatives.

Introduction

‘Living-It-Up’ (LiU) is a 3-year initiative aiming at promoting the development, deployment and uptake of digital health and well-being services across Scotland.¹ LiU is conceptualised as a radical transformation of health and social care delivery, with an emphasis on promoting preventative care, healthy lifestyles and independent living. This transformation is motivated by the dramatic increase in the number of people affected by complex, co-morbid conditions,² with the associated unsustainable cost burden for the health and social care systems.³ However, the decreasing cost of digital technologies suggests that alternative and cost-effective models of care are potentially feasible.⁴ Implementing these at scale has, however, proven an incredibly challenging endeavour.^5–7 Various possible explanations have been put forward as to why it has been so difficult to scale-up technologies. Key barriers previously identified include inadequate engagement with stakeholders, the impact of new technologies on roles and responsibilities and perhaps the insufficient attention allocated to appraisal mechanisms within organisations.^8,9 Some have suggested that effective adaptation to local contexts, accessibility and strong stakeholder collaboration can operate as important facilitators to assisted living technology deployment.¹⁰ However, these factors may, therefore, be more difficult to replicate and sustain at scale. Identifying factors which have shaped technologies and how new services have evolved over time is crucial in order to understand key implementation lessons in this sphere. Therefore, the LiU programme in Scotland provides a valuable opportunity to study the implementation processes involved throughout the large-scale deployment of digital health and well-being technologies.

Overview of LiU

The LiU programme is led by NHS 24, Scotland’s National Health Service telehealth and telecare organisation. It is part of the wider Delivering Assisted Living Lifestyles at Scale (dallas) programme, which aims to promote the roll-out of digital health and well-being technologies at scale across the United Kingdom.¹ LiU has developed a national delivery plan across Scotland consisting of the following four distinct deployment phases (see Table 1):

Table 1.

Living-It-Up consortia.

Health sector	Scottish Centre for Telehealth and Telecare, NHS 24, NHS Lothian, NHS Western Isles, NHS Forth Valley, NHS Grampian, NHS Highland
Voluntary sector	Carers Scotland
Industry sector	Highlands and Islands Enterprise, Atos, Philips, Intersystems, STV, STV Healthcare, Maverick TV, Ernst &Young, Scottish Enterprise, Digital Life Sciences, Interlate, Sitekit, Looking Local
University and academia	Glasgow School of Art
Local authorities and state services	The Scottish Government, Kirklees Council, The Highland Council, Argyll and Bute Council, Moray Council, West Lothian Council, East Lothian Council, Edinburgh Council, Falkirk Council, Stirling Council, Clackmannanshire Council, Midlothian Council
Other organisations	Joint Improvement Team, Health Alliance Scotland

• Phase 1 – solution exploration: this initial phase aims to focus on community engagement and iterative co-design with target users, partners and stakeholders;

• Phase 2 – prototyping: this phase focuses on testing the platform initially developed with early users, stakeholders and partners;

• Phase 3 – development of product and services: this phase focuses on developing the services underpinning the LiU digital platform;

• Phase 4 – large-scale roll-out: the final phase of the programme focuses on recruitment and the large-scale roll-out of products and services.

This article aims to describe the implementation activities that occurred during the initial phases (1 and 2) of the deployment (i.e. from approximately October 2012 to January 2014). For the interested reader, further details on the range of LiU services have also been reported elsewhere.¹¹

Materials and methods

Ethical approval for this study was obtained from the University of Glasgow, College of Medicine, Veterinary and Life Sciences Ethics Committee (REF: 2000130029). This study used a combination of qualitative methods, including ethnographic and observational techniques as well as semi-structured interviews.¹² Data collected included (see Table 2) semi-structured interviews (N = 6), participant observation sessions (N = 5) and meetings with key stakeholders (N = 3). One researcher (R.A.) attended meetings organised by the LiU Community Engagement Team (CET) throughout the ‘co-design and user engagement events’ (Phase 1) and ‘prototyping workshops’ (Phase 2). These events ranged from small sessions (e.g. 10–15 participants) to large public meetings (e.g. with up to 200 people). During these meetings, the researcher took field notes and collected documents used throughout the engagement sessions. Primary data collection using participant observation was conducted by two researchers (R.A. and M.M.-L.). These sessions provided useful insight into the activities and discussions which took place in the early phases of the LiU programme design. The meetings − lasting between 1 and 2 h − were audio-taped with the participants’ consent, transcribed and used as primary data as part of this study analysis. The semi-structured interviews (N = 6) were carried out over the phone with a purposive sample of individuals responsible for driving the LiU implementation forward. We developed a semi-structured interview framework based on Normalisation Process Theory (NPT), a social theory of the implementation processes at play during the deployment of innovative services or technologies.^8,13,14 Each interview lasted an hour approximately. Interviews were digitally recorded with the participants’ consent, transcribed verbatim and corrected prior to analysis.

Table 2.

Qualitative data collected.

Data collection method	Instance and description	Location and time frame	Hours of audio recordings
Participant observation	N = 1, co-design and pop-up user engagement event	Livingston, February 2013	N/A, field notes
	N = 3, co-design stakeholder meetings	Edinburgh and Falkirk, November–December 2012 and February 2013	N/A, field notes
	N = 1, executive showcase supporting knowledge transfer across sectors and professional networks	Liverpool, March 2013	N/A, field notes
Meetings with stakeholders	N = 3, ad hoc meetings with programme representatives to discuss status of ongoing implementation	Glasgow, October 2012, February and May 2013	5 h
Semi-structured interviews	N = 6, in-depth telephone interviews	Glasgow, January 2014	6 h

Data analysis and theoretical framework

Transcripts and notes were coded and analysed using an inductive approach. Emerging concepts and themes relating to the social change processes involved in the early phases of the LiU deployment were then mapped to a NPT framework (see Table 3, Appendix 1). NPT provides a robust conceptual framework to understand the complex social processes involved in implementing, embedding (making practices routine) and integrating (sustaining embedded practices) new technologies and services.^8,13,14

NPT refers to four key mechanisms: coherence (how individuals or groups ‘make sense’ of new ways of working), cognitive participation (the ‘engagement’ work to ensure participants ‘buy into’ and ‘sustain’ an intervention), collective action (the work of putting operationalising an intervention) and reflexive monitoring (the work of appraising an intervention).^13,14 One researcher (R.A.) coded all the data, and categories were subsequently discussed, iteratively refined and validated during ‘data clinics’ attended by other researchers with extensive experience of qualitative data analysis (M.M.-L. and F.M.) in order to develop a comprehensive view of the implementation issues arising from the LiU programme. The following section describes the main findings of our implementation processes analysis during the initial phase of the LiU programme.

Results: factors affecting the large-scale implementation of digital health and well-being technologies

Theme 1: cultural shift in mindsets and work practices

It became clear from the programme outset that a fundamental shift in focus, organisational behaviours and ways of working was necessary in order to enable the development and integration of new technologies into existing services:

All this is perhaps going to take a new way of working and a new way of thinking and perhaps new job roles within statutory and voluntary services to help people help each other out. (LiU Respondent 01A)

Developing a ‘shared understanding’ of the overall goals of the project is very much an iterative process, which requires time and effort from all parties, including professionals and potential consumers of the services.

One respondent highlighted that some people were not ‘technology-minded’ and hence were perhaps reluctant – or at least unenthusiastic – at the prospect of using new technologies. It appeared that some users chose to voluntarily disengage themselves from using technology perhaps because they were unprepared or unwilling to embrace change:

So they sort of dismiss it, ‘no it’s not for me!’ and you know … you’ve got to respect people’s opinions … (LiU Respondent 02A)

Theme 2: establishing roles, responsibilities and setting collective aims

Clarifying individual and collective roles and responsibilities when working within such a large and diverse consortium naturally raised substantial difficulties. There were clear challenges in determining and deciding ‘Whose voice counts?’, particularly in the context of the large public engagement activities which took place throughout the initial phase of the programme. Stakeholders’ views varied widely and tensions sometimes emerged as each ‘interest-group’ attempted to position and define an identity for themselves within the wider LiU project:

We are calming people down … we need to take this stage by stage and we are telling people ‘don’t worry, you are still at the table but you don’t need to be doing everything now, until we learn what we actually need’ … so it’s a difficult juggling act. (LiU Respondent 03A)

Hence, it was important and crucial for stakeholders to identify a collective purpose in meeting the objectives of the programme:

Just getting everybody in the room … to develop their relationships across the different organisational structures and to develop a shared vision. So … what is it we are really trying to do? How does that meet everybody’s agendas? Are you engaging with us and are you going to drive it forward from your different organisational perspectives? (LiU Respondent 03A)

Initial roles and responsibilities were not always understood initially within the consortium. This on occasions led to a mismatch between roles and expertise, with some people taking on responsibilities which did not necessarily closely match their personal or organisational expertise:

“There was a kind of confusion or blurring of roles and responsibilities” … I think sometimes the term ‘co-design’ or ‘co-production’ has been confused and what sometimes has happened is that we’ve failed to acknowledge or to draw on the expertise that particular people have. (LiU Respondent 04A)

So you have all these people who that is not their key area, that’s not their area of specialism at all who are coming in and putting their tuppence worth … which is fine, right … but when the decision is made, it’s then made somewhere else […]. (LiU Respondent 05A)

… In hindsight, it would have been good to have … had much more clarity on who was responsible for what. (LiU Respondent 05A)

Theme 3: person-centred versus service-centred design

LiU spent a considerable amount of time and effort engaging with potential end-users from the outset, and service designers were keen to design solutions collaboratively, so that services were more likely to be adopted by end-users:

Our particular aim is to encourage people to get involved, to join in and to co-design these services, to come along to our events that we put on and encourage people to give us their views … (LiU Respondent 06A)

One of the key principles of LiU is the use of familiar technology. So we are not going to find new gadgets to buy: we are thinking about what people already have and are using … and how do we interoperate that back into statutory services … (LiU Respondent 03A)

Adopting a more person-centred approach to services implies attempting to remove some of the existing cultural or organisational barriers:

Users are saying that … ‘what we don’t want is more information but what we do want is personalised information’ for their needs, at the right time, on the right device that suits them. (LiU Respondent 03A)

However, this is easier said than done:

How can we push information to someone if we don’t know anything about someone? … so that’s a constraint! If we want to really drive the personalisation agenda and integrated services, we need to collect data and that’s where we are going for a fairly light and basic profile … because if you do that, we will be able to enrich the user experience through LiU. (LiU Respondent 03A)

The latter point naturally raises the complex and difficult issue of personalisation versus data privacy when attempting to develop truly person-centred services, a challenge which can perhaps operate as a further barrier to uptake.¹⁵

Theme 4: user engagement, participation and enrolment versus sustainability

The CET developed a number of service principles to underpin the LiU programme design and deployment. A key driving force within LiU was to recognise and harness people’s individual skills throughout the community as well as facilitating skill-sharing and upskilling. This was also seen as a way to reach the ‘digitally excluded’:

We’ve also got a lot of work going on where people or organisations are doing technology workshops and training sessions for people. So we are trying to link in with them so that they can introduce them to Living-it-Up and just finding other ways that people can access the technology. They might not be able to do it from their home but if they are coming in … could they do it in a local library or in a community centre … (LiU Respondent 05A)

A key theme which emerged from our data was the tensions which existed when attempting to develop innovative services but at the same time recruiting at scale:

Users may be recruited in their thousands but it is vital to keep them engaged. (LiU Respondent 03A)

I think one of the implementation lessons learned is … the programme isn’t long enough! and I’ve said this all back … but the programme isn’t long enough: because really … you spend years developing and innovating and then recruitment should follow … (LiU Respondent 03A)

While the co-design process was aimed at promoting engagement and ‘buy-in’ from potential users and consumers in the initial phase of the programme, the lack of an available ‘finished product’ within a reasonable amount of time proved to be a risk to sustaining interest in the programme. What this furthermore suggests is that it remains unclear to what level – if any – the initial co-design initiative subsequently translated into sustained service uptake.

So how do we measure if co-design is useful or not?’ because we all want to use it but how can we tell that the intervention is good? People may say it is good, but why is it good ‘Oh because we co-designed’ but how do you measure that though? (LiU Respondent 06A)

Theme 5: resource allocation, funding and infrastructure

LiU members emphasised the importance of having adequate resources and the necessary infrastructure in order for support the implementation. Staffing, division of labour, cost and infrastructure were recurrent issues.

People are struggling to do their work with less resources than what they have had before … then it’s very difficult for them to start to think about new ways of doing things. (LiU Respondent 07A)

I think staffing in each of the local areas still do need bigger teams, with focus on different things and whether that’s like buying-in services … (LiU Respondent 04A)

Serving a particular catchment area with differing needs was also expressed as a factor which need to be monitored or could negatively affect the implementation:

‘We have a different geographical set up here so there is not necessarily the same structures or resources in place’ […] ‘Lots of areas that don’t have broad-band at all … we do have some real challenges, even in some of our towns, we have some challenges with this broadband access as well … so as far as the technology is concerned: yes we have had lots of problems!’. (LiU Respondent 08A)

Theme 6: measuring effectiveness and benefits realisation

An important aspect is how the LiU programme benefits are assessed in order to support the programme long-term sustainability. During the initial stages, LiU invested considerable efforts in developing strong links and communication mechanisms between service designers, providers and the target users. This certainly had some positive impact on the programme implementation. In terms of the ongoing evaluation of the project, aligning evaluation strategies between the wider dallas evaluation and individual dallas communities was seen as an important issue. The LiU evaluation team referred to the need to encapsulate the interventions as a whole. This in turn seems to have an impact on the future sustainability of the project, and the question remains how this element can be appraised effectively to inform future policy and similar projects:

You are not looking at the implementation of an intervention; you are looking at the intervention of a programme, and we need to think of it in that context of the bigger picture. (LiU Respondent 06A)

Discussion

The LiU programme is ambitious as it aims to radically transform how health, social care and community services are delivered and consumed. It is clear that developing digital, person-centred services require substantial changes in the culture and behaviour of organisations and individuals involved: from both the perspectives of service providers as well as ‘consumers’ − if only for people to start thinking of themselves as proactive stakeholders in their health and well-being as opposed to passive recipients of care (i.e. patients).

A recurrent stressor thought that the programme deployment was the persistent conflicting priorities of technological innovation versus service roll-out at scale. Our study so far suggests that LiU is being positively received by consumers. Our data suggest that the extensive co-design and engagement efforts which have been undertaken within the LiU programme are promoting and developing a shared sense of understanding of the programme across a broad range of stakeholders. We have reported elsewhere how the ground work of sustaining engagement with stakeholders proved to be a key factor in enabling the successful deployment of a large-scale eHealth initiative (electronic clinical portal) within one health-board in Scotland.¹⁶ Hence, the broad consultation processes and communication mechanisms established as part of the LiU co-design processes may yet reap dividends in future. However, the consultative and iterative nature of co-design activities also means that it has taken a considerable amount of time before finished products and services were eventually available to potential consumers. Despite the current widespread advocacy of co-design methodologies, we still know very little with regard to the impact of such activities on the subsequent uptake of technologies.

The sheer size and diversity of the LiU consortium also presented important challenges in itself. Developing a shared vision among partners with different priorities, cultures and ways of working has required a great deal of effort. This need for a shared view of the direction of travel for the programme cannot be overemphasised as it has implications for all aspects of the deployment, for example, from determining priorities to agreeing service specifications. A great deal of effort had to be put into clarifying and allocating clear roles and responsibilities. Finally, identifying and assessing tangible benefits while the programme is still at a very early stage of implementation is naturally very difficult.

Study strengths and limitations

Our data analysis focuses on the early stages of the programme implementation, and hence, the long-term benefits or dis-benefits of the programme are not yet apparent. Clearly, there is much more learning to be gained from this large-scale digital health and well-being implementation. A second limitation of this study lies in the fact that we are only reporting the perspectives of implementers. However, we are currently undertaking a range of focus groups with end-users of dallas services, and we will also be reporting on service users’ perspectives in due course once our data analysis is completed. A key strength of our study has been the use of a range of qualitative approaches, underpinned by a sound and robust, conceptual framework, NPT, to underpin our data analysis and interpretation.

Conclusion

The deployment of LiU services is ongoing, but our results to date suggest that in order to be successful, the roll-out of digital health and well-being technologies at scale requires a delicate and pragmatic trade-off between co-design activities, the development of innovative services and the efforts allocated to widespread marketing and recruitment initiatives.

Appendix 1

Figure 1.

Conceptual model of Living It Up, co-design and user engagement toolkit.

Table 3.

NPT coding framework used for qualitative data analysis.

Coherence (sense-making)	Cognitive participation (buy in)	Collective action (enacting work)	Reflexive monitoring (appraisal)
Differentiation: Is there a clear understanding of how the Living-It-Up programme differs from previous efforts within implementation?	Enrolment: Do service users actively participate and ‘buy into’ the services within the Living-It-Up programme?	Skill-set workability: How does the implementation of Living It Up affect division of labour of work practices, training or roles and responsibilities?	Reconfiguration: Do participants develop a ‘work-around’ or try to alter a technology, product or service?
Communal specification: Do the stakeholders, service users and service providers develop a shared understanding of the aims and objectives of the Living-It-Up programme?	Activation: Can service users who participate in the Living-It-Up project sustain its implementation?	Contextual integration: Is there organisational support for resource allocation? In terms of ways that users enact a new set of practice	Communal appraisal: How do service user groups judge and determine the value of the Living-It-Up programme?
Individual specification: Do all stakeholders have a clear understanding of their specific roles and responsibilities in achieving the vision of the Living-It-Up programme?	Initiation: Are individuals willing to drive the implementation forward? Who are they? Local Champions?	Interactional workability: Does the Living-It-Up programme make existing routines of practice easier?	Individual appraisal: How do individual participants appraise the effects of the implementation of Living It Up on them and their work environment?
Internalisation: Do all stakeholders understand the significance of this intervention and its future value?	Legitimation: Do participants believe it is right for them to be involved? Do participants feel they can make a valid contribution?	Relational integration: Do participants have confidence in using Living-It-Up technologies, products and services?	Systematisation: How do participants determine the effectiveness (benefits and limitations) or usefulness of Living It Up? And how can this be measured?