Skip to main content
BMJ Open logoLink to BMJ Open
. 2022 Nov 23;12(11):e065778. doi: 10.1136/bmjopen-2022-065778

Stakeholders barriers and facilitators for the implementation of a personalised digital care pathway: a qualitative study

Florence Heijsters 1,2,, Jesse Santema 1, Margriet Mullender 1, Mark-Bram Bouman 1, Martine de Bruijne 3,4, Femke van Nassau 3
PMCID: PMC9685003  PMID: 36418140

Abstract

Objective

A prerequisite for patient-centredness in healthcare organisations is offering patients access to adequate health information, which fits their needs. A personalised digital care pathway (PDCP) is a tool that facilitates the provision of tailored and timely information. Despite its potential, barriers influence the implementation of digital tools in healthcare organisations. Therefore, we investigated the perceived barriers and facilitators for implementation of the PDCP among stakeholders.

Design

A qualitative study was conducted to acquire insight into perceptions of the stakeholders involved in the implementation of a digital care pathway in three diverse patient groups.

Setting

This study is part of the PDCP research project in a large academic hospital in the Netherlands.

Participants

Purposive sampling was used to recruit internal stakeholders (eg, healthcare professionals, employees of the supporting departments) and external stakeholders (eg, employees of the external PDCP supplier). In addition, existing semistructured interviews with patients involved in pilot implementation (n=24) were used to verify the findings.

Results

We conducted 25 semistructured interviews using the Consolidated Framework for Implementation Research. Content analyses yielded four themes: (1) stakeholders’ perceptions of the PDCP (eg, perceived usefulness); (2) characteristics of the individuals involved and the implementation process (eg, individuals express resistance to change); (3) organisational readiness (eg, lack of resources); and (4) collaboration within the organisation (eg, mutual communication, multidisciplinary codesign). The main barriers mentioned by patients were duration of first activation and necessity for up-to-date content. In addition, the most facilitating factor for patients was user-friendliness.

Conclusion

Our findings emphasise the importance of gaining insights into the various perspectives of stakeholder groups, including patients, regarding the implementation of the PDCP. The perceived barriers and facilitators can be used to improve the PDCP implementation plan and tailor the development and improvement of other digital patient communication tools.

Keywords: PLASTIC & RECONSTRUCTIVE SURGERY, Paediatric plastic & reconstructive surgery, Dermatological tumours, Sexual and gender disorders


Strengths and limitations of this study.

  • This qualitative study took into account the diverse perspectives of all types of stakeholders.

  • A codesign approach was used to enhance successful implementation.

  • Due to the COVID-19 pandemic, interviews were conducted both digitally and physically.

  • Participants were recruited using snowball sampling techniques, which could have resulted in selection bias.

  • Since the interviews were conducted and analysed by multiple researchers, investigator triangulation was applied.

Introduction

In recent years, the emphasis on patient-centred care has increased in the Dutch healthcare system.1 Patient-centredness is a key element of high-quality care and entails collaboration between the healthcare professional (HCP), the patient and their families, in which the needs, values and preferences of patients are the focus of care.2 3 Patients no longer want to be passive recipients of care but increasingly want and need to proactively manage their own health. They also wish to be empowered and involved in decision making that relates to their care, which can contribute to patient-centred care.4–9 To achieve this, it is important that HCPs and patients share the same information.10 11

To achieve patient-centred care including shared decision making, health information should be tailored. Personalised health information includes details about the diagnosis and treatment options of the individual and practical information about their care pathway.12 13 This concerns information about possible choices and the advantages and disadvantages of these choices, along with outcomes and uncertainties.8 13 14 Information provision should match the patient’s wishes, needs and their ability to process information, which ensures a better experience for the patient.15 In addition, optimally dosing and timing the information provision is crucial to prevent patients from an information overload.7 15 16 Also, health literacy studies show that general health information is frequently not understood to a sufficient extent.17

Both patients and HCPs have expressed their willingness to contribute to patient-centred care but often have insufficient knowledge on how to put this into practice.4 5 7 Therefore, HCPs should be instructed on how to provide their patients with suitable information.7 18 Providing the HCP and patient with tools to improve information provision facilitates effective communication.4 11 A personalised digital care pathway is an example of a tool that facilitates HCPs and their patients in accessing adequate information.4 16 19 This can be described as a digital tool providing personalised dosed information and an overview of appointments for the coordination of care, tailored to a certain patient group, which leads to greater information symmetry between the HCP and patient.20

Despite the potential of digital care pathways, there appear to be barriers to the implementation of digital tools in healthcare organisations.11 21–24 An innovation that has been poorly implemented by HCPs may end up not being used in daily routines or used in the wrong way, which results in a low uptake by end users.24 25 Gathering information about the organisation, such as the context and responses to change before and during implementation, can determine the factors that affect implementation.25–27 All stakeholders act within their own contexts and expectations.28 Therefore, to implement a digital care pathway in practice, it is essential to explore the perceptions of all stakeholders involved including end users.29

Information about implementation is often expressed in barriers and facilitators.27 30 This information contributes to selecting tailored implementation strategies, which in turn can help overcome the hurdles of implementation.27 31 As example, previous studies have shown that low health literacy and inadequate staffing were barriers to the implementation of digital tools. However, the perceived usefulness of an intervention and good multidisciplinary communication were identified as facilitating for implementation.3 19 32 However, some of these studies mainly focused on implementation in one specific patient group and the perceived barriers and facilitators from an organisational perspective. As a result, there is a lack of insight into the hospital-wide embedding in diverse patient groups and visions of all the different stakeholders involved, including end users. Therefore, the aim of this study was to identify the perceived facilitators and barriers among various internal and external stakeholders (ie, patients, healthcare professionals, non-medical professionals and external supplier) regarding the implementation of personalised digital care pathways within a large academic hospital.

Methods

Study design

A qualitative study was conducted to acquire insight into perceptions of the stakeholders involved in the implementation of a digital care pathway in three diverse patient groups.

Context and setting

This study is part of the ‘Personalised Digital Care Pathway’ (PDCP) research project. As basis for initiating this patient-centred project, we used previous patient-reported experience measurements and patient participation sessions during the start of the value-based healthcare (VBHC) programme at Amsterdam UMC, a large academic hospital in Amsterdam, the Netherlands.33 Patients indicated that information was outdated and not easily accessible. This was mentioned among all three selected patient groups, all of them were part of the VBHC programme. Furthermore, they experienced a low level of self-management with regard to their care. Development and subsequent implementation of a PDCP was anticipated to remedy these shortcomings. The project included two phases:

  1. Adaptive development including pilot implementation, based on experience-based codesign (out of scope in this study). PDCPs were created in close collaboration with: (1) patients with excessive scars (scar clinic), (2) cleft lip and/or palate (cleft care) and (3) people who experience gender dysphoria (gender care) and their healthcare professionals, communication advisors and the eHealth team of our electronic health record service centre (EvA-SC).

  2. Evaluation of implementation in practice of the PDCPs at Amsterdam UMC for the three patient groups as described previously.

We conducted this qualitative study at the end of phase 1.

Personalised digital care pathway

In this study, we define a PDCP as a digital tool that provides patients and their healthcare professionals an overview of a personal care pathway, with adequate and dosed information at appropriate time points. An example is shown in figure 1 (web and mobile version). In this customised tool, relevant content will become available gradually to end users as the care pathway progresses over time, including appointments and practical information. Patients can access the PDCP tool after a two-factor authentication via an app or as a web-based tool.34 Access is given after their first intake to the hospital, and after the initial authentication, patients and/or parents can enter the tool at any time. Healthcare professionals have access to the patients’ PDCP via the electronic health record (EHR). The IT system used for this project was developed by an external supplier (Soulve Innovations).35

Figure 1.

Figure 1

An example of personalised digital care pathway (web and mobile version) in Amsterdam UMC.

Study participants

Stakeholders were recruited by purposive sampling. To identify further relevant stakeholders, we used snowball sampling.36 Internal stakeholders were healthcare professionals of the medical specialties involved (plastic surgery, otorhinolaryngology, psychology and gynaecology) and employees of the supporting departments, divided into managers and team members. Employees of the external supplier participated as external stakeholders. All stakeholders were contacted through email. Once the stakeholder had agreed to participate, an interview was scheduled, digitally via Microsoft Teams or on location. Informed consent was signed after the participant was informed about the purpose of the study. Verbal consent for audio recording was obtained from every participant.

Theoretical framework

We used the Consolidated Framework for Implementation Research (CFIR), as this framework can help to explain why implementation of the PDCP may or may not be successful.37 It provides a practical guide for evaluating perceived facilitators and barriers. In this framework, the context, complexity, multilevel aspects and interaction of the implementation are considered.31 37 The five domains of the CFIR framework are: intervention (eg, advantage, adaptability), outer setting (eg, patient needs, external policies), inner setting (eg, culture, readiness for implementation), the individuals involved (eg, knowledge and beliefs, self-efficacy) and the implementation process (eg, engaging, executing).37 38

Data collection

We conducted individual, semistructured, in-depth interviews guided by CFIR. We ordered the themes in a way that was consistent with the care and implementation process, incorporating all components of CFIR. The list of topics (online supplemental appendix 1) was amended minimally for stakeholders who work as healthcare professionals. All audio-recorded interviews were conducted by trained researchers (JS, JG, FH and FvN) and transcribed verbatim.

Supplementary data

bmjopen-2022-065778supp001.pdf (344.1KB, pdf)

Patient and public involvement

Patients were actively involved during adaptive development of the PDCP (phase 1). Patients were selected via their healthcare professional, consultation appointment or inpatient admission. In this study, pragmatic semistructured interviews with patients (n=24), which had been conducted after taking part in the pilot implementation, were used for verification of our findings about the facilitators and barriers for implementation of the PDCP. The purpose of these interviews was to collect patients’ experiences concerning the content and use of the application during the pilot implementation and verify the gathered findings. These data were also collected as part of the PDCP research project during phase 1 and will be used for further implementation.

Data analysis

To ensure data saturation, data analysis was initiated during data collection, so that missing information, themes or perspectives could be gathered during upcoming interviews.39 This process was repeated until no new themes emerged from the data and we mainly heard information we gathered before. Data were analysed using content analysis in Atlas.ti V.9 (Berlin, Germany). First, three researchers (FH, JS and FvN) coded six transcripts openly and inductively. During several meetings (FH, JS and FvN) codes were discussed, grouped in overarching codes and revised to reach a consensus and ensure quality of the analysis. The codes used in Atlas.ti V.9 were described in a final codebook (online supplemental appendix 2), which was used to analyse the remaining transcripts by two independent researchers (FH and JS). After coding the transcripts, categories were formed by deductive axial coding. These categories were used to form themes and subthemes, described in the results section. In the final step, quotes were selected for representation. All data were analysed and presented pseudonymously.

Supplementary data

bmjopen-2022-065778supp002.pdf (227.6KB, pdf)

Results

In total, 25 interviews were conducted between November 2020 and June 2021. Most of the interviews were held digitally due to the COVID-19 pandemic (n=22), and three interviews were held face to face. Interviews lasted on average an hour (37min; 67 min max). Table 1 shows characteristics of the interviewees. The group of ‘non-medical professionals’ included supporting staff of the implicated medical departments level (decentral), supporting staff of the main organisation, for example, strategy, communication and EHR department (central), and employees of the external supplier as external participants. Median employment duration of stakeholders in their current position was 2 years (min 1 – max 20 years).

Table 1.

Characteristics of study participants (n=25)

Gender, n (%) Professional roles in organisation, n (%) Employment context of participants, n (%)
Male 7 (28) Healthcare professionals Total 10 (40) Excessive scars
(scar clinic)
4 (16)
Female 18 (72) Doctor 5 (20) Cleft lip and/or palate
(cleft care)
4 (16)
Nurse (specialist) 3 (12) Gender incongruence
(gender care)
6 (24)
Other roles 2 (8) Internal and external communication 2 (8)
Non-medical professionals Total 15 (60) EvA service centre 4 (16)
Decentral staff Consultant 2 (8) Strategy and Innovation 3 (12)
Other roles 2 (8) External PDCP supplier 2 (8)
Central staff Manager 3 (12)
Application specialist 2 (8)
Consultant 4 (16)
External staff Other roles 2 (8)

EvA-SC, electronic health record service centre; PDCP, personalised digital care pathway.

In addition, the 24 interviews with patients were conducted after taking part in the pilot implementation (phase 1) and were included in the results. An interview validity check was used for verification of the themes and findings. Quotes were used for representation.

Content analyses yielded four main themes, subdivided in 24 subthemes (table 2).

Table 2.

Themes and perceived barriers and facilitators regarding implementation of a personalised digital care pathway

Themes Subthemes
1. Stakeholders’ perceptions of the PDCP Barriers Duration of first activation
Not suitable for every patient group
Necessity of up-to-date content
Still unclear effectiveness
Facilitators User friendliness
Perceived usefulness by patients
Potential efficiency in practice
Contribution to patient-centred care
2. Implementation by HCP in daily practice Barriers Individuals express resistance to change
Time-consuming implementation
Non-optimal facilities
Facilitators Possibility of incorporation in daily practice
Providing feedback on adoption
Created support in the medical department
by enlisting codesign
3. Organisational readiness Barriers Lack of long term capacity and resources
Rigidity of organisation
Ongoing merger
Facilitators Focus of national policy on digital healthcare
Support from different levels in organisation
Patients providing incentives for change
4. Collaboration within the organisation Barriers Mutual communication
Lack of clear process agreements
Facilitators Multidisciplinary codesign
Appointing a clinical and operational lead

HCP, healthcare professional; PDCP, personalised digital care pathway.

Stakeholders’ perceptions of the PDCP

Patients willing to use the PDCP for the first time needed different applications (apps) for first activation, including a two-factor authentication. The time-consuming process and that patients experienced practical difficulties were identified as barriers by patients and HCPs. It was also mentioned that this may affect its suitability for patients in acute care settings or for end users lacking digital skills. In addition, to access the PDCP through the EHR as a healthcare professional, the HCP must first manually install the tool.

We want them [patients] to create and activate a MyChart [patient portal] account, but they also have to create another, separate account for MediMapp [PDCP tool]. (HCP)

For the PDCP to permanently match the needs of the patients, one patient indicated that it is necessary for the application to remain up-to-date:

It [the application] has to stay up-to-date. It should not be another tool that is produced but never updated. That is often what tends to happen with these kinds of innovations. (Patient, gender care)

The final barrier relates to the unknown effectiveness of the PDCP innovation in the context of an academic hospital. Respondents mentioned that the precise value of the PDCP tool for the organisation is still unclear. This meant that some HCPs and supporting staff were unconvinced by the innovation, which made it difficult to assess whether is it worth the investment in time and resources during implementation.

One of the main experienced facilitators was the user friendliness of the PDCP. All stakeholders, including patients, regarded the PDCP as visually attractive with a user-friendly interface and an easily understandable overview of the care pathway.

It [MediMapp] looks clear and welcoming, you know exactly where to find what you need. (Patient, cleft care)

Mainly HCPs and patients expressed that the perceived usefulness would facilitate PDCP implementation. The innovation met the information needs of patients and created better accessibility of high quality and dosed information. In addition, the PDCP is linked to the EHR, which ensures completeness and reliability for end users.

The integration with electronic health records is one of the unique selling points of this tool, because that allows patients access to their own [personal] app environment. (HCP)

HCPs indicated that using a PDCP might improve efficiency in their daily practice. First, patients know what to expect and what a consultation entails. Second, a PDCP could reduce patient questions, since patients can access information before and after a consultation.

HCPs and supporting staff perceived that implementing a PDCP contributed to the provision of patient-centred care within the hospital. Important reasons were: (1) placing the patient at the centre of care and using digital tools to support this aligns with the strategy of Amsterdam UMC, (2) the PDCP may ensure a higher level of involvement by patients in their treatment and (3) it may facilitate better interaction between the patient and the HCP.

In my opinion, the relationship between the patient and the specialist will actually improve … They [patients] know that the specialist is offering the best possible option at that moment, because they [patients] can read up on the available information themselves. (HCP)

Implementation by HCP in daily practice

As indicated by a HCP, changing people’s behaviour is difficult and resistance may occur. Since HCP’s individual change is necessary for implementation of PDCP, this resistance was perceived as a barrier. Breaking routines and demonstrating that the new way is better was perceived as a difficult transition:

Doctors are often creatures of habit, and … you [HCP] often have a routine that works effectively for yourself. Changes can then be more difficult to implement. (HCP)

Since each department and organisation has its own routine in daily practice, it was mentioned that the innovation must also fit into the routine, including the departments’ character and setting:

The risk is that there is no room for innovation, because of the kind of person or doctor, and the type of work you do. The setting does not immediately allow innovation. (HCP)

In relation to this, it also often takes time to embed and apply an innovation into practice. Since using the tool required an initial (manually) action by the HCP, which costs time and motivation, this was perceived as a barrier. Moreover, some HCPs indicated that if the consultation room facilities were not optimal, this hindered use in practice.

The screen cannot always be turned to face the patient because it is wired up with locked cables. … This can make it quite a challenge to ensure a good view of the screen for both the HCP and patient. … This raises the question: if I cannot show the screen to the patient, what added value does using it [PDCP] offer me? If I cannot show the patient my screen, it makes no sense to use it [PDCP]. (HCP)

However, most HCPs concluded that the consultation room facilities are currently adequate to apply the PDCP in practice.

Due to mainly routine work, especially in the outpatient clinic, it should be possible to implement the PDCP in their daily practice in the consultation room.

To ensure long-term adoption, almost all stakeholders suggested that providing feedback on adoption can facilitate the implementation for HCPs. It was also mentioned by HCPs that sharing positive experiences, preferably those of their own colleagues, can also enthuse non-users by showing added value for patients. Sharing experiences could also be a way of explaining how the application works in practice. This can be promoted by internal ambassadors. In addition, HCPs indicated that reminders may be needed to stimulate users to continue using the app.

If I could notice a couple of times that the tool really helped a patient, then it would be sold to me. My own experiences would really contribute, but the experiences shared by colleagues would make a big difference. (HCP)

Lastly, as mentioned by HCPs, the experience of being involved during the process of developing the tool from an early stage would work as a facilitator. An internal staff member also indicated that it would be conducive to ensure commitment from the department via a financial contribution.

What I like about this [development of the PDCP], is how we have been closely involved in the design phase and content development. … I would have difficulty in adopting something new if I felt that my input was not taken seriously. (HCP)

Organisational readiness

Almost all stakeholders mentioned that in order to innovate, an organisation must provide sufficient capacity and resources. One contradicting finding was that most HCPs suggested that long-term implementation support from central departments can facilitate adoption and maintenance, but these central departments indicated that due to a lack of capacity and resources, this was not sustainable. This dilemma was perceived as a barrier:

The realisation that you need to make significant investments in digital support to achieve the level of ambitions has not yet penetrated our governance structure and the Board of Directors… For example, the eHealth team has to devote quite some time and effort to managing the PDCP. (Organisational staff)

Furthermore, certain restrictive characteristics of this large organisation were mentioned as an obstacle to implementation. Due to the rigidity and bureaucratic structure, the stakeholders, including the external supplier, stated that decision making in the organisation was difficult and that all processes took a lot of time. These aspects negatively influenced the agility and ability to change of the organisation:

I know Amsterdam UMC as an extensive organisation, where many administrative processes must be proceeded through to effect relatively minor changes … In my opinion, it [Amsterdam UMC] can be rigid and unwieldy. I hope this will not be the case regarding the implementation of the PDCP. (HCP)

Another hindering characteristic was the current, ongoing merger of the two locations of Amsterdam UMC. This was mainly perceived as a barrier for implementation. When mandatory change is imposed by an organisation, employees perceived this as not having a choice. The merger implied numerous changes (eg, working at new locations, observing new medical protocols and working with new colleagues) to which employees were expected to adapt. In view of the extent of the impending changes, one HCP expressed that it would be challenging to expect additional changes to be accepted too. However, it was also mentioned by a HCP that as so much change was already taking place, it would be better to introduce all the changes at once:

We are already experiencing so much change, we can cope with this change as well. (HCP)

Frustration was observed among some HCPs. As the merger was creating insecurity concerning their position as employees, this made additional changes even harder because they need to focus on themselves first before focusing on changes in the organisation. In contrast, internal staff stated that the merger was also having a positive impact on the organisation. Since the formerly two hospitals had different cultures, merging them had a positive influence on collaboration:

Combining the teams at the VUmc and AMC is actually quite a relief. Bringing the two cultures together creates positive energy … I always characterised the culture of AMC as more individualistic and the culture of the VUmc more as a group … the merger has been very healthy. (Organisational staff)

With regard to facilitators, as mentioned by the external supplier, the national policy of the Dutch Ministry of Health, Welfare and Sport was focused on digital healthcare at the time of this study. This created momentum and drove a sense of urgency to innovate for healthcare organisations, as illustrated by the following quote:

As soon as there is a sense of urgency, you see that change suddenly takes place. That was also the case with COVID-19, digitalisation was rapidly embraced. (Organisational staff)

From the organisation itself, at department and division level, support was perceived as a facilitator for the stakeholders involved:

The ‘gender’ board and different department heads widely support the implementation of the PDCP. In addition, attention was given to the project on the policy day. This support is very visible, which I think is very important. (Organisational staff)

To ensure sustainable implementation, it became clear that support is needed at various levels, including from the Board of Directors. As a final facilitator, patients expressing a clear desire and need for more digital information by requesting access to patient information in this way could also act as a major incentive to implementation. Within these three patient groups, the majority of the patients are relatively young and digital natives. This generates a stronger demand for more digitalisation, which ultimately creates the motivation to innovate and change.

In my opinion, the gender outpatient clinic is one of the outpatient clinics that already has a high level of digitalisation, in terms of video consultation. (HCP)

Collaboration within the organisation (support)

At the start of the PDCP project, the extent of collaboration between all stakeholders involved was inadequate, and there was a certain unwillingness to open up to cooperation with others.

Due to the sheer size of Amsterdam UMC, I think that we still tend to work from individual, isolated perspectives. (Organisational staff)

In addition, this was also reinforced by disparities in the definitions of common terms used in mutual communication. Stakeholders gave different definitions of important interpretive concepts such as implementation and maintenance.

To ensure successful implementation and embedding in the organisation, clear process agreements were necessary between the external supplier, EvA service centre, ICT (information and communication technology) and the end users. Most of the supporting staff regarded these agreements to be lacking during the pilot implementation. It was experienced as important to also describe the division of roles and ownership. Moreover, it was often unclear who held responsibility for what part of the process and which tasks were assigned to which department. This emphasises the importance of agreeing and coordinating these points beforehand:

I think we should have paid more attention to project adoption and assurance from the beginning. We should have appointed someone within the project team to be responsible for this adoption and assurance. (Organisational staff)

At a certain point in phase 1, the project team had been formed with the appropriate representatives from medical and supporting staff to collaborate in the design of the PDCP. This collaboration was perceived as very useful and resulted in regular multidisciplinary meetings during the implementation phase, in which both positive findings and issues were shared. This ensured a very approachable collaboration.

The communication, care support, strategy and innovation departments, and the EvA service centre worked together … This greatly contributed to making this project a success, because all stakeholders were involved. … This is a very positive development and should be repeated in future projects. (Organisational staff)

A complementary clinical and operational leader were both assigned from the start, based on personal motivation and availability. As indicated by multiple stakeholders, this facilitated good cooperation, both substantively and operationally with prospective implementation in mind.

Discussion

We investigated factors influencing the implementation of a digital care pathway that was developed using experience-based codesign. We identified four themes: (1) ‘stakeholders’ perceptions of the PDCP’ (eg, duration of first activation, perceived usefulness PDCP); (2) ‘implementation by healthcare professionals in daily practice’ (eg, individuals express resistance to change, providing feedback on adoption); (3) ‘organisational readiness’ (eg, lack of resources, patients providing incentives for change); and (4) ‘collaboration within the organisation’ (eg, mutual communication, multidisciplinary codesign). Main barriers mentioned by patients were duration of first activation and necessity for up-to-date content. In addition, the most facilitating factor for patients was user-friendliness.

There were several common factors among stakeholders (eg, user friendliness, lack of resources and rigidity of the organisation). For example, all stakeholders agreed that more resources are needed for sustainable implementation. However, what resources were needed depended on the stakeholder (eg, funds, time, workforce). It was also expressed that this funding should be made available by the central board.

Nilsson et al also stated that it is recommended to have sufficient support from the board of directors and align the organisation from the initial stage of the development and implementation of an innovation to embed the innovation well in the organisation. In line with that, the board of directors should facilitate sufficient capacity and resources.40 These findings imply that it is vital to invest in sufficient resources from the start of such a project.3 Previous literature has also shown that resistance to change was not only found among HCPs, but also among the broader stakeholders group.41 Since resistance to change could cause an implementation to fail, the impact of this perceived barrier should be minimised.42 Explaining what the precise changes are for stakeholders, including end users, emphasising the added value, and involving different stakeholder groups, including patients, during the design from an early stage can help alleviate resistance.32 42–49

The perception of rigidity of the organisation, mainly due to the size of the organisation, was also mentioned by different stakeholders. This rigidity negatively influenced the decision-making speed and action taking. This was also experienced in the collaboration by the external supplier. Granja et al50 stated that this rigidity is typical for a healthcare organisation and adapting, and adjusting to eHealth interventions is not suited for these kind of organisations. Our findings nuanced this, as there was a sense of urgency that created the first steps of organisational readiness and willingness of HCPs to implement this innovation, since the added value was certainly recognised and acknowledged by almost all stakeholders. However, as also mentioned by Threapleton et al, implementing a change in an organisation can take several years.51 Therefore, transformation requires organisational readiness for change among healthcare professionals, including cultural change.52–55

There were also remarkable differences between stakeholders, for example, regarding the impact of the ongoing merger. Some stakeholders perceived this as an opportunity for change, while others, due to the merger, sometimes felt it was too much change at once. These differences were also reflected in terms of the facilities required.

Strengths and limitations

A major strength of this study is that we incorporated the perspectives of all stakeholders, including the non-medical (supporting) staff. We collected views and opinions of those directly involved in implementation from an organisational perspective and those of external stakeholders (patients and supplier). The authors chose to include the patient’s perspective through the verification of findings with data from existing patient interviews, since these interviews had already been conducted within the same scope. Other recent studies only focused on the organisational perspective or just on the experiences with eHealth implementation and adoption among healthcare professionals and patients. The inclusion of all perspectives contributed to the representativeness of this study, which took place in the complexity of a large organisation.3 56–58

Other strengths related to the fact that this study adds to the limited implementation studies by providing insight into the development and the subsequent implementation process in daily practice.59 The use of codesign in this project enhanced successful implementation as it ensured input and feedback by stakeholders and end users and created commitment for further implementation. Furthermore, the data were conducted according to the framework of CFIR for examining the numerous influences during the implementation of complex innovations, making comparison with other studies possible.60 The final strength of this study is that investigator triangulation was assured, since the interviews were conducted by multiple researchers.61

This study also has some limitations. First, participants were recruited using snowball sampling techniques, which could have resulted in selection bias.62 However, this effect was minimised by including stakeholders both familiar and unfamiliar with the PDCP tool. Second, analyses of qualitative data relies on the subjective interpretation of researchers. To reduce this bias, two researchers independently analysed the data that positively affected the validity.63 Third, due to the COVID-19 pandemic, the interviews were conducted both by video call and face to face, which may have led to differences in understanding and data interpretation between the interviews. Lastly, CFIR’s model included the relevant domains, intervention, outer setting, inner setting, individuals involved and the implementation, but does not include the patients’ perspectives as a separate domain, which is an identified gap in CFIR.60 To include this domain, Flottorp’s model could be considered for implementation factors in future comparable studies.64 In this study, the authors aspired to add this perspective by using data from patient interviews.

Practical implications

Based on the results of this study, we have formulated three practical implications to improve and enhance implementation of a PDCP. First, it is essential to translate the facilitating aspects into evidence-based implementation strategies.65 For example, the added value and effectiveness of the PDCP for HCPs and patients should be made clear to every stakeholder involved, in a manner tailored to stakeholder. Second, it is vital to establish a multidisciplinary team comprising a wide selection of stakeholders (eg, patients, technicians, HCPs and communication experts) right from the start of the project. This facilitates effective collaboration in the subsequent adoption and implementation phase. In addition, it is recommended to create change readiness and take advantage of momentum if change readiness has already been created. This may take the form, for instance, of implementing innovations in parallel or as part of ongoing organisational changes, such as the hospital’s merger.

Future research

At the time of data collection, the PDCP was implemented as a pilot phase in three departments. The results of this research contributed to the further cocreation and implementation process and were used to formulate appropriate implementation strategies. Subsequent process and effect evaluation research is planned to assess the full implementation of the PDCP. In addition, conducting research on comparable implementation processes in other departments or organisations to elaborate on the generalisability of our findings is also recommended. This research showed the importance of a tool being user-friendly and useful. However, more research is needed on usability for a diverse range of users.

Conclusion

In conclusion, this qualitative study has identified factors facilitating or hindering the implementation of a PDCP in a large Dutch academic hospital. There were several similarities between the experienced facilitators or barriers among all stakeholders (eg, user friendliness, lack of resources and rigidity of the organisation). Influential factors were related to the perceived usefulness of PCDP yet need for keeping the content up to date. Resistance to change and expected time investment hindered implementation, while the possibility of incorporation in daily practice worked as facilitator. Organisational readiness worked both as facilitator and barrier, and clear process agreements and communication are needed in place for strong collaboration. In our case, the cocreation process during adaptive development facilitated this collaboration. Findings were echoed by patients, and their main barriers were duration of first activation and necessity for up-to-date content. Our findings emphasise the importance of gaining insight into the various perspectives of stakeholder groups, including patients. It is recommended to tailor implementation strategies for each stakeholder group, adjusted to their perceived facilitators and barriers. Our findings can be used to improve and enhance PDCP implementation and tailor the development and improvement of other digital patient communication tools.

Supplementary Material

Reviewer comments
Author's manuscript

Acknowledgments

The authors would like to thank all stakeholders for their participation in this study.

Footnotes

Contributors: Each author made substantial contributions to the work and writing of this manuscript: FH, FvN, MM, MB and MdB participated in the design of the study; FH and JS were responsible for data collection; FH, JS and FvN participated in the analysis of the interviews; writing the initial draft: FH and JS; review and editing: FH, FvN, MM, MB and MdB. All team members approved the version of the manuscript submitted for publishing and have agreed to be accountable for all aspects of the work. All authors had full access to all the study data and take responsibility for the integrity of the data and accuracy of the data analysis.

Funding: This study was supported by ZonMw program ‘Actieonderzoek Innovatieve Zorg’ (grant number: 516006007), the Netherlands.

Disclaimer: The funder had no role in data collection, data analysis, the decision to publish or manuscript preparation.

Competing interests: None declared.

Patient and public involvement: Patients and/or the public were involved in the design, or conduct, or reporting, or dissemination plans of this research. Refer to the Methods section for further details.

Provenance and peer review: Not commissioned; externally peer reviewed.

Supplemental material: This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

Data availability statement

No data are available. The character and identifiability of the qualitative data do not allow for distribution. All data relevant to the study are included in the article or uploaded as supplementary information.

Ethics statements

Patient consent for publication

Not applicable.

Ethics approval

The need for approval for this study was waived by the Medical Ethical Committee at Amsterdam UMC, Vrije Universiteit Amsterdam (2019.651). Participation was voluntary, and all respondents gave verbal and written informed consent before taking part in our project. Participants gave informed consent to participate in the study before taking part.

References

  • 1.van Weert N. Personalized specialty care: value-based healthcare Frontrunners from the Netherlands: Springer nature.
  • 2.Nyhof BB, Jameel B, Dunn S, et al. Identifying strategies to implement patient-centred care for women: qualitative interviews with women. Patient Educ Couns 2020;103:1422–7. 10.1016/j.pec.2020.02.008 [DOI] [PubMed] [Google Scholar]
  • 3.Kooij L, Groen WG, van Harten WH. Barriers and facilitators affecting patient portal implementation from an organizational perspective: qualitative study. J Med Internet Res 2018;20:e183. 10.2196/jmir.8989 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Liljamo P, Säilynoja H, Tuomikoski K, et al. Patient-Centered development of a digital care pathway for arrhythmia patients. Stud Health Technol Inform 2020;275:127–31. 10.3233/SHTI200708 [DOI] [PubMed] [Google Scholar]
  • 5.Mühlbacher AC, Nübling M. Analysis of physicians' perspectives versus patients' preferences: direct assessment and discrete choice experiments in the therapy of multiple myeloma. Eur J Health Econ 2011;12:193–203. 10.1007/s10198-010-0218-6 [DOI] [PubMed] [Google Scholar]
  • 6.Lundberg N, Koch S, Hägglund M, et al. My care pathways - creating open innovation in healthcare. Stud Health Technol Inform 2013;192:687–91. [PubMed] [Google Scholar]
  • 7.Epstein RM, Duberstein PR, Fenton JJ, et al. Effect of a patient-centered communication intervention on Oncologist-Patient communication, quality of life, and health care utilization in advanced cancer: the voice randomized clinical trial. JAMA Oncol 2017;3:92–100. 10.1001/jamaoncol.2016.4373 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Harnas S, van der Kraan J, Knops A, et al. Samen Beslissen Met de zorgverlener over welke zorg Het beste past, 2017. [Google Scholar]
  • 9.Barry MJ, Edgman-Levitan S. Shared decision making — the pinnacle of patient-centered care. N Engl J Med Overseas Ed 2012;366:780–1. 10.1056/NEJMp1109283 [DOI] [PubMed] [Google Scholar]
  • 10.Elwyn G, Frosch D, Thomson R, et al. Shared decision making: a model for clinical practice. J Gen Intern Med 2012;27:1361–7. 10.1007/s11606-012-2077-6 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Stiggelbout AM, Pieterse AH, De Haes JCJM. Shared decision making: concepts, evidence, and practice. Patient Educ Couns 2015;98:1172–9. 10.1016/j.pec.2015.06.022 [DOI] [PubMed] [Google Scholar]
  • 12.Tseng EK, Hicks LK, Care VB. Value based care and patient-centered care: divergent or complementary? Curr Hematol Malig Rep 2016;11:303–10. 10.1007/s11899-016-0333-2 [DOI] [PubMed] [Google Scholar]
  • 13.Elwyn G, Dehlendorf C, Epstein RM, et al. Shared decision making and motivational interviewing: achieving patient-centered care across the spectrum of health care problems. Ann Fam Med 2014;12:270–5. 10.1370/afm.1615 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Slade M. Implementing shared decision making in routine mental health care. World Psychiatry 2017;16:146–53. 10.1002/wps.20412 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Astin F, Closs SJ, McLenachan J, et al. The information needs of patients treated with primary angioplasty for heart attack: an exploratory study. Patient Educ Couns 2008;73:325–32. 10.1016/j.pec.2008.06.013 [DOI] [PubMed] [Google Scholar]
  • 16.Connell A, Black G, Montgomery H, et al. Implementation of a digitally enabled care pathway (Part 2): qualitative analysis of experiences of health care professionals. J Med Internet Res 2019;21:e13143. 10.2196/13143 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Friis K, Lasgaard M, Osborne RH, et al. Gaps in understanding health and engagement with healthcare providers across common long-term conditions: a population survey of health literacy in 29,473 Danish citizens. BMJ Open 2016;6:e009627. 10.1136/bmjopen-2015-009627 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.van der Weijden T, Post H, Brand PLP, et al. Shared decision making, a buzz-word in the Netherlands, the PACE quickens towards nationwide implementation…. Z Evid Fortbild Qual Gesundhwes 2017;123-124:123-124:69–74. 10.1016/j.zefq.2017.05.016 [DOI] [PubMed] [Google Scholar]
  • 19.Pithara C, Farr M, Sullivan SA, et al. Implementing a digital tool to support shared care planning in community-based mental health services: qualitative evaluation. J Med Internet Res 2020;22:e14868. 10.2196/14868 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Everink IHJ, van Haastregt JCM, Kempen GIJM, et al. [Challenges in geriatric rehabilitation: the development of an integrated care pathway]. Tijdschr Gerontol Geriatr 2015;46:104–12. 10.1007/s12439-015-0125-5 [DOI] [PubMed] [Google Scholar]
  • 21.Ossebaard HC, Van Gemert-Pijnen L. eHealth and quality in health care: implementation time. Int J Qual Health Care 2016;28:415–9. 10.1093/intqhc/mzw032 [DOI] [PubMed] [Google Scholar]
  • 22.Nilsen ER, Stendal K, Gullslett MK. Implementation of eHealth technology in community health care: the complexity of stakeholder involvement. BMC Health Serv Res 2020;20:395. 10.1186/s12913-020-05287-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Seckler E, Regauer V, Rotter T, et al. Barriers to and facilitators of the implementation of multi-disciplinary care pathways in primary care: a systematic review. BMC Fam Pract 2020;21:113. 10.1186/s12875-020-01179-w [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Vreugdenhil MMT, Ranke S, de Man Y, et al. Patient and health care provider experiences with a recently introduced patient portal in an academic hospital in the Netherlands: mixed methods study. J Med Internet Res 2019;21:13743. 10.2196/13743 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Damschroder LJ, Hagedorn HJ. A guiding framework and approach for implementation research in substance use disorders treatment. Psychol Addict Behav 2011;25:194–205. 10.1037/a0022284 [DOI] [PubMed] [Google Scholar]
  • 26.Evans-Lacko S, Jarrett M, McCrone P, et al. Facilitators and barriers to implementing clinical care pathways. BMC Health Serv Res 2010;10:182. 10.1186/1472-6963-10-182 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Nilsen P. Making sense of implementation theories, models and frameworks. Implement Sci 2015;10:53. 10.1186/s13012-015-0242-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28.Swinkels ICS, Huygens MWJ, Schoenmakers TM, et al. Lessons learned from a living lab on the broad adoption of eHealth in primary health care. J Med Internet Res 2018;20:e83. 10.2196/jmir.9110 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Frambach RT, Schillewaert N. Organizational innovation adoption: a multi-level framework of determinants and opportunities for future research. J Bus Res 2002;55:163–76. 10.1016/S0148-2963(00)00152-1 [DOI] [Google Scholar]
  • 30.Greenhalgh T, Wherton J, Papoutsi C, et al. Beyond adoption: a new framework for theorizing and evaluating Nonadoption, abandonment, and challenges to the scale-up, spread, and sustainability of health and care technologies. J Med Internet Res 2017;19:e367. 10.2196/jmir.8775 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Seljelid B, Varsi C, Solberg Nes L, et al. A digital patient-provider communication intervention (InvolveMe): qualitative study on the implementation preparation based on identified facilitators and barriers. J Med Internet Res 2021;23:e22399. 10.2196/22399 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Hulter P, Pluut B, Leenen-Brinkhuis C, et al. Adopting patient portals in hospitals: qualitative study. J Med Internet Res 2020;22:e16921. 10.2196/16921 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.Heijsters FACJ, van Breda FGF, van Nassau F, et al. A pragmatic approach for implementation of value-based healthcare in Amsterdam UMC, the Netherlands. BMC Health Serv Res 2022;22:550. 10.1186/s12913-022-07919-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34.Amsterdam UMC LV . Inloggen in Mijn dossier Met DigiD, 2021. [Google Scholar]
  • 35.Innovations S . MediMapp care pathway n.d. Available: https://www.soulve.nu/medimapp-care-pathway/
  • 36.Sharma G. Pros and cons of different sampling techniques. International journal of applied research 2017;3:749–52. [Google Scholar]
  • 37.Damschroder LJ, Aron DC, Keith RE, et al. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implementation Science 2009;4. 10.1186/1748-5908-4-50 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Breimaier HE, Heckemann B, Halfens RJG, et al. The consolidated framework for implementation research (CFIR): a useful theoretical framework for guiding and evaluating a guideline implementation process in a hospital-based nursing practice. BMC Nurs 2015;14:43. 10.1186/s12912-015-0088-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39.Fusch P, Ness L. Are we there yet? data saturation in qualitative research. The Qualitative Report 2015;20:1408–16. 10.46743/2160-3715/2015.2281 [DOI] [Google Scholar]
  • 40.Nilsson K, Bååthe F, Erichsen Andersson A, et al. The need to succeed - learning experiences resulting from the implementation of value-based healthcare. Leadersh Health Serv 2018;31:2–16. 10.1108/LHS-08-2016-0039 [DOI] [PubMed] [Google Scholar]
  • 41.Nilsen ER, Dugstad J, Eide H, et al. Exploring resistance to implementation of welfare technology in municipal healthcare services - a longitudinal case study. BMC Health Serv Res 2016;16:657. 10.1186/s12913-016-1913-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42.Damschroder LJ, Aron DC, Keith RE, et al. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci 2009;4:50. 10.1186/1748-5908-4-50 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 43.Mair FS, May C, Finch T, Murray E, et al. Understanding the implementation and integration of e-health services. J Telemed Telecare 2007;13:36–7. 10.1258/135763307781645112 [DOI] [Google Scholar]
  • 44.Vonk Noordegraaf A, Huirne JAF, Pittens CA, et al. eHealth program to empower patients in returning to normal activities and work after gynecological surgery: intervention mapping as a useful method for development. J Med Internet Res 2012;14:e124. 10.2196/jmir.1915 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 45.Ariens LF, Schussler-Raymakers FM, Frima C, et al. Barriers and facilitators to eHealth use in daily practice: perspectives of patients and professionals in dermatology. J Med Internet Res 2017;19:e300. 10.2196/jmir.7512 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 46.Safi S, Thiessen T, Schmailzl KJ. Acceptance and resistance of new digital technologies in medicine: qualitative study. JMIR Res Protoc 2018;7:e11072. 10.2196/11072 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 47.Gotlib Conn L, McKenzie M, Pearsall EA, et al. Successful implementation of an enhanced recovery after surgery programme for elective colorectal surgery: a process evaluation of champions' experiences. Implement Sci 2015;10:99. 10.1186/s13012-015-0289-y [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 48.Herbert G, Sutton E, Burden S, et al. Healthcare professionals' views of the enhanced recovery after surgery programme: a qualitative investigation. BMC Health Serv Res 2017;17:617. 10.1186/s12913-017-2547-y [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 49.Svedberg P, Arvidsson S, Larsson I, et al. Barriers and Enablers affecting successful implementation of the electronic health service Sisom: multicenter study of child participation in pediatric care. J Med Internet Res 2019;21:e14271. 10.2196/14271 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 50.Granja C, Janssen W, Johansen MA. Factors determining the success and failure of eHealth interventions: systematic review of the literature. J Med Internet Res 2018;20:e10235. 10.2196/10235 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 51.Threapleton DE, Chung RY, Wong SYS, et al. Care toward the end of life in older populations and its implementation facilitators and barriers: a scoping review. J Am Med Dir Assoc 2017;18:1000–9. 10.1016/j.jamda.2017.04.010 [DOI] [PubMed] [Google Scholar]
  • 52.Lee TH. Putting the value framework to work. N Engl J Med 2010;363:2481–3. 10.1056/NEJMp1013111 [DOI] [PubMed] [Google Scholar]
  • 53.Weiner BJ. A theory of organizational readiness for change. Implement Sci 2009;4:67. 10.1186/1748-5908-4-67 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 54.Erichsen Andersson A, Bååthe F, Wikström E, et al. Understanding value-based healthcare – an interview study with project team members at a Swedish university hospital. Journal of Hospital Administration 2015;4:64. 10.5430/jha.v4n4p64 [DOI] [Google Scholar]
  • 55.van der Nat PB. The new strategic agenda for value transformation. Health Serv Manage Res 2022;35:189-193. 10.1177/09514848211011739 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 56.Cijvat CD, Cornet R, Hägglund M, Ms H. Factors influencing development and implementation of patients' access to electronic health Records-A comparative study of Sweden and the Netherlands. Front Public Health 2021;9:621210. 10.3389/fpubh.2021.621210 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 57.Askari M, Tam JLYY, Aarnoutse MF, et al. Perceived effectiveness of clinical pathway software: a before-after study in the Netherlands. Int J Med Inform 2020;135:104052. 10.1016/j.ijmedinf.2019.104052 [DOI] [PubMed] [Google Scholar]
  • 58.McGinn CA, Grenier S, Duplantie J, et al. Comparison of user groups' perspectives of barriers and facilitators to implementing electronic health records: a systematic review. BMC Med 2011;9:46. 10.1186/1741-7015-9-46 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 59.Stone AB, Yuan CT, Rosen MA, et al. Barriers to and facilitators of implementing enhanced recovery pathways using an implementation framework: a systematic review. JAMA Surg 2018;153:270–9. 10.1001/jamasurg.2017.5565 [DOI] [PubMed] [Google Scholar]
  • 60.Morgan D, Kosteniuk J, O'Connell ME, et al. Barriers and facilitators to development and implementation of a rural primary health care intervention for dementia: a process evaluation. BMC Health Serv Res 2019;19:709. 10.1186/s12913-019-4548-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 61.Gray DE. Doing research in the real world. 2 ed. Londen, UK: Sage, 2019. [Google Scholar]
  • 62.Parker C, Scott S, Geddes A. Snowball sampling. SAGE research methods foundations, 2019. [Google Scholar]
  • 63.Mason J. Qualitative researching: SAGE publications, 2017. [Google Scholar]
  • 64.Flottorp SA, Oxman AD, Krause J, et al. A checklist for identifying determinants of practice: a systematic review and synthesis of frameworks and taxonomies of factors that prevent or enable improvements in healthcare professional practice. Implementation Science 2013;8:1–11. 10.1186/1748-5908-8-35 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 65.Powell BJ, Waltz TJ, Chinman MJ, et al. A refined compilation of implementation strategies: results from the expert recommendations for implementing change (ERIC) project. Implement Sci 2015;10:21. 10.1186/s13012-015-0209-1 [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplementary data

bmjopen-2022-065778supp001.pdf (344.1KB, pdf)

Supplementary data

bmjopen-2022-065778supp002.pdf (227.6KB, pdf)

Reviewer comments
Author's manuscript

Data Availability Statement

No data are available. The character and identifiability of the qualitative data do not allow for distribution. All data relevant to the study are included in the article or uploaded as supplementary information.


Articles from BMJ Open are provided here courtesy of BMJ Publishing Group

RESOURCES