Table 2.

Identified current barriers for the KLIK PROM portal implementation using the CFIR-ERIC matching tool

CFIR domain	CFIR barrier	What is already done?	Why still a barrier?
Intervention characteristics	1. Relative advantage	Overall, clinicians working with the KLIK PROM portal think it is a valuable tool to monitor PROMs in their patients [12, 29, 35] The advantages about the KLIK PROM portal and discussing PROMs in the consultation room are spread during conferences and in scientific papers	Some stakeholders are reluctant to change and do not see the advantages of using PROMs, or suggest that alternative solutions (e.g., administering PROMs using EHRs or on paper) can be used
Intervention characteristics	2. Adaptability	KLIK is a very flexible system where many individual wishes of the multidisciplinary teams can be met (e.g., different PROMs for different patient groups at different timeslots, for different ages, various forms of feedback different for specific clinicians) To make KLIK as user friendly as possible. For example, KLIK is available in different languages and proxy reported PROMs are offered for patients with disabilities [13]	Clinicians prefer the intervention as tailored as possible. A standard set of PROMs is currently being offered to patients automatically based on age and patient group, not yet on an individual patient level (selecting specific PROMs per individual patient per visit)
Intervention characteristics	3. Complexity	The KLIK PROM portal is easy to use as a result of its origin in pediatrics. A recent evaluation study shows that the majority of clinicians (72%) think KLIK is easy to use [35] Together with the use of the KLIK PROM portal, hospitals receive support and advise from the KLIK expert team during all steps of the implementation (see Fig. 3)	For some clinicians KLIK remains complex to use, for example if they are not familiar with ICT. In addition, it requires additional actions, because clinicians need to actively motivate patients and sometimes send out the PROMs to their patients
Intervention characteristics	4. Cost	KLIK is being offered at low costs, as we are a non-profit organization, and alternative portals are often more expensive	Within healthcare there are often insufficient financial resources. Therefore, some teams still decide to refrain from using PROMs because of the additional costs
Inner setting	5. Structural characteristics	In general, KLIK is being implemented bottom-up, where small multidisciplinary teams show their interest in using KLIK	Hospitals are large organizations, and obtaining permission to change existing workflows can be a long process The board of the hospital might not be aware of bottom-up processes and can, therefore, be perceived as a barrier in larger scale implementation
Inner setting	6. Tension for change	Champions (clinicians who are enthusiastic about using KLIK) can explain the added value of using PROMs in clinical practice and persuade colleagues in trying out KLIK as well	Some clinicians do not see the current situation (not using PROMs in clinical practice) in a need of change
Inner setting	7. Compatibility	At the start of the KLIK implementation, the KLIK expert team advices on how to fit KLIK best into the existing workflow Recently, in four hospitals, a front-end integration with KLIK and the EHR (Epic© and HiX©) is realized	A study showed that a perceived barrier for stakeholders was compatibility (24% of clinicians indicated that the KLIK method did not fit well with current routines) [29]. To make it better fit with existing workflows, KLIK should be integrated into the EHR in all hospitals
Inner setting	8. Organizational incentives & rewards	Clearly communicate incentives (e.g., communication tool, improvement of quality of care, data can also be used for scientific purposes) of using PROMs in clinical practice for both patients and clinicians	Sometimes there are no incentives in the opinion of multidisciplinary teams and they, therefore, do not promote the use of PROMs in clinical practice
Inner setting	9. Leadership engagement	License agreements are signed by an authorized signatory and it therefore approves the implementation	Key organizational leaders or managers could show more commitment and involvement in KLIK by promoting it actively. In addition, in the current situation, they are not held accountable for implementation of the innovation
Inner setting	10. Available resources	KLIK has received several grants for the implementation and developed a business model to provide financial resources for the KLIK expert team in addition to the external resources	There is no structural funding yet for the KLIK expert team. To continue implementing KLIK, we are currently working on a new business model where we are not dependent on external funds, but can provide the use of KLIK at low costs For clinicians, money or time to discuss PROMs can be a barrier in implementing PROMs
Characteristics of individuals	11. Individual stage of change	When clinicians experience benefits from implementing PROMs, they become more enthusiastic By training clinicians, the skills necessary to implement and discuss PROMs are provided During the annual evaluation meetings we identify clinicians that do not perceive enough benefits or forget using KLIK. These meetings keep the clinicians focused on the goal of discussing PROMs From the perspective of patients, information letters, flyers, and educational videos are provided to give them the skills to complete and discuss PROMs. In addition, focus groups are held to explore their experiences regarding KLIK, in order to further optimize KLIK	Clinicians that do not feel skilled or enthusiastic about using the innovation in a sustained way are resistant to use the intervention. Feedback from patients includes that PROMs are not discussed by the clinician, they sometimes do not see the added value, and PROMs can be long and repetitive
Process of implementation	12. Champions	Most teams have a champion (an individual who support the KLIK implementation in a way that helps to overcome indifference of resistance by key stakeholders) who is motivated to start implementing KLIK for their patients	Some champions seem to have insufficient influence to convince their colleagues
Process of implementation	13. Engaging (Key stakeholders)	Clinicians are involved in the entire implementation process Also patients are more and more involved in the KLIK PROM portal, e.g., by asking their opinion in both qualitative and quantitative studies, developing educational videos to prepare them for the outpatient consultation, and by collaborating with patient associations	Patient engagement can be increased, for example, at the start of the implementation to explore the relevant PROs for patients