Toward hybrid mental health care: an early health technology assessment based on stakeholder analysis and economic simulations

Abstract

Objectives

Combining digital and traditional treatment elements as part of a hybrid care delivery model may strengthen the sustainability of mental health care. However, implementation problems permeate efforts of digitalizing routine care. The objective of this formative study is to explore current unmet stakeholder needs and the potential value of hybrid care models for the treatment of severe mental disorders, in order to guide further research and inform practical implementation strategies.

Methods

We applied Step Up, an early health technology assessment (eHTA) framework. We performed (a) a template analysis of qualitative data from a stakeholder workshop mapping unmet needs, potential effects of hybrid care models, and conditions relevant for implementation and (b) exploratory economic simulations of the impact of increasing video consultation use in routine services on quantitative sustainability parameters.

Results

The template analysis revealed stakeholder-anticipated value from hybrid care models across all domains: patient, clinician, and next-of-kin user experiences; clinical variables; organizational setup; and economy. Two recommendations for enhancing implementation were identified: promoting nuanced perceptions of digitalized care and developing clinician incentives. The simulations demonstrated that a moderate increase in video consultations may impact clinical capacity, as well as economic and environmental sustainability.

Conclusions

This study provides early decision support concerning hybrid care models during the conceptual stage of service innovation. It reveals that conservative changes toward digitalized practices can create new opportunities for organizing services and have a substantial impact on sustainability parameters. Successful implementation across clinical contexts requires continuous consideration of the clinician user perspective, starting from the conceptual phase.

Introduction

Globally, public mental health systems struggle with scarce resources while the demand for services is increasing (1). Moreover, there are significant disparities in access to care (2–4) and widespread challenges with recruitment and retainment of clinicians (5–7). There is a growing need for more sustainable models of organizing and delivering services (8).

Digitalization is a strategy to reshape care models within current resource constraints (9–13). Existing literature indicates great promise for the integration of accessible digital tools in mental health, such as video consultations to deliver therapy (14–16) and mobile apps to support therapeutic interventions or monitor clinically relevant variables (17–20).

The potential of digitalization extends to services targeting severe mental disorders, such as psychotic and bipolar disorders, but there is still a lack of knowledge concerning these patient populations (21–24). Findings from previous work suggest that no service users should be excluded from the target groups of digitalized care based on diagnosis alone (25;26). Insights gained from studying this population are often applicable to individuals with less severe conditions, making this group a valuable reference point for advancing digitalized mental health care overall.

However, public health care is characterized by slow implementation processes, (27) and adoption of digitalized tools in routine care remains low (28–31). Many health care innovation projects are costly, and there is a high rate of failure (32–34). This indicates great losses related to the waste of resources spent on research, development, and implementation efforts (8;35). Moreover, digitalized care is commonly perceived as a deviation from in-person treatment and triggers resistance toward adoption, especially among clinicians, who commonly hold more critical views than service-users (36;37).

Hybrid care models combine digital and traditional formats. Since they offer benefits from both types of care delivery, digital elements may be perceived as more feasible and acceptable (36). Hybrid care models facilitate more flexible and need-based treatment and new ways of allocating resources and structuring workflows (38–41). Nevertheless, they are uncommon in Norwegian routine mental health care and remain at a conceptual stage of service innovation. Services must make informed decisions about further research and implementation to reduce the risk of failed investments, but conclusive evidence regarding the value of different hybrid care models and how to successfully implement them is currently lacking (42).

Adaptations of traditional research approaches are needed to efficiently guide the next steps for such service innovations early on. There is a rapidly growing academic interest in early health technology assessment (eHTA) (43;44), a forward-looking methodology that enables analyses while existing evidence is scarce. eHTA allows flexible compositions of various methods and generates formative decision support, which can help advance the further innovation processes of new concepts, such as hybrid care models.

The objective of this study is to explore unmet needs of service users, clinicians, and other actors affected by the way health care services are organized, estimate the potential value of hybrid care models in treatment of severe mental disorders through eHTA methodology, and guide further research and inform practical implementation strategies.

Methods

The structure of this manuscript adheres to the main elements of the SQUIRE guidelines (35), a framework adapted to reporting research on systemic health care improvements.

Context

The general context of the study is the Norwegian public specialist health care for severe mental disorders. Norway is a high-income country with a comprehensive welfare system and low private health expenditure (45). The specialist psychiatric services have a mandate to serve the population equally across social strata and geography. To access specialist health care, service users must experience moderate to severe symptoms and receive a referral from their general practitioner or another primary health actor (46). The sector struggles with limited availability of clinicians, and health authorities call for new sustainable strategies of organizing and delivering care (47;48).

The background of this assessment is a series of three pilot studies conducted by the research group to investigate the integration of a mobile app into treatment for severe mental disorders (20;25). The pilots revealed a sharp contrast between the readiness expressed at research sites prior to the research project and the low levels of activity during the study period. This was also observed at the third site, which was characterized by beneficial practical circumstances and a professional environment explicitly supportive of digitalization. There were reports of concrete problems that partly explained challenges with coherent participation, such as technical bugs and unforeseen changes in the health condition of patients. More importantly, however, there seemed to be fundamental systemic barriers to integrating technology into routine services. Although these observations were made during pilots, representing much earlier phases than implementation, they were interpreted as symptoms of the recurrent implementation problems in the digital mental health field and termed an implementation paradox. See Supplementary Material 1 for a description of this phenomenon and details about the pilot study procedures.

The research group’s paradoxical experiences with the unsuccessful integration of technology highlighted the need to revisit the innovation process. Consequently, we chose to proceed with eHTA to gain a better understanding of how to address these challenges and effectively implement hybrid care models.

Hybrid care models

We define hybrid care models as routine services that integrate digital tools in their delivery of care based on clinician evaluation and current needs of the service user. The concept focuses on creating a new digitalized service that enables more flexible communication and treatment through the purposeful use of accessible technologies. We selected the integration of video consultations and mobile apps in treatment for severe mental disorders as a case for the evaluation to explicitly demonstrate and discuss potential value and implementation challenges. Although this case includes two digital tools, this assessment explores hybrid care models in general, rather than evaluating video consultations and mobile apps specifically.

Methodological framework

eHTA is a methodology used to evaluate the potential value of health technologies from early phases of development, while scientific evidence is scarce. eHTAs also support simultaneous iterative testing with a heavy emphasis on aligning the innovation with user needs. The output of eHTAs is formative decision support, guiding the next steps in the development, research, implementation, or discontinuation of health technology based on validated estimates. This process reduces risk related to research waste and implementation failure. eHTAs can combine available data sources and methods that are suitable for specific knowledge gaps (49;50). Stakeholder involvement is typically a core element and can comprise a range of different data collection activities, such as workshops, surveys, and key informant interviews. In addition, methods such as scenario building and analysis, economic simulations/sensitivity analyses, and usability tests can comprise elements in an eHTA process.

We applied the eHTA framework Step Up (51) in this study. Step Up is an evidence-based method that assesses innovations across four domains: user, clinical, organizational, and economic. The framework includes tools designed to gather insights from a diverse range of stakeholders affected by the innovation while also considering potential risks and barriers. Step Up is designed to be applied in a proactive and iterative manner, developing formative knowledge that advances the innovation and clarifies what is needed for successful implementation and realization of intended value. Step Up is flexible and can be used in various processes, ranging from early need-mapping and problem-defining, to informing procurement strategy. The Step Up handbook and material supporting workshop activities are publicly available from the Norwegian Centre for E-health Research’s website (52).

Co-creation is integral in this framework and may be defined as the process in which end-users of a service are included as active partners in defining challenges and designing the service, based on their experience and competences (53). Co-creation challenges the traditional top-down governance models in health care, including service-users in the design, delivery, and maintenance of health innovations (54). In this eHTA, we used scenario drafting to construct realistic and tangible examples of hybrid care models prior to the main data collection. Scenarios were co-created with two experts by experience recruited through the Bipolar Association Norway, who applied insights concerning unmet service user needs in current care models. The process resulted in several hypothetical scenarios illustrating enhanced user journeys based on digitalized care models. See Supplementary Material 2 for a description of the scenario drafting process and examples of complete scenarios. The scenarios were validated by a broad range of other stakeholders during the main workshop and applied throughout the assessment as a tool to evaluate hybrid care models as a concept. In addition to contributing to the construction of scenarios, the experts by experience also participated in a panel that provided feedback during planning and writing of the manuscript.

Data collection and analysis

The eHTA is based on two data sources (Figure 1).

Figure 1.

Flowchart of data sources. Note: Insights developed from our previous research comprise the context for the stakeholder workshops. Output from the workshops fed variables to the exploratory economic simulation model.

Figure 1. long description.

Stakeholder involvement

We involved a broad range of stakeholders to map and validate potential areas of application, anticipated effects, and implementation issues related to hybrid care models. Stakeholders included researchers and clinicians with experience from various specialist services for severe mental disorders, a leader of a specialist health service, experts by experience as next-of-kin and through personal experience with severe mental disorders, a specialized nurse with mercantile competencies, a leader of economics in specialist health services, and clinicians working in municipal services. A participant with experience with occupational therapy for severe mental disorders was prevented from attending due to technical issues and provided their viewpoints at a later point in written form. The stakeholder workshop (n = 11) was conducted on 28 February 2025 and led by KJK, LNS, and MS. It was conducted using Microsoft Teams and lasted two and a half hours. The workshop resulted in an audio recording, a text chat log with stakeholder comments, and Step Up worksheets in Microsoft PowerPoint. All participants received a summary of the workshop material for review.

The workshop included two Step Up activities (51), an impact analysis, and a benefit planning. During the impact analysis, participants identified potential beneficial effects and risks of implementing hybrid care models. They assessed the impact of each effect across different domains. They also reported current unmet needs, discussed how hybrid care models could address these needs more efficiently, and identified conditions relevant for implementation. We performed a thematic analysis of these data using King and colleagues’ template analysis techniques (55), adapted to the Step Up framework. See Supplementary Material 3 for a detailed description. During the benefit planning, stakeholders discussed which potential effects should be prioritized and identified suitable methods for investigating these in real-world settings. These data were later used to construct quantitative scenarios for the exploratory economic simulations.

Economic simulations

We performed economic simulations to quantitatively demonstrate the potential value of implementing video consultations as part of hybrid care models. Importantly, the simulations of this study were performed in an early conceptual phase characterized by a scarce data foundation and a considerable degree of uncertainty. The simulations are formative and exploratory, not a full cost-effectiveness analysis.

We collected and processed data in two phases to build the economic simulation model: First, we utilized the results from the benefit planning, extracting potential short- and longer-term beneficial effects that were identified by stakeholders. The selection of potential effects (i.e., scenarios) to be included in our model was based on a) stakeholder prioritization: perceived relevance and impact, b) quantitative operationalizability, c) accessibility of the necessary data, and d) that scenarios could be used as proxies to study potential impact on different pillars of sustainability (economic, environmental, and service-related, i.e., clinical capacity). We selected the following scenarios: 1) increased outpatient capacity through increasing the number of video consultations, 2) reduction in cancelled consultations, and thus, increased clinical activity levels, 3) increased revenue through enhanced activity, and 4) reduced travel costs for patients and clinicians and environmental impact in terms of reduced CO₂ emissions. A prerequisite for the scenarios was that video consultations are used either as a supplement to in-person consultations (as clinically required) or as an integral component of a predefined service-user pathway.

Second, we collected publicly available registry data, internal cost data, and relevant public reports and guidelines, supplying the model with real data from local settings. Assumptions concerning anticipated effects of the integration of video consultations into routine services on increased clinician capacity for hybrid employees, reduced cancellations due to more accessible services for service-users, and following changes in travelling patterns, were based on expert opinion from the local clinical and research environments and stakeholder insights. The local context for the simulations was the Division of Mental Health and Addiction at Oslo University Hospital. Oslo University Hospital is located in the capital and is the largest hospital in the country. The hospital is highly specialized and delivers secondary and tertiary services, serving several districts of Oslo as a local hospital while also having multiregional and nation-wide responsibilities (56). Supplementary Material 4 contains information about other preconditions and assumptions underpinning the scenarios, as well as descriptive sensitivity analyses demonstrating more radical and moderate alternative scenarios.

The economic model can be validated after actual implementation at a later point through a socioeconomic analysis based on a randomized controlled trial or case study data, testing whether the selected assumptions align with empirically observed sustainability outcomes.

Ethical considerations

The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on research involving human subjects and with the Helsinki Declaration of 1975, as revised in 2013. All stakeholder workshop participants signed a written consent form. The procedure was approved by the data protection officer at Oslo University Hospital (24/14691 - 34477779).

Results

Potential effects of implementing hybrid care models

Combined data analysis demonstrated potential effects across the user, clinical, organizational, and economic domains of the Step Up framework (Table 1).

Table 1.

Stakeholders’ anticipated effects of implementing hybrid care models

Table 1. long description.

User domain

+ Service users and next-of-kin are more satisfied with treatment: feel safer, practical, and timesaving + Facilitates the combination of service-users’ everyday life and attending treatment + Offers more alternatives to traditional treatment settings, which can reach more people + Easier access to services reduces the pressure on the next-of-kin + Clinicians are more satisfied with their working conditions: predictable, flexible, working from home, and variation + Supports efficient involvement of several actors - Clinician fatigue from repeated digital communication during a workday - Loss of the therapy room: more than a physical space, offers a context for care - Fundamental prerequisites: technical skills and equipment, language demands, and suitable spaces - Roles become less clear: a need to redefine these - Attitude of digitalized care as secondary to traditional care models impacts the experience for all users

Clinical domain

+ Increased access - the right help at the right time improves prognosis: early intervention, need-based care + Increased continuity: more alternatives reduce the threshold for showing up, increasing engagement + Access to ecologically valid information about clinical parameters that guide further treatment + Allows new ways of delivering therapy that improves effect: e.g. short check-ins in exposure therapy - Loss of clinically relevant information obtained in face-to-face encounters, easier to mask symptoms - Service users must handle emotional reactions from remote therapy on their own - Facilitates social withdrawal and other forms of avoidance - Digitalized care models are not suitable for all therapy orientations

Organizational domain

+ Increased predictability: less unnecessary cancellations and last-minute changes + Smarter need-based resource allocation when services complete planned activities to a greater degree + Supports implementation of new methods and ways of designing services: e.g. patient-led booking + Supports new workflows: working from home, shifting tasks between professional groups + Clinical competencies not limited by geography + Public hospitals perceived as more attractive: more sustainable clinician recruitment and retention + Supports smarter documentation and communication across actors and services - Lack of sufficient infrastructure: added workload, frustration - Clinicians pressured into working as hybrid therapists: threat to autonomy, threat to working environment

Economic domain

+ Hybrid services comprise other forms of services, which can result in increased income for hospitals + Socioeconomic benefits from higher participation in work life and education + Reduced sick leave from mild illness for both clinicians and service-users + Less traveling results in reduced costs and environmental impact + Reduced need for large office spaces - Investment in suitable equipment for providing (and receiving) digitalized care

Note: Anticipated beneficial effects (+) and risks (−) of implementation of hybrid care models, extracted from the first and second level thematic levels of the template analysis.

The exploratory economic simulations estimated that increased use of video consultations as part of hybrid care model implementation had a positive impact on sustainability pillars across the four scenarios (Table 2). In a descriptive sensitivity analysis, we explored the effect of moving from a moderate estimate assuming an increase of one outpatient consultation per clinician per week to the increase of two and three consultations per clinician per week. This resulted in a 12, 25, and 37 percent increase in clinical activity, respectively. Regarding capacity loss due to service users not showing up to scheduled sessions, the estimated reduction in cancellations ranged from 3 to 13 percent. Strengthened clinical capacity and resource allocation combined were estimated to result in a 19 percent increase in production, representing 1.9 million euro in hospital income. Finally, the sensitivity analysis showed that the simulated shift towards more digitalized patient pathways and hybrid therapist workflows resulted in annual travel cost savings ranging from 326 000 to 978 000 euro, as well as a reduction of CO₂ emissions ranging from 82 to 247 tons. See Supplementary Material 4 for more details.

Table 2.

Results summary from the exploratory economic simulations

Table 2. long description.

Outcome	Description
Increased outpatient capacity by 12%	Video consultations may save time. One additional consultation per week per therapist yields 22,500 more consultations annually, based on 500 practitioners performing one more consultation per week for 45 weeks
Reduction in cancellations by 7%	Video consultation may offer a more flexible therapy form with the potential to reduce the number of no-shows and cancelled appointments. Among the 4,000 monthly consultations that are not being carried out in the clinic, the scenario showed how 12, 000 additional consultations may be carried out annually with a 25% reduction in no-shows and cancelled appointments
Increased revenue for the hospital of 19% through enhanced capacity	Based on the increased capacity and reduction in cancellations from the former scenarios, the total capacity may increase by 19%. The increased production will, in this scenario, result in increased revenue for the hospital of 1.9 mil euro, considering a unit cost of 329 euro with a DRG of 0.157–0.185 and 34,500 additional consultations
Annual travel cost savings of 652 000 euro, and annual reduction of CO2 emissions of 165 tons	Video consultations may save trips to the hospital for both therapists and patients. In this scenario, 500 therapists save one trip per week for 45 weeks, and 22,500 trips are saved annually. Assuming each therapist has five patients who also save one trip per week, resulting in 112,500 trips annually. The scenario is based on travel distances of 30 km for therapists and 20 km for patients, with travel habits in Oslo of 40% car and 60% public transport. This would result in financial savings based on less traveling to the hospital of 651 708 euro and savings in CO2 emissions of 165 tons CO2/km

Note: Descriptions of the simulated outcomes of four scenarios. The exploratory simulation models are based on local registry data from Oslo University Hospital and data from publicly available reports.

Stakeholder recommendations

Change through incentives

The thematic template’s first integrative theme Change Through Incentives highlighted the necessity to closely adhere to not only needs but also preferences and ambitions of clinicians, from early developmental phases to succeed in the implementation of new care models. Stakeholders suggested that care models should result in improved workflows and working conditions for clinicians. The possibility to work from home, increased everyday autonomy, and greater variation in tasks are examples of stakeholder-validated incentives that can trigger a change in practices and thus facilitate organizational shifts toward hybrid care models. Stakeholders also emphasized the relevance of incentives in creating attractive workplaces to ensure sustainable recruitment and retainment.

Nuancing novel care models

The second integrative theme, Nuancing Novel Care Models, illustrated how a nuanced understanding of digitalized care is needed for users to accurately evaluate the feasibility and value of reshaping existing care models into hybrid ones. Stakeholders expressed that digitalization is commonly connoted with fully digital services and a permanent opt-out from in-person encounters. Such all-or-nothing conceptualizations may result in doubtful attitudes, heightened attention toward value that is lost in digitalized alternatives, and thus, maintain resistance. Conversely, understanding the concept of hybrid care models as a collection of varied digitalized care model designs with flexibility at their core may result in more optimistic attitudes and spark creativity concerning local adaptation and implementation.

Discussion

This eHTA explored the potential value of hybrid care models in health care for severe mental disorders while addressing current unmet user needs and implementation challenges. Estimations based on stakeholder involvement and exploratory economic simulations demonstrated potential beneficial consequences from implementing hybrid care models across all domains included in the assessment, indicating great promise of hybrid care delivery in strengthening the sustainability of mental health care.

Although the optimistic findings of this assessment are mirrored in the literature (10–12;38), realization of anticipated value depends on successful implementation, which is not a given outcome. The starting point of this assessment, the implementation paradox observed in our previous work, aligns with well-documented implementation gaps in the digital mental health field (29;30) and in the public health sector more generally (27). Digitalized innovations repeatedly gain scientific support from technical trials, feasibility studies, and controlled randomized trials; however, their part in reshaping care models remains modest. Research that contributes to bridging the persistent gap between the theoretical value of digitalization and its realization by producing actionable and contextualized insights is increasingly requested (34;35;57). eHTA provides the opportunity to identify future potential benefits early in the implementation process, allowing for stakeholder-validated adjustments to be made along the way to enhance the chances of successful implementation. Moreover, the methodology allows a broad scope, studying services, rather than technology in isolation, and including the overarching systems that the service innovation is part of. To our knowledge, this study is the first eHTA that assesses a mental health service and the first study that uses eHTA methodology to address digital mental health implementation. The demonstration of eHTA application in this study may comprise an academic contribution, as it can inspire the methodological thinking in future work across both fields of research.

The assessment highlights the clinicians as a user category to consider from conceptual to implementation-related stages of innovative care models. The patient perspective has traditionally been prioritized in the design of care models. However, service providers comprise a key user category and often play the role of the gatekeeper in the implementation of digitalized service innovations (37). The service design approach is an integral part of the Step Up framework, which incorporates service users, next-of-kin, and clinicians within the user domain (58). Accordingly, the stakeholders of this assessment indicated that desired changes in the practices of clinicians toward delivering digitalized care should be linked with clinician incentives. In the pilot studies conducted prior to this assessment, which were characterized by implementation failure, the clinicians did not experience any direct incentives related to the digital innovation. Instead, across the three sites, integrating the app seemed to represent an additional task, and few clinicians gained much experience with its beneficial clinical effects. In the stakeholder-validated clinician scenario, however, incentives were a core prerequisite for the new, enhanced workflow. The strategy of boosting practice change through incentives is supported by existing literature, which underlines that the effects of incentives vary between contexts and must be adapted according to local needs, preferences, and culture (59–61). Moreover, research supports investments in clinician incentives to enhance the attractiveness of mental health services as workplaces and improve recruitment and retention rates (5;6;62).

The assessment also underlines the relevance of subjective perceptions of digitalized care models in willingness to change practices and contribute to implementation. Developing a nuanced and pragmatic view on digitalization has been suggested as a precondition for implementation in the literature (36;37). Importantly, more positive attitudes are associated with direct experience with a digitalized method of care delivery (63;64). However, there is a tendency to prefer the current situation rather than engage in change, that is the status quo bias (65). This represents a threshold for gaining initial experiences and maintains all-or-nothing thinking for both clinicians and decision-makers and may be an important target in the planning of future implementation processes. There is a need to provide inspiration, augment credibility, and fuel willingness to gain initial experiences as hybrid service providers. An enhanced focus on how even low-threshold changes can lead to substantial systemic and quantitative changes may represent a core message in this communication. Demonstrating different types of evidence for this notion may be effective for this purpose. Evidence can comprise material such as the early evidence produced in this study. Moreover, it should include direct accounts from clinicians with hybrid care experience from real clinical settings, such as those identified in our previous work and in other existing literature (26;66).

eHTA methodology is also concerned with unintended effects (44). Through the stakeholder analysis, we identified risks related to all user categories, including technical barriers and loss of therapeutic context, clinicians’ digital fatigue, service users’ maintenance of avoidance strategies, as well as unclear roles for next-of-kin. Some of these risks are well documented in existing literature (67–69), underlining their relevance. Risks experienced from the next-of-kin perspective, however, remain a less explored research area. In accordance with service design principles and the Step Up method (51;70), we argue that searching for hidden risks, developing a comprehensive understanding, and then addressing them, is at least equally important as direct efforts to realize potential benefits.

We encourage applying the empirical and methodological insights developed through this study in multiple small-scale case studies across clinical contexts, testing various hybrid care models and implementation strategies. The clinician user perspective must be emphasized in each distinct context. We also encourage research on direct impact and ripple effects of the potential changes in user experiences, organizational setup, and clinical practice related to hybrid care implementation, both negative and beneficial ones. This includes consequences of establishing clinician incentives, e.g., working from home setups; expansions of current treatment alternatives, e.g., service-user-led booking of consultations; and shifts in relational dynamics for different user categories. As hybrid care models remain in a conceptual phase, eHTA methodology will continue to play a useful role in advancing the innovation process. Moreover, eHTA methodology is still evolving, and there is a need to develop and test additional methodological features, such as including sustainability parameters (71) or adapting frameworks to the specific stage of the innovation subject to assessment (72).

Limitations

This eHTA is a formative evaluation and represents only one of several steps in the process of evaluating, further developing, and implementing hybrid care models. The study design is explorative and produces validated estimates, rather than aiming to draw firm conclusions. Accordingly, we refer to value as potential, rather than empirically documented. A limitation of the study is its conceptual level of analysis and its lack of focus on specific clinical environments. Another important limitation that is important to acknowledge is the authors’ starting point, as well as their active, interpretive approach to conducting the study, which also shape the angle of the assessment.

Conclusions

This assessment provides early decision support concerning hybrid care models. Hybrid care models hold potential in meeting needs of stakeholders to a greater extent than current models and in improving system sustainability. The eHTA estimates that conservative changes toward digitalized practices result in a substantial impact on quantitative sustainability parameters and create new opportunities for organizing services. The eHTA emphasizes that implementation problems should be expected across mental health contexts and must be addressed early. Successful implementation requires continuous consideration of the user perspective of clinicians, starting from the conceptual phases of innovation. Promoting nuanced perceptions of digitalized care and developing clinician incentives that prompt praxis change are stakeholder-validated recommendations. eHTA methodology can be applied to align further research with critical knowledge gaps and to ensure that care models are reshaped according to stakeholder needs.

Supporting information

Skoge et al. supplementary material 1

Skoge et al. supplementary material

Skoge et al. supplementary material 2

Skoge et al. supplementary material

Skoge et al. supplementary material 3

Skoge et al. supplementary material

Skoge et al. supplementary material 4

Skoge et al. supplementary material

Skoge et al. supplementary material 5

Skoge et al. supplementary material

Long descriptions

Figure 1. Long description

The flowchart is organized into three vertical columns that feed into a final horizontal box at the bottom.

1. Left Column (Context for the assessment): Titled ‘Pilot series investigating the clinical integration of a mobile app’. It contains three dashed boxes connected by ‘Iteration’ arrows: ‘Proof of concept study’ leads to ‘Feasibility study’, which leads to ‘Pilot study’. This sequence feeds into a final dashed box: ‘Ecologically valid insights into the implementation paradox of digitalized innovations and critical knowledge gaps’.

2. Middle Column (Main data source): Titled ‘Broad stakeholder involvement and co-creation’. An arrow from the left column's pilot study feeds into a dashed box: ‘Scenario drafting workshop’. This leads down to a solid box: ‘Stakeholder workshop’, which then leads to a solid box: ‘Stakeholder-validated unmet needs, hybrid care scenarios, and anticipated value of hybrid care models’.

3. Right Column (Secondary data source): Titled ‘Hybrid care scenarios translated into quantitative value’. An arrow from the middle column's stakeholder workshop feeds into a dashed box: ‘Model construction based on validated scenarios, local registry data, and reports’. This leads down to a solid box: ‘Economic simulations’, which leads to a final solid box: ‘Estimated economic, environmental, and service-related impact of increased use of video consultations’.

4. Bottom Horizontal Box: Arrows from the final boxes of the middle and right columns point to a large grey box at the base titled: ‘Early health technology assessment of hybrid care models for severe mental health disorders’.

Navigate back to Figure 1..

Table 1. Long description

The table is divided into four primary domains.

1. User domain

* Benefits: Service users and next-of-kin report higher satisfaction, safety, and time savings. It facilitates balancing treatment with everyday life and offers alternatives to traditional settings. Clinicians benefit from flexible working conditions and home-based work.

* Risks: Clinician fatigue from digital communication, loss of the physical therapy room context, and unclear role definitions. Prerequisites like technical skills and language demands are noted as barriers.

2. Clinical domain

* Benefits: Improved prognosis through early intervention and need-based care. Increased engagement and access to ecologically valid clinical parameters. Allows for new therapy delivery methods like short check-ins.

* Risks: Loss of face-to-face information leading to masked symptoms. Service users must manage emotional reactions alone. Potential for increased social withdrawal and unsuitability for certain therapy orientations.

3. Organizational domain

* Benefits: Reduced cancellations and smarter resource allocation. Supports new workflows like patient-led booking and task shifting. Clinical competencies are no longer limited by geography, aiding recruitment.

* Risks: Lack of infrastructure leading to added workload and pressure on clinicians, which may threaten autonomy.

4. Economic domain

* Benefits: Potential for increased hospital income and socioeconomic benefits from higher workforce participation. Reduced sick leave, travel costs, environmental impact, and office space requirements.

* Risks: Significant initial investment required for digital care equipment.

Navigate back to Table 1..

Table 2. Long description

The table consists of two columns: Outcome and Description.

* Row 1: Outcome is Increased outpatient capacity by 12 percent. Description states video consultations may save time. One additional consultation per week per therapist yields 22,500 more consultations annually, based on 500 practitioners performing one more consultation per week for 45 weeks.

* Row 2: Outcome is Reduction in cancellations by 7 percent. Description states video consultation may offer a more flexible therapy form. Among 4,000 monthly consultations not carried out in the clinic, a 25 percent reduction in no-shows and cancellations could result in 12,000 additional consultations annually.

* Row 3: Outcome is Increased revenue for the hospital of 19 percent through enhanced capacity. Description states total capacity may increase by 19 percent based on previous scenarios. This results in increased revenue of 1.9 million euro, based on a unit cost of 329 euro with a D R G of 0.157 to 0.185 and 34,500 additional consultations.

* Row 4: Outcome is Annual travel cost savings of 652,000 euro and annual reduction of C O sub 2 emissions of 165 tons. Description states 500 therapists and their patients save trips. This results in 22,500 therapist trips and 112,500 patient trips saved annually. Calculations are based on travel distances of 30 kilometers for therapists and 20 kilometers for patients, with Oslo travel habits of 40 percent car and 60 percent public transport, totaling 651,708 euro in savings and 165 tons of C O sub 2 per kilometer.

Navigate back to Table 2..

Supplementary material

The supplementary material for this article can be found at http://doi.org/10.1017/S0266462326103985.

Competing interests

The authors declare none.