The evaluation of the role of technology in the pathways to comorbidity care implementation project to improve management of comorbid substance use and mental disorders

Abstract

In substance use treatment settings, comorbid mental health problems can occur in up to 70% of people. An integrated approach for managing comorbidity, implementing evidence-based intervention in drug and alcohol settings, remains problematic. Technology can help in adopting evidence-based practice to implement effective treatment healthcare pathways. This study sought to examine aspects of tailored portal utilization (barriers and facilitators) by participants taking part in a program aimed at improving the implementation of evidence-based practice for comorbidity management Pathways to Comorbidity Care (PCC).

Method

A self-report questionnaire and a semi-structured interview were designed to measure clinician satisfaction with the PCC portal and e-resources throughout a 9-month intervention. An adapted version of the “Non-adoption, Abandonment, Scale-up, Spread and, Sustainability” (NASSS) framework facilitated discussion of the findings.

Results

Twenty participants from drug and alcohol services responded to all measures. Facilitators included: (i). clinician acceptance of the portal; (ii). guidance from the clinical supervisor or champion to encourage e-resource use. Barriers included: (i). complexity of the illness (condition); (ii). participants’ preference (adopter system) for face-to-face resources and training modes; and, (iii). lack of face-to-face training on how to use the portal (technology and organization).

Conclusion

Based on the NASSS framework, we identified several barriers and facilitators of the use of the portal including the complexity of illness, lack of face-to-face training, and clinician preference for training mediums. Recommendations include ongoing organizational support, in-house clinical supervision, and consultation with clinical providers to assist in the development of tailored e-health resources and open training opportunities on how to operate and effectively utilize these resources.

Background and significance

In substance use treatment settings, comorbid mental health problems can occur in up to 70% of people.^1,2 Despite this high prevalence, service provision is often segregated, resulting in accessibility challenges for clients. ³ Integrated care, whereby staff identify and provide evidence-based management for comorbidity for those not responsive to a substance use-focused intervention, may be effective. ⁴ While clinical guidelines for treating comorbidity recommend an integrated approach, rates of evidence-based interventions in drug and alcohol settings remain low. ⁵

Implementation science has been defined as the study of the systematic uptake of research findings and evidence-based practices translated into routine procedures in health services. ⁶ Effective implementation may require distinct technologies for translating treatments into practice. Technology can help accelerate the adoption of evidence-based practice, with a growing number of initiatives utilizing this to successfully implement treatment healthcare pathways. ⁷ Technology can enhance a program’s sustainability and is increasingly used for the evaluation of implementation projects. ⁸ The field of implementation science is progressively benefitting from technological advances. ⁸ A number of projects using e-health resources to foster research-practice partnerships and facilitate research translation have been established in public health settings worldwide.^8,9 This “e-health” can be broadly defined as the use of a variety of information and communication technologies in the delivery of health services (and evaluation), or to support patients’ self-management. Examples include electronic medical records (EMRs) replacing hard copy records, healthcare information systems, telehealth, online clinical decision support tools, and/or in-service training for health professionals. ⁸ Technology can enhance communication and collaboration among providers, fostering integrated care, and aiding evidence-based interventions. This potentially expands access to care, improving care coordination, extending reach to individuals who are resistant to engaging in traditional treatment settings, providing outcomes, and recovery monitoring. ¹⁰

Nonetheless, difficulties regarding new technologies in healthcare have been documented;^11,12 several being faced by public systems worldwide. ¹² Unsuccessful e-health initiatives can be inflexible, with a highly centralized approach and failing to involve local staff. ¹¹ Introducing reforms and associated expenditure may not represent value for money. ¹² One argument is that changes in how programs are delivered and reductions in systems delivered could, ultimately, impact clinical care. ¹²

Staff-related factors or complexity of technologies have been shown to be responsible for non-effective e-health implementation. ¹³ A framework for understanding the processes involved when implementing initiatives in health systems has been developed by Greenhalgh et al. ¹³ The Non-adoption, Abandonment, Scale-up, Spread, and Sustainability (NASSS) framework aims to address all aspects regarding non-adoption and abandonment of technologies, and the challenges in moving from a small-scale project to scale-up program, transferable to new settings and sustainable. ¹³ In brief, the NASSS model comprises six elements: condition; technology; value proposition; adopter system; health or care organization; and wider context and embedded adaption. This framework has been widely utilized in the evaluation of technological solutions in health systems,^14,15,16 such as assessing the implementation of internet-delivered cognitive behavior therapy (iCBT) for insomnia in psychiatric health care; ¹⁴ analyzing the experiences of health providers around the implementation of technology-supported person-centered care for people with long-term care needs; ¹⁵ and describing an initiative to introduce new technology, HealthTracker, into Australian primary care (TORPEDO) and the large empirical dataset it generated. ¹⁶ Importantly, the NASSS framework has served to advance knowledge regarding categorization of challenges experienced during the pilot implementation of new technology and how a complexity analysis can be used to develop an implementation strategy for scaling up pilot implementations.

Objectives

This study evaluated the implementation of a multi-modal training program, with a focus on the Pathways to Comorbidity Care (PCC) portal, to improve participants’ capacity to manage comorbid substance use and mental health outcomes in public health drug and alcohol settings. The study sought to examine aspects of the PCC portal utilization by gathering participants’ opinions and other relevant data regarding the use of online components of the PCC training package. We aimed to determine: (1) facilitators of technology utilization; and (2) barriers of using technology when managing comorbidity in clinical care. We used an adapted version of the NASSS as a framework for the evaluation of the online portal.

Method

Procedure and participants

The PCC training package was designed to improve rates of utilization of evidence-based practice in treating comorbid mental health and substance use problems. Participants were staff employed as drug and alcohol counselors in drug and alcohol treatment services Local Health Districts in New South Wales (NSW), Australia. NSW Public Health services 7.5 million Australians in urban and regional areas. ¹⁷ In 2017–18, NSW Health alcohol and other drug services provided 45,824 closed treatment episodes to 27,407 clients. ¹⁸

This intervention was delivered over 9 months as a controlled before-and-after study across six outpatient drug and alcohol services in NSW (metro, outer metro/inner regional, and regional local health districts) with three active PCC training sites and three control sites. It involved multiple modes of training (over a 3-month period), such as didactic seminars, group workshops run by a local clinical champion, individual clinical consultation with a senior clinical psychologist, access to the PCC portal containing comorbidity resources, and booster sessions to facilitate learning and enhance sustainability. Evaluation of the multi-modal package was conducted after all the elements of the initiative were completed.

The study was approved by the Human Ethics Review Committee of the of the Sydney Local Health District (X16-0440 & HREC/16/RPAH/624). For the analysis, only data from participants receiving the intervention was used. All participants provided written informed consent.

Pathways to comorbidity care portal

A detailed description of the full training package is reported elsewhere. ¹⁷ For this study, we focused on technology resources in the PCC portal; e-health resources were used with the aim to optimize learning and to evaluate key aspects. Training included an online portal containing a range of web-based resources for evidence-based treatment of comorbid mental health and substance use disorders (www.pccportal.org.au). Portal content was established in consultation with independent psychologists via a focus group to gauge opinions and needs, and to present designs, structure and overall function.

The portal includes: (i) up-to-date information regarding comorbidity and evidence-based treatments; (ii) online manuals, materials, and assessment tools for screening, monitoring, and treatment; (iii) filmed webinars; (iv) national guidelines, policy documents, and online tutorials; (v) referral pathways; (vi) and booster sessions relating to seminar content. Other e-resources were sought, such as analytics to track website usage.

Measures

An assessment battery, including a self-report questionnaire and a semi-structured interview, was designed to measure satisfaction with the PCC portal and e-resources. Tracking analytics and website usage, and demographic characteristics of the participants, were also collected.

9-month self-report survey: Satisfaction was measured via the statements: “Please provide feedback about each component of the PCC: - Manuals and written resources loaded in the portal, Navigating the PCC Portal and, - Assessment Tools Loaded in the Portal.” Items were rated on a 5-point Likert scale with response options: “1 = Excellent,” “2 = Good,” “3 = Average,” “4 = Poor,” and “5 = Very poor.”

Online Comorbidity Tutorials (“Guidelines on the management of co-occurring alcohol and other drug and mental health conditions in alcohol and other drug treatment settings, NHMRC Centre of Research Excellence in Mental Health and Substance Use, National Drug and Alcohol Research Centre University of New South Wales”) (at 9-month follow-up): To assess completion participants were asked: “Did you complete the online training program in the last 4 months?”. Responses were: “1 = Yes, I completed the training,” “2 = I started the training, but didn’t complete it,” “3 = I enrolled, but never started the training,” and “4 = No.”

To evaluate participants’ perceptions regarding the online tutorials, they were asked “How useful have you found the online training program?”. A 4-point Likert scale included response options: “1 = Slightly,” “2 = Moderately,” “3 = Quite a bit,” “4 = Extremely.” Participants were asked “How satisfied are you with the online training program?” Response options were: “1 = Mildly satisfied,” “2 = Indifferent,” “3 = Mostly satisfied,” “4 = Very satisfied.” And, “How often have you referred to the content of the online training program to assist you with your clinic?”. Response options were: “1 = Never,” “2 = occasionally,” “3 = Sometimes,” “4 = Often.”

Analytics

Portal usage, completed online tutorials (%) and evaluation of resources were obtained. Usage was based on trackage information recorded in the PCC portal, measured by the number of clicks over 9-months.

9-month semi-structured interview with participants (open-ended responses): At the end of the intervention, participants were interviewed individually to gather attitudes about the PCC portal. Examples of the questions were “What did you think of the PCC portal?”, “Did the portal assist you with managing comorbidity in your clinical work? If so, how?”, and “Did you have any problems when accessing the portal?”.

Data analysis

Quantitative analysis

Correlational analyses using R (v3.5.1 Feather Spray) was performed. We aimed to determine any difference in the number of clicks that were linked and trackable to unique log-in details assigned to each participant, and demographic variables (gender, age, years since graduation). As our outcome variable (number of clicks) was not normally distributed (mean = 39.4, SD = 62.6, range = 0–271; Shapiro-Wilk = 0.621, p > .001), we logarithmically transformed clicks (ln + 1) or used non-parametric tests where appropriate.

Framework analysis

An adapted version of the NASSS framework, originally developed by Greenhalgh et al., ¹³ was used to discuss the components of technology encountered during the PCC project’s implementation and research stages. The original NASSS framework was designed to fill a key gap in the literature on technology implementation. It aimed to produce an evidence-based, theory-informed, but also accessible framework to enable those seeking to design, develop, implement, scale-up, spread, and sustain technology-supported health or social care programs, to identify and address key challenges in different domains and their interactions. ¹³ Importantly, this framework is intended for use when introducing or evaluating technology-supported health or care programs. Its development also reflectively helps generate ideas, not as a checklist. ¹³ A thematic analysis of the interviews was performed using NVIVO software.

This study employed applicable components to report on the most salient experiences, described as follows:

• 1. “Condition” addresses the clinical, comorbidities and sociocultural aspects (e.g., nature of the illness).

• 2. “Technology” addresses material and technical features. Features such as size, sounds, aesthetics, and “clunkiness” had significant impacts on the technology’s actual and perceived usability and appropriateness. This also refers to the knowledge generated or made visible by technology, the knowledge and support needed to use it, and to sustainability issues raised by the supply model (how it was developed and will be maintained over time—web developers, licenses or maintenance).

• 3. “Value proposition” refers to whether a new technology is worth developing in the first place and for whom it generates value.

• 4. “Adopter system” refers to adoption (and continued use) of the technology by staff. Evidence suggests that, at times, staff have difficulties engaging with new technologies due to threats to their scope of practice, or patients’ safety and welfare or a fear of job loss. Non-adoption or abandonment of the technology can also be explained by its attributes (e.g., ease of use).

• 5. “Organisation” component addresses the capacity (to embrace any service-level innovation) and readiness (for a specific technology) respectively, such as the uptake and internal scale-up of technology-supported programs as well as the adoption decision, typically a budget (board-level decision). This also refers to the extent to which established work routines will be disrupted or made brittle by the new technology.

• 6. “Wider context” relates to the institutional and sociocultural context, which is often key to explaining an organization’s failure to move from a successful demonstration project (heavily dependent on champions and informal workarounds) to a fully mainstreamed service (scale-up); widely transferable (spread) and persists long term (sustainability).

• 7. “Interaction between domains and adaptation over time” refers to the relationships that any technology implementation project (at an empirical level) and its components are inextricably interlinked and dynamically evolving, often against a rapidly shifting policy context or evolution of the technology.

Strengthening the reporting of observational studies in epidemiology guideline

The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) guidelines for the reporting of an observational study are included with this report in Supplemental file 1.

Results

Sample

Twenty participants from three NSW Health drug and alcohol services that received the PCC training program responded to all measures. Of the sample 75% were female. The mean age was 51.53 (SD = 8.14) and the average number of years since graduation was 16.10 (SD = 9.13). Forty-five percent were psychologists, 15% were social workers, 15% were counselors, 10% were nurses and 5% were community workers. 35% had more than 15 years’ experience working with substance use and mental health-related problems.

Portal analytics

Data analytics from participants accessing to PCC e-resources are presented in Table 1. Between subjects t-tests revealed that females were more likely to use the website than males (mean clicks women = 52.7 vs. men = 0.2; t (18) = −5.865, p > .001). To determine a link between clinician’s age/years since graduation and use, a Spearman’s Rho correlation found no link between the number of clicks and age (r _s = −0.266; p = .303) or years since graduation (r _s = −0.217; p = .314). Number of clicks and number of users by month are depicted in Figures 1 and 2.

Table 1.

Usage of e-tools for clinicians enrolled in the PCC training program.

Measures	Post-intervention	%
9-month follow-up survey (self-reported)
Manuals and written resources loaded in the portal
Excellent	4	20
Good	11	55
Average	5	25
Navigating the PCC portal
Excellent	1	5
Good	8	40
Average	10	50
Poor	1	5
Assessment tools loaded in the portal
Excellent	1	5
Good	11	55
Average	7	35
Poor	1	5
Comorbidity guidelines tutorial completion		-
Yes	6	30
Training started but not completed	3	15
Enrolled but never started	1	5
No	10	50
Comorbidity guidelines tutorial evaluation
Usefulness of the comorbidity guidelines tutorial
Slightly	4	50
Moderately	2	25
Quite a bit	1	12.5
Extremely	1	12.5
Satisfaction with the comorbidity guidelines tutorial
Mildly satisfied	5	62.5
Indifferent	1	12.5
Mostly satisfied	1	12.5
Very satisfied	1	12.5
Reference to the content of the online training program
Never	2	25
Occasionally	4	50
Sometimes	1	12.5
Often	1	12.5
PCC portal usage (number of clicks/times accessed to e-resources; mean, SD)
First trimester	12.90 (21.67)	-
Second trimester	21.80 (49.90)	-
Third trimester	.94 (3.29)	-
Forth trimester	2.63 (4.27)	-

ⁱMay not add to 20 due to missing data.

Abbreviations; PCC: Pathways to Comorbidity Care; SD = Standard deviation.

Figure 1.

Number of clicks.

Figure 2.

Number of users by month.

9-month self-report survey

Overall, 55% of participants thought the portal’s manuals, resources and assessment tools were of good quality; however, they also reported some difficulty when navigating the website, classifying this item as “average” or “poor.” Half did not complete the online guidelines. However, for those who completed or started the online tutorials (45%), 50% thought that this resource was sightly useful, 62% were mildly satisfied and 60% used it occasionally.

Non-adoption, abandonment, scale-up, spread, and sustainability framework

The analysis identified several themes relevant to the use of technology in the PCC project by participants. These are presented under the NASSS domains below.

An overview of the findings is presented in Table 2.

Table 2.

Summary of themes, facilitators, and barriers, sorted under the (applicable) NASSS domains.

Domain	Theme	Facilitator/s	Barrier/s
1. Condition	A. The complexity of health and social needs B. Lack of appropriate referral pathways	Not reported	A. Complex psychosocial care needs of clients and lack social care support B. Challenging service navigation
2. Technology	A. Time constraints B. Supporting IT systems C. Buy-in and the outlook of the portal design D. Training	A. Positive perception of the overall design of the portal and its innovative features B. Source of good quality resources and very comprehensive	A. Lack of time to be involved with technology B. Slow internet or hospital network blocking websites C. Lack of formal training on how to use the portal
3. Value proposition and 4. Adopter system	A. Clinicians’ perception B. The role of supervisor	A. The clinical supervision role is seen as a mentor that encouraged the use of the portal	A. Negative perception from clinicians regarding the use of the portal to treat comorbidity
5. The organization	A. Management endorsement B. Supportive mentoring	A. Management endorsed and provided workload time for participants to engage in the components of the multi-modal package B. in-house mentoring	Not reported

Figure 3 depicts the components relevant to the PCC project using the NASSS framework. This is not exhaustive and has been tailored to reflect and guide discussion.

Figure 3.

Application of the Non-adoption, Abandonment, Scale-up, Spread and, Sustainability’ Framework to the Pathways to Comorbidity Project.

Condition

Complexity of health and social needs theme

Comorbidity management imposes several challenges for clinical staff, who are required to deal with their patients’ complex psychosocial treatment needs. For example, dealing with heavily intoxicated clients and their risk of self-harm has been acknowledged. Interview #1 participant reported “I’m just thinking more about our clients, in terms of dealing with suicide risk. Especially in the context of intoxication, it’s really challenging. And then we’ve got, clients with significant mental health comorbidities. That is often an issue.”

Homelessness and lack of social supports have been reported as risk factors or the result of comorbid substance use and mental illness. Interview #2 participant reported “It depends on what’s going on for the client. If the client is homeless then, you know, they’re going to struggle giving up drugs, I’ll help them look up refuges and contact numbers for housing all that sort of stuff.”

Referral pathways theme

Participants also reported challenges regarding how to best refer clients and find the right service based on their needs (mild, moderate or severe presentations). Interview #3 participant reported “A lot of our clients are very complex so finding the right service can be a bit tricky.”

Interview #4 participant reported “I guess it’s the more severe end that we struggle with. That we might see as severe, but they might not reach the threshold for mental health intervention. Especially with our amphetamine using clients, when psychosis merges and we are worried about them but they don’t want to access treatment. They are often the ones we struggle with too, and I don’t know if that whole issue of methamphetamine use and drug-induced psychosis and so on – I think that’s a whole area in mental health that we see a lot of and I’m sure that’s the same with a lot of services and that’s the struggle with our clients who we can see are experiencing psychosis. But getting them treatment for that is really hard.”

Technology

Introduction of a training package to improve clinical care by training staff pertains some considerations. Online resources can be daunting for participants already dealing with challenges, given the variety of illness complexities.

Health information technology systems and structure theme

Some participants (25%) stated “they did not use it enough.” Some (35%) reported difficulties with access due to slow internet or hospital network blocking websites.

Training theme

Some participants also reported the need (or lack) of formal training on how to access, download and navigate the portal. Interview #6 participant reported that “Yeah I would have liked some more explanation at the beginning with someone going through and saying this is what’s here because I just kind of discovered it as I went, and that might have partly been as well because it did take a bit of a backseat because we were all so busy at that time. But just maybe a bit more clarity about what’s actually was on there and then probably would’ve helped with my interest early on I think.”

Portal outlook theme

In contrast, there were some positive perceptions regarding the resources in the portal, with 55% of participants reporting that “it looked good,” “had some good resources and links” and was “very comprehensive.” Moreover, 35% reported that “it is good to know that is there” and “made me focus on making sure I am up to date.”

Interview n#7 participant reported “I used some assessment screening tools, such as the ADHD and trauma tools. The suicide one was quite a thorough one which I thought was quite good. Then also some of the workbooks like the anxiety or different mental health issues with substance use workbooks to go through which looked quite good. The fact sheets from the webinars were quite good as well in the portal. There was a good one on opiates which was good for training participants.”

Value proposition and 4. Adopter system

Participants’ perceptions regarding whether the PCC portal was worth developing and the level of adoption and use of the portal was mediated by the following factors:

Time constraints theme

Participants’ perceptions regarding the training and its impact on their clinical care varied greatly. Specifically, 35% indicated that time constraints within their current workload prevented the use of the resources (e-tools, assessments). Interview n#5 participant stated (regarding the portal), “It didn’t make a huge difference, maybe if I had more time to look into what else it had to offer but the one time that I did have the one particular thing that I was looking for didn’t come up.”

Clinician general opinion theme

Overall, 40% of participants reported benefits in downloading resources from the PCC portal. While some were supportive of the platform’s value-add, over 50% reported that it did not assist them in managing comorbidity.

The role of supervisor theme

The role of the clinical supervisor was deemed very important for encouraging use of the resources, whereby the majority of participants used the portal following advice during supervision. This is evidenced in a parallel study conducted as part of the PCC evaluation ¹⁹ reporting that following clinical supervision there were significantly increased rates of reported use of mental health screening and assessment tools as well as treatment strategies, which can be linked with an increased endorsement of the e-resources (tools and assessments in the portal) as encouraged by the clinical supervisor.

Organisation

Management endorsement

At all sites, formal organizational and endorsed organizational support existed for training package delivery to staff from clinical directors and supervisors. Managers allocated time for participants to engage in training and the evaluation component (i.e., questionnaires, interviews).

Supportive mentoring

During implementation, participants were encouraged to use the resources and participate in the components of the multi-modal package. This has been complemented with the in-house supervision that all participants received while engaging with the overall PCC training package as evidenced in the PCC clinical supervisor evaluation study ¹⁹ linked to the same overall PCC project.

Discussion

This study explored the facilitators and barriers of technology utilization during the Pathways to Comorbidity Care (PCC) implementation project, with a PCC portal focus, using the NASSS framework.

Our results demonstrated that facilitators included: (i). clinician acceptance of the PCC portal; (ii). guidance from the clinical supervisor, who encouraged the use of e-resources; and (iii). positive perception of the portal (e.g., had some good resources and links). Portal use was more frequent during the clinical supervision and champion workshop phase (3–6 months into the training). We observed that increased portal use was predicted by gender; female participants being more frequent users. However, our sample was composed predominately of female participants, likely a reflection of services, with a high concentration in counseling and psychology roles. No other associations between participants’ demographics and use of the portal were found. This is in line with evidence that such associations are difficult to establish, with many studies reporting unclear findings.^20,21

We also observed that barriers to e-health resources uptake included complexity of the illness (condition) and participants’ preference (adopter system) for face-to-face resources and training modes (e.g., clinical supervision, workshops). Co-occurrence of substance use disorder and mental illness is difficult to manage due to factors such as the overlap in symptoms of intoxication, withdrawal, and mental illness. ²² This imposes a real challenge for delivering therapeutic interventions. However, there is a large body of evidence (including participants’ perception) supporting the notion that didactic or workshop training (involving clinical champions) with ongoing feedback and supervision can facilitate the provision of a gold standard practice in the fields of mental health and addiction treatment. ²³

Barriers also included limited perception regarding benefits of the resources (value proposition), technological and interoperability aspects, time constraints, and lack of face-to-face training on use of the portal (technology and organization). Several studies demonstrated that a common reason for the unsuccessful implementation of e-health resources is that the tools may not fit well with daily clinical work or practices through disruptions to workflow, roles and responsibilities. ²⁴ A lack of knowledge and limited understanding of the benefits can also act as implementation barriers.^25,26

Conversely, the study demonstrated that portal usage was largely driven by advice from the clinical supervisor and clinical champion, and the organization’s commitment to e-resources (the adopter system and organization). Several studies report that facilitators such as leadership engagement and management support,^20,27,28 at all stages of development and implementation of e-health systems, are vital for success.

Accordingly, we offer the following recommendations. Firstly, ongoing consultation with staff to improve and modify the website according to evolving needs. Evidence suggests that involving health professionals is crucial and can facilitate future endorsement of new technologies. ²⁰ Secondly, provision of face-to-face training to improve digital literacy, including navigation of resources (e.g., tools and manuals). Education has been demonstrated to increase staff acceptance of e-health systems including around anticipated benefits. ²⁸ Since our results demonstrate that staff members are influenced by their supervisor and clinical champions’ suggestions to access the portal, we propose that encouragement must facilitate uptake of new technologies during implementation.

Limitations of this study include the small sample size. A convenient sample of participants providing counseling working in NSW Health addiction treatment facilities was used and is unlikely to represent all staff treating addictions. Further, the impact of the participants’ portal usage on patient health outcomes was not evaluated. While there is extensive evidence regarding the uptake of technologies such as EMRs or tracking patient devices in health and addiction medicine generally,^29–31 less literature investigates the utilization of e-health to better train staff in comorbidity management. In addition, it would be beneficial to include other professionals such as physicians, psychiatrists, and other primary care providers beyond allied health professionals into this type of initiative.

In addition, future research and implementation would benefit from the use of the NASSS framework to plan and design programs including the use of new and revised e-resources prospectively.

Strengths include being the first such program aimed at training counselors in drug and alcohol services to improve comorbidity management using a multi-modal package specifically with e-health resources. Moreover, this study demonstrated how flexible the NASSS framework has proven to be, by allowing a retrospective analysis of the themes and categories gleaned from the data collected. Importantly, we were able to adapt and use the framework to draw conclusions for the implementation of technologies in this space and for its sustainability over time.

Conclusion

Overall, introducing “e” into the health paradigm requires significant adjustments within systems and in transforming clinician attitudes. Based on the NASSS framework, we identified several barriers and facilitators when using e-health resources in the management of comorbidity in drug and alcohol services, with respect to “the condition, technological, the adopter system, value proposition and organization.” Recommendations include ongoing organizational support, in-house clinical supervision and consultation with clinical providers to assist in the development of tailored e-health resources and open training opportunities on how to operate and effectively utilize these resources.

Clinical relevance statement

Use of e-health resources can improve service provision for staff working in drug and alcohol treatment settings given that use of evidence-based practices is severely underutilized and this can compromise patient care. Engagement with stakeholders to develop better integration of e-health solutions for increasing evidence-based practices is highly recommended.

ORCID iD

Maria Gabriela Uribe Guajardo https://orcid.org/0000-0001-5708-3984