Barriers and facilitators to the implementation of a novel web-based sleep apnea management platform

Abstract

Study Objective

Obstructive sleep apnea (OSA) is a highly prevalent yet underdiagnosed disorder affecting US military Veterans. The Remote Veterans Apnea Management Platform (REVAMP) is a web-based OSA management program created to improve access to care. REVAMP was launched within the Veterans Health Administration (VHA) in July 2017, with variable patient recruitment rates (from 0 to 573 patients per site) at the first 10 Veterans Affairs (VA) medical centers (Wave-1 sites). This study aimed to examine the contextual circumstances surrounding the implementation of REVAMP from the provider perspective to inform strategies to increase its uptake at future rollout sites.

Methods

A purposive sample of REVAMP site leaders from the Wave-1 sites was recruited with additional staff members being solicited as well. Semi-structured interviews were conducted. Two independent coders reviewed individual transcripts using content analysis to identify emerging themes.

Results

Fifteen individuals from Wave-1 sites were interviewed. Implementation of REVAMP was facilitated by the presence of leadership support, staff, and time dedicated to REVAMP, and perceived usefulness of REVAMP by staff as well as positive feedback from the Veterans using REVAMP. The difficulty of supporting Veteran creation of login credentials to the program and integrating REVAMP into the existing workflow were major barriers to its implementation.

Conclusion

Improving leadership engagement, simplifying the enrollment process, and enhancing the medical staff experience through shared best practice alerts were identified as actions needed to improve the penetration of REVAMP at future rollout sites.

Statement of Significance.

This study evaluated the barriers and facilitators to implementation of a novel web-based sleep apnea management platform in the VA healthcare system. The outcomes of this study will inform future strategies to increase REVAMP penetration at future rollout sites.

Introduction

Obstructive sleep apnea (OSA) is a prevalent yet underdiagnosed and undertreated disorder affecting more than 1 million US military Veterans [1]. It is a systemic disorder that is especially salient for Veterans as male gender, obesity, and increasing age are its primary risk factors [2]. OSA is associated with conditions common in Veterans such as cardiovascular diseases [3–9], mood disorders [10], post-traumatic stress disorder [10], neurodegenerative diseases [11, 12], and cancer [13, 14]. Unfortunately, OSA is underdiagnosed in part due to problems with Veterans’ access to sleep care. Distances to the closest Veterans Affairs (VA) sleep centers and wait times for appointments can be long [15]. Within the Veterans Health Administration (VHA), about half of all Veterans live more than 25 miles from a VA hospital, including 6% of Veterans living more than 100 miles away [16]. In some cases, Veterans may have to wait over a year to see a sleep specialist [15]. Thus, it is imperative to improve access to care by decreasing the distance traveled and wait times.

The Remote Veterans Apnea Management Platform (REVAMP) was launched within the VHA in July 2017. It is an interactive web-based sleep apnea management program designed to remotely diagnose and treat Veterans with sleep apnea. Currently, REVAMP has been deployed at 62 VA medical centers and satellite clinics. However, the rate of patient recruitment for REVAMP over the first year of deployment in the first 10 VA medical centers (Wave-1 sites) was extremely variable, from 0 to 573 patients per site. Although we do not know how many patients at each site were referred for evaluation of OSA and would have been eligible for enrollment in REVAMP, all of the Wave-1 sites were comprehensive, relatively high volume sleep centers.

The purpose of this study was to evaluate the contextual circumstances surrounding the implementation of REVAMP at the Wave-1 sites. As perceptions of healthcare providers are significant factors in the uptake of new technology [17, 18], this study focused primarily on the perceptions of the sleep center clinical staff toward REVAMP and its implementation. The findings of this study provide data to improve implementation of REVAMP at future rollout sites, and the results have broad implications for implementation of web-based interactive applications for population-based healthcare of OSA as well as remote management of other chronic diseases.

Methods

Intervention

REVAMP is a novel web-based management platform designed to remotely diagnose and treat Veterans with OSA. REVAMP is housed behind the VA firewall and has a patient-facing and provider-facing platform. Patients are able to access REVAMP by mobile phone, tablet, laptop, and PC. Providers access REVAMP from a VA networked computer. The program is offered to Veterans with OSA who have email accounts and internet access. The provider must create an account for the Veteran, and the Veteran accesses the platform using a secure username and passcode. The Veteran completes symptom-related validated questionnaires on REVAMP that take approximately 20 min to complete. The responses are reviewed by either a sleep physician or a nurse practitioner. The provider then interviews the patient via telemedicine link, telephone, or in-person. The information entered by the patient on the platform is auto-exported to templated progress notes that can be entered to the patient’s electronic medical record. When positive airway pressure (PAP) device from any manufacturer is prescribed, mask on-time, air leakage, and residual apnea–hypopnea index (AHI) are wirelessly transmitted to the website. These objective measures of treatment use and efficacy are available for both the provider and the patient to review. The patient portal feature of the website contains educational videos and frequently asked questions [19]. Previous work by Kuna et al. [20] has shown that PAP adherence is significantly greater in patients with Web access to information on their treatment usage data.

A site administrator, usually a sleep physician, and a site clinical champion were identified at each facility prior to REVAMP deployment. The staff was trained to use REVAMP through in-person meetings and training webinars. Weekly conference calls among the Wave-1 sites took place for updates and to address any concerns regarding the program and its deployment. At the time our interviews were conducted, 3 of the 10 Wave-1 sites were participating in a VA-funded randomized controlled trial comparing in-person versus REVAMP-based management of Veterans with sleep apnea. The research study funded a study coordinator at each of the three sites who was responsible for enrolling patients in REVAMP and using it to monitor their clinical management. For the rest of the Wave-1 sites that did not participate in the randomized control trials, it was up to the discretion of the clinical teams at each site to determine how to integrate REVAMP into their general clinical workflow based on their individual needs and circumstances.

Study design

A purposive sampling method was employed. Specifically, medical staff who served as leaders in launching REVAMP at the 10 Wave-1 sites was contacted by email or phone call for an interview, and names of additional staff members working with the REVAMP program were solicited. The interviews were conducted between July 2018 and October 2018.

The interview questions were developed based on previous semi-structured interviews used to pilot the program and in collaboration with one of the authors (RF) at Portland VA Medical Center (Supplementary File). The questions were revised based on the recommendations of an expert in qualitative research (JS). A semi-structured approach was used to ensure that the same topics were explored with each participant, but the interviewer had the liberty to ask more probing questions when appropriate. Interviews were conducted by one interviewer (YC) who was trained in qualitative methodology and interviewing techniques. The interviewer was familiar with the features of REVAMP but was not using REVAMP for patient care. The interview questions were guided by the Consolidated Framework for Implementation Research (CFIR), a conceptual framework that is frequently used to assess components to successful implementation of an intervention and is composed of five major domains, including intervention characteristics, inner setting, outer setting, characteristics of the individuals, and the process of implementation [21]. All interviews were audio-recorded and transcribed verbatim. One participant asked that the recording be stopped after 3 min but allowed the interview to continue. The median time for interviews was 20 min (range: 3–33 min).

Content analysis was used to identify emerging themes while reviewing the individual transcripts. A codebook was developed to help ensure the integrity and reliability of the coding and to ensure that all the coders had a shared understanding of the meaning and context of each code. Two of the authors (YC, BS) read the transcribed text and met regularly to identify themes that came up during the analysis. Five transcripts were coded independently by two coders (YC, BS), and nine transcripts were coded by one coder (BS) using NVivo 12.0.0.71, a qualitative analysis software (Doncaster, Victoria, Australia). The median intercoder reliability, measured by Cohen’s kappa, was 0.98 (95% CI, 0.93–0.97). The differences in coding were discussed and resolved by consensus. One author (YC) refined the categories and developed themes, regularly reviewing them with team members. Author YC maintained an audit trail of the data processing and a record of how themes emerged from the data. The Portland VA Medical Center and Crescenz VA Medical Center Institutional Review Boards determined the study to be a VA quality improvement project. Therefore, it was exempt from ethical assessment.

Results

Nineteen individuals from the 10 Wave-1 sites were solicited; 3 individuals did not respond, and 1 no longer worked at the site. Therefore, 15 participants from 9 sites were interviewed. Participants included physicians, physician’s assistants, nurse practitioners, respiratory therapists, researchers, research coordinators, and sleep case managers. Years of experience in caring for sleep apnea patients ranged from 1 to 23 years. Exposure to telemedicine technology before using REVAMP ranged from no experience at all to using clinical video teleconferencing and store-and-forward technology. The staff members either volunteered to participate in the REVAMP pilot program or were specifically recruited to use REVAMP (Table 1). Three of the CFIR domains influencing the implementation process stood out: intervention characteristics, inner setting, and characteristics of individuals.

Table 1.

Interview Participant Demographics

Position	Years of practice	Previous experience with telemedicine
Sleep physician	7	Clinical and Research REVAMP
Sleep physician	20	Telephone visits Electronic consults
Sleep physician	15	Medical school Telehealth seizure clinic Telephone visits
Sleep physician	23	CPAP and HSAT set up
Sleep physician	4	No prior experience
Physician’s assistant	6	Minimal CVT experience Store-and-forward technology
Sleep physician	7	Minimal CVT experience
Research coordinator	2	Limited experience
Sleep physician	12	CVT Telephone clinics Secure messaging In-home video clinics
Respiratory therapist	5	No prior experience
Research Coordinator	1	No prior experience
Respiratory therapist	1	No prior experience
Sleep Case Manager	15	No prior experience
Nurse practitioner	1 year	Minimal CVT experience
Researcher	15	CVT Store-and-forward technology

CPAP, continuous positive airway pressure; CVT, Clinical Video Telehealth; HSAT, Home Sleep Apnea Test; REVAMP, Remote Veterans Apnea Management Platform.

Factors influencing REVAMP implementation

Intervention characteristics

Intervention characteristics describe the features of the REVAMP program that influence its implementation. The three most commonly cited attributes of REVAMP that hindered its implementation were (1) the technical difficulties with Veterans’ REVAMP sign-on process, (2) the volume of the intake questionnaires, and (3) REVAMP’s lack of integration with the existing VA electronic health record, Computerized Patient Record System (CPRS). Veterans needed a DS Logon username and password to logon to REVAMP. DS Logon is an identification issued by the Department of Defense (DoD) that allows Veterans to access VA and DoD websites [22]. DS Logon proved to be one of the major difficulties of implementing REVAMP, as one respondent stated,

The difficult part was with Veterans, you know, trying to get them enrolled into it. We’ll tell them about it, but the difficult part with REVAMP [was] that they need their DS logon, and to get the DS logon was difficult in the beginning.[…] Going on to the site where it says, “how to obtain my DS Logon,” the series of questions was really difficult and trying to get in contact with the help desk on that end was very difficult. That was very hard.

Veterans were asked to complete the intake questionnaires online prior to their visits so the providers could review their responses before the visit. Interestingly, clinicians perceived the sleep questionnaires as both enablers and barriers to the implementation and utilization of REVAMP. Some interviewees felt the questionnaires allowed the clinicians to collect detailed sleep histories in a systematic and standardized fashion and helped to manage potentially complicated patients. As one respondent answered,

[What] I was surprised at is how readily they would fill out a bunch of questionnaires and there’s a lot of questionnaires and it makes for some nice initial data and it kind of exposed how complex even these young people with sleep disorders are. So, I think it unearthed a lot of issues and Veterans welcomed the opportunity to expand on the questions that REVAMP asked that no one else may never have asked before.

Other interview participants noted that they were able to personalize each visit for the Veterans based on the questionnaire responses. They observed that the Veterans using REVAMP were more knowledgeable about their condition and were more engaged with their care:

They come to their follow up appointments a little bit more knowledgeable. They’ll be able to explain to me what’s going on.

In contrast, several of the interview participants expressed their concerns about the number and lengths of the questionnaires as well as the challenges of motivating Veterans to complete the questionnaires ahead of time; they felt it was cumbersome for the patients. As one respondent answered,

[…] the patient [would say], “fine, I’ll fill’em out,” and then 3 or 6 months later, “ugh, gotta fill’em out again?” And then we are like, “please fill them out,” and that degrades the utility of REVAMP. REVAMP does put a lot on the patients at least in terms of the questionnaires.

A few of the respondents felt that REVAMP did not offer any relative advantage over their existing clinical tools, saying “[we] have gone without using their questionnaires with our monitors and we tend not to collect a lot of information before making a decision about testing.” Some felt using REVAMP created redundant work due to the lack of integration of REVAMP with CPRS that required additional steps by staff to use online REVAMP data for a clinic visit within CPRS. As one provider put it, “I see it functioning as a separate [electronic medical record (EMR)] that doesn’t communicate with our actual EMR. So, it’s kind of double the work.” Another provider summarized her thoughts as follows,

I haven’t been really pushing the enrollment into REVAMP, because I’m kind of waiting for them to improve the site functionality a little more, improve the log on, let us customize it a little bit, reduce the questionnaires to what we feel would be beneficial for our patients and for us when we’re following up with the patient and just overall make it a little bit of an easier process. Otherwise, I don’t think we can scale it up that much because then it would become cumbersome for us.

Inner setting

Inner setting refers to different factors within an organization that can interact and influence implementation of an intervention [21]. The key enablers to implementing REVAMP within the inner setting included the organization’s readiness for implementation, such as having adequate leadership support and resources, and an implementation climate that placed REVAMP over other competing items within the institutional priorities. The readiness for program implementation and a supportive implementation climate seemed to be more likely within the context of a research setting than in a clinical environment.

In a research setting, one respondent commented on the resource availability and clarified where REVAMP lies within institutional and personal priorities:

REVAMP is very high in my priority. Partly or largely because I’m involved in the REVAMP research project. So that’s where a lot of my research, a lot of my REVAMP experience, and most of my REVAMP patients are through research because I have the research coordinators to stay on top of these people.

In contrast to the research setting, lack of leadership support and dedicated time and resources for REVAMP were some of the major barriers to implementing REVAMP in the clinical environment. One participant commented, “it was very difficult to implement REVAMP from a leadership perspective, not having REVAMP be supported by [the VA medical center] leadership here.” Several respondents commented on the inadequate implementation climate for REVAMP, admitting that REVAMP was low on their list of priorities. One respondent recalled how the loss of the REVAMP site administrator at her site negatively affected her work: “I lost help when my last boss left. Seeing the same number of patients with loss of help is difficult, so it is lower on my priority because of lack of time.” Another respondent stated, “we have had tremendous challenges related to staffing and growth of the program, especially as we’ve been implementing some of the higher priority clinics like the [telemedicine clinics] and home testing clinics. So those took priority in terms attention and resources in getting stood up.”

Within the implementation climate, compatibility refers to the degree of tangible fit between the value attached to the intervention by involved individuals and how the intervention fits with existing workflows and systems [21]. The difficulty with integrating REVAMP into existing clinic workflow served as a barrier to its implementation, as one respondent explained,

It has not fit well with the current workflow. Initially I was offering REVAMP to patients that worked or didn’t want to travel, people who are already on PAP therapy that I was seeing in clinic, but it has become difficult to carve out dedicated time periods in the week that aren’t interrupted by meetings and cross-cover, other things or other calls that come up. So it’s been very challenging to map out the time for it on a regular basis.

In addition, the medical staff was less inclined to modify the existing workflow to integrate REVAMP if the existing work structure already allowed clinicians to access OSA treatment adherence data and “[arrange] sleep studies without the patient coming in to clinic beforehand.” Site-specific conditions such as high volume of patients, lack of certain technology capabilities such as the ability to perform video teleconferencing visits, and the lack of personnel to coordinate REVAMP follow-ups were mentioned as some of the barriers to using REVAMP in clinics. One participant summarized the differences in implementing REVAMP between a research setting and a clinical setting as follows:

It’s much easier to do this with research patients. With non-research patients, because they do not have a research coordinator following them or a nurse navigator following them or anything like that, for clinical patients, it’s just very hard to get going on the website, fill out the questionnaires, there are a lot of them, and actually have the patience to follow through and do that. So, it’s easier with research right now because that comes with more staff. Clinically at least the way our lab is set up, we are working on ways to get this fixed, but with the staffing right now, it’s hard to get the support to keep reminding patients to do the questionnaires, [because if] they don’t do the questionnaires it’s not very useful.

Characteristics of individuals

Because organizational change starts with individual behavior change, one of the constructs measured by the CFIR framework is the individual’s knowledge and beliefs toward the intervention [21]. This construct refers to the staff’s attitudes toward and familiarity with REVAMP. Perceived usefulness of REVAMP and positive feedback from Veterans facilitated the staff’s use of REVAMP as noted by these respondents:

Veterans’ responses have been pretty good. I think they really appreciate the access to their data. I think there’s something about feeling like that they are being monitored on a more frequent basis. We’re able to make proactive calls that, maybe, weren’t done before and . . . more structure. Those are the main things we are hearing from the Veterans about the REVAMP program.

I think it’s been great because they also feel like they have access to the data that they didn’t have before. I know that some of the more motivated patients might have used one of the manufacturer apps, but right now, it feels like… with the access to the data, I think it’s meeting their needs.

In contrast, the staff’s negative experience with Veterans who were less familiar with web-based technology served as barriers to its utilization. Some interview participants admitted recruiting younger Veterans specifically given their comfort with newer technologies and “they are more likely to be able to use an iPad and enter data and so forth.” Another participant commented, “you get other patients who are either older or they’re not tech savvy, and then you’re on the phone for 30 minutes,” which discouraged the staff from recruiting older Veterans to REVAMP.

One staff member explained the lack of staff buy-in was stemming from viewing REVAMP as “something that just added on to the usual existing crushing workload” and, “for the physicians who are taking care of patients and not concerned about the clinical research implications, they have little enthusiasm for taking up a new system that they couldn’t see how it was going to improve their practice.” One participant expressed her thoughts as follows:

I guess I’ve been hesitant to enroll too many people just because I’ve been hearing about how cumbersome it is and how patients are getting frustrated, and if we have too many patients with this problem then that’s gonna take up a lot more time to help them solve those problems and then also kind of apologize to the patients and that kind of stuff… I don’t want to get our patients frustrated, when we’re dealing with kind of a sensitive topic anyway. I mean it’s hard to get people to use CPAP in the first place, so I don’t wanna add extra frustrations on top of that.

Discussion

This study explores the contextual circumstances surrounding variations in the uptake of REVAMP among the Wave-1 sites. Of the several implementation constructs within the CFIR framework, the inner setting, the intervention characteristics, and the characteristics of individuals stood out as key factors. Within the inner setting, the institutions’ readiness for implementation and its implementation climate in particular had an over-arching effect on REVAMP implementation. REVAMP being implemented in a research setting was the most notable facilitator, as it included (1) readiness for implementation with strong leadership engagement, (2) dedicated resources and a predetermined design that already integrated REVAMP into workflow, and (3) a more mature implementation climate with REVAMP being prioritized higher in the organizational and individual list of priorities. In clinical settings, REVAMP was often low in the list of institutional priorities, and there was a general lack of leadership engagement, both from the VA medical center administration as well as from clinical leadership, and a lack of incentive as well as resources to utilize REVAMP. The medical staff often did not perceive the need for a change in their existing practice pattern, or the clinic lacked the capacity to absorb REVAMP into their practice. Of the intervention characteristics, the technical difficulty with the sign-on process stood out as one of the major barriers of implementation, whereas the perceived usefulness of the intake questionnaires and positive responses from Veterans served to be major enablers of utilizing REVAMP. Regarding individual characteristics, the lack of staff buy-in and negative attitudes toward REVAMP served as barriers to its use.

The concentration of implementation factors within the constructs of intervention characteristics, inner setting, and characteristics of individuals is similar to that of work done by Williams-Roberts et al. [23] who studied barriers and facilitators to collection of sociodemographic data in Canadian healthcare settings. As in our study, their participants emphasized the importance of resources such as staff capacity, time, and funding to support implementation [23]. Additionally, clinicians in our study and others are resistant to integrate health information technology (HIT) into their care setting when they have concerns regarding the potential negative effects of the technology on clinical workflow and productivity [24, 25]. Particularly, the resistance to use REVAMP was compounded by the technical difficulty with enrolling patients to REVAMP, and the lack of system interoperability with CPRS and clinic workflow [26]. Previous work by Ingebrigtsen et al. [27] and Hikmet et al. [28] illustrated that having leadership familiar with the information technology, had clear vision of the implementation outcome, and perceived the technology as useful was associated with successful HIT implementation. It is unclear if the lack of awareness or knowledge about REVAMP within the organizational leadership of VA medical centers contributed to the lack of engagement and support of the program. However, it is notable in our study that the sites with strong leadership engagement were more inclined to incorporate REVAMP into their practice.

Perceived ease of use and perceived usefulness are strong and significant influences on physicians’ intentions to use a technology, with some studies favoring the effect of one over the other [18, 29–33]. In our study, both forces seemed to affect implementation, albeit to a different degree depending on the site and the individuals involved. In VA clinical settings, several clinicians found REVAMP’s ability to systematically collect complex sleep history and individualize treatment for Veterans to be useful. Some felt that REVAMP helped to facilitate management of complicated patients and improved patient engagement. Although technical difficulties sometimes discouraged its use, many staff members expressed the intention to use REVAMP in the future when “they work out the kinks.” One of the staff members felt that REVAMP will meet the needs of older Veterans once the Veteran sign-on process is simplified. Staff members were more inclined to implement REVAMP within their clinic when the staff had positive experiences with REVAMP, such as receiving encouraging feedback from Veterans or having little difficulty enrolling patients. Having positive experiences in patient care and clinic flow has been shown to improve clinicians’ confidence to adopt HIT for patient care [33]. Future studies evaluating Veterans’ perspectives on using REVAMP and its effect on medical staff perceptions of the program and its uptake will be informative.

Health service providers share concerns of increasing staff capacity and reorienting service designs to support integration of data collection [23]. Leadership engagement as well as adequate resource and dedicated time to enroll and follow patients must be ensured for clinicians to reliably integrate REVAMP into their general clinic workflow. Overall, clinicians may view REVAMP as a complex intervention involving a radical change from existing clinical practice requiring significant restructuring of daily protocols because it represented a departure from existing technologies and medical records systems [34]. The resulting departure from the existing practice pattern hinders REVAMP utilization [21, 34]. Boonstra et al. [35] noted that the problems that occur during the change process, such as lack of proper organizational culture or leadership or individual resistance, serve as mediators to barriers to implementation. Therefore, for successful REVAMP implementation, it is imperative to have physician champions, administrative leadership support, and a team effort to establish how the technology fits into the day-to-day operation and organizational goals [36].

Our study is the first to qualitatively evaluate VA medical staff’s perceived facilitators and barriers to adopting REVAMP. The field of sleep medicine is unique in that a substantial component of sleep apnea care relies significantly on remote management technologies, such as wirelessly available treatment usage data and the ability to perform sleep apnea testing at the patient’s home. With the rapidly worsening prevalence of OSA among Veterans [1], it is critical to develop strategies that utilize technologies to diagnose and treat Veterans with sleep apnea in a timely and efficient manner [37]. This study shed light on various components affecting REVAMP utilization and implementation at different VA medical centers and outpatient clinics, with patient recruitment rate serving as a proxy to its use by staff members. Ultimately, whether REVAMP becomes widely adopted or not, we believe that web-based applications will have an increasingly important role in management of patients with chronic diseases. This study focused on the cultural change that is necessary for acceptance of such telehealth technologies. If we can facilitate acceptance of web-based patient/population management for sleep disorders, then we open the door to a new era of patient care that would expand beyond any geographical limitation, regardless of the software or platform utilized.

There are limitations to our study. The small sample size is of concern, given that we were only able to interview 15 individuals. However, we feel that we were able to obtain adequate representation of the Wave-1 sites given that we recruited 1–2 individuals from 9 out of the 10 sites. Of note, we reached thematic saturation by the time we interviewed the last (15th) person. Selection bias is another concern, given that those who are more invested in a program are more willing to be interviewed. However, we feel that this concern was addressed by reaching the majority of the individuals at Wave-1 REVAMP sites who were directly involved with the project.

Conclusion

Finally, leadership engagement, adequate resources, and positive staff engagement toward REVAMP facilitated its use. Improving the penetration of REVAMP at future rollout sites can be facilitated by: (1) Improving leadership engagement through iterative program evaluation, (2) simplifying the enrollment process, and (3) enhancing the medical staff experience and perceptions of the program through shared best practice alerts and sharing Veterans’ perceptions of the program.

Funding

This study was funded by Veterans Health Administration Office of Rural Health, Veterans Health Administration Office of Connected Care, the Master of Science Program in Health Policy Research of University of Pennsylvania, National Institutes of Health T32 Grant HL07713, P01 HL094307, and Health Services Research & Development HX001094.

Disclosure statement

Financial Disclosure: None.

Non-financial Disclosure: none.

Conflict of interest statement. None declared.