Abstract
Objectives:
Falls are persistent among community-dwelling older adults despite existing prevention guidelines. We described how urban and rural primary care staff and older adults manage fall risk and factors important to integration of computerized clinical decision support (CCDS).
Methods:
Interviews, contextual inquiries, and workflow observations were analyzed using content analysis and synthesized into a journey map. Sociotechnical and PRISM domains were applied to identify workflow factors important to sustainable CCDS integration.
Results:
Participants valued fall prevention and described similar approaches. Available resources differed between rural and urban locations. Participants wanted evidence-based guidance integrated into workflows to bridge skills gaps.
Discussion:
Sites described similar clinical approaches with differences in resource availability. This implies that a single intervention would need to be flexible to environments with differing resources. Electronic Health Record’s inherent ability to provide tailored CCDS is limited. However, CCDS middleware could integrate into different settings and increase evidence use.
Keywords: falls, prevention, evidence-based practice, person centered care, clinical decision support
Introduction
Approximately 25% of older adults, those over 65, fall annually in the United States (US) (Bergen et al., 2016). This trend has persisted with no significant change between 2012 and 2018 (Moreland et al., 2020). Community falls were associated with 950,000 hospitalizations and 32,000 deaths in 2018 (Moreland et al., 2020). Falls are widespread, impacting 29.5% of rural older adults and 27% of urban older adults and are the leading cause of fatal and nonfatal injuries, costing the US $50 billion annually (Centers for Disease Control and Prevention, 2015; Moreland et al., 2020).
Agencies have developed evidence-based fall prevention guidelines including the US Preventive Services Task Force, American Geriatric Society, and British Geriatrics Society (Ganz & Latham, 2020). However, research shows inconsistent implementation of guidelines, resulting in up to 92% of adults not receiving appropriate preventive care (Borsky et al., 2018). A study evaluating preventive care and chronic disease management found only 50% of recommended services were delivered (Tung et al., 2010). This is further supported by the 2017 Medicare Health Outcomes Survey which found 51.5% of Medicare respondents at increased fall risk reported receiving an intervention (Assurance, 2021). Computerized Clinical Decision Support (CCDS)–focused research has shown it can increase provider and patient engagement with preventive services, like fall screening and evidence-based prevention (e.g. balance and strength exercises) (Bae et al., 2017; Sutton et al., 2020). CCDS is technology aimed at improving care by providing timely, patient-specific information, typically through point of care automation (Bryan & Boren, 2008; Clinical Decision Support, 2018). Fall prevention CCDS, like Fall Tailoring Interventions for Patient Safety, have been developed for hospital use (Dykes et al., 2017). However, information about adoption of fall prevention guidelines in primary care represents a gap.
Research regarding adoption of prevention guidelines cites common barriers including lacks of time, awareness, and skills for behavioral change interventions (Krist et al., 2015; Kurth et al., 2018). CCDS could mitigate barriers through automation and actionable recommendations (Kawamoto et al., 2005). In addition, contextual information about studies is needed to implement findings (Trinkley et al., 2020). Context is important within primary care due to the longitudinal nature and complex coordination inherent in this environment (Prathivadi et al., 2022). Sociotechnical Systems Theory (STS) and the Practical Robust Implementation Science Model (PRISM) can help organize important contextual details. Socio technical systems theory theory describes complex relationships between human social systems and technical systems and emphasizes joint optimization of both (Oosthuizen & Pretorius, 2016). PRISM is a process, evaluation, and deterministic model, which provides insight factors influencing implementation and links specific factors to implementation outcomes (Trinkley et al., 2020). When considering a new technology, like CCDS, interactions between the technology and other factors should be considered for effective implementation.
The purposes of this paper are as follows: 1) describe how community-dwelling older adults and primary care staff in urban and rural settings engage in fall risk management, and 2) apply sociotechnical and PRISM conceptual domains to identify CCDS requirements that support successful integration of fall prevention evidence in these settings.
Methods
Approach
Data reported here are part of a larger sequential mixed methods study designed to assess primary care fall prevention, identify current evidence-based interventions, and develop an interoperable CCDS tool. Individual semi-structured interviews provided information about stakeholders’ perceptions and self-reported behaviors (Flick, 2018). Contextual inquiries (CIs) built on knowledge from interviews and provided insight into thought processes as staff engaged in fall prevention including experiences using their current system (Viitanen, 2011). Lastly, observations validated staff reported workflow from CIs (Tanzini et al., 2021). A combined PRISM and STS conceptual model was created for this study to highlight the design and implementation of the planned system, see Figure 1. The combined model was used to identify aspects of workflow important to CCDS implementation. Model application included literature review of STS and PRISM domains followed by reflecting on how each domain emerged from our data.
Figure 1.
Fall Prevention Care Planning Journey Map. Any item marked with * denotes items specific to the urban site. ** denotes items specific to the rural site. All other items were consistent across both locations.
After all data sources were analyzed, they were synthesized into a journey map by the lead author to visually depict the fall risk management process in different primary care settings (Gibbons, 2018). Journey maps have been used in healthcare to better understand care processes plus the motivations and thoughts of those engaged in the process (Gibbons, 2018). The CCDS development team used journey maps to holistically understand the process and consider workflow integration. They can also serve as training aides to provide deeper understanding of a larger process by individual participants.
Design, Setting, and Participants
Participants were recruited using purposive sampling from two sites with differing Electronic Health Record (EHR) systems to assess site-specific differences. Specifically, the approach to fall prevention and how it is done within respective EHRs. Utilization of two sites provided the necessary environment to assess feasibility of an interoperable solution. The first site and primary IRB is a large, urban system with multiple primary care locations serving the Boston area, uses the EPIC EHR, and is physician-led. The second site is a single small nurse-led federally designated rural health clinic, uses the Athena EHR, is in north-central Florida, and is affiliated with a neighboring academic medical center. Participants included primary care staff and older adults. Staff at site-associated primary care clinics who care for older adults were eligible to participate. Eligible older adults were enrolled at site-associated clinics, spoke English and were ambulatory. Participants received a $50 gift card. The study was funded by the Agency for Healthcare Research and Quality (U18HS027557).
Enrollment
Primary care clinic medical directors were key informants and provided information for eligible staff. Eligible staff were then e-mailed study information. Additionally, staff were provided postcards to give older adults meeting inclusion criteria. Following interviews, participants were asked for referrals. At the rural site, research staff were onsite, so interested older adults could participate without additional transportation burden. The patient-family advisory council was contacted to recruit older adults at the urban site.
Data Collection and Analysis
Interview, CI, and observation guides were developed with input from fall prevention researchers, CCDS researchers, and human factors experts. Questions were designed to understand how participants approach fall prevention, including 1) current guideline awareness, 2) when and how fall risk is addressed, 3) engagement factors for staff and older adults, and 4) desired features for CCDS. Staff and older adult interview guides were unique, see supplemental materials. Broad engagement with clinical staff and older adults was completed in accordance with CCDS best practices and usercentered design (Trinkley et al., 2020). Interviews were conducted in a private room at the rural site, and virtually at the urban site due to COVID restrictions. Because the primary purpose of CIs was understanding how staff interacted with their EHRs they were only conducted with staff participants and were completed virtually to record on-screen navigation. During CIs, participants shared their fall prevention related workflow including associated activities, steps, and thought processes for a recently seen patient. Participants were then given a scenario and asked to demonstrate how they would interact with the EHR related to falls. In-person workflow observations were planned for both sites but only completed at the rural site due to COVID restrictions. Observations served to validate self-reported workflows in real-world context. Interviews and CIs were recorded, professionally transcribed, and reviewed for accuracy. During team meetings impressions of interviews, CIs, and observations were shared. Once the team agreed that we had reached saturation we moved to the next mode of data collection and ultimately terminated data collection (Saunders et al., 2018).
Transcripts were analyzed in aggregate to develop a comprehensive understanding of fall prevention inclusive of all perspectives. Following aggregate analysis, intersite and intergroup results were compared for differences potentially important to CCDS development and implementation. Conventional content analysis was completed starting with first pass review of transcripts (Hsieh & Shannon, 2005). Sections of transcripts relevant to research aims were highlighted and bracketed (Hsieh & Shannon, 2005; Saldana, 2021). Bracketed labels were categorized into similar concepts creating a hierarchical framework with concepts at the highest level, followed by categories, subcategories, and codes. The research team reviewed and revised the codebook until consensus was reached. Coding was completed using NVivo12. Simultaneous coding was employed due to project complexity because it allows for the application of more than one code to a datum (Saldana, 2021). Complexity stemmed from seeking to understand the processes from multiple viewpoints including any informatics tools currently used and desired capabilities for future tools. To ensure reliability two team members coded one transcript together followed by independent coding of 10% of transcripts. Due to number of nodes and use of simultaneous coding, percent agreement was used to evaluate internal validity (Burla et al., 2008). Intercoder agreement across the 183 nodes applied in overlapping transcripts ranged between 83.72 and 99.99% with mean agreement of 97.72%. Remaining transcripts were then coded independently.
Results
Forty-six transcripts were analyzed including 38 interviews (20 staff, 18 older adults), eight CIs (5 Urban, 3 Rural), and nine observations. See Table 1 for demographics. Staff participants included primary care providers (Physicians, Advance Practice Registered Nurses, Physician Assistants), Medical Assistants (MA), Registered Nurses (RN), and Licensed Practical Nurses (LPN). While there were nominally more urban staff participants (n = 14), all eligible rural staff (n = 6) participated. Due to the limited number of staff at the rural site, CIs and workflow observations were completed with staff who also completed interviews. See Table 2 for quotes describing barriers and enablers. Quotes labeled “U” represent urban participants and labels starting with “R” represent rural participants. When analyzing results, there were far more consistencies among groups than differences. Results are therefor reported in aggregate unless specifically labeled otherwise to reduce redundancies. The following results are organized by study purposes.
Table 1.
Demographics.
Staff (n = 24) | |
---|---|
Female | 18 |
Male | 6 |
Age in years | Mean: 51 Median: 51 |
Race | White: 16 Asian: 6 Black: 2 |
Urban site | 19 |
Rural site | 6 |
Provider type | MD: 11 APRN: 4 PA: 3 MA: 3 LPN: 2 |
Years in current profession | Mean: 19 Median: 17 |
Patient care hours/Week | Mean: 18 Median: 28 |
Compared to your peers how do you rate yourself for helping prevents falls? | Above average: 4 average: 17 below average 3 |
Older adults (n = 18) | |
Female | 13 |
Male | 5 |
Age in years | Mean: 75 Median: 74 |
Race | White: 14 Asian: 0 Black: 4 |
Urban site | 8 |
Rural site | 10 |
Afraid of falling? | Yes: 6 no: 12 |
Fallen 2 or more times in past year? | Yes: 4 no: 14 |
Fall injury in past year? | Yes: 5 no: 13 |
Table 2.
Barriers and enabler quotes.
Fall Risk Factors | Urban | Rural | Total |
---|---|---|---|
Extrinsic Assistive devices |
12 | 4 | 16 |
Stairs | 9 | 6 | 15 |
In-home support | 9 | 2 | 11 |
Rugs | 7 | 3 | 10 |
Cluttered walkways | 6 | 1 | 7 |
Weather | 4 | 1 | 5 |
Carrying items | 2 | 3 | 5 |
Intrinsic Co-morbidities |
16 | 9 | 25 |
Sensory impairment | 10 | 2 | 12 |
Balance | 9 | 4 | 13 |
Age | 8 | 3 | 11 |
Fall history | 8 | 2 | 10 |
Lack of attention | 7 | 3 | 10 |
Moving quickly | 3 | 5 | 8 |
Strength | 4 | 4 | 8 |
Medications | 6 | 2 | 8 |
Available resources Clinical resources Physical therapists |
19 | 8 | 27 |
MA/LPN | 11 | 8 | 19 |
RN | 13 | 0 | 13 |
Fall prevention education resources | 7 | 2 | 9 |
Clinical pharmacists | 4 | 0 | 4 |
Telehealth capabilities | 2 | 2 | 4 |
Community resources Group exercise |
10 | 1 | 11 |
Senior centers | 7 | 1 | 8 |
Virtual resources | 7 | 1 | 8 |
Current practices Assessment elements Questionnaires |
15 | 12 | 27 |
Environmental assessment | 19 | 6 | 25 |
Fall history | 14 | 7 | 21 |
Medications | 15 | 6 | 21 |
Gait assessment | 13 | 5 | 18 |
Assistive devices | 10 | 5 | 15 |
Orthostatic evaluation | 12 | 0 | 12 |
Targeted assessment | 7 | 4 | 11 |
Appointment type | 8 | 7 | 15 |
Other practices Conflicting demands |
3 | 3 | 6 |
Osteoporosis management barriers | 2 | 1 | 3 |
Willingness to disclose | 3 | 5 | 8 |
Engagement factors Pre-existing relationship |
12 | 9 | 21 |
History of falls | 10 | 2 | 12 |
Older adults Intervention Individualized |
9 | 5 | 14 |
Support systems | 12 | 10 | 22 |
Intrinsic motivation | 10 | 4 | 14 |
Experience/Habit | 8 | 5 | 13 |
Fear loss of independence | 12 | 2 | 14 |
Easy to do anywhere | 2 | 2 | 4 |
Cognitive impairment | 1 | 1 | 2 |
Cost | 2 | 3 | 5 |
Insurance coverage | 7 | 3 | 11 |
Transportation | 1 | 7 | 8 |
Staff Previous fall |
7 | 1 | 8 |
Personal experience | 1 | 2 | 3 |
Mostly older adult panel | 2 | 1 | 3 |
Fall etiology | 10 | 1 | 11 |
Unclear resource navigation | 10 | 1 | 11 |
Fall risk visibility | 4 | 1 | 5 |
Patient provider communication Motivational interviewing |
4 | 0 | 4 |
Multi-modal | 23 | 14 | 37 |
Paper handouts | 22 | 13 | 35 |
Demonstration | 6 | 3 | 9 |
Verbal | 6 | 2 | 8 |
Patient portal | 8 | 5 | 13 |
Recommendations Physical therapy |
17 | 6 | 23 |
Assistive devices | 9 | 2 | 11 |
Walking | 9 | 4 | 13 |
Strength exercises | 8 | 3 | 11 |
Balance exercises | 6 | 1 | 7 |
Referrals | 5 | 1 | 6 |
Non-traditional exercises | 4 | 0 | 4 |
Staff communication EHR |
14 | 3 | 17 |
Future state Team-based care |
6 | 4 | 10 |
Actionable information | 6 | 4 | 10 |
Exercise guidance | 9 | 1 | 10 |
Purpose 1
See Table 3 for a site comparison of factor frequency.
Table 3.
Site comparison of factor frequency.
Theme | Subthemes | Barrier | Enabler |
---|---|---|---|
| |||
Fall risk factors | |||
Extrinsic | Someone ditches their walker, and then they go to sit down on something and either they fall getting to the thing or when they go to stand up. They can’t actually reach their walker, so they’re falling in between those two places. (US3) I had it in my arms, and 1 went the steps, and 1 missed a step. (RA2) I have patients who have fallen on the sidewalk because there are all kinds of obstacles there, stumps, roots, potholes, and so on. (US2) |
Yes, that’s what 1 plan to do, get myself a cane but 1 probably wouldn’t use it all the time period I’ll just use it when I think I need it. (RA6) I’ve learned not to take things with me that’s in both of my hands. These are things that I’ve learned through falling but not hurting myself... I don’t take laundry down. I actually have my kids take it down or I will throw it down the stairs beforehand. (UA2) |
|
Intrinsic | That’s part of the problem, his legs are weak period he tripped and slipped on something and was stuck in the yard. (RS5) I ‘ve never had good balance, but I’m a very strong person. You know what 1 mean? 1 should probably, working more on my balance, especially now because I’m not at the gym. 1 know the balance stuff you can do. I just haven’t done it. (UA3) |
I think the knee arthritis is a real barrier, and even to the extent where I’m sending patients to orthopedics or the pain clinic to get injections in their knees. Then those patients, for the next 6 months, are a little more pain free, and they say, “Yeah, okay. It doesn’t hurt so much I can go for a walk” (USI6) I have friends who I’ve seen fall, and I understand. I also understand that as you get older, you have to pay more attention to where you’re going and where you’re stepping because it’s not as automatic as it used to be. (RA8) |
|
Available resources | |||
Clinical | We had the STRIDE program before at the «clinic», and that was, I think, quite helpful.” (US2) I have had patients who have a geriatrician and they have done a very standardized approach. ... There was something called the mobile observation unit where an NP would go out to the house for patients who we’re concerned about. That was more of a holistic, providing primary care, but then, potentially, they would also just do the home safety eval as well. (US 12) Typically, «LPN» and «LPN» are really excellent. They will start the fall risk questionnaire with them. (RS2) |
||
Community | There is a Senior Center up the road here, but I’m actually not sure they do much other than have lunch. (RSI) | Through the «city» Senior Center, has an unbelievable exercise program they have several actually, but their participation of the seniors that go to them is tremendous. (UA5) | |
Current practices | |||
Assessment elements | They ask you those routine questions, the Medicare kind of questions. I just go through them. “I’m fine, fine, fine, fine.” (UA3) Traditionally before the pandemic, 1 would go out to the waiting room to greet my patient and escort them into my office, which represented a significant part of the examination. 1 got a lot of sense of how stable they were. (US 13) |
We talked about what I do for exercise, and I told them I did usually walk two to three times a week. I’m honest with her. If I haven’t did it, I’d say, you know what? I haven’t did it in a couple weeks. She says, well you have to get back to walking or and keep up with it. (UAI) I look at vital signs a lot. 1 love the vital sign bar to be quite honest, so I can look to see is someone’s blood pressure floating dangerously low that they’re actually orthostatic and that’s why they’re losing their balance. (US3) |
|
Education reported | I spend a lot of time talking about medications and timing of medications. (US5) One of the things that I talked to my seniors with is their home setting, and particularly throw rugs. If they have throw rugs at home, I tell them to please get rid of them. (RS2) |
||
Osteoporosis management | Anybody over the age of 65 without a doubt- well, let me caveat that. Women. I’ve had a difficult time getting any men approved for getting DEXA scans. It’s very frustrating. Then, women under 65 who are high risk, so i.e. those who are on steroids, oral steroids chronically, or for those women who have a history of breast cancer, and they’re on hormone blockers, those type of patients. (RS2) I’m skeptical about these new drugs «bisphosphonates». Obviously, there’s the osteonecrosis of the jaw, these eight typical fractures, right, that had been reported. Probably from risk/benefit, I probably am under using these drugs. (US 16) |
There’s a particular template that has patient medical history...I’ll tend to put their last date of their dexa scan, what the T score was, and then I will commonly put when they started whichever type of medication they started so that I can keep an eye on it. (RS2) | |
Engagement factors | |||
Older adults | Well, the physical therapy, the one we referred her to...and then daughter said that’s too much. She needs put her in a wheelchair and everything. She really wants somebody come to her house, so then we will call the home health, do you take this insurance, can you do the physical therapy at home. Then I will make phone call according to their insurance. It’s all varies, so that’s going to be a long phone call. (RS4) | I figured maybe 1 won’t be able to do those exercises unless I do them again. Whatever I did yesterday, I’m going to do it again today. (UA4) A lot of times, we ask permission for the patients to have a family member either come to the appointment or be part of the phone visit or get permission to talk with the family member. We talk about some of the things that the families are noticing and strategies that they can use to prevent falls. That tends to be a little more helpful because families are usually around a lot more and can help reinforce the plan. (US8) |
|
Staff | They’ve now just been decreased again. All visits doesn’t matter the reason are 30 minutes. (US3) I would say that just as a caveat, my patient population is not skewed older. It’s mainly younger folks, so more often I’m seeing this in somebody who fell off their bike. I think I’ve been conditioned a little bit not to worry about it as much as somebody who sees more older patients. (USI I) |
If the patient reports a fall, I think that’s the time that we take a step back and assess what the internal and external risks are for the patient, and we could have a little discussion as to how we can try to prevent the next fall. We like to see the patient identify the reasons why they are falling, and see what solutions they have to prevent this. (US2) | |
Patient communication | |||
Approach | I Guess the biggest technique I use, or the most common one I use, is motivational interviewing, which has good evidence behind it. I’m sure you know what that is, but just to be clear, it’s sort of asking a series of questions to elicit patients’ goals, and then being really concrete about establishing those goals with numbers behind it to give more motivation. (USI I) | ||
Recommendations | I Like PT. Some patients just don’t wanna do it, or they’ve exhausted their PTs. (US 10) I don’t give people balance handouts ‘cause that would honestly scare me. (US3) The only time I may give them a list of exercises would be if they would have a hard time getting to PT on a regular basis. Even then, I’m really concerned, if they have weakness, then I’m definitely gonna do home PT anyway. Even for healthy young people who injure themselves or have an overuse injury, I tend to send to PT just because I feel like that’s their expertise. (US4) |
The most common thing for me is getting them some sort of a device, like a cane, or a walker, or a wheelchair, or something like that. That’s the most common thing that we do. (RS5) Some seniors are amazingly fit and agile and able to do things like yoga. There’s a particular yoga instructor that 1 love on YouTube named Adriene. She’s free. (RS2) I do like pilates because if they strengthen their core, even if they tripped, they can catch themselves. (US 10) | |
Staff communication | It would be the providers that are not in EPIC that are the hardest ones to follow’cause you just don’t know unless you talkto a family member or the patient to find out what’s going on. (US7) I don’t know if we have a true process for if that screen is positive, how it gets communicated to the provider. Some practices, I actually float to different practices, and some of them, they actually bring me a piece of paper that says they answered yes to a fall risk Others, nothing. Nada. (US4) |
In the HPI,... if there’s anything in there that I feel is important for the providers to know about, especially if we’re busy and I don’t get to see them in passing, I type that in there. (RS3) | |
Desired future state | The medical assistant in our practice administers the fall screening program, but they don’t really go beyond that other than just documenting the risk for falls. Part of the reason is because they really don’t have much time to go further into the risk factors for the patient. I suppose if, in an ideal world, if you want to be more aggressive in the program we could train them to actually go over the factors that contribute to their fall « risk» and start the conversation. (US2) | A phone call in 3 weeks by someone in the office would be a good thing. The biggest barrier is right at the beginning, getting started, following the referral” (US9) “I would love to have access in a way that’s really easy to remember so I’m not searching wildly. (USI I) I think we’re lacking in that is what to do after it’s scored. That’s where I’m hoping that we can tailor some of the interventions to the scores and say here’s the score, these are the things that should be done, ... kind of like a checklist of things to guide us and make sure we’re following the guidelines. (RS5) |
Fall Risk Management Process
Fall Risk Factors.
Participants described factors as extrinsic or intrinsic. Extrinsic factors included assistive devices, stairs, in-home support availability, rugs, cluttered walkways, weather (e.g. ice, leaves), and carrying items. Intrinsic factors included co-morbid conditions (e.g. diabetes, arthritis), sensory impairment, balance, age, fall history, lack of attention, moving quickly, strength, and medications.
Available Resources.
Participants discussed available clinical and community resources. Clinical resources included physical therapists, MAs or LPNs, clinic RNs, fall prevention education resources, clinical pharmacists, and telehealth capabilities. Community resources cited included group exercise, senior centers, and virtual resources. When rural (continued) participants were asked about community resources, they mostly spoke about a lack of them, whereas urban participants described robust senior centers, in-person classes, and virtual classes.
Current Practices.
All participant groups mentioned assessment elements, assessment frequency, and fall-related education as current practices. Common assessment elements included questionnaires, environmental assessment, fall history, medications, gait, assistive devices, and orthostatic evaluation. More participants described assessment frequency as targeted to wellness visits or triggered by clinical judgment. This tailored approach was reported across sites and participant groups. However, some urban staff (n = 3) and rural older adults (n = 3) reported assessing fall risk as part of the screening for every visit regardless of reason for visit, and two participants (1 urban staff, 1 rural older adult) reported not assessing fall risk at all.
Additional practices influencing fall prevention discussed by staff included appointment type, conflicting demands, and osteoporosis management at both locations. Appointment type made a difference in whether staff addressed fall risk. They reported engaging more often in the context of annual wellness visits because Medicare requires fall risk screening (Medicare Wellness Visits, 2021). Conflicting demands, like chronic disease management, resulted in a lack of time to adequately address falls. When discussing osteoporosis, staff voiced barriers to screening including, changing guidelines, concerns about strength of evidence, and difficulty getting insurance approval for screening at-risk males. Electronic Health Record alerts supported staff to screen older women for osteoporosis. Older adults’ willingness to disclose a fall varied across sites. While some were willing to disclose a fall that resulted in injury or fall for an unknown reason, others stated they would not disclose a fall if they knew what caused it.
Engagement Factors.
Some factors likely to increase engagement with fall prevention were expressed by both staff and older adults, including preexisting clinical relationships and history of falls.
Older Adults.
Older adults were more likely to engage with interventions like exercise and physical therapy if recommendations were individualized to their abilities and interests. One provider described their experience as “When you do motivational work around this and you actually ask people how motivated or not motivated, one of the biggest issues is they actually just don’t have any buy-in” (UP9). Factors positively associated with intervention engagement included intrinsic motivation or desire to improve, personal exercise experience, fear of losing independence, and interventions easily done anywhere. Those with support systems, including peers, family, or community members, were more likely to engage with recommendations. Conversely, cognitive impairment, out-of-pocket costs, lack of insurance coverage, or recommendations requiring frequent transportation were described as barriers to engagement with recommendations. Seven out of eight participants who were concerned about transportation were rural participants. Notably, cost was mentioned more often mentioned by rural participants and the impact of insurance coverage was mentioned more often by urban participants.
Clinical Staff.
Staff stated they were more likely to engage in fall prevention if a patient had previously fallen, but were less likely if the patient was at risk and had not yet fallen. Staff with personal experience (e.g. parents) or who saw mostly older adults were more likely to engage. They were less likely to address easily explained falls (e.g. trip, slip) that appeared isolated. Staff who shared experiences of unclear resource navigation or challenges with risk visibility were less likely to engage in prevention. Unclear resource navigation included difficulty finding or a lack of tailored fall prevention or uncertainty about connecting patients to community resources. Some urban providers commented that fall risk is not visible in their typical workflow because another team member completes screening. Screeners used paper-based communication tools as a work around to convey fall risk results to providers.
Patient-Provider Communication
Approach.
Providers spoke about developing patient-centered fall prevention plans based on the patient’s motivation and acknowledgement that falling was a concern. Some providers used motivational interviewing to address falls. One provider described their approach as, “I think starting off real small is typically the best that I can do. If I say to you hey, what do you think if we start off—we just do 10 minutes of walking 3 times a week? Do you think that’s something that you could potentially do?” (RS2)
Providers and patients preferred multiple modes to communicate fall prevention information. Paper handouts (e.g. exercise/medication handouts) were discussed most often. Other modes of communication included demonstration, verbal discussion, and patient portal use.
Recommendations.
The most common recommendations reported included physical therapy, assistive device use (e.g. cane, walker, grab bars), walking, strength exercises, balance exercises, and non-traditional exercises like gardening, and referrals to evaluate medications or home safety. Staff expressed reluctance to recommend specific exercises without expert guidance from physical therapy. One provider reported “I usually let PT do that because I want more of an eval of what would be best for them and what would be safe for them. I really don’t want them trying to do things on their own because I’m concerned they’re going to hurt themselves.” (US4)
Staff Communication.
Electronic Health Record documentation was the most frequent communication mode between staff. Twenty-five percent of staff participants reported using paper documentation to communicate. Staff responsible for screening also described verbal communication as important for communicating. Staff reported different modes of communication with outside agencies. One provider described their experience as “it all depends on the agency. I’ve had good rapport with them and VNAs that will probably let me know anything, and then there’s other ones I’m chasing them down.” (US7) Providers at the rural site described needing to login to a separate EHR to view specialty notes to ensure all providers are “on the same page.” (RS6)
Desired Future State.
Participants varied about how often fall risk should be assessed. Responses ranged from quarterly to annually. Staff participants wanted more team-based care in which staff could complete tasks such as assisting with referrals or evaluating care plan progress. Staff also expressed a desire for a system that supported actionable evidence-based recommendations and included expert guidance for exercise. A report of end-user needs for fall risk management CCDS is reported elsewhere (Rice et al., 2022).
Fall Management Journey
Interviews, CIs, and workflow observations were synthesized to create a journey map, see Figure 2. Many fall prevention activities were similar across rural and urban sites. All locations reported visit preparation followed by screening, the provider visit, and then general follow-up. Differences between sites were limited to visit preparation resources, how staff provided handouts to patients, and printing capabilities. The urban site reported using the patient portal or phone to collect screening information before wellness visits. Rural participants reported insufficient staff to collect data before visits. Urban participants described having more clinical resources such as geriatricians to co-manage patients with and registered nurses. Visit durations varied with urban participants typically having 30 minutes and rural participants having 60 minutes. Despite this variation, both sites expressed similar barriers like competing clinical demands and unclear resource navigation. Urban participants were more likely to be unsure of community resource availability, while rural participants described a lack of community resources and fragmented clinical resources.
Figure 2.
Integration of sociotechnical systems theory with PRISM domains. R = recipients I = intervention ISI = implementation sustainability infrastructure. E = external environment RE-AIM = Reach Effectiveness Adoption Implementation and Maintenance.
Purpose 2
STS and PRISM domains.
Understanding how participants are currently managing fall risk is important to prepare for implementation of planned CCDS. This includes how technical systems are used (e.g. EHRs and patient portals) and how they would like to see these systems function in the future. To highlight important contextual factors results of aim 1 were applied to the combined STS and PRISM model.
Social System Domain
Organizational Structure.
This domain consists of organizational hierarchy, reward systems, and organizational values and differed between sites (Oosthuizen & Pretorius, 2016). Due to smaller size and flatter organizational structure the rural site had more flexibility in day-to-day operational decisions like appointment durations. Most urban staff stated they had 30 minutes regardless of visit type. Conversely, the rural site was able to adjust scheduling templates allowing 60 minutes for annual wellness visits. Rural participants stated this allowed time to address preventive measures that otherwise compete with chronic disease management during other appointment types.
Recipients.
In this study, this domain addresses characteristics of clinical staff and older adults (demographics, knowledge, attitude towards falls). Across sites all participant types agreed that fall prevention is valuable. Most staff also agreed that they could improve their fall prevention practices. Older adults at both sites tended to underestimate their own risk with only 1/3 citing concern for falling. Most participants also felt that disclosure of falls with clear etiology and without injury was unimportant.
Intervention.
This domain incorporates intervention elements and perspectives of the organization, staff, and older adults (Feldstein & Glasgow, 2008). We reviewed evidence strength for community-based fall prevention interventions and determined that strength and balance exercises, deprescribing of fall-risk-increasing-drugs, and use of bisphosphonates for those with osteoporosis or osteopenia had the strongest evidence. Staff participants agreed that fall prevention CCDS should be brief, and evidence-based. All participants preferred CCDS capable of tailoring recommendations because they felt individualization would increase older adult engagement.
Technical System Domain
Physical Systems.
Electronic Health Records, hardware, and facilities (e.g. buildings) are all part of this domain (Oosthuizen & Pretorius, 2016). Technical systems in this study (e.g. EHRs) differed by site, with the urban site using EPIC and rural site using Athena. CCDS being developed will need to be interoperable or EHR agnostic. It should also be able to pull relevant information from the EHR to minimize time needed to use the tool. One way to communicate with different systems is to use an information exchange standard (e.g. Fast Healthcare Interoperable Resources). Fast Healthcare Interoperable Resources standardizes data in a modular way which allows integration with various technical systems (Braunstein, 2018).
Task.
This domain refers to workflow or tasks associated with or impacted by the intervention (Oosthuizen & Pretorius, 2016). To minimize resource navigation issues, a barrier at both sites, future CCDS need flexibility to integrate with existing workflows at individual sites. This could be accomplished by integrating the launch button into the EHR system where providers will see it during current workflows. If CCDS systems are designed to send information back to the EHR then individual sites should have flexibility to direct content to the best location for that system. For example, the rural site could more easily print patient handouts, whereas the urban site had more difficulty printing due to printer locations and complex printer routing. Conversely, the urban site often sent handouts electronically using the patient portal.
External Environment.
Elements of this domain (e.g. regulatory requirements, community resources, public reporting of performance metrics) interact with STS elements within the complex environment. While all participants were associated with primary care practices they are in disparate communities. Urban participants described a wider array of community and clinical resources. Urban staff were more likely to have RNs and clinical pharmacists. Urban older adults also had more access to community resources (e.g. exercise programs, senior centers). In addition, the rural site has different regulatory requirements due to its federally qualified rural health clinic designation. Development teams must be mindful of environmental differences when developing CCDS for disparate environments. Otherwise, the CCDS may conflict with needs and capabilities resulting in abandonment.
Discussion
In this study we sought to understand how urban and rural primary care practices approach fall prevention, if a single CCDS solution could be applied in both settings, and used STS and PRISM domains to highlight important factors for future implementation. This combined framework highlights the need to tailor integration and implementation by site even with the same core intervention. We identified four important findings while assessing the fall prevention approach in primary care. First, we found a skills gap among staff related to providing evidence-based fall interventions, particularly for those at risk of falling, but have not yet fallen. We also found that despite potential harm providers were recommending walking alone for fall prevention. Compared with current literature we also had higher levels of fall prevention engagement. Based on the application of the STS and PRISM domains we learned that despite having similar clinical approaches the workflow and available resources will require flexible implementation approaches that can adapt to local resources.
Half of older adult participants reported discussing falls with their primary care team. While this level of engagement is lower than desired, it is nearly double the 28% reported in the 2017 Medicare health outcomes survey (Assurance, 2021). This level of reported engagement is encouraging especially considering our study included a broader population compared with the Medicare survey, which was limited to older adults already identified at risk for falls. Our study also found that primary care staff were more likely to engage patients about falls if they had a history of falls. This finding is similar to a 2015 RCT that found staff were more likely to prioritize fall prevention for those with recurrent falls [OR 2.2] (Jansen et al., 2015). The Jansen et al. study also found staff prioritized fall prevention for older adults with severe fear of falling. While our staff participants also acknowledged a fear of falls as a reason to act, they were unsure what to recommend for those at risk but who had not yet fallen. This demonstrates a skills gap that needs attention. When staff participants made recommendations, they overwhelmingly recommended physical therapy, but some were reluctant to provide specific exercises without expert guidance, further supporting a skill gap. For those who recommended exercises, they most often recommend walking. However, studies have found that recommendations of increased walking alone do not reduce falls and may increase them among older adults (Avin et al., 2015). Most older adults expressed willingness to engage with physical therapy and exercise. However, participants noted barriers to accessing physical therapy (e.g. cost, transportation, appointment availability). Our findings suggest that CCDS that incorporates expert recommendations from physical therapy and tailors’ recommendations to account for individual safety could help bridge the skill gap. If primary care providers had access to expert recommendations at the point of care, they may be more likely to provide evidence-based recommendations rather than walking alone, which may cause harm.
Both rural and urban participants identified similar approaches to fall prevention. Annual wellness visits were identified as the most common time that falls are addressed, and participants identified common fall risk factors (e.g. fall history, FRIDS, stairs, co-morbidities). Characteristics of recipients were also common across sites with older adults reluctant to disclose falls not resulting in injury. Both staff and older adults described a trusting relationship as important to addressing fall risk and that addressing fall risk was important. When participants identified desired intervention elements, they consistently requested brief CCDS that was evidence-based and provided tailored recommendations. Based on common approaches to the core task of fall prevention, development of a common core intervention seems feasible.
The journey map depicting fall risk management revealed few differences between sites. The most common differences impacting CCDS integration were related to system usage and capabilities (e.g. printing and patient portal use). Urban staff were able to attach patient handouts electronically to the after-visit summary which were shared through the patient portal. Many urban staff participants used this workflow and expressed concerns about printing handouts directly due to printer location and issues with printer mapping. Conversely, the rural site was not able to electronically attach handouts to the visit summary and did not express printing concerns. These differences will need to be addressed as part of site-specific implementation to ensure workflow integration because interventions that do not easily integrate into existing workflows are often abandoned.
In addition to workflow compatibility, CCDS interventions leveraging patient-specific EHR information are preferred by staff and patients. The ability to pull information into the CCDS and use it to make recommendations decreases cognitive burden on staff and decreases the number of times the same questions are asked. However, as of 2015 only 40% of EHRs could use EHR data to make tailored recommendations (Braunstein, 2018). One way to address this barrier would be to develop CCDS as middleware that can pull selected information (e.g. age, gender, diagnoses) from the EHR into the CCDS resulting in individualized recommendations. By developing CCDS as middleware, these tools can be used across different EHRs and data could be pulled from the EHR using FHIR services and could be used to generate tailored recommendations. This would support a common intervention or CCDS that could be used in differing settings and technical systems.
Our study found notable differences in organizational structures that would impact CCDS implementation. The urban site was part of a larger hierarchical organization that valued uniformity and pushed policies from the top down, including standardized appointment durations regardless of appointment type. In contrast, the rural site had a smaller flatter organizational structure and was able to determine appointment durations based on appointment type. This allowed the rural site more time to address preventative items like fall risk management during annual wellness visits because they had 60 minutes for this appointment type compared with 30 minutes at the urban site. Differences in appointment duration and regular patient portal utilization by urban older adults may have influenced the urban site to send screening questionnaires ahead of annual wellness visits allowing more time to address screener results during the appointment. Other differences were found in the technical system related to how the different EHRs displayed fall risk and system capabilities in sending education materials to patients. These differences highlight the importance of developing site-specific implementation plans even for the same core intervention.
Strengths
This study used multiple methods to triangulate information and reduce participant recall bias (Flick, 2018). To our knowledge, ours is the only study that compares fall prevention practices between urban and rural primary care practices. Inclusion of both staff and older adults also provides a more comprehensive picture of the fall prevention process compared with studies that focus on a single stakeholder group. Lastly, using theoretical frameworks that support implementation organizes our results in a way that highlights findings important to sustainable intervention development and implementation.
Limitations
While our sampling methodology was appropriate for qualitative data collection, our use of non-probabilistic sampling may have introduced some selection bias (Daly & Lumley, 2002). Our sampling methodology and size may also limit generalizability. Utilization of rural and urban sites was a strength. However, it did limit the number of possible participants due to the rural site staff size. This limitation meant that the same rural staff participants were asked to participate in each data collection method. While the urban site had more potential participants, site-specific COVID restrictions prevented in-person workflow observations. In addition, due to the US healthcare system being more specialist focused our results may differ from countries that emphasize primary care and prevention.
Conclusion
Overall, primary care staff and older adults reported valuing fall prevention. However, appointment durations, competing clinical demands, and lack of clinical or community resources were barriers to evidence-based fall prevention. CCDS tools that engage both staff and older adults, integrated into existing EHRs and clinical workflows could increase fall prevention engagement and bridge identified skills gaps. When developing CCDS for differing settings, teams must balance standardization with the ability to tailor tools to specific clinical situations, technical systems, and external environments. Creating middleware leveraging interoperability standards is one potential solution.
Supplementary Material
What this paper adds
Comparison between urban and rural primary care fall prevention practices
Identifies specific fall prevention skills gaps in primary care
Understanding of fall prevention experiences and future desires from both primary care staff and older adults
Applications of study findings
Important factors to be aware of when implementing CCDS and prevention guidelines
Recommended solutions to develop sharable CCDS for different settings
Acknowledgments
Dykes and Lucero contributed equally to the development of the manuscript as co-senior investigators.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Agency for Healthcare Research and Quality [grant number U18HS027557].
Declaration of Conflicting Interests
The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Kristen Shear recently completed her PhD with funding from the U.S. Army’s Long-term health and education program. This work was submitted as part of her dissertation and fulfill some requirements for graduation. Request to re-print as part of her dissertation will be requested at a future date. “The views expressed herein are those of the author(s) and do not reflect the official policy or position of Brooke Army Medical Center, the Department of Defense, or any agencies under the U.S. Government”.
Footnotes
IRB protocol approval number
Mass General Brigham: 2020P002075
University of Florida: CED000000426
Supplemental Material
Supplemental material for this article is available online.
References
- Assurance N. C. f. Q. (2021). Fall risk management. National Committee for Quality Assurance. Retieved 4/25/2022 from https://www.ncqa.org/hedis/measures/fall-risk-management/ [Google Scholar]
- Avin KG, Hanke TA, Kirk-Sanchez N, McDonough CM, Shubert TE, Hardage J, & Hartley G, Academy of Geriatric Physical Therapy of the American Physical Therapy Association (2015). Management of falls in community-dwelling older adults: Clinical guidance statement from the academy of geriatric physical therapy of the American physical therapy association. Physical Therapy, 95(6), 815–834. 10.2522/ptj.20140415 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bae J, Hockenberry JM, Rask KJ, & Becker ER (2017). Evidence that electronic health records can promote physician counseling for healthy behaviors. Health Care Management Review, 42(3), 258–268. 10.1097/hmr.0000000000000108 [DOI] [PubMed] [Google Scholar]
- Bergen G, Stevens M, & Burns ER (2016). Falls and fall injuries among adults aged≥ 65 years—United States, 2014. MMWR. Morbidity and Mortality Weekly Report, 65(37), 993–998. 10.15585/mmwr.mm6537a2 [DOI] [PubMed] [Google Scholar]
- Borsky A, Zhan C, Miller T, Ngo-Metzger Q, Bierman AS, & Meyers D. (2018). Few Americans receive all high-priority, appropriate clinical preventive services. Health Affairs (Project Hope), 37(6), 925–928. 10.1377/hlthaff.2017.1248 [DOI] [PubMed] [Google Scholar]
- Braunstein ML (2018). Health informatics on FHIR: How HL7’s new API is transforming healthcare. In (pp. 13–29). Springer. [Google Scholar]
- Bryan C, & Boren S. (2008). The use and effectiveness of electronic clinical decision support tools in the ambulatory/primary care setting: A systematic review of the literature. Informatics in Primary Care, 16(2), 79–91. 10.14236/jhi.v16i2.679 [DOI] [PubMed] [Google Scholar]
- Burla L, Knierim B, Barth J, Liewald K, Duetz M, & Abel T. (2008). From text to codings: Intercoder reliability assessment in qualitative content analysis. Nursing Research, 57(2), 113–117. 10.1097/01.NNR.0000313482.33917.7d [DOI] [PubMed] [Google Scholar]
- Centers for Disease Control and Prevention. (2015). Falls among older adults: An overview. Disease Control. Retrieved June 20, 2015 from http://www.cdc.gov/ [Google Scholar]
- Clinical Decision Support. (2018). The Office of the national coordinator for health information technology. Retrieved 3/7/2021 from https://www.healthit.gov/topic/safety/clinicaldecision-support
- Daly J, & Lumley J. (2002). Bias in qualitative research designs. Australian and New Zealand Journal of Public Health, 26(4), 299–300. 10.1111/j.1467-842x.2002.tb00174.x [DOI] [PubMed] [Google Scholar]
- Dykes PC, Duckworth M, Cunningham S, Dubois S, Driscoll M, Feliciano Z, Fevrin FE, Lyons S, Lindros ME, Monahan A, Paley MM, Jean-Pierre S, Scanlan M, & Ferrazzi M. (2017). Pilot testing fall TIPS (tailoring interventions for patient safety): A patient-centered fall prevention toolkit. Joint Commission Journal on Quality and Patient Safety, 43(8), 403–413. 10.1016/j.jcjq.2017.05.002 [DOI] [PubMed] [Google Scholar]
- Feldstein AC, & Glasgow RE (2008). A practical, robust implementation and sustainability model (PRISM) for integrating research findings into practice. Joint Commission Journal on Quality and Patient Safety, 34(4), 228–243. 10.1016/s1553-7250(08)34030-6 [DOI] [PubMed] [Google Scholar]
- Flick U. (2018). In Steele M. (Ed.), Designing qualitative research (2nd ed.). Sage. [Google Scholar]
- Ganz DA, & Latham NK (2020). Prevention of falls in community-dwelling older adults. The New England Journal of Medicine, 382(8), 734–743. 10.1056/NEJMcp1903252 [DOI] [PubMed] [Google Scholar]
- Gibbons S. (2018, December 9, 2018). Journey mapping 101. Nielsen Norman Group. Retrieved 03/10/2021 from. [Google Scholar]
- Hsieh HF, & Shannon SE (2005). Three approaches to qualitative content analysis. Qualitative Health Research, 15(9), 1277–1288. 10.1177/1049732305276687 [DOI] [PubMed] [Google Scholar]
- Jansen S, Schoe J, van Rijn M, Abu-Hanna A, Moll van Charante EP, van der Velde N, & de Rooij SE (2015). Factors associated with recognition and prioritization for falling, and the effect on fall incidence in community dwelling older adults. BMC Geriatrics, 15, 169. 10.1186/s12877-015-0165-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kawamoto K, Houlihan CA, Balas EA, & Lobach DF (2005). Improving clinical practice using clinical decision support systems: A systematic review of trials to identify features critical to success. Bmj, 330(7494), 765. 10.1136/bmj.38398.500764.8F [DOI] [PMC free article] [PubMed] [Google Scholar]
- Krist AH, Baumann LJ, Holtrop JS, Wasserman MR, Stange KC, & Woo M. (2015). Evaluating feasible and referable behavioral counseling interventions. American Journal of Preventive Medicine, 49(3 Suppl 2), S138–S149. 10.1016/j.amepre.2015.05.009 [DOI] [PubMed] [Google Scholar]
- Kurth AE, Krist AH, Borsky AE, Baumann LC, Curry SJ, Davidson KW, Doubeni CA, Epling JW Jr, Fan T, Garćıa FAR, Herzstein J, Phillips WR, Pignone MP, Tseng, & Weinstein R. (2018). U.S. Preventive services task Force methods to communicate and disseminate clinical preventive services recommendations. American Journal of Preventive Medicine, 54(1s1), S81–s87. 10.1016/j.amepre.2017.07.004 [DOI] [PubMed] [Google Scholar]
- Medicare Wellness Visits. (2021). Medicare learning network. Retrieved 1/5/2022 from https://www.cms.gov/Outreach-and-Education/Medicare-Learning-Network-MLN/MLNProducts/preventive-services/medicare-wellness-visits.html
- Moreland B, Kakara R, & Henry A. (2020). Trends in nonfatal falls and fall-related injuries among adults aged ≥65 Years — United States, 2012–2018. MMWR. Morbidity and Mortality Weekly Report, 69(27), 875–881. 10.15585/mmwr.mm6927a5 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Oosthuizen R, & Pretorius L. (2016). Assessing the impact of new technology on complex sociotechnical systems. South African Journal of Industrial Engineering, 27(2). 10.7166/27-2-1144 [DOI] [Google Scholar]
- Prathivadi P, Buckingham P, Chakraborty S, Hawes L, Saha SK, Barton C, Mazza D, Russell G, & Sturgiss E. (2022). Implementation science: An introduction for primary care. Family Practice, 39(1), 219–221. 10.1093/fampra/cmab125 [DOI] [PubMed] [Google Scholar]
- Rice H, Garabedian PM, Shear K, Bjarnadottir RI, Burns Z, Latham NK, & … Dykes PC (2022). Clinical Decision Support for Fall Prevention: Defining End-User Needs. Appl Clin Inform, 13(3), 647–655. 10.1055/s-0042-1750360 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Saldana J. (2021). In Seaman J. (Ed.), The coding manual for qualitative researchers (4th ed.). Sage. [Google Scholar]
- Saunders B, Sim J, Kingstone T, Baker S, Waterfield J, Bartlam B, Burroughs H, & Jinks C. (2018). Saturation in qualitative research: Exploring its conceptualization and operationalization. Quality and Quantity, 52(4), 1893–1907. 10.1007/s11135-017-0574-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Sutton RT, Pincock D, Baumgart DC, Sadowski DC, Fedorak RN, & Kroeker KI (2020). An overview of clinical decision support systems: Benefits, risks, and strategies for success. NPJ Digital Medicine, 3, 17. 10.1038/s41746-020-0221-y [DOI] [PMC free article] [PubMed] [Google Scholar]
- Tanzini M, Westbrook JI, Guidi S, Sunderland N, & Prgomet M. (2021). Measuring clinical workflow to improve quality and safety. In Donaldson L, Ricciardi W, Sheridan S, & Tartaglia R. (Eds.), Textbook of patient safety and clinical risk management (pp. 393–402). Springer International Publishing. 10.1007/978-3-030-59403-9_28 [DOI] [PubMed] [Google Scholar]
- Trinkley KE, Kahn MG, Bennett TD, Glasgow RE, Haugen H, Kao DP, Kroehl ME, Lin CT, Malone DC, Matlock DD, & Matlock DD (2020). Integrating the practical robust implementation and sustainability model with best practices in clinical decision support design: Implementation science approach. Journal of Medical Internet Research, 22(10), Article e19676. 10.2196/19676 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Tung EE, Vickers KS, Lackore K, Cabanela R, Hathaway J, & Chaudhry R. (2011). Clinical decision support technology to increase advance care planning in the primary care setting. The American Journal of Hospice and Palliative Care, 28(4), 230–235. 10.1177/1049909110386045 [DOI] [PubMed] [Google Scholar]
- Viitanen J. (2011). Contextual inquiry method for user-centred clinical IT system design. Studies in Health Technology and Informatics, 169, 965–969. [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.