Abstract
Summary
We assessed the context in which a hip Fracture Liaison Service was implemented. We conducted semi-structured interviews with 21 key informants at two time points to understand organizational readiness, facilitators, and barriers to change. We identified strategies important to successful implementation, particularly in the context of change fatigue.
Purpose
Fracture Liaison Service (FLS) is effective for secondary fracture prevention. Two hospital sites implemented FLS for hip fracture patients, 50 + years, in Alberta, Canada. We assessed organizational readiness, facilitators, and barriers to change to better understand the context in which the FLS was implemented to inform its potential spread provincially.
Methods
We recruited individuals involved in FLS implementation at provincial and site levels to participate in telephone interviews at baseline and 16 months post-implementation. Interviews were transcribed and analyzed using thematic content analysis. In addition, site-level participants were invited to complete the Organizational Readiness to Implement Change tool at baseline.
Results
We conducted 33 semi-structured interviews (20 at baseline; 13 at post-implementation) with 21 key informants. Participants included managers (24%), FLS physicians/clinical nurses (19%), operational/leadership roles (19%), physicians/surgeons (14%), pharmacists (10%), nurse practitioners (10%), and social work (5%). Seventeen site-level participants completed the ORIC tool at baseline; all participants scored high (71%) or neutral (29%). We found that the use of several strategies, including demonstrating value, providing resources, and selecting appropriate sites, were important to implementation, particularly in the context of change fatigue. Participants perceived the FLS as acceptable and there was evidence of facilitated learning rather than simply monitoring implementation as intended.
Conclusions
An effective change management approach neutralized change fatigue. This approach, if maintained, bodes well for the potential spread of the FLS provincially if proven effective and cost effective. Change readiness assessment tools could be used strategically to inform the spread of the FLS to early adopter sites.
Electronic supplementary material
The online version of this article (10.1007/s11657-020-0692-0) contains supplementary material, which is available to authorized users.
Keywords: Osteoporosis, Fracture liaison service, Change readiness, Qualitative
Introduction
Osteoporosis is associated with increased morbidity and mortality and decreased quality of life. [1, 2] Osteoporosis care reduces risk of future fractures by about 50%. [3, 4] Regardless, fewer than 20% of adults, 50 years of age and older, are tested or treated for osteoporosis after a low-trauma fragility fracture [3, 4] generating a substantial and unwarranted “care gap.” Fracture liaison services (FLS) are an evidence-based and effective approach to secondary fracture prevention utilizing a 3i approach to secondary fracture prevention (i.e., Identify those at risk, Investigate using bone mineral density tests, and Initiate appropriate treatment when needed). [5–7]
The FLS model has been found effective for secondary fracture prevention in Canada and internationally [7, 8]. Despite promotion of FLS with a best practice framework by the International Osteoporosis Foundation [6, 9] and advocacy by Osteoporosis Canada [7], implementation of a coordinator-based post-fracture model of care is not without barriers. Implementing change can be difficult. Implementation outcomes serve as indicators of implementation success and, when implementation is successful, are the effects of purposeful and deliberate actions to implement new treatments, practices, and services [10]. When a newly implemented healthcare innovation fails to achieve the expected benefits, it is often attributable to ineffective implementation rather than issue with the innovation itself [11]. Challenges can emerge from individual and organizational factors and obstacles in the local environment [11–13]. Therefore, understanding the context in which implementation occurs can shed light on the implementation outcomes.
Rogers’ diffusion of innovations theory [14] proposed categories to describe the adoption of innovations. Innovators and early adopters lead the way, followed by the early and late majority, and finally the laggards take up the innovation. When implementing a new healthcare service, investigating the experience of the early adopters can help to identify the facilitators and barriers to implementation and provide valuable lessons learned ahead of the spread of the innovation through the rest of the system.
As part of an ongoing collaboration to improve secondary fracture prevention in Alberta, the Bone and Joint Health Strategic Clinical Network (BJH SCN), Alberta Health Services (AHS), and the Alberta Bone and Joint Health Institute (ABJHI) partnered with 2 hospital sites to implement a Type A [15] FLS. In brief, the FLS identified patients age 50 or older with a hip fracture within the hospital setting, commenced appropriate osteoporosis care, bridged the acute-to-post-acute period and the return to pre-fracture living environment, and, finally, transitioned care back to the family physician. The FLS utilized the expertise and continuity of clinical nurses and a physician at each site to serve as a common source of information and connection to the healthcare system, with the clinical nurses following patients for 1-year following the hip fracture. In advance of broader service rollout, we assessed the 2 early adopter sites to understand the context in which the FLS was implemented, including readiness to implement change as well as facilitators and barriers to implementation, to inform its potential spread provincially (Fig. 1).
Fig. 1.
Perspectives represented by participants when commenting on the context in which the H-FLS was implemented. (Note: Due to rounding error, percentages do not equal 100)
Methods
This qualitative study was situated as part of a larger comprehensive, pragmatic mixed-methods evaluation of three secondary fracture prevention initiatives in Alberta, Canada. Focused on the implementation of FLS, in this study, we conducted a qualitative contextual assessment of the service as it unfolded naturally at two early adopter sites. The qualitative design allowed us to elicit and understand the experience and perceptions of individuals directly involved in FLS implementation and service delivery.
Data collection
We used purposeful and snowball sampling to identify key informants with knowledge of the implementation and/or a clinical role in the FLS at the provincial level and at the two sites. Potential key informants were identified in collaboration with the Fragility and Stability Program Manager and the Unit Managers at each site, who also sent an introductory email about the study. Interested participants contacted the research team by telephone or email with questions and to arrange participation. Participants were invited to participate in telephone interviews at 2 time points: at baseline during the initial implementation phase and at approximately 16 months post-implementation. In addition, participants were asked to identify other individuals with knowledge of the FLS; newly identified individuals were invited to participate in the study using the same strategy outlined above. Lastly, individuals at the site level who participated in baseline interviews were invited to complete the Organizational Readiness for Implementing Change (ORIC) tool [13]. ORIC is a 10-item tool which measures readiness for change on a 5-point Likert scale and includes the sub-domains of change commitment and change efficacy. Participants received the ORIC questionnaire by email following their baseline interview, with instructions to complete and return the questionnaire.
A trained researcher (LAW) conducted the telephone interviews using semi-structured interview guides at baseline and post-implementation (Online Resource). The interview guides were refined as data analysis progressed. Interviews lasted up to 90 min. Interviews were digitally recorded and transcribed for subsequent analysis and verified for accuracy. The study received Research Ethics Board approval from the Universities of Alberta and Calgary and all participants provided written informed consent.
Data analysis
We used thematic analysis [16, 17] to analyze the data. One researcher (LAW) identified emerging codes and concepts and discussed with the study lead (HMH). The research team held regular meetings to review code definitions, emerging concepts, and memos and discussed discrepancies to reach consensus. We conducted data collection and analysis until thematic (and role) saturation was achieved [18] as was appropriate based on the design and objective of the study [19]. Qualitative data were managed and analyzed using ATLAS.ti Version 7 [20]. Descriptive statistics were conducted for the ORIC questionnaire, with data entered and managed using SPSS Statistics for Windows Version 25 [21]. Total and sub-scale scores were calculated by the sum across items. Guided by the questionnaire response scale anchors, we categorized participant scores as low (10–29), neutral (30–39), or high (40–50), with higher scores representing more favorable Organizational Readiness for Implementing Change through the FLS. We implemented several strategies throughout the research process to ensure rigor including methodological coherence [22], peer debriefing [23], and maintaining an audit trail [23]. In addition, we synthesized and presented the findings using thematic statements, which is a strategy to make findings clear and actionable. [24]
Results
In total, we conducted 33 semi-structured telephone interviews with 21 key informants at the 2 time points. We conducted 20 baseline interviews between July 2015 and January 2016, and 13 post-implementation interviews between October 2016 and March 2017. Participants represented a variety of roles, including managers (24%), FLS team members (i.e., physicians and nurses) (19%), those in leadership or operational roles (19%), physicians or surgeons (14%), pharmacists (10%), nurse practitioners (10%), and social work (5%). Of those who participated in a baseline interview, 18 were invited to complete the ORIC, 17 of which returned the questionnaire. Two participants were not eligible to complete a questionnaire due to their role or because they did not participate at baseline and 1 participant was unresponsive. All 17 participants who completed the questionnaire scored their organization’s readiness to implement change as high or neutral. There was no difference in ORIC scores between the two sites (t (15) = − 0.62, p = 0.54). We provide a summary of the participant characteristics and ORIC scores in Tables 1 and 2.
Table 1.
Organizational Readiness for Implementing Change (ORIC) scores
Respondent ORIC scores | Count (%) | |
High (40–50) | 12 (71) | |
Neutral (30–39) | 5 (29) | |
Low (10–29) | 0 (0) | |
Site ORIC Scores |
Site 1 Mean (SD) |
Site 2 Mean (SD) |
ORIC score | 41.1 (6.5) | 42.9 (4.9) |
Change commitment sub-scale | 20.7 (3.7) | 21.8 (3.0) |
Change efficacy sub-scale | 20.4 (3.1) | 21.1 (2.4) |
ORIC scores were calculated as the sum of 10 questions scored on a 5-pt Likert scale (where 1 = disagree and 5 = agree), for a maximum score of 50
Table 2.
Examples of participant quotes in support of identified themes
Codes | Sub-codes | Supporting quotes |
---|---|---|
Theme 1—Use of strategies in context of change fatigue | ||
Demonstrating the need for or value of the H-FLS | Communicating the benefits for patients and/or providers |
I think if you are able to convey the need and provide the evidence and help people understand there a significant care gap. Even though this is an additional program and additional step…people are willing to come on board and be enthusiastic to support the program (Site 2, Participant 3). It’s important that people see how this actually supports the patient and I think they do (Site 1, Participant 6). People need to know why something is being done in order to have buy in. They need to be able to have something that you can latch on to. To say, “Ok, this thing - an extra step, extra employee, and extra set of paperwork is worth it because of this benefit” and I think with implementation as it is, we have seen that (Site 2, Participant 1). Good education, information of what’s in it for them, why it’s good for the patient, why it’s a necessity to move forward, and change. Those are all primary things for effective change management (Site 2, Participant 7). |
Target patient population had become a priority |
The FLS is part of the Bone and Joint Health Strategic Clinical Network and it’s also the Bone and Joint Network signature project and the FLS falls under that. So we have had good support from a leadership perspective across the province and I think that will continue because it has been made a priority (Provincial Stakeholder, Participant 1). I think across the board everyone realized that osteoporosis is a big and costly issue for both patients and the system (Site 2, Participant 8). I think even from a higher level it’s having the FLS and osteoporosis care identified as a care gap by the AHS was what really started all of this. And having them focus on osteoporosis care and providing their funding to run the program like this and continue to fund it in the future if there is benefit to the program is more important than the local factors at the PLC (Site 2, Participant 3). The Bone and Joint SCN have gone to them [beta hospital] and asked them to become the center of excellence in orthopedic care in this area so that in itself, you know, when somebody sets it as a priority area, it becomes a facilitator (Site 1, Participant 8). |
|
Providing resources so as not to increase workload of others |
I do not think that the FLS is adding on to anyone else’s workload. If anything I think it’s helping other people’s work load so I think that’s a key thing with this initiative, we are not piling anything onto anyone else (Site 1, Participant 1). It’s someone else’s job and it’s not being added to the plate to the current staff (Site 1, Participant 4). It’s when changes occur that creates more work for them that they do not see value in it (Site 1, Participant 9). I think the one thing is that it does not add work load to other people and so it in fact does not impact other people at all. So that’s helpful in that it’s not yet one more thing someone has to do or learn or whatever (Site 2, Participant 5). It’s a fairly non-invasive program. It’s not one that demands a lot of buy in or extra work from the staff on the ground level (Site 2, Participant 8). We’ve been very mindful of anything that we have put in cannot add additional workloads to them (Provincial Stakeholder, Participant 1). |
|
Appropriate site selection | Early adopters or culture open to change |
Well, I’ve been told that [name of site] is a popular site for trying pilot projects, which shows that they are open to change or new initiatives (Site 1, Participant 3). I think we are one hospital that is really keen on trying out new projects initiated… I think it’s just that culture and the staff of course willing to participate in and initiate its own programs to improve the overall patient care (Site 2, Participant 6). It’s the people. The administration team at the [site name] was all for it. Let us go for it…. it’s just a group that’s open and willing to look at something or introduce something or participate in something when they know it could be of benefit to the patient (Site 2, Participant 7). We volunteered. We said, “Let us try it”, right? And having somebody say, “You guys need to do this now. It’s your turn”. That’s not always attractive (Site 1, Participant 2). |
Being committed to the patient population or content area |
You really do have to find someone [referring to FLS clinical nurse role] who has a passion, not only for osteoporosis therapy and management but for kind of the overall care of patients. You have to recognize that it’s not just a prescription or it’s not just an order, that these are people (Site 1, Participant 1). I often think it’s people, not positions because if you have somebody who’s, you know, just like our FLS team, it’s going to fly. You could have somebody with the same position and not have any drive or passion and it would just flop. So I always believe it’s people and what they believe in and what they want to bring. It’s their passion and their commitment to what they want to do for the patients (Site 1, Participant 2). There’s a bunch of different stake holders. There’s the SCN, there’s the physician, there’s the orthopedic service, there’s the hospital admin. And they just generally have an understanding or a will to want to do it (Site 1, Participant 3). I’ve said this in the past that I’m not so sure why everyone worries about the bird flu. We’ve already had a pandemic. It’s called osteoporosis (Site 1, Participant 9). I have a great team. Most of them are just so passionate about hip fractures that they are open to all of these changes (Site 2, Participant 1). When they recruited the nurse clinician of the [name of 2nd site], she was just so passionate about the program. So you are passionate, you just get things going (Site 2, Participant 6). |
|
Importance of champions of the service |
We had champions at those sites that were willing to stand up and say I want to lead this. We when we do program planning [we] recognize the barriers for making clinical change and lack of a champion is probably the biggest risk to any implementation project. So the fact that we had champions at those two sites really to stand up and take ownership of the program and to run the program, it raised those two sites to the top of the list (Provincial Stakeholder, Participant 2). Well you need champions… I think that particularly that physician champion is essential (Site 1, Participant 1). I’ve been helping with identifying stakeholders, communicating how the plans will work, how to implement it on site and providing support to the FLS nurse and whatever issues that pop up with trying to identify patients. Or communication with staff in acute care. But then also meeting with the primary care physicians to try and help with that transition or communication between our team and primary care (Site 1, Participant 3). We had a champion on board… having that engaged physician, elder-friendly care physician, willing to participate and the FLS nurse (Site 2, Participant 7). We have a physician and we have a nurse and they were the main primary drivers of getting this thing up and rolling… I’m involved in the provincial working groups and the [name of city] zone working groups and [the FLS clinical nurse] always there at the table with us. So to be able to have her to speak to the program as well, she was like a promoter (Site 1, Participant 2). The nurse practitioner herself was a champion (Site 2, Participant 6). |
|
Learned from past experiences with change and applied learnings to H-FLS implementation |
It’s been foundational to how FLS has been designed from the get go. We’ve borrowed very heavily from past experience on implementing the Hip and Knee Care Pathway and implementing Catch a Break. All of these programs have influenced how we designed and rolled out the FLS right from site selection to the fact that we are doing a staged roll out through to how we communicate, how we engage with our stakeholders or how we even organized the content development and delivery of this program (Provincial Stakeholder, Participant 2). As long as things are as simple as possible for everyone else that really helps with implementation; we have learned that in other change as well (Site 2, Participant 1). But basic learnings from regular change management in terms of good communication, good education or information of what’s in it for them, why it’s good for the patient, why it’s a necessity to move forward, and change. Those are all primary things for effective change management (Site 2, Participant 7). |
|
Context of change fatigue; however, it was not perceived as a barrier to implementing the H-FLS | Current context of change fatigue independent of the H-FLS |
There’s a certain fatigue of having these pilots come in and once the funding runs out, having them canceled (Site 1, Participant 3). I think its health care in general. There has been so much change even in the last two years and it just never stops coming…So there is a sense among staff I think throughout healthcare that, you know, we are burned out, we cannot do anymore (Site 1, Participant 4). There’s a lot going on. A lot of different initiatives (Site 1, Participant 6). The ortho world has had so many different things implemented in it over the last couple of years in terms, you know, we have implemented pathways, we have implemented forms and check lists and all kinds of things and I think we can learn a lot from that (Site 2, Participant 1). |
Change fatigue was not perceived as a barrier to H-FLS implementation |
I have not witnessed any change fatigue in this area...It wasn’t that they were fatigued with change. They were fatigued with change that did not go anywhere (Provincial Stakeholder, Participant 2). We’ve had some changes recently that I’m bucking hard against. But this is one [referring to the FLS], that I’m most certainly not opposing at all (Site 1, Participant 7). In AHS, we are used to change so I do not think it would have much of an influence. I think people would be accepting of a new service like FLS (Site 2, Participant 4). |
|
Theme 2—The FLS was perceived as acceptable | ||
Clear and consistent understanding of the intent of the H-FLS | 3i model |
The 3i’s are to identify the case, to investigate through them, and then to help implement some of the treatments (Site 1, Participant 3). The FLS nurse [is] doing the assessment to identify the patients who qualify for the FLS program and she is essentially doing all the work up to determine whether osteoporosis therapy is required (Site 2, Participant 4). So the FLS is really designed to identify these patients while they are still in hospital and really get them on the care pathway for osteoporosis treatments and make sure they are getting the right diagnostics, make sure they are getting the right drug therapy and the proper imagining and the necessary tests are being ordered (Provincial Stakeholder, Participant 2). |
Secondary prevention of falls or fractures |
That they could try to prevent a new fracture from re-occurring (Site 1, Participant 2). Generally the intent is to improve post fracture osteoporotic treatment as well as overall reduce future falls or risk (Site 1, Participant 3). Hopefully preventing further fractures (Site 1, Participant 10). If we are just focusing on osteoporosis therapy we are missing kind of a bit of a bigger picture and so you know we are also looking at fall prevention…we certainly want to prevent fractures which not only includes osteoporosis therapy but also fall prevention (Site 2, Participant 5). |
|
To bridge a care gap |
It’s not that they [primary care providers] do not want to provide osteoporosis therapy or teach. But sometimes they do not have the time or the ability to recognize the need or the ability to provide that education. So I think that having the FLS there is helping the providers (Site 1, Participant 1). I suspect there was a big gap there from when patients were discharged as far as flagging that for primary care physicians. Then to initiate treatment. And even then, there are some assessments there that just did not happen (Site 1, Participant 3). My understanding of the FLS is to bridge a gap in terms of ensuring those patients have osteoporosis or fragility fractures that they are started on the appropriate treatments when they present to hospital and before they leave the hospital. Because we know that a significant majority of these patients whether they have been diagnosed, they have not been treated with the appreciate medications (Site 2, Participant 3). There’s been a huge care gap with patients. The population that we are dealing with is the hip fracture patients. So the orthopedic surgeons are terrific at fixing the fracture but then they do not do any follow up as far as cause of fracture i.e. osteoporosis. So that’s a huge care gap as patients who have fractured and especially hip fracture are at high risk or re-fracturing again (Site 2, Participant 5). There are these monumental gaps in communication, coordination in services, as a hip fracture patient recovers from their hip fracture and they transition out of an acute care environment (Provincial Stakeholder, Participant 2). |
|
Transition patients to primary care |
And then in respect to communications with transitions, the FLS does provide communications to the family physicians which I would think that they would appreciate it. I think sometimes they are surprised to hear that their patient has been in the hospital with a hip fracture, that they find out months later and then are kind of left wondering what to do. And here we are sending them a letter saying, “your patient has had a hip fracture and this is some of the things that we have done. We’ve started this medication; we are doing these investigations”, to help them out because they may not often be comfortable with orthopedic care or osteoporosis management (Site 1, Participant 1). When they were sent back to their community living arrangement there was a plan of care for the primary care provider to understand what the therapy was to continue on in their future (Site 2, Participant 2). The care that is provided to the patients will be passed on to their primary care physician to ensure seamless care and treatment of patients with osteoporosis (Site 2, Participant 4). Whatever we do initially whether that’s starting therapy in hospital or sending patients home with a prescription…we send a letter to their GP [general practitioner] and then also at 3, 6, 9 and 12 months. We’ll also send another letter to their GP and just let them know what we have done and the plan is that at 3 months we’ll do a Vit D level as well as a bone mineral density test if they have not had one in the last couple of years. And those two a copy of those will go to their GP (Site 2, Participant 5). We’ve done a year of this focused work of getting [patients] on the right trajectory. Now we do a hand off to primary care saying this patient now has a detailed care plan. It’s really now under primary care accountability to manage that chronic disease and manage that osteoporosis similar to how they have managed their diabetes or cardio vascular health or any other chronic disease situation…the FLS is really designed to be that transition period (Provincial Stakeholder, Participant 2). |
|
Expanded scope beyond 3i model |
My role is basically to help assess and guide therapeutic plan with regards to these patients post hip fractures, especially with their osteoporosis care, medications to address that. As well as flagging any geriatric syndromes…from my geriatric point of view is our success rate with getting these people into other services (Site 1, Participant 3). They’re [FLS physician and clinical nurse] reviewing not just the patient’s bone health, but just overall medical history and providing suggestions and recommendations. And if the patient does not require a full geriatric assessment he [FLS physician] will provide those suggestions - perhaps outpatient geriatric referral’, ‘perhaps rehab referral’ or ‘perhaps a day program referral’ for PT in the community. So he would provide some of those suggestions or if he had to catch the patient who is having ongoing delirium post-operatively, he can provide some preliminary recommendations, not involving the patient geriatric consult service but he’s able to try some suggestions on delirium management. So looking at the bone health and also looking at overall the patients’ health (Site 1, Participant 11). The other piece about the FLS service is beyond the osteoporosis management which is also identifying patients with reoccurring falls. So especially in the 65 [years old] and above. Identifying geriatrics syndrome, in cognitive impairment, delirium, dementia, depression or poly pharmacy…the experience in [name of beta site city] certainly has been, you know, not just identification of osteoporosis and treatment but also looking at all that complex medical problems from the pharmacy. And were seeing the same thing in [name of second site city] (Site 2, Participant 6). We decided to make ours a little bit broader in the sense that we did not want it just to be a medication type program. We also wanted to include some aspects of geriatric care because of the model that we developed and we were using care of the elderly physicians. So that was another piece to it. So we expanded the scope to include having physicians to start to identify those underlined geriatric syndromes that may be contributing to the fall initially. And then we also obviously deal with the osteoporosis medication and treatment piece of it but there is that added geriatric component (Provincial Stakeholder, Participant 1). |
|
Aligned with organizational values | Providing patient-centered care |
From the first day they realized that this could not be a cookie cutter approach. Not everyone was going to go on medication, not everyone wants to go on medication…You need to make sure that they are first educated and aware of their options. So I think that it’s wonderful why the FLS goes for a year. Because sometimes at the point of care, surgery, people aren’t ready to make some of these decisions. Maybe at 3 or 6 or 9 months down the road, when they are a little bit back to their normal self, maybe they can make some of these decisions (Site 1, Participant 1). It certainly helps to provide that greater education for patients so that then they can make informed decisions; patients and families and then their wants. We are also having to be patient centered, so giving them the options with the risks and benefits and then respecting whichever choice they choose to make (Site 1, Participant 4). We are still trying to ultimately preserve lives but then respect decisions of the patients themselves (Site 1, Participant 10) For me the FLS really aligns well with that because it is still patient centered (Site 2, Participant 1). I mean the rule of the [name of hospital] again that saying, “If it’s good for the patient, if its patient centered if it will improve patient care” we’ll buy into it (Site 2, Participant 6). The model of the program, it’s about encouraging patients to participate in osteoporosis care and prevention. To accept resources in the community. But there are certain situations – we are always balancing things off - the financial picture of the patient, their ability to access services in the future, and their ability to pay for medication. And work with the patient to come up with the best game plan for the patient that incorporates that information…you cannot force a patient to take a medication. That takes away their preference. You can provide the information and the supports available for the best courses of treatment (Site 2, Participant 7). |
Collaborative or multi-disciplinary approach to providing care |
Again the culture of our multidisciplinary approach and the very high likelihood that this is an intervention that we can do and make a difference for people (Site 1, Participant 7). We are very collaborative multidisciplinary team including rehab, PT, OT, the transition coordinator, pharmacy, the clinical associate/physician, and our FLS [clinical nurse]. We work together collaboratively to try to make sure that we are placing not just fractured hip patients but all of our patients on the right [path of care] (Site 1, Participant 9). There was a little bit of a [barrier] in terms of not crossing into people’s territories. But a wonderful collaborative relationship happened a lot with geriatric medicine. It’s amazing. I mean, we see notes from geriatrics saying, “FLS to recommend osteoporosis management” and I smiled when I first saw that (Site 2, Participant 6). |
|
Evidence-based but adapted or tailored | Evidence-based model adapted to Alberta context |
It’s encouraged in other guidelines like the NICE guidelines in the UK. It’s considered best practice as far as having it in your post-op post hip fracture care. So just for all the reasons as far as consensus amongst even the British orthopaedic and geriatrics societies have agreed upon this as the gold standard of treatment. So it’s one of the initiatives that seem to be an accepted standard in hip fracture care (Site 1, Participant 3). It’s specifically addressing the fact that this is a recommended guideline research evidenced based therapy that should be administered to most of the patients (Site 2, Participant 2). The evidence base for this program is very strong. We are not testing an unproven method here. We’re actually applying strong evidence that we know works. So this is one of those few areas in health care where there’s not a lot of controversy and not a lot of dispute over the value of this program. That’s a very unique situation. Most of our programs, they are far greyer in terms of the economic and quality of impact on health service delivery. Here we have got pretty cut and dried strong evidence based, a lot of that evidence also is made in Alberta. We’ve got lots of research history in this area here in Alberta (Provincial Stakeholder, Participant 2). Clinicians in the fragility and stability group have been excellent in getting this going. And this includes clinicians that extend past [this group], such as other physicians in the osteoporosis clinic. They have come together to provide knowledge…so that we could pick their brains as to what is important to put in [our] FLS. What is important to send to family physicians? I do not think we would have come up with all the different algorithms and tools that we are currently using in the FLS. I think having broad representation from people across the province, whether they are directly involved or not, has been just so important for this project (Site 1, Participant 1). They had a committee with of a lot of experience and education getting behind them. “Ok, here you go. These are all the specs we want you to include”. They developed their algorithms and then double checked it with all the medical guidelines and then proceeded with it (Site 2, Participant 2). |
H-FLS tailored to site-level contexts |
This is a programme that was started up for the FLS in [name of beta site], so we literally took their template. Adjusted a few things for [name of city of second site] (Site 2, Participant 6). A little bit of negotiation, in terms of here’s what we are doing, how does it fit into your model? Because, you know, each of the sites is different…I really relied on those site leads to tell me what makes sense at their hospital site (Provincial Stakeholder, Participant 1). I suppose it’s then how even a unit runs, so when they do their rounds, when’s a good time to talk with the multidisciplinary team. Each site is so different and so that will be important going forward, is knowing what are their routines so that you can kind of almost again build yourself into their routine without disrupting it too much. Because, you know, you want it to be an add-on service; you do not want to be a hindrance to anything that they are doing (Site 1, Participant 1). The advice I would have for [new sites] is you have the model, you have the goal, you know what you want to achieve but it can look slightly different on how it unfolds at your particular site. So the first step is just doing a deep dive into your area, reviewing the resources that are available, reviewing who would interact with the clinic and just kind of defining slightly how it’s going to look at your site. Because it can look different. We’ve got the base model, the same goals we want to achieve but it can look different and probably will look different in your particular area. And I think if you looked at the three places that are set up now achieving the same goal, you can say it’s fulfilled the 3i model however things do look a little bit different between the three centres so far (Site 2, Participant 7). |
|
Theme 3—Evidence of facilitated learning, not merely monitoring implementation | ||
Willingness to share and accept lessons learned | During implementation |
The general structure, how the 3i model was rolled out, I think [it] went through a couple of modifications at the [name of beta site]. And that’s just learning from learning. So it was easier for us being the second team to go because they had already ironed out a couple of kinks (Site 2, Participant 7). We have learnt so much from [city of beta site] already so we had worked with [beta site] to figure out a medical algorithm to figure out the work sheet and [name of FLS nurse], who’s the nurse up in [name of city] she’s hugely supportive and had lots of really good tips for me for when I started. And when I started I could call her and say, “Ah, what do you do for this or that” and so lots of support that way (Site 2, Participant 5). This is a programme that was started up for the FLS in [city of beta site], so we literally took their template. Adjusted a few things for [city of 2nd site]. So, we learned from their mistakes, their good areas. We kind of learned from those. We learned from their successes, so that’s why I think we did well...they shared what worked well, what did not work well and what we should not do (Site 1, Participant 6). |
To inform spread |
I am the only nurse in [city of beta site] who knows this program so, you know, it’s a great opportunity to mentor new people (Site 1, Participant 1). Draw on our site to help promote that program in other sites as well (Site 1, Participant 2). When I do orientation and when I do think about what to tell new people that start, it’s like, you know, assessments are so important. And just the knowledge that I now know about the medications, I tell the new nurses rather than them having to figure it out for themselves (Site 2, Participant 5). |
|
Establishment of a provincial steering committee to facilitate learning in addition to monitoring implementation of the H-FLS across the province |
I think that mere fact that it was a staged roll out and they were looking at tools that were used in Alberta for several months but had been tweaked and revised based on learnings as the program matured. But the planning steps for that second site were probably half the time, half the energy level of what the original initial sites were. And as we roll out to [name of city] as a 3rd site, I’m seeing already that they are starting on step 3 not step 1. So it’s really reducing the planning stage. And that definitely contributes to simplifying implementation….We’re walking in with a tool kit ready to go turnkey implementation almost. You take it, you turn it to your needs and go forward. That has defiantly simplified the approach greatly (Provincial Stakeholder, Participant 2). A provincial FLS meeting twice a year and it’s that chance to get everyone together, say what’s going at your site, and we learned a lot from those (Provincial Stakeholder, Participant 1. There’s now been developed an FLS committee province-wide. So now we have other peers to run through decision-making with, and there are lots of decisions where we decide, okay, we are going to make this standardized between the teams. So that helps, having colleagues. Also, some of the local resources for osteoporosis care, but it’s hard to program, like, the nuts and bolts (Site 1, Participant 3). Thanks to [name of ABJHI PM] and her organization ability is meeting regularly as a provincial group. So, I think it’s about every six months that we meet. And it gives us a time to really look at things like our medical algorithm, our processes, our letters, and just have a really frank discussion with the players in the FLS to see, you know, what’s working well, what needs, to be tweaked, what are your challenges. So, I think ongoing support within the group (Site 2, Participant 5). |
Three major themes emerged. First, the FLS was implemented using several strategies that were essential to implementation success, particularly in the context of change fatigue. Second, participants perceived the FLS as acceptable in meeting the needs of this patient population. Third, there was clear evidence of facilitated learning that went beyond monitoring implementation of the FLS as intended. We detail each thematic statement below. We provide a list of codes, sub-codes, and additional quotes for each statement in Table 2. For all quotes, we included the study ID number for each participant.
Theme 1—Use of strategies in the context of change fatigue
There was evidence that the FLS was implemented at the two sites through the use of strategies that were vital to successful implementation, particularly within the context of reported change fatigue. This included demonstrating the need or value of the service, providing the necessary resources to implement it, and selecting appropriate sites. Demonstrating the need for or value of the FLS involved communicating the benefits of the service for patients and/or healthcare providers. One participant commented on communicating the value of FLS,
I think if you are able to convey the need and provide the evidence and help people understand there a significant care gap. Even though this is an additional program and additional step…people are willing to come on board and be enthusiastic to support the program (Site 2, Participant 3).
In addition, many participants indicated at baseline that the target patient population (i.e., hip fracture patients and/or seniors) had become a priority (e.g., identified by the BJH SCN and ABJHI as a priority, identified by AHS as a care gap in osteoporosis, and the introduction of senior-friendly initiatives). Another important strategy was providing the resources (i.e., human and/or financial) needed to implement the FLS, thereby not increasing the existing workload of others. Participants commented on the need to be mindful of staff workloads with comments such as,
I don’t think that the FLS is adding on to anyone else’s workload. If anything I think it’s helping other people’s work load so I think that’s a key thing with this initiative, we’re not piling anything onto anyone else (Site 1, Participant 1).
It’s a fairly non-invasive program. It’s not one that demands a lot of buy in or extra work from the staff on the ground level (Site 2, Participant 8).
However, some participants reported some role confusion at baseline, such as concerns over duplication of services or territorial issues. In addition, there were some concerns regarding the workload and coverage for the FLS physician and clinical nurse. Finally, appropriate site selection appeared to have been another important strategy used in implementing the FLS. This included partnering with sites with high change commitment and change efficacy as demonstrated by their ORIC scores, where participants scored their sites’ readiness to implement change as high (71%) or neutral (29%) at baseline, and as reported by participants during interviews. For example, participants described the people or sites involved as “early adopters” or as having a culture open to change. In speaking of the culture of openness to new ideas, one participant described the site as follows,
It’s the people. The administration team at the [site] was all for it. Let’s go for it…. it’s just a group that’s open and willing to look at something or introduce something or participate in something when they know it could be of benefit to the patient (Site 2, Participant 7).
Commitment to the patient population or content area was also identified as a facilitator to successful implementation, as was the importance of having champions of the service to support communication and implementation, as well as having learned from previous experiences with change and applying those learnings to the implementation of the FLS. In the following quotation, a participant described the learning process,
It's been foundational to how FLS has been designed from the get go. We’ve borrowed very heavily from past experience on implementing the [care pathway] and implementing [2i model for secondary fracture prevention]. All of these programs have influenced how we designed and rolled out the FLS right from site selection to the fact that we’re doing a staged roll out, through to how we communicate, how we engage with our stakeholders, or how we even organized the content development and delivery of this program (Provincial Stakeholder, Participant 2).
Of note, several participants described change fatigue independent of the FLS, including the introduction of several pilot programs and/or new initiatives related to the hip fracture patient population over a relatively short period of time. Regardless of the amount of change experienced, participants did not believe that change fatigue was/would be a barrier to implementing the FLS, often citing the strategies outlined above as the rationale for this opinion. The following participant commented,
In AHS, we are used to change so I don’t think it would have much of an influence. I think people would be accepting of a new service like FLS (Site 2, Participant 4).
Theme 2—Perceived acceptability of the FLS
Participants involved in the implementation and delivery of the FLS perceived it to be acceptable. That is, they had a clear understanding of its intent, believed it was aligned with organizational values, and appreciated that it was evidence-based but tailored to their local context. Participants described a clear and consistent understanding of the intent of the FLS related to the 3i model of “Identify” eligible patients, “Investigate,” and “Initiate” treatment, mainly medication, with the purpose of secondary prevention of fractures or falls or to bridge a care gap. In addition, many participants identified the transition of patients back to primary care, including sharing clinical information, as a key part of the FLS. In describing the letter sent to the primary care physician of FLS patients, this participant identifies communication of the occurrence of a fracture and the tests and treatments initiated as being of potential benefit to the patient’s continuing care,
And then in respect to communications with transitions, the FLS does provide communications to the family physicians which I would think that they would appreciate it. I think sometimes they are surprised to hear that their patient has been in the hospital with a hip fracture, that they find out months later and then are kind of left wondering what to do. And here we are sending them a letter saying, “your patient has had a hip fracture and this is some of the things that we’ve done… we’ve started this medication, we’re doing these investigations”, to help them out because they may not often be comfortable with orthopedic care or osteoporosis management (Site 1, Participant 1).
Of note, several participants indicated that the scope of the FLS was either intended to move beyond or had expanded beyond the 3i model, with a focus on medication initiation, to include identification of geriatric syndromes or issues (e.g., fall risk, urinary incontinence, dementia, and swallowing dysfunctions) with referral (or triage or red flagging) to additional services (e.g., geriatric unit in hospital or community, fall prevention program in community, home care, or osteoporosis clinics). The following participant described the broader focus,
We decided to make ours a little bit broader in the sense that we didn’t want it just to be a medication type program. We also wanted to include some aspects of geriatric care because of the model that we developed and we were using care of the elderly physicians. So that was another piece to it. So we expanded the scope to include having physicians to start to identify those underlined geriatric syndromes that may be contributing to the fall initially. And then we also obviously deal with the osteoporosis medication and treatment piece of it but there is that added geriatric component (Provincial Stakeholder, Participant 1).
In addition, participants indicated that the FLS was aligned with organizational values including providing patient-centered care and using a collaborative or multi-disciplinary approach, as described by these patients,
It certainly helps to provide that greater education for patients so that then they can make informed decisions; patients and families and then their wants. We are also having to be patient centered, so giving them the options with the risks and benefits and then respecting whichever choice they choose to make (Site 1, Participant 4).
We are very collaborative multidisciplinary team including rehab, PT [Physical Therapy], OT [Occupational Therapy], the transition coordinator, pharmacy, the clinical associate/physician, and our FLS [clinical nurse]. We work together collaboratively to try to make sure that we are placing not just fractured hip patients but all of our patients on the right [path of care] (Site 1, Participant 9).
Lastly, participants appreciated that the FLS was evidence based and/or informed by Alberta-based experts,
The evidence base for this program is very strong. We are not testing an unproven method here. We’re actually applying strong evidence that we know works. So this is one of those few areas in health care where there’s not a lot of controversy and not a lot of dispute over the value of this program. That’s a very unique situation. Most of our programs, they’re far greyer in terms of the economic and quality of impact on health service delivery. Here we’ve got pretty cut and dried strong evidence based, a lot of that evidence also is made in Alberta. We’ve got lots of research history in this area here in Alberta (Provincial Stakeholder, Participant 2).
Further, participants appreciated that they were able to build from the evidence base for the service but that they were able to tailor implementation of the service to the site-level context, and recommended this approach to the spread of the FLS service to future sites within the province. The following participant quotation describes the approach taken to tailor implementation,
A little bit of negotiation, in terms of “here's what we're doing, how does it fit into your model?” Because, you know, each of the sites is different…I really relied on those site leads to tell me what makes sense at their hospital site (Provincial Stakeholder, Participant 1).
Theme 3—Evidence of facilitated learning, not merely monitoring implementation
There was clear evidence of facilitated learning in the implementation and delivery of the FLS at the 2 sites. This was demonstrated by a willingness to share and accept lessons learned during FLS implementation and the establishment of a provincial FLS steering committee. Participants reported a willingness of people to share, accept, and apply lessons learned about the FLS during its implementation, particularly from the first early adopter site to the second site that was afforded by the staggered implementation timing. One participant spoke of the value of being able to draw on the experience of the team at the first beta site,
She’s [the FLS Nurse at the first beta site] hugely supportive and had lots of really good tips for me for when I started. And when I started I could call her and say, “Ah, what do you do for this or that” and so lots of support that way (Site 2, Participant 5).
In addition, several participants offered or recommended that new sites, if the FLS is spread, should draw upon their experiences and expertise, including mentorship for FLS teams, promoting the program to interested sites, providing resources and tools (e.g., assessment format, medical algorithms), and offering opportunities for networking to share experiences. In addition, a few participants reported the establishment of a provincial steering committee to facilitate learning throughout the life of the service, beyond merely ensuring implementation unfolded as intended. One participant commented on the benefits of the steering committee as including peer sharing and the opportunity to discuss standardization,
There’s now been developed an FLS committee province-wide. So now we have other peers to run through decision-making with, and there are lots of decisions where we decide, okay, we’re going to make this standardized between the teams. So that helps, having colleagues. Also, some of the local resources for osteoporosis care, but it’s hard to program, like, the nuts and bolts (Site 1, Participant 3).
Other participants commented that activities of the steering committee could include the following: the development of an orientation and training program (e.g., trainers and materials developed/formalized); providing shadowing opportunities with an FLS clinical nurse, a toolkit for processes, and a list of physical/equipment requirements (e.g., office space, fax machine); and database training. Facilitated learning supported FLS implementation at the 2 early adopter sites.
Discussion
In this qualitative study examining the context in which a FLS was implemented at two early adopter sites, participants described a favorable implementation environment and had positive views of organizational readiness for implementing this change. An effective change management approach, including the use of effective strategies, the perceived acceptability of the service, and facilitated learning, neutralized change fatigue. Indeed, others can learn from this Alberta-based experience when implementing or spreading health interventions.
Participants demonstrated a clear understanding of the intent of the 3i model for post-fracture osteoporosis care. They perceived the FLS to be an acceptable approach for meeting the needs of hip fracture patients. Perceived acceptability of the FLS is an important facilitator for implementation, because modifications to the way in which care is organized has been identified as challenging in healthcare settings [25]. The attribute of acceptability is one of six identified dimensions of health service quality [26] and has been conceptualized as an implementation outcome [10]. Acceptability of the FLS contributed to the context of its successful implementation. Participants’ views of the acceptability of the FLS speak to the overall intent of the FLS as meeting the needs of older hip fracture patients.
Related to participants’ perceptions of acceptability of the FLS was their reported appreciation of the ability to tailor the service to their local site. Adaptation of the FLS to the local context was identified by Osteoporosis Canada as a strategy for the implementation of FLS [7]. The ability to fine-tune the FLS was facilitated by the learning orientation of the early adopter sites, which has been identified as an antecedent for implementation effectiveness [11]. The willingness to learn from the experience of others allowed sites to reflect on what had been done elsewhere and the potential suitability for their site. Teams actively participated in sharing their experiences and accepting suggestions from others. This transferability of learnings from elsewhere positively contributed to the context in which the FLS was implemented. The continued facilitation of a culture of learning will be essential to the successful spread of the FLS, and potentially other interventions for this patient population, provincially.
FLS implementation sponsors could consider using change readiness assessment tools, such as the ORIC, to inform the strategic selection of future sites to implement and spread the FLS, or any other intervention. For instance, an assessment tool could be used to identify and select sites with a culture of innovation or early adoption, similar to the sites in this study. The assessment tools could also assist in identifying committed individuals at the site level to act as champions of the innovation advocating for and assisting with the implementation process. As such, the implementation or spread of interventions, like the FLS, would be strategic by selecting early adopter sites until a “tipping point” of acceptance was reached, where late majority and laggards sites would be more likely to implement an intervention and succeed.
Participants were overwhelmingly positive in their perceptions of the implementation at their site. The interview guide did ask about barriers and challenges to implementation, and we used within interview discussion prompts to elicit feedback on the challenging aspects or obstacles to implementation to inform future roll-out. Despite explicitly asking for comments on such implementation barriers, we did not hear substantial criticism or negative feedback from participants. The predominantly positive orientation of participants reinforces the perspective that the initial selection these two sites as early adopters was correct. The organizational and team composition was ready and willing to implement the FLS at their site.
This study is not without limitations. First, we purposively invited participation from those with knowledge of and/or a clinical role in the FLS at the two early adopter sites. We also used a snowball recruitment strategy to identify any additional individuals whom participants felt would be appropriate for inclusion. This strategy provided us with a pool of individuals informed about the FLS and its implementation. It is possible, however, that our recruitment did not capture any detractors on the periphery of the service, people who might negatively view or influence implementation of the FLS. Second, we quantitatively assessed preparedness for implementing organizational change at baseline using the ORIC tool. We did not quantitatively assess perceptions of change at follow-up. Future studies might wish to investigate perceptions of change both before and following implementation to determine how the organizational context for the change may wax or wane with time.
Nonetheless, this study provides valuable insights into the context in which the FLS was implemented at two early adopter sites in Alberta. Participants in this study described how the strategies that helped to counteract change fatigue, the perceived acceptability of the service, and the incorporation of lessons learned aided in the successful implementation of the FLS. This study sheds light on the successful implementation context and readiness to implement change that unfolded at two early adopter sites and informs the subsequent wide-spread provincial implementation. The implementation approach bodes well for the potential spread of the FLS provincially if proven clinically and cost effective.
Electronic supplementary material
(PDF 48.1 kb)
Acknowledgments
We gratefully acknowledge the Seniors Health Strategic Clinical Network™, Alberta Health Services, for investigator time in support of this work (HMH).
Funding information
This work was supported by a peer-reviewed research grant from Alberta Innovates Partnership for Research and Innovation in the Health System (AI-PRIHS) [grant number 201400391]. LAB holds the Dr. David Magee Endowed Chair in Musculoskeletal Research and is supported by the Canadian Institutes of Health Research (CIHR) as a New Investigator. SRM held the Endowed Chair in Patient Health Management funded by the Faculties of Medicine and Dentistry and Pharmacy and Pharmaceutical Sciences of the University of Alberta during the term of this grant.
Compliance with ethical standards
Conflicts of interest
None.
Ethical approval
All procedures performed in studies involving human participants were in accordance with the ethical standards of the University of Alberta (Ethics ID Pro00056376) and the University of Calgary (Ethics ID REB15-1217) and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.
Footnotes
Sumit R. Majumdar is deceased. This paper is dedicated to his memory.
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
References
- 1.Little EA, Eccles MP. A systematic review of the effectiveness of interventions to improve post-fracture investigation and management of patients at risk of osteoporosis. Implement Sci. 2010;5:80–97. doi: 10.1186/1748-5908-5-80. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Sujic R, Gignac MA, Cockerill R, Beaton DE. A review of patient-centred post-fracture interventions in the context of theories of health behaviour change. Osteoporos Int. 2011;22(8):2213–2224. doi: 10.1007/s00198-010-1521-x. [DOI] [PubMed] [Google Scholar]
- 3.Majumdar SR. A T-2 translational research perspective on interventions to improve post-fracture osteoporosis care. Osteoporos Int. 2011;22(Suppl 3):471–476. doi: 10.1007/s00198-011-1700-4. [DOI] [PubMed] [Google Scholar]
- 4.Papaioannou A, Morin S, Cheung AM, Atkinson S, Brown JP, Feldman S, Hanley DA, Hodsman A, Jamal SA, Kaiser SM, Kvern B, Siminoski K, Leslie WD. 2010 clinical practice guidelines for the diagnosis and management of osteoporosis in Canada: summary. CMAJ. 2010;182(17):1864–1873. doi: 10.1503/cmaj.100771. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Eisman JA, Bogoch ER, Dell R, Harrington JT, McKinney RE, McLellan A, Mitchell PJ, Silverman S, Singleton R, Siris E, Fractur ATFS. Making the first fracture the last fracture: ASBMR task force report on secondary fracture prevention. J Bone Miner Res. 2012;27(10):2039–2046. doi: 10.1002/jbmr.1698. [DOI] [PubMed] [Google Scholar]
- 6.International Osteoporosis Foundation ((N.D.)) Capture the fracture: best practices Fracture Liaison Services. http://www.capturethefracture.org/fracture-liaison-services. Accessed October 12 2017
- 7.Osteoporosis Canada (2013) Make the first break the last with fracture liaison services. Toronto, Ontario
- 8.Mitchell P. Fracture liaison services: the UK experience. Osteoporos Int. 2011;22(Suppl 3):S487–S494. doi: 10.1007/s00198-011-1702-2. [DOI] [PubMed] [Google Scholar]
- 9.Åkesson K, Marsh D, Mitchell PJ, AR ML, Stenmark J, Pierroz DD, Kyer C, Cooper C, Group IFW Capture the fracture: a best practice framework and global campaign to break the fragility fracture cycle. Osteoporos Int. 2013;24(8):2135–2152. doi: 10.1007/s00198-013-2348-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, Griffey R, Hensley M. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Admin Pol Ment Health. 2011;38:65–76. doi: 10.1007/s10488-010-0319-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Klein KJ, Knight AP. Innovation implementation: overcoming the challenge. Curr Dir Psychol Sci. 2005;14(5):243–246. doi: 10.1111/j.0963-7214.2005.00373.x. [DOI] [Google Scholar]
- 12.Green LW, Kreuter MW. Health program planning: an educational and ecological approach. New York: McGraw-Hill; 2005. [Google Scholar]
- 13.Shea CM, Jacobs SR, Esserman DA, Bruce K, Weiner BJ. Organizational readiness for implementing change: a psychometric assessment of a new measure. Implement Sci. 2014;9:7. doi: 10.1186/1748-5908-9-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Rogers EM. Diffusion of innovations. 5. New York: Free Press; 2003. [Google Scholar]
- 15.Ganda K, Puech M, Chen JS, Speerin R, Bleasel J, Center JR, Eisman JA, March L, Seibel MJ. Models of care for the secondary prevention of osteoporotic fractures: a systematic review and meta-analysis. Osteoporos Int. 2013;24(2):393–406. doi: 10.1007/s00198-012-2090-y. [DOI] [PubMed] [Google Scholar]
- 16.Patton M. Qualitative research and evaluation methods. 3. Thousand Oaks: Sage Publications; 2002. [Google Scholar]
- 17.Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. 2006;3(2):77–101. doi: 10.1191/1478088706qp063oa. [DOI] [Google Scholar]
- 18.Morse JM. "Data were saturated…". Qual Health Res. 2015;25(5):587–588. doi: 10.1177/1049732315576699. [DOI] [PubMed] [Google Scholar]
- 19.Ritchie J, Lewis J, Elam G, Tennant R, Rahim N. Designing and selecting samples. In: Ritchie J, Lewis J, McNaughton Nicholls C, Ormston R, editors. Qualitative research practice: a guide for social science students and researchers. Thousand Oaks: Sage Publishers, Inc.; 2014. [Google Scholar]
- 20.Scientific Software Development GmbH (2014) Atlas.ti: Qualitative Data Analysis. http://atlasti.com/. Accessed June 30, 2017
- 21.IBM Corp. (2017) IBM SPSS statistics for windows, version 24.0. IBM Corp., Armonk
- 22.Morse JM, Barret M, Mayan M, Olson K, Spier J. Verification strategies for establishing reliability and validity in qualitative research. Int J Qual Methods. 2002;1(2):2. doi: 10.1177/160940690200100202. [DOI] [Google Scholar]
- 23.Mayan MJ. Essentials of qualitative inquiry. Qualitative Essentials. Walnut Creek: Left Coast Press; 2009. [Google Scholar]
- 24.Sandelowski M, Leeman J. Writing usable qualitative health research findings. Qual Health Res. 2012;22(10):1404–1413. doi: 10.1177/1049732312450368. [DOI] [PubMed] [Google Scholar]
- 25.Grol R, Grimshaw J. From best evidence to best practice: effective implementation of change in patients’ care. Lancet. 2003;362:1225–1230. doi: 10.1016/S0140-6736(03)14546-1. [DOI] [PubMed] [Google Scholar]
- 26.Health Quality Council of Alberta (2005) Alberta quality matrix for health. Edmonton
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
(PDF 48.1 kb)