Abstract
Background
The importance of physical activity and the orientation of exercise therapy in rehabilitation has changed for many chronic health conditions. Exercise therapy is the most applied therapy form within multidisciplinary rehabilitation programs for almost all chronic health conditions. Despite the scientifically discussed need to refine exercise therapy, there is relatively little knowledge of how exercise therapy is actually conducted. This study protocol describes the methodological procedure used in the project “Exercise therapy in medical rehabilitation: a survey at facility and practitioner level”, which aims to take a national survey of exercise therapy in rehabilitation practice in Germany.
Methods
The project was implemented using an explanatory sequential mixed method design. Quantitative and qualitative methods were integrated in two consecutive project phases. Phase 1 used a standardized, quantitative written survey of the heads of exercise therapy departments to compile a national overview of concepts and process features of exercise therapy of individual rehabilitation facilities. Phase 2 recorded individual perspectives and opinions concerning exercise therapy goals, content and methods and current developments in the rehabilitation context (e.g., physical activity promotion, interdisciplinarity, standardization) of exercise therapy practitioners. Over the course of two one-and-a-half day workshops, central themes were introduced and prepared with standardized written individual surveys from Phase 1 and combined with qualitative surveys using facilitated group discussions (focus groups in mixed methods design).
Discussion
The project generates a comprehensive picture of exercise therapy in medical rehabilitation at facility level and inserts further information at the practitioner level into this context. The chosen methodology of a mixed method design combines the perspective of the facility with that of the practitioner, thus allowing for a complex and multifaceted description of the status quo in exercise therapy practice and makes it possible to identify facilitators and barriers for the refinement of exercise therapy in specific everyday rehabilitation. These findings form the basis for the systematic development of quality exercise therapy in rehabilitation, in particular in terms of the refinement, implementation and dissemination of biopsychosocial concepts of exercise therapy.
Keywords: Physical activity, Health services, Chronic disease, Therapy concept, Therapy process
List of abbreviations: DGRW, Deutsche Gesellschaft für Rehabilitationswissenschaft (German association for rehabilitation science); DRV, Deutsche Rentenversicherung Bund (Germany Statutory Pension Insurance Scheme); ICF, International Classification of Functioning, Disability and Health; KTL, Klassifikation therapeutischer Leistungen (classification of therapeutic services system)
1. Background
The project “Exercise therapy in medical rehabilitation: a survey at facility and practitioner level” aims to take a national survey of exercise therapy in rehabilitation practice in Germany. The focus is on a) concepts and process features in rehabilitation facilities and b) individual perspectives and opinions concerning exercise therapy goals, content and methods and current developments in the rehabilitation context of exercise therapy practitioners. This study protocol contains a detailed description of the methodology of explanatory sequential mixed method design [1,2].
The importance of physical activity and orientation of exercise therapy in rehabilitation has changed for many chronic health conditions: Firstly, the increasing lack of exercise in the general public has been identified as an independent risk factor in the emergence and development of chronic non-communicable diseases [3]. Secondly, there is now strong evidence for the positive effects of physical activity in the rehabilitation of chronic diseases [4,5]. Thirdly, exercise therapy has become the most commonly applied form of therapy for almost all chronic health conditions within multidisciplinary rehabilitation programs in Germany [6]. Further, the objectives of exercise therapy have advanced and become more differentiated. An extensive system of biopsychosocial objectives of the exercise therapy working group of Deutsche Gesellschaft für Rehabilitationswissenschaft (German association for rehabilitation science; DGRW) [7] includes traditional biomedical objectives (e.g., regaining physical functioning) but also pedagogical psychological aspects. Finally, among other goals, committing to increasing physical activity is of central importance [[7], [8], [9]], as changes in favor of a physically active lifestyle often fail [10] and the prevalence of physical inactivity among people with chronic diseases is high [[11], [12], [13], [14]].
Contemporary exercise therapy concepts and processes address, above all else, empowering persons to adopt and maintain physically active lifestyles starting after medical rehabilitation and to use physical activity and exercise to enhance health resources and to manage illness. This implies the need for the refinement of traditionally dominant (body) function-oriented training approaches towards elaborate bio-psychosocial therapy concepts [[15], [16], [17]].
1.1. Status of research
In scientific works in the fields of physical therapy [18,19] and exercise therapy [7] there is more and more emphasis on a biopsychosocial approach aligned with the International Classification of Functioning, Disability and Health (ICF) [20]. This is linked to an increased interest in psychosocial and behavioral goals of exercise therapy as a whole. Further, there is also an emphasis on the goal of positively influencing the physical activity behaviour over the long term and systematically developing corresponding content and methods [21]. In Germany, the DGRW exercise therapy working group recently – based on the projects funded by Germany's Statutory Pension Insurance Scheme (Deutsche Rentenversicherung Bund; DRV) 1 – prepared evidence-based recommendations for exercise therapy with the goal of encouraging patients to adopt a physically active lifestyle [9,22,23].
However, international studies show that interventional models with a biomedical focus and a primary goal of increasing physical fitness continue to be commonplace. Physical therapy and exercise therapy-related research [18,24], the education for physical and exercise therapy professions [25,26], along with the targets and actions of the therapy staff [[27], [28], [29]] all focus on the (short-term) improvement of physical functioning. Correspondingly, various studies show that exercise therapy content working towards behavioral and psychosocial goals in a targeted manner are seldom used and that many exercise therapy practitioners are also unaware of such content [17,30,31].
Despite the scientifically discussed need to refine exercise therapy, there is relatively little knowledge of how exercise therapy is actually implemented in German rehabilitation practice regarding content, methods and didactic-methodological use. As far as representative information on the provision of exercise therapy is concerned, at a national level there are “only” the current quality assurance tools and the related documentation practice of the DRV, which do not provide deep understanding of the mentioned aspects.
To date there has not been a systematic survey of Germany-wide exercise therapy practice within medical rehabilitation that could provide insights into the concepts and process features. In addition, there is only a rudimentary knowledge of how individual perspectives are catered for in terms of goals, content and methods of exercise therapy practitioners [32] and how these perspectives (can) inform therapeutic action within the framework conditions of a specific rehabilitation facility.
1.2. Questions and aims of the study
As a result, the first question asks how exercise therapy concepts and processes are actually implemented in individual rehabilitation facilities across Germany as they constitute the basis for refinement of existing concepts. The second key question asks which individual perspectives do physical and exercise therapists have concerning exercise therapy goals, content and methods as it is the therapists who ultimately structure and “live with” potential change processes and refinements in the rehabilitation facility. The survey aimed to include therapists (e.g., physical, exercise therapist) who are responsible for the implementation of physical activity, exercise or physical training as therapy content as described in the German classification system of therapeutic services (Klassifikation therapeutischer Leistungen; KTL). This standardized classification system by the DRV must be used in rehabilitation facilities to document the therapeutic services provided.
In addition to the therapeutic services listed in the areas A “Sport and exercise therapy” and B “Physical therapy”, it lists information and training (C), clinical social work and social therapy (D), ergotherapy (E). In this respect, we have decided to hereinafter refer to as exercise therapist or in terms of the therapy form as exercise therapy.
The project was implemented in two consecutive phases as follows (Table 1): Phase 1 involved a quantitative Germany-wide baseline survey of exercise therapy concepts (Topic A1) and of process features in exercise therapy (Topic A2) at the level of individual rehabilitation facilities.
Topic A1 addresses concepts of exercise therapy practice in rehabilitation facilities:
-
-
What characteristics do exercise therapy concepts and processes have in rehabilitation practice in relation to target groups, goals, content and methods?
-
-
What problems are seen amongst rehabilitation patients and how is exercise therapy seen to be capable of influencing these problems?
-
-
What written exercise therapy concepts are available in rehabilitation facilities?
-
-
How do exercise therapy concepts differ for different health conditions?
Table 1.
Overview of study phases including methodological actions.
| Phases | Data collection | Data analysis | Products |
|---|---|---|---|
|
Phase 1 01/2015–12/2015 |
Quantitative data from the national cross-sectional survey | Descriptive and analytical statistics and Latent Class Analysis |
|
|
Phase 2 1/2016–06/2016 |
Qualitative-quantitative data of focus groups participants including individual surveys | Criteria-led content analyses |
|
|
Phase 3 07/2016–08/2017 |
None | Interpretation and Integration of quantitative and qualitative data |
|
Topic A2 relates to process features of exercise therapy in rehabilitation facilities:
-
-
How is the planning and control of exercise therapy carried out? What assessment methods are used in relation to the allocation and control of exercise therapy, and what information is passed on by other professionals in the interdisciplinary rehabilitation team to the exercise therapy team?
-
-
What organizational forms exist in the provision of exercise therapy (individual therapy versus groups), what is the global content gearing (practice, theory, linking theory and practice) and how high is the respective level of standardization of exercise therapy interventions?
-
-
How significant is content to promote a physically active lifestyle considered to be, and how are measures for long-term promotion of physical activity implemented methodologically and didactically? For example, how is information on the health effects of physical activity disseminated (presentation, group discussions, one-to-one discussions, integrated in practice) and are media aids (therapist and participant materials) used for this purpose?
-
-
In view of the multiple objectives and the implementation of rehabilitation measures in multi-professional rehabilitation teams, how is the collaboration within exercise therapy teams and inter-professional collaboration structured?
Phase 2 (qualitative-quantitative) explored individual perspectives and opinions of the exercise therapy practitioners in terms of exercise therapy goals, content and methods (Topic B).
Topic B thus focuses on the following areas:
-
-
Weighting of exercise therapy goals and assessments on individual forms of therapy in relation to exercise therapy goals with a focus on promoting physical activity
-
-
Barriers to and facilitating factors for the implementation of exercise therapy interventions promoting physical activity
-
-
Didactic-methodological focus areas in the implementation of individual forms of therapy and relevant rationale (e.g., assumed influencing factors)
Phase 2 also involved liaising with exercise therapists to determine consequences for developing the quality of exercise therapy in view of current developments and requirements for rehabilitative measures (e.g., patient-centredness, standardization, inter-professional collaboration of exercise therapy practitioners). The corresponding Topic C aims to answer the following questions:
-
-
What are the opportunities and the risks of current development trends in rehabilitation (patient-centredness, standardization and inter-professional collaboration) for the refinement of exercise therapy?
-
-
What needs to change at the level of the rehabilitation facilities
-
-
What further training needs for exercise therapists are seen to exist at a personal level, e.g., in relation to beneficial factors for or barriers to translating scientific findings into exercise therapy practice.
-
-
What are the barriers and facilitating factors in terms of a qualitative refinement of exercise therapy?
2. Methods
The project was implemented using an explanatory sequential mixed method design [1,2]. Quantitative and qualitative methods were integrated in two consecutive project phases (Table 1). Phase 1 used a standardized, quantitative written survey of the heads of exercise therapy departments to compile a comprehensive national overview of conceptual features of exercise therapy at the level of individual rehabilitation facilities. Based on this questionnaire-based cross-sectional survey, Phase 2 involved recording of individual perspectives and opinions of exercise therapy practitioners. To do this, two one-and-a-half day workshops with 60 exercise therapists from 60 different facilities were carried out. Based on Scheer et al. [33], central topics were introduced and prepared with standardized individual surveys and combined with qualitative surveys using facilitated group discussions (focus groups in mixed methods design).
The planning and implementation of all methodological steps was carried out in close collaboration with the interdisciplinary (medical, physiotherapy, psychology, sports science) DGRW working group “Exercise therapy” as the associated expert group. A project advisory board comprising representatives from rehabilitation theory and practice also provided advice.
The content and structure of this study protocol for Phase 1 of the project are based on the guidelines for reporting observational studies [34,35]. Phase 2 of the project is based on the guidelines for reporting qualitative studies and the guidelines for the implementation and reporting of mixed method studies [36,37].
2.1. Setting
The study was set in medical rehabilitation clinics across Germany. With around one million medical rehabilitation services per year, DRV is by far the country's largest service provider [38]. Medical rehabilitation services are primarily provided on an in-patient basis, with only 10–15% offered as out-patient services [38]. Out-patient and in-patient rehabilitation are considered as equal alternatives in the project – as also described in the conceptual framework for medical rehabilitation of the DRV [39].
2.2. Phase 1: Germany-wide baseline survey of exercise therapy concepts and processes (primarily quantitative study)
Phase 1 involved a cross-sectional questionnaire-based baseline survey of concepts and processes of exercise therapy practice in medical rehabilitation Germany-wide.
2.2.1. Study population and sample size
The entirety of the 1558 exercise therapy departments from 1146 adult medical rehabilitation facilities taking part in the quality assurance process of DRV were considered eligible for Phase 1. In terms of the range of illnesses treated with medical rehabilitation [38], all health conditions were included in the Germany-wide baseline survey in Phase 1.
2.2.2. Measuring instruments
A questionnaire was developed covering exercise therapy concepts and processes using a rational construction strategy [40]. The following process was used to prepare the questionnaire: In a first step, existing documents and instruments that record concepts and process features in rehabilitation were searched and analyzed. The findings were supplemented by features based on expert knowledge and questions from the project application. This resulted in a comprehensive pool of preliminary items. The second step involved designing quality dimensions, quality-relevant action/content areas of exercise therapy and allocating items. For this purpose, relevant quality dimensions and quality-relevant action/content areas of exercise therapy (based on the cybernetic model of therapy planning; [41]) were selected and higher-level quality dimensions [42,43] were systematically inspected. The third step involved preparing a first pilot version of the questionnaire and evaluating the questionnaire. The pilot version of the questionnaire was sent to experts from the DGRW exercise therapy working group (n = 14) and to selected executives in exercise therapy from rehabilitation practice (n = 11; covering all health conditions) to evaluate the individual items. The evaluation questionnaire contained questions on the areas of completeness, answerability, acceptance and understandability. The final version of the questionnaire was prepared based on content-analytical evaluation of the comments and suggestions for improvement (n = 156, on average ten comments per person).
The final questionnaire addresses conceptualization, content, methods and organizational process features of exercise therapy departments. These include, among others, questions regarding target groups, goals, content, methods, scope, assessment methods, therapy referral and control, documentation, manualization, standardization and evaluation, and the qualifications of staff. Table 2 provides an overview of the items allocated to the quality dimensions and action/content areas. The questionnaire comprises 38 higher-level questions. The final questionnaire can be found in Appendix A (English version) and in Appendix B File 2 (German version).
Table 2.
Overview of quality dimensions and quality-relevant content areas.
| Quality dimension | Quality-relevant action/content areas of exercise therapy |
||||||
|---|---|---|---|---|---|---|---|
| Assessment | Therapy goals | Content, methods, media |
Working method | Implemen-tation | Therapy control |
Referral to exercise therapy | |
| Theory base | 21, 22, 23, 24 | 11 | 15a, b, d 17a, b, d |
35 | |||
| Evidence base | 11b | 18a, 18b | 35 | ||||
| System-related/ sustainability |
12 | ||||||
| Manualization/ standardization |
32 | 13 | 32 | 15c, 17c | 32 | 29, 30, 32 | |
| Patient-centredness | 20, 25, 26 | 13 | 35 | 30 | |||
| Interdisciplinary | 27, 28 | 13 | 33, 34, 35, 36, 37, 38 |
31 | |||
| Quality assurance/ quality improvement |
20 | 35 | 20 | ||||
The figures refer to the respective item in the developed questionnaire (Appendix A). Items 1–10 contain basic information about the facility and are thus not listed in the table.
2.2.3. Data collection process
The questionnaire was sent out from May 2015 onwards using a coding list via DRV to 1558 exercise therapy departments of 1146 rehabilitation facilities. Departments that had not returned the questionnaires by mid-June 2015 (n = 1244) were sent a reminder in early July 2015 and the questionnaire again. Responders were requested to return the questionnaire by the 29th of July 2015. To increase the rate of participation, the postage was paid in advance. The questionnaire was returned pseudonymized to the leading scientific research institute in Erlangen.
The questionnaire-based baseline survey contains a selection of questions on concepts and process features. For further analyses, an enquiry was made about the willingness to provide any media and materials available in the facilities confidentially for document analysis. The departments surveyed thus had the option to provide their contact details and were contacted afterwards regarding the sending of documents by the research centers.
At the beginning of the recruitment process, the heads of relevant departments were asked to participate (“informed consent”). In addition, each letter provided information on the voluntary nature of the study, the right to revoke consent, the anonymity of data processing and assurance that there would be no disadvantages if they chose not to participate. For Phase 1, the participating heads of the departments gave their consent to taking part in this research project by returning the questionnaire to the academic research institute.
2.2.4. Return of questionnaire
Fig. 1 shows an overview of the process from sending the questionnaires, to the return of the questionnaires, and of the final questionnaires included. Six hundred eighty five of the 1146 facilities contacted (60%) responded. Of that number, 541 facilities provided information on their exercise therapy department (734 questionnaires). Of the other 144 facilities, 90 facilities responded that they do not have any exercise therapy department, 45 facilities did not wish to take part in the survey (e.g., for reasons of time, unclear cost-benefit ratio, management not in agreement) and nine facilities sent back the uncompleted questionnaire without any comment. Twenty one questionnaires were excluded because they were incomplete. Seven hundred and thirteen questionnaires provide the basis of data for the following analyses.
Fig. 1.
Flow chart on sending and return of questionnaires.
2.2.5. Data analysis
Descriptive analyses of the examined quality dimensions or quality-relevant content areas of exercise therapy will be carried out. To present the heterogeneous nature of the clinics in terms of the exercise therapy concepts and process features recorded, an important result of Phase 1 – based on the methodology used in the MeeR project [42] – was an illness-specific clinic categorization of the facilities surveyed using latent class analysis. The last analysis was the basis for phase 2 recruitment process. SPSS and Mplus was used for statistical analysis.
2.3. Phase 2: An in-depth examination of concepts and process features in exercise therapy practice (qualitative-quantitative study)
In Phase 2, two one-and-a-half day development workshops were held with exercise therapists responsible for the implementation of the exercise therapy services. The aim was to explore organizational and content process features as well as features of the practitioners in the rehabilitation team in more depth.
2.3.1. Sampling of the study population and sample size
In conjunction with the project advisory board, we decided to make six major health conditions with high relevance for medical rehabilitation as a subject of discussion during the workshops: Psychosomatic, Addiction, Orthopedics - total hip/knee replacement, Orthopedics - back pain, Oncology, Neurology. Due to practical reasons, we organized one focus group for each of this six health conditions with the maximum recommended size of ten persons. Thus the workshop was to be carried out with a total of 60 exercise therapists from 60 different facilities. Participants in Phase 2 were recruited from the exercise therapy departments that took part in the questionnaire survey in Phase 1 of the project (sequential nested sampling) [44].
The invitation to the workshops aimed to include the entire range of rehabilitation facilities along the illness-specific clinic categorization from Phase 1. The sampling of each focus group was purposive that means an effort was made to put together heads of exercise therapy departments with a maximal diversity of opinions and values seen in the concept and process features recorded. Therefore, based on the findings from the first project phase, the exercise therapy departments were categorized using latent class analysis [45], taking into account the following selected content-conceptual features: perceived ability to influence problem situations (see Appendix A, Question 11, p 5); weighting of exercise therapy practice versus knowledge transfer (see Appendix A, Question 14/15, p 7), evaluation of specific exercise therapy contents to promote physical activity (see Appendix A, Question 18 p 9). The latent class analysis generated different classes of exercise therapy facilities with different values in the named concept and process features. The classes differed for example with regard to knowledge transfer for patients during exercise therapy or the arrangement of exercise therapy content to promote physical activity. The next step was to form one focus group for each of the six health conditions selected by randomly assigning heads of the according exercise therapy departments so that they were spread evenly across the heterogeneous classes created with the latent class analysis.
2.3.2. Measuring instruments
The development workshops centered on carrying out focus groups that comprised exercise therapists for the same health conditions. These were combined with quantitative individual surveys.
2.3.2.1. Quantitative individual surveys
Four overarching areas of exercise therapy goals were determined from the findings from Phase 1 by factor analysis (physical/motor skills, psychosocial, uncertainty/deconditioning as well as transfer of physical activity to everyday life). Based on the methodology described in Finger et al. [28] and the goal taxonomy developed for the evaluation of exercise therapy [7,46], relevant aims, content and methods of current exercise therapy care relevant for the case examples were explored in more depth.
For the individual surveys, illness-specific ICF-based “case examples” formed the basis for weighting therapy goals [32]. A typical problem constellation was described for each of the six health conditions. In line with the ICF, the health condition, the functioning level as well as associated environmental and personal context factors were described in detail. To come up with “condition-treatment pairs”,2 the first step for the participants was to weight exercise therapy goals for their respective illness-specific case examples in terms of their significance for the respective problem constellation. Next, for each goal they were asked to briefly describe the three most important exercise therapy services to achieve the therapy goal in an open question format. The weighting of therapy goals was thus carried out on a case-by-case basis using the analytic hierarchy process [47]. The case examples as well as this specific individual survey were also tested in advance with eight exercise therapists from rehabilitation practice.
2.3.2.2. Facilitated group discussions (focus groups)
Within the development workshops, three focus groups were carried out for each health condition.
Focus group 1: The central goal of “Adopting and maintaining physically active lifestyles” was explored in more detail. Using discussion stimulus, participants were asked to explain what helps to enable patients to commit to more physical sporting activity in the long term in their exercise therapy practice. Table 3 shows the interview guide including a lead question, follow up questions to keep the conversation going or steer the conversation, and a list of potential questions if the discussion falters.
Table 3.
Interview guide for focus group 1 (Promoting physical activity in exercise therapy).
| Type of applied question and stimuli | Examples | |
|---|---|---|
| Lead question | Introduction and discussion stimulus | A central aim of rehabilitation is “Adopting and maintaining physically active lifestyles”. There are different approaches for pursuing this aim … |
| Prompt: | … what helps in your exercise therapy to enable patients to commit to more physical activity in the long term, i.e. beyond the in-patient rehabilitation period? | |
| Questions to steer or keep the conversation goinga | Option A - Keep the conversation going: | “Can you tell us more about this aspect?”/“And how do you do that exactly?”/How exactly is that implemented?” |
| Option B - Steer the conversation: | What do the others think?/What other possibilities are there? | |
| Option C - return to lead question 1: | As soon as an aspect has been explored in depth/sufficiently, return to lead question: “What else do you do to get your patients to commit to more physical activity in the long term?” | |
| Specific questionsb | Formulate summary and ask specifically about a mentioned aspect from the priority list: | “You mentioned the point _________. Could you tell more about this specific aspect? “ |
| Ask specifically about obstacles: | “Do you see certain obstacles here that prevent the goal from being achieved?” | |
| Ask specifically about beneficial factors: | “What helps to achieve this goal?” | |
| Optional Questionsc | Ask specifically about one of these factors from the priority list: | “Could you tell more about this specific aspect of _________?” 1. Content (exercise practice/passing on knowledge/linking theory & practice) 2. Heterogeneity (dealing with heterogeneity and/or different prerequisites) 3. Patient centredness (active role of the patient/participative objective or planning/preferences are taken into account/biopsychosocial perspective/patient-therapist relationship 4. Media and materials (patient materials/therapist materials/films, presentations) 5. Methodological implementation (group versus individual/experience-based versus evidence-based) |
It was a central goal of the focus group interview to foster an in-depth exploration of mentioned aspects.
As soon as named aspect is part of the priority list.
Only if these aspects of the priority list are not mentioned anyway & the discussion falters.
Focus group 2: In preparation for the second focus group, participants received a 40-min presentation on current developments in medical rehabilitation. The quality dimensions of patient-centredness, interdisciplinary nature and manualization/standardization were discussed, which had already served to structure the questionnaire from Phase 1. For each of the quality dimensions named, the participants then individually assessed how significant they consider these dimensions to be for exercise therapy, and how they rate each dimension in terms of its implementation in their own facility. This was shown in a graphical presentation of a chart with four fields with the axes “Significance” [low-high] and “Implementation” [low-high] graded into eleven levels. Next, all of the participants explained their weightings for the dimensions in relation to significance and implementation using a large screen that all participants could see. The resulting charts formed the basis for the subsequent focus group discussion. The group explored, in depth, how participants arrived at their assessment and asked about barriers and facilitating factors (see Table 4).
Table 4.
Interview guide for focus group 2 (Quality dimensions or development trends in rehabilitation) and focus group 3 (Wishes for the future).
| Type of applied question and stimuli | Examples | |
|---|---|---|
| Lead question | Introduction and discussion stimulus | You have been shown developments in the rehabilitation environment and assessed three dimensions (patient centredness, interdisciplinary, standardization/manualization) in terms of their personal significance and current implementation. The chart shows that you rate dimension ___ (see above) as a) most significant, b) least implementation, c) most reliable weighting. |
| Prompt: | Can you tell us how you arrived at this assessment? | |
| Questions to steer or keep the conversation goinga | Option A - Keep the conversation going: | “Can you tell us more about this aspect?”/“How is this currently managed where you work?”/How did this development/introduction/change come about? |
| Option B - Steer the conversation: | “What do the others think?” | |
| Specific questionsb | Ask specifically about significance: | “You mentioned the point _________. Why do you consider that so important/unimportant?” |
| Ask specifically about obstacles: | “Why do you think it might be that it is difficult to implement?” | |
| Ask specifically about beneficial factors: | “In your opinion, what would have to happen to improve implementation?”/“How is it implemented for those who have already integrated it?” | |
It was a central goal of the focus group interview to foster an in-depth exploration of mentioned aspects.
Mandatory questions – in-depth exploration of the most reliable weighting.
Focus group 3: In the third and final focus group, possibilities for optimization as well as the need for refinement for exercise therapy were explored. The lead question for the session asked the participants what they wanted, e.g., from science, training systems, insurers or the structural framework conditions (“What do you wish for the future, e.g., from science, training/education system, health or pension insurance schemes).
To ensure the quality of the workshops and the focus groups in particular, the following steps were taken in advance: 1.) The design of the workshops was agreed upon with the expert panel of DGRW exercise therapy working group. 2.) Focus group training measures were held for the facilitators and co-facilitators (concerning ground rules, guidelines, dealing with “difficult” situations in discussions etc.). In addition, two pilot tests were carried out in advance for focus groups 1 and 2 to allow the facilitator to practice and to test the survey methods.
2.3.3. Recruiting for the workshops and informed consent
To present the range of exercise therapy departments recorded, exercise therapists from all six classes were invited who with at least 80% probability belonged to the respective class. The first workshop dealt with the health conditions Orthopedics total hip/knee replacement, Orthopedics chronic unspecific back pain and Neurology, while the second workshop covered the areas Oncology, Psychosomatic and Addiction. Thirty therapists were expected to take part in each workshop, distributed evenly across the respective three indications.
Invitations were sent via DRV. For Phase 2, the heads of the exercise therapy departments were asked to provide a written declaration of consent when registering for the development workshop. This signed declaration of consent was a prerequisite for taking part in Phase 2. Potential participants had the opportunity to contact the research institutes involved with any queries before giving their consent.
After receiving the invitations, the therapists registered directly with the research institutes involved using the registration form. For the first workshop, 38 of the 87 people invited registered, while 35 of the 79 persons invited to the second workshop registered to take part.
2.3.4. Implementation of the two development workshops
The two development workshops took place in April 2016. Both workshops were carried out in rooms belonging to the Hessen state sports association in Frankfurt/Main. Ultimately, 58 heads of exercise therapy departments took part, resulting in three focus groups with ten persons each and one group each with eight, nine and eleven persons respectively. Thirty practitioners took part in Workshop I, while Workshop II was attended by 28 persons as two registered participants cancelled due to illness. The therapists did not incur any costs from taking part. Their accommodation and meals were paid for, and their travel expenses were reimbursed.
2.3.5. Data analysis
In the analysis phase, quantitative and qualitative-content findings obtained in Phases 1 and 2 were combined with a view to: a detailed identification of the starting conditions for the development of quality criteria for exercise therapy; facilitating factors and barriers for the refinement and practical implementation of scientifically proven, evidence-based exercise therapy; the analysis of the perception and assessment of the collaboration in the exercise therapy team and of inter-professional team work.
The data analysis required for this purpose follows the standards in the evaluation of explorative design. The recordings from the interviews and the focus groups are transcribed and the content analyzed based on specific criteria [33,48] using MaxQDA software. The focus groups were processed and interpreted using the seven stages of structuring qualitative content analysis [49]. The data collected with the questionnaire were evaluated in terms of frequencies, link to indication, clinical features, therapist features with the help of descriptive and interference-statistical analysis methods.
3. Discussion
In Phase 1 the project delivers quantitative data at facility level regarding conceptual features and processes of exercise therapy that go considerably beyond the mere evaluations of the scope of exercise therapy services according to KTL documentation currently available. Phase 1 makes it possible to describe the quantitative values in relation to central structural and process features of exercise therapy. The clear quantitative gearing with a comprehensive sample in Phase 1 ensures high statistical generalizability and national representativeness.
The survey performed in Phase 2 at the level of exercise therapists sheds light on their content-conceptual and didactic-methodological frameworks for action and their therapeutic scope. These are decisive factors in structuring and implementing high-quality exercise therapy care. The qualitative-quantitative survey based on a sample using class analytics with maximum heterogeneity in relation to central features of exercise therapy ensures that different cases, rich in information, are taken into account. This facilitates a detailed exploration, explanation and analysis of mechanisms behind the quantitative findings and thus high analytical generalizability [50]. As far as methodology is concerned, focus groups were chosen as they are suitable for recording complex behaviors and attitudes in multilayered subject areas, in particular relating to frameworks for action in interaction with others [51,52].
Thus, the project first generates a macroscopic picture of exercise therapy in medical rehabilitation at facility level and inserts individual-related information at practitioner level into this context. The chosen methodology in the mixed method design combines the perspective of the facility with that of the practitioner, thus allowing for a complex and multifaceted description of the status quo of exercise therapy practice. It also makes it possible to identify facilitators and barriers for the refinement and practical implementation of scientifically proven, evidence-based exercise therapy in specific everyday rehabilitation. On the whole, the chosen methodology leads to a deeper understanding, better interpretability and ultimately to increased significance of the findings [53].
Furthermore, this research project provides important information on the current status of institutional and personal conditions in exercise therapy care. The analysis of specific conceptual gearings and methodological-didactic forms of structuring exercise therapy forms – in according with findings from translation research [54,55] – form the basis for systematic quality development of exercise therapy in rehabilitation. This is especially true with regard to the refinement, implementation and dissemination of elaborated biopsychosocial concepts of exercise therapy. Based on the findings of the study, specific recommendations for action for the quality development of exercise therapy will be drafted that will help improve the concept and process quality of evidence-based and sustainable exercise therapy.
Ethics approval and consent to participate
The study is carried out in accordance with the recommendations of the World Medical Association (Declaration of Helsinki) [56] and the Guideline for Good Clinical Practice (informed consent, voluntary nature, data protection etc.). The study protocol and the data privacy concept were examined by the independent Ethics Commission of the Medical Faculty of Friedrich-Alexander-University Erlangen-Nuremberg (Invoice no. 182_16B) and approved without objection.
Availability of data and material
The data generated during Phase 1 are available from the corresponding author on reasonable request. Interview data of Phase 2 may be linked to individuals interviewed and as such is not available open use. Should anyone wish to have access or is interested in further exploration of the data, you may contact the author: wolfgang.geidl@fau.de.
Competing interest
The authors declare that they have no competing interests.
Funding
The study is funded by the DRV (subsidy number: 0421/40-64-50-47). The project management responsible at DRV sent the questionnaire-based measuring instruments in Phase 1 and the invitations to the workshops for data collection in Phase 2. The funder is represented by an individual from the expert group and the project advisory board. Otherwise, the funder was not involved in planning the methodology, in data management, analysis and interpretation nor in the writing and submission of the manuscript.
Authors' contribution
All authors were involved in every phase of this study. All authors provided substantial contribution to design and analysis of the study and interpretation of findings, drafting the paper and revising it critically for important intellectual content. All authors have read and approved the final manuscript.
Acknowledgements
The authors wish to thank DRV for the financial support provided for this study. We extend our sincere thanks to all exercise therapists who took part in the study. Thanks to the members of the DGRW's exercise therapy working group and the expert advisory board for their valuable advice and support in carrying out the studies.
Footnotes
Project to develop evidence-based concepts for exercise therapy in rehabilitation, see http://forschung.deutsche-rentenversicherung.de/ForschPortalWeb/contentAction.do?key = main_reha_ep_bewegung.
In order to avoid the illness-centered term “problem-treatment pairs” with its negative connotations (Mittag et al., 2007), the term “condition-treatment pairs” is introduced here.
Supplementary data related to this article can be found at https://doi.org/10.1016/j.conctc.2018.05.004.
Contributor Information
Wolfgang Geidl, Email: wolfgang.geidl@fau.de.
Judith Deprins, Email: Judith.deprins@uni-tuebingen.de.
René Streber, Email: rene.streber@fau.de.
Nina Rohrbach, Email: nina.rohrbach@tum.de.
Gorden Sudeck, Email: gorden.sudeck@uni-tuebingen.de.
Klaus Pfeifer, Email: klaus.pfeifer@fau.de.
Appendix A.
-
-
File format: PDF
-
-
Title: Final Questionnaire (English Version)
-
-
Description: This is the translated version of the finale questionnaire that was used in the Phase 1 of this study.
Appendix B.
-
-
File format: PDF
-
-
Title: Final Questionnaire (German Version)
-
-
Description: This is the original version of the finale questionnaire that was used in the Phase 1 of this study.
Appendix C. Supplementary data
The following are the supplementary data related to this article:
References
- 1.Creswell J.W., Klassen A.C., Plano Clark V.L., Smith K.C. Best practices for mixed methods research in the health sciences. https://pdfs.semanticscholar.org/867f/2d9a5491411530fa9b79fa020d4c63e83025.pdf 31 August 2017.
- 2.Teddlie C., Tashakkori A. A general typology of research designs featuring mixed methods. Res. Sch. 2006;13:12–28. [Google Scholar]
- 3.Lee I.-M., Shiroma E.J., Lobelo F., Puska P., Blair S.N., Katzmarzyk P.T. Effect of physical inactivity on major non-communicable diseases worldwide: an analysis of burden of disease and life expectancy. Lancet. 2012;380:219–229. doi: 10.1016/S0140-6736(12)61031-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Pedersen B.K., Saltin B. Exercise as medicine - evidence for prescribing exercise as therapy in 26 different chronic diseases. Scand. J. Med. Sci. Sports. 2015;25(3):1–72. doi: 10.1111/sms.12581. [DOI] [PubMed] [Google Scholar]
- 5.Börjesson M., Hellenius M.L., Jansson E., Karlson J., Leijon M., Staehle A., Sundberg C.J., Taube T. Physical activity in the prevention and treatment of disease. http://fyss.se/wp-content/uploads/2011/06/fyss_2010_english.pdf 31 August 2017.
- 6.Brüggemann S., Sewöster D., Kranzmann A. Bewegungstherapeutische Versorgung in der medizinischen Rehabilitation der Rentenversicherung – eine Analyse auf Basis quantitativer Routinedaten. Rehabilitation. 2017;57:24–30. doi: 10.1055/s-0043-102556. [DOI] [PubMed] [Google Scholar]
- 7.Pfeifer K., Sudeck G., Brüggemann S., Huber G. DGRW-Update: Bewegungstherapie in der medizinischen Rehabilitation – Wirkungen, Qualität, Perspektiven. Rehabilitation. 2010;49:224–236. doi: 10.1055/s-0030-1261909. [DOI] [PubMed] [Google Scholar]
- 8.Dean E., de Andrade A.D., O'Donoghue G., Skinner M., Umereh G., Beenen P., Cleaver S., Afzalzada D., Delaune M.F., Footer C., Gannotti M., Gappmaier E., Figl-Hertlein A., Henderson B., Hudson M.K., Spiteri K., King J., Klug J.L., Laakso E.-L., LaPier T., Lomi C., Maart S., Matereke N., Meyer E.R., M'kumbuzi V.R.P., Mostert-Wentzel K., Myezwa H., Olsen M.F., Peterson C., Petursdottir U., Robinson J., Sangroula K., Stensdotter A.-K., Tan B.Y., Tschoepe B.A., Bruno S., Mathur S., Wong W.P. The Second Physical Therapy Summit on Global Health: developing an action plan to promote health in daily practice and reduce the burden of non-communicable diseases. Physiother. Theory Pract. 2014;30:261–275. doi: 10.3109/09593985.2013.856977. [DOI] [PubMed] [Google Scholar]
- 9.Geidl W., Semrau J., Pfeifer K. Health behaviour change theories: contributions to an ICF-based behavioural exercise therapy for individuals with chronic diseases. Disabil. Rehabil. 2014;36:2091–2100. doi: 10.3109/09638288.2014.891056. [DOI] [PubMed] [Google Scholar]
- 10.Newsom J.T., Huguet N., McCarthy M.J., Ramage-Morin P., Kaplan M.S., Bernier J., McFarland B.H., Oderkirk J. Health behavior change following chronic illness in middle and later life. J. Gerontol. B Psychol. Sci. Soc. Sci. 2011;67:279–288. doi: 10.1093/geronb/gbr103. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Arne M., Janson C., Janson S., Boman G., Lindqvist U., Berne C., Emtner M. Physical activity and quality of life in subjects with chronic disease: chronic obstructive pulmonary disease compared with rheumatoid arthritis and diabetes mellitus. Scand. J. Prim. Health Care. 2009;27:141–147. doi: 10.1080/02813430902808643. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Courneya K.S., Katzmarzyk P.T., Bacon E. Physical activity and obesity in Canadian cancer survivors. Cancer. 2008;112:2475–2482. doi: 10.1002/cncr.23455. [DOI] [PubMed] [Google Scholar]
- 13.Kersten R.F., Stevens M., van Raay J.J., Bulstra S.K., van den Akker-Scheek I. Habitual physical activity after total knee replacement. Phys. Ther. 2012;92:1109–1116. doi: 10.2522/ptj.20110273. [DOI] [PubMed] [Google Scholar]
- 14.Zhao G., Ford E.S., Li C., Mokdad A.H. Are United States adults with coronary heart disease meeting physical activity recommendations? Am. J. Cardiol. 2008;101:557–561. doi: 10.1016/j.amjcard.2007.10.015. [DOI] [PubMed] [Google Scholar]
- 15.Bithell C. Developing theory in a practice profession. Physiother. Res. Int. 2005;10:iii–v. doi: 10.1002/pri.26. [DOI] [PubMed] [Google Scholar]
- 16.Shepard K.F. Are you waving or drowning? Phys. Ther. 2007;87:1543–1554. doi: 10.2522/ptj.2007.mcmillan.lecture. [DOI] [PubMed] [Google Scholar]
- 17.O'Donoghue G., Doody C., Cusack T. Physical activity and exercise promotion and prescription in undergraduate physiotherapy education: content analysis of Irish curricula. Physiotherapy. 2011;97:145–153. doi: 10.1016/j.physio.2010.06.006. [DOI] [PubMed] [Google Scholar]
- 18.Dean E., Al-Obaidi S., de Andrade A.D., Gosselink R., Umerah G., Al-Abdelwahab S., Anthony J., Bhise A.R., Bruno S., Butcher S. The first physical therapy summit on global health: implications and recommendations for the 21st century. Physiother. Theory Pract. 2011;27:531–547. doi: 10.3109/09593985.2010.544052. [DOI] [PubMed] [Google Scholar]
- 19.Probst A. Modell der menschlichen Bewegung in der Physiotherapie. physioscience. 2007;3:131–135. [Google Scholar]
- 20.World Health Organization . World Health Organization; 2001. International Classification of Functioning, Disability and Health: ICF. [Google Scholar]
- 21.Arbeitsgruppe "Bewegungstherapie Ziele und Aufgaben der Arbeitsgruppe „Bewegungstherapie “in der Deutschen Gesellschaft für Rehabilitationswissenschaften (DGRW) Rehabilitation. 2009;48:252–255. doi: 10.1055/s-0029-1234051. [DOI] [PubMed] [Google Scholar]
- 22.Geidl W., Pfeifer K. Körperliche Aktivität und körperliches Training in der Rehabilitation des Typ-2-Diabetes. Rehabilitation. 2011;50:255–265. doi: 10.1055/s-0031-1280805. [DOI] [PubMed] [Google Scholar]
- 23.Geidl W., Hofmann J., Göhner W., Sudeck G., Pfeifer K. Verhaltensbezogene Bewegungstherapie–Bindung an einen körperlich aktiven Lebensstil. Rehabilitation. 2012;51:259–268. doi: 10.1055/s-0031-1280803. [DOI] [PubMed] [Google Scholar]
- 24.Goldstein M.S., Scalzitti D.A., Craik R.L., Dunn S.L., Irion J.M., Irrgang J., Kolobe T.H.A., McDonough C.M., Shields R.K. The revised research agenda for physical therapy. Phys. Ther. 2011;91:165–174. doi: 10.2522/ptj.20100248. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Shepard K.F. Are You waving or drowning? Phys. Ther. 2007;87:1543–1554. doi: 10.2522/ptj.2007.mcmillan.lecture. [DOI] [PubMed] [Google Scholar]
- 26.Foster N.E., Delitto A. Embedding psychosocial perspectives within clinical management of low back pain: integration of psychosocially informed management principles into physical therapist practice - challenges and opportunities. Phys. Ther. 2011;91:790–803. doi: 10.2522/ptj.20100326. [DOI] [PubMed] [Google Scholar]
- 27.Allet L., Cieza A., Bürge E., Finger M., Stucki G., Huber E.O. Intervention categories for physiotherapists treating patients with musculoskeletal conditions on the basis of the International Classification of Functioning, Disability and Health. Int. J. Rehabil. Res. 2007;30:273–280. doi: 10.1097/MRR.0b013e3282f14458. [DOI] [PubMed] [Google Scholar]
- 28.Finger M.E., Cieza A., Stoll J., Stucki G., Huber E.O. Identification of intervention categories for physical therapy, based on the international classification of functioning, disability and health: a Delphi exercise. Phys. Ther. 2006;86:1203–1220. doi: 10.2522/ptj.20050134. [DOI] [PubMed] [Google Scholar]
- 29.Mittrach R., Grill E., Walchner-Bonjean M., Scheuringer M., Boldt C., Huber E.O., Stucki G. Goals of physiotherapy interventions can be described using the International Classification of Functioning, Disability and Health. Physiotherapy. 2008;94:150–157. [Google Scholar]
- 30.Frerichs W., Kaltenbacher E., van de Leur J.P., Dean E. Can physical therapists counsel patients with lifestyle-related health conditions effectively? A systematic review and implications. Physiother. Theory Pract. 2012;28:571–587. doi: 10.3109/09593985.2011.654179. [DOI] [PubMed] [Google Scholar]
- 31.Mohan N., Collins E., Cusack T., O'Donoghue G. Physical activity and exercise prescription: senior physiotherapists' knowledge, attitudes and beliefs. Physiother. Pract. Res. 2012;33:71–80. [Google Scholar]
- 32.Mittag O., Müller V., Bjarnason-Wehrens B., Böhmen S., Karoff M., Maurischat C. Interdisziplinäre Behandlungskonzepte in der kardialen Rehabilitation: eine Erhebung von Ist-und Soll-Zustand anhand von Indikationsprofilen („problem-treatment-pairs”), Physikalische Medizin, Rehabilitationsmedizin. Kurortmedizin. 2007;17:320–326. [Google Scholar]
- 33.Scheer D., Konrad W., Scheel O., Ulmer F., Hohlt A. Fokusgruppen im Mixed-Method-Design: Kombination einer standardisierten und qualitativen Erhebung. In: Schulz M., Mack B., Renn O., Mack B., editors. Fokusgruppen in der empirischen Sozialwissenschaft: Von der Konzeption bis zur Auswertung. Springer; Wiesbaden: 2012. pp. 148–167. [Google Scholar]
- 34.von Elm E., Altman D.G., Egger M., Pocock S.J., Gøtzsche P.C., Vandenbroucke J.P., Initiative S. The strengthening the reporting of observational studies in epidemiology (STROBE) statement: guidelines for reporting observational studies. Int. J. Surg. 2014;12:1495–1499. doi: 10.1016/j.ijsu.2014.07.013. [DOI] [PubMed] [Google Scholar]
- 35.Vandenbroucke J.P., von Elm E., Altman D.G., Gøtzsche P.C., Mulrow C.D., Pocock S.J., Poole C., Schlesselman J.J., Egger M., Initiative S. Strengthening the reporting of observational studies in epidemiology (STROBE): explanation and elaboration. PLoS Med. 2007;4(10):e297. doi: 10.1371/journal.pmed.0040297. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Leech N.L., Onwuegbuzie A.J. Guidelines for conducting and reporting mixed research in the field of counseling and beyond. J. Counsel. Dev. 2010;88:61–69. [Google Scholar]
- 37.Leech N.L., Onwuegbuzie A.J. Beyond constant comparison qualitative data analysis: using NVivo. Sch. Psychol. Q. 2011;26(1):70. [Google Scholar]
- 38.Rentenversicherung Deutsche, Reha-Bericht 2015. Die medizinische und berufliche Rehabilitation der Rentenversicherung im Licht der Statistik. http://www.deutsche-rentenversicherung.de/Allgemein/de/Inhalt/6_Wir_ueber_uns/03_fakten_und_zahlen/03_statistiken/02_statistikpublikationen/02_rehabericht_2015.pdf?__blob=publicationFile&v=4 Berlin. 31 August 2017.
- 39.Rentenversicherung Deutsche. Leistungen zur Teilhabe am Arbeitsleben (LTA) Rahmenkonzept der Deutschen Rentenversicherung. http://www.deutsche-rentenversicherung.de/cae/servlet/contentblob/207034/publicationFile/2130/rahmenkonzept_lta_datei.pdf 31 August 2017.
- 40.Jonkisz E., Moosbrugger H., Brandt D.-P.H. Planung und Entwicklung von Tests und Fragebogen. In: Moosbrugger H., Kelava A., editors. Testtheorie und Fragebogenkonstruktion. Springer; 2012. pp. 27–74. [Google Scholar]
- 41.Werle J., Woll A., Tittlbach S. Kohlhammer; Stuttgart: 2006. Gesundheitsförderung: Körperliche Aktivität und Leistungsfähigkeit im Alter. [Google Scholar]
- 42.Stamer M., Zeisberger M., Kleineke V., Brandes I., Meyer T. MeeR Merkmale einer guten und erfolgreichen Reha-Einrichtung im Auftrag der Deutschen Rentenversicherung Bund. https://www.mh-hannover.de/fileadmin/institute/epidemiologie/epi/Forschung/Projekte/MeeR/Meer_Abschlussbericht.pdf 31 August 2017.
- 43.Ströbl V., Friedl-Huber A., Küffner R., Reusch A., Vogel H., Faller H. Vorbereitung eines Zentrums Patientenschulung - Abschlussbericht. http://www.zentrum-patientenschulung.de/verein/berichte/Abschlussbericht_Zentrum_Patientenschulung_2007.pdf 31 August 2017.
- 44.Onwuegbuzie A.J., Collins K.M.T. A typology of mixed methods sampling designs in social science research. Qual. Rep. 2007;12:281–316. [Google Scholar]
- 45.Collins L.M., Lanza S.T. Wiley; Hoboken, NJ: 2010. Latent Class and Latent Transition Analysis: with Applications in the Social, Behavioral, and Health Sciences. [Google Scholar]
- 46.Sudeck G., Pfeifer K. Evaluation bewegungsbezogener Interventionen in Therapie und Rehabilitation. In: Woll A., Mess F., Haag H., Alkemeyer T., editors. Handbuch Evaluation im Sport. Hofmann, Schorndorf; 2010. pp. 89–111. [Google Scholar]
- 47.Saaty T.L. How to make a decision: the analytic hierarchy process. Eur. J. Oper. Res. 1990;48:9–26. doi: 10.1016/0377-2217(90)90060-o. [DOI] [PubMed] [Google Scholar]
- 48.Mayring P., Mayring P. Beltz; Weinheim: 2008. Qualitative Inhaltsanalyse: Grundlagen und Techniken. [Google Scholar]
- 49.Kuckartz U. Juventa; Weinheim: 2016. Qualitative Inhaltsanalyse: Methoden, Praxis, Computerunterstützung, Beltz. [Google Scholar]
- 50.Miles M.B., Huberman A.M., Huberman A.M., Saldaña J. Sage; London: 2013. Qualitative Data Analysis: a Methods Sourcebook. [Google Scholar]
- 51.Bloor M. Addressing social problems through qualitative research. In: Silverman D., editor. Qualitative Research: Issues of Theory, Method and Practice. Sage; Los Angeles: 2011. pp. 399–415. [Google Scholar]
- 52.Dürrenberger G., Kastenholz H., Behringer J. Integrated assessment focus groups: bridging the gap between science and policy? Sci. Publ. Pol. 1999;26:341–349. [Google Scholar]
- 53.Collins K.M.T., Onwuegbuzie A.J., Sutton I.L. A model incorporating the rationale and purpose for conducting mixed methods research in special education and beyond. Learn Disabil. 2006;4:67–100. [Google Scholar]
- 54.Glasgow R.E., Vogt T.M., Boles S.M. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am. J. Publ. Health. 1999;89(9):1322–1327. doi: 10.2105/ajph.89.9.1322. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 55.Stirman S.W., Kimberly J., Cook N., Calloway A., Castro F., Charns M. The sustainability of new programs and innovations: a review of the empirical literature and recommendations for future research. Implement. Sci. 2012;7:17. doi: 10.1186/1748-5908-7-17. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 56.World Medical Association Declaration of Helsinki: ethical principles for medical research involving human subjects. J. Am. Med. Assoc. 2013;310(20):2191–2194. doi: 10.1001/jama.2013.281053. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.

