Research activities contributing to evidence-based practice implementation in Japanese rehabilitation professionals

Abstract

[Purpose] This study aimed to determine whether certain research activities improve the attitude of rehabilitation professionals towards evidence-based practice and its implementation in Japan. [Participants and Methods] We included physical, occupational, and speech therapists currently working in clinical settings. We employed hierarchical multiple regression analyses to assess the attitude of rehabilitation professionals towards evidence-based practice and research activities. Scores of the five dimensions of the Health Sciences-Evidence Based Practice questionnaire were considered the dependent variables. The five dimensions were as follows: Dimension 1, attitude towards evidence-based practice; Dimensions 2–4, evidence-based practice implementation; and Dimension 5, work environment related to evidence-based practice barriers–facilitators. The four sociodemographic variables (gender, academic degree, clinical experience, and the number of therapists at work) were initially included, following which self-reported research achievements were supplemented as independent variables (the number of case studies, literature reviews, cross-sectional studies, and longitudinal studies). [Results] We analyzed data from 167 participants. In addition to sociodemographic variables, the research achievements that statistically increased F-values of the modeling were case study achievements in Dimensions 2–3, cross-sectional study achievements in Dimensions 2 and 4, and longitudinal study achievements in Dimension 5. [Conclusion] Case studies and cross-sectional studies could improve evidence-based practice implementation among rehabilitation professionals in Japan.

INTRODUCTION

Evidence-based practice (EBP) is a key indicator of high-quality patient care¹⁾. However, large surveys of 1,000 participants revealed that Japanese physical and occupational therapists lacked self-efficacy in EBP implementation^{2, 3)}. Accordingly, factors that improve attitude toward EBP and its implementation among rehabilitation professionals must be investigated in order to improve the quality of rehabilitation in Japan.

Using a survey of Australian physical therapists, a previous systematic review⁴⁾ proposed attitude toward involvement in research activities as a possible facilitator of EBP implementation into clinical practice⁵⁾. Previous Japanese studies also have found that experience with research activities may improve attitude and adherence toward EBP. For example, Fujimoto et al.²⁾ conducted a survey among 1,000 physical therapists randomly selected from members of a certain prefectural physical therapist association and investigated their involvement in research activities on a 3-point Likert scale (yes, partially, and no) as well as their positive attitude and behavior toward EBP on a 5-point Likert scale ranging from “strongly agree” to “strongly disagree”. The 5-point Likert scale was divided into two groups: an “agree group”, which included “strongly agree” and “agree”, and a “disagree group”, which included the remaining three responses. There were statistically significant differences in the odds ratio of the agree group on 10 items (e.g., “I am confident in my ability to critically review professional literature”)²⁾. In addition, several questions on attitudes and behaviors toward EBP and clinical practice guidelines (CPGs) were found to be statistically related to gender, clinical experience duration, academic degrees, number of people at workplace, and certification or specialized qualification²⁾. Tomotaki et al.⁶⁾ also used the Evidence-Based Practice Questionnaire to survey 843 clinical nurses at two university hospitals in Japan⁷⁾. According to the researchers⁶⁾, the Evidence-Based Practice Questionnaire scores were positively associated with experience with ≥2 research activities, duration of clinical experience, advanced qualification, current position, academic degrees, education of research method, and education on EBP. These findings suggest that, in addition to the demographic variables of gender, years of clinical experience, academic degrees, and the number of people at work, participation in research activities may promote EBP implementation. However, it is unknown what types of research activities promote EBP attitudes and implementation.

There was no comprehensive measure to evaluate EBP attitudes and implementation until recently, so only ambiguous relationships between research activities and EBP attitudes and behaviors could be verified, as in the study by Fujimoto et al²⁾. Recently, a multidimensional questionnaire called health sciences evidence-based practice (HS-EBP)^8,9,10) was developed to comprehensively assess attitudes and implementation of EBP, and normative scores were known in Spain¹¹⁾. Under such circumstance, the purpose of this study was to determine which research activities influence rehabilitation professionals’ attitudes and implementation of EBP.

PARTICIPANTS AND METHODS

In this study, hierarchical multiple regression models with possibly relevant sociodemographic variables were developed to assess Japanese rehabilitation professionals’ attitudes toward EBP and research activities. An anonymous mail survey was used to collect data from July 2021 to August 2021. Because of the anonymous mail survey, no written informed consent was required, which was approved by an institutional research committee (Saitama Prefectural University, No. 20098).

Participants were chosen based on the following criteria: (1) a member of the Japanese Society of Allied Health and Rehabilitation and (2) currently undertaking clinical work as a physical, occupational, or speech therapist. Authors and members of educational or research institutions were among those who were barred.

As dependent variables, the Japanese HS-EBP¹⁰⁾, a reliable and validated five-dimensional questionnaire on EBP^8,9,10), was used. Dimension 1 focuses on beliefs and attitudes and includes 12 items (e.g., “I consider EBP improves the quality and results of interventions”). Dimensions 2–4 are concerned with EBP implementation. Dimension 2 relates to scientific research findings and includes 14 items (e.g., “I ask myself questions in such a way that they can be answered through results from research”, and “I usually assess the quality of the methodology used in the research studies I find”). Dimension 3 focuses on professional practice development and includes 10 items (e.g., “I integrate the preferences, values, and expectations of the patient in my interventions”). Dimension 4 is about evaluating results and includes 12 items (e.g., “I use standardized measures, based on scientific evidence, to assess the results of my interventions”). Dimension 5 has 12 items that relate to work environment perceived as barriers/facilitators to implementing EBP in daily practice (e.g., “Keeping up-to-date with results from research is a priority in my workplace”). Each item was rated on an 11-point numerical rating scale by participants, and a mean score in each domain was calculated. A higher score indicates a more positive attitude toward EBP in Dimension 1, more EBP implementation in Dimensions 2–4, and a more supportive environment toward EBP in Dimension 5. Sum scores in each domain were also computed.

This study employed eight independent factors, four of which were sociodemographic variables previously linked to attitudes toward EBP²⁾ and four of which were self-reported research achievements over the last 5 years (case studies, literature reviews, cross-sectional studies, and longitudinal studies). Gender, highest academic degree (career college, junior college, college, master’s degree, doctoral degree), duration of clinical experience as a therapist (<3 years, 3–5 years, 6–10 years, 11–15 years, ≥16 years), and number of therapists at work (<3 people, 3–5 people, 6–10 people, 11–15 people, ≥16 people) were the four sociodemographic variables studied. According to previous study²⁾, the highest academic degree classifications were divided into three categories: career college, junior college or college, and master’s or doctoral degree. Participants were asked to nominate the number of research achievements as a coauthor and first author in relation to papers or conference presentations for case studies, literature reviews, cross-sectional studies, and longitudinal studies over the last 5 years. The total number of research achievements was calculated regardless of whether the author was first or coauthor, whether it was a paper or a conference presentation, and whether it was presented in English or Japanese.

Participants in this study were also asked to answer 31 questions about EBP and CPGs from a previous study²⁾ in order to better understand attitudes toward EBP and CPGs among Japanese rehabilitation professionals. Seven constructs were chosen from prior study question items used to make up the 31 questions^12,13,14), whose response scales included Likert scales except item 31 with multiple choices: (1) attitude toward EBP (items 1–9), (2) EBP education (items 10–11), (3) EBP-related behavior (items 12–16), (4) attitude toward CPGs (items 17–26), (5) knowledge of EBP and CPGs (item 27; a–j), (6) CPG-related behavior (items 28–30), and (7) obstacles in using CPG (item 31).

A hierarchical multiple regression model was constructed in each dimension of the HS-EBP by first using the enter method to include the four sociodemographic variables and then using the stepwise method to include the four variables of self-reported research achievements. The estimated sample size for sufficient modeling was 105, with 15 samples per variable¹⁵⁾, which was assumed to be achievable given a general response rate of 27%–53% in previous studies^{16, 17)}. The F-values were evaluated after the addition of the independent variable to investigate the independent contribution of each of the four variables of self-reported research achievements to the dependent variable. To validate the results of the multicollinearity tests, the tolerance and the variance inflation factors were applied. The tolerance for multicollinearity was set at ≤0.10 and the variance inflation factor was set at ≥10¹⁸⁾. The following R²-value was interpreted as follows: <0.3, a none‒very weak effect size; 0.3‒0.5, a weak‒low effect size; 0.5‒0.7, a moderate effect size; and >0.7, a strong effect size¹⁹⁾. The Statistical Package for the Social Sciences (version 21.0, IBM Corporation, Armonk, NY, USA) was used to conduct statistical analyses with a 5% statistical significance. Descriptive statistics were used to summarize the characteristics of the participants, as well as their responses to the HS-EBP and the 31 questions about EBP and CPGs. To understand difference in EBP attitude and implementation between Japan and Spain, Hedges’ g of the sum scores in each HS-EBP domain was calculated using normative data of the HS-EBP from the Spanish physiotherapist population (n=419)¹¹⁾. To calculate sum scores, data imputation was performed using a median value when the missing value was <5%^{20, 21)}. The effect size of Hedges’ g was interpreted as small at 0.2, moderate at 0.5, and large at 0.8.

RESULTS

Out of 389 eligible participants (328 physical therapists, 55 occupational therapists, and 6 speech therapists), 167 completed the survey (42.9%) (Table 1), meeting the estimated minimum sample size of 105. For the HS-EBP, there were seven missing data points. Data imputation was not used for the regression modeling, but a median value was used to calculate Hedges’ g values between Japan and Spain. There were 99 missing data points for the 31 questions regarding EBP and CPGs, and no data imputation was performed. Supplementary material 1 details research achievements over the last 5 years.

Table 1. Summary of independent variables.

Variable	N=167
Gender	110 (65.9%) males and 57 (34.1%) females
Highest degree	Career college, Junior college, or College: 158 (94.6%)
	Master degree or Doctoral degree: 9 (5.4%)
Clinical experience as a therapist	<3 years, 20 (12.0%)
	3–5 years, 62 (37.1%)
	6–10 years, 33 (19.8%)
	11–15 years, 26 (15.6%)
	≥16 years, 26 (15.6%)
Number of therapists at work	<3 people, 4 (2.4%)
	3–5 people, 1 (0.6%)
	6–10 people, 7 (4.2%)
	11–15 people, 6 (3.6%)
	≥16 people, 149 (89.2%)
Number of case studies during the last 5 years (number)	0.6 ± 2.0
Number of literature reviews during the last 5 years (number)	0.2 ± 1.0
Number of cross-sectional studies during the last 5 years (number)	0.3 ± 1.1
Number of longitudinal studies during the last 5 years (number)	0.2 ± 1.0

Table 2 summarizes the HS-EBP scores. Except for Dimension 5, there were moderate or large effect sizes of differences indicating a poorer EBP attitude and implementation in Japan than in Spain.

Table 2. Summary of the Health Sciences Evidence-based practice scores of the 167 participants.

Dimension	0–10 mean score	Sum scores	Spanish sum scores11)	Hedges’g†
Dimension 1: Beliefs and attitudes	7.50 ± 1.37	90.01 ± 16.45	97.77 ± 13.88	−0.53
Dimension 2: Results from scientific research	4.82 ± 1.68	67.46 ± 23.56	92.28 ± 25.78	−0.99
Dimension 3: Development of professional practice	6.37 ± 1.40	63.71 ± 14.01	78.45 ± 11.28	−1.22
Dimension 4: Assessment of results	5.77 ± 1.45	69.22 ± 17.44	86.05 ± 19.74	−0.88
Dimension 5: Barriers/facilitators	4.89 ± 1.63	58.69 ± 19.61	61.58 ± 23.82	−0.13

Mean ± standard deviation.

Supplementary material 2 presents results of each multiple regression modeling. In the HS-EBP Dimension 1, none of the eight dependent variables was statistically significant contributing variable (R²=0.038, analysis of variance p=0.173, Durbin–Watson=1.795). In the HS-EBP Dimension 2, statistically significant contributing variables were the number of case studies (unstandardized coefficient=0.216, p=0.001) and cross-sectional studies (unstandardized coefficient=0.269, p=0.003), respectively (R²=0.143, analysis of variance p<0.001, Durbin–Watson=1.642). In the HS-EBP Dimension 3, only the number of cross-sectional studies was a statistically significant contributing variable (unstandardized coefficient=0.196, p<0.001, R²=0.100, analysis of variance p=0.004, Durbin–Watson=1.889). In Dimension 3 modeling, there was one outlier where the predicted value of the measured value exceeded ± 3 standard deviations (SD). In Dimension 4 of the HS-EBP, only the number of cross-sectional studies was a statistically significant contributing variable (unstandardized coefficient=0.272, p=0.010, R²=0.067, analysis of variance p=0.046, Durbin–Watson=1.947). There was one outlier in which the predicted value exceeded the measured value by more than ± 3 SD. In the HS-EBP Dimension 5, statistically significant contributing variables were gender of women (unstandardized coefficient=0.659, p=0.014) and the number of longitudinal studies (unstandardized coefficient=0.390, p=0.003), respectively (R²=0.089, analysis of variance p=0.010, Durbin–Watson=1.630). There was no significant multicollinearity because none of the tolerance and variance inflation factor values in each model met the predefined criteria. The R²-values in each model indicated a none‒very weak effect size.

Supplementary material 3 contains a summary of the responses to the 31 questions about EBP and CPGs. The majority of participants (82.4%) agreed that EBP application should be used in clinical practice, while only a few (9.1%) said that they had learned the fundamentals of EBP as part of their academic preparation.

DISCUSSION

To the best of our knowledge, this is the first study to investigate what research activities can promote EBP implementation, despite the fact that participation in research activities has long been thought to be beneficial in promoting EBP^{2, 5)}. Notably, case study and cross-sectional study achievements were associated with high Dimensions 2–4 scores on the HS-EBP, indicating greater EBP implementation. To promote EBP, two steps regarding attitudes toward EBP (Step 0, cultivate a spirit of inquiry; and Step 6, disseminate EBP results) and five steps (Step 1, ask clinical questions in the PICOT format; Step 2, search for the best evidence; Step 3, critically appraise the evidence; Step 4, integrate the evidence with clinical expertise and patient preferences and values; and Step 5, evaluate the outcomes of the practice decisions or changes based on evidence) are important²²⁾. The dimensions of the HS-EBP do not exactly correspond to the steps of this EBP, but the questions in Dimension 2 appear to be relatively close to Steps 1–2, Dimension 3 to Steps 3–4, and Dimension 4 to Step 5. In this study, case study achievement influenced Dimension 2–3 scores, indicating the significance of conducting case studies to improve competency of EBP in clinical practice and reflects the fact that EBP is required for conducting high-quality case studies. Cross-sectional study performance, on the other hand, contributed to Dimensions 2 and 4 scores. When designing cross-sectional studies and developing discussions, this may reflect the experience of searching for information, summarizing previous findings, and selecting appropriate outcomes.

In this study, we compared the HS-EBP to the normative values of Spanish physiotherapists. Except for Dimension 5, there were moderate or large effect sizes of differences indicating a poorer EBP attitude and implementation in Japan than in Spain. This result is a clear indication of the inadequate status of Japan’s EBP implementation and is consistent with the low number of PEDro searches in Japan²³⁾. In particular, the large effect size observed for Dimensions 2–4, which would indicate the ability of EBP implementation, can be interpreted as indicating the importance of comprehensive education of Steps 1–5 of EBP. In contrast, the lack of significant differences in Dimension 5 suggests that the promotion of EBP in Japan should focus on pre- and postgraduate education rather than the work environment. In fact, none of the ten strategies that reached consensus in a study that examined specific strategies for promoting EBP using the Delphi method included anything related to the work environment²⁴⁾.

Except for gender in Dimension 5, four sociodemographic factors identified in previous preliminary studies^{2, 6)} were incorporated into the model in this study, but none of them were statistically significant contributing factors. This finding is consistent with a previous study in which postgraduate degrees and years of clinical experience were not contributing factors to the attitude of approaching low back pain from a biopsychosocial framework, which is an EBP-compliant concept²⁵⁾. Furthermore, in Dimension 1, neither research achievement was a contributing factor. This means that factors other than the four sociodemographic factors and research achievement included in the model influence attitudes toward EBP. More research on modifiable factors that can assist Japanese rehabilitation professionals in improving their attitudes toward EBP is required. Interestingly, Dimension 5 listed longitudinal study performance as a statistically significant contributing factor. In terms of ethics and study duration, longitudinal studies are more difficult to conduct than other research designs. The supportive environment of the EBP appears to allow for longitudinal studies, which are otherwise relatively difficult.

The results of the 31-question survey appear to be similar to those of a previous study conducted among physical therapists in 2017²⁾. Although the majority of participants had positive attitudes toward EBP (items 1, 2, 4, and 6), only 15.8% of them chose “disagree” or “strongly disagree” in item 3, indicating that incorrect perception of EBP remains prevalent. Furthermore, 53.8% of participants said it took them too long to read the guidelines (item 31), implying that CPGs are possibly misunderstood. Furthermore, 34.9% of participants stated that EBP implementation was not encouraged in their workplace. Moreover, only 9.1% of participants said they learned the foundations for EBP as part of their academic preparation, compared to 10.4% in the previous study in 2017²⁾. Regarding item 27 for statistics understanding, more than half of the participants responded “do not understand/do not know term in itself”. However, the results of items 28 and 29 show that Japanese rehabilitation professionals have no difficulty accessing relevant databases and the internet at home or their workplace. Since 2019, EBP education has been explicitly stated in the Japanese educational curriculum in physical and occupational therapy and included in training programs for new professionals in Japanese physical and occupational therapy associations; thus, these data will be used to assess the effectiveness of the new curriculums.

There are two limitations in this study. The first limitation is due to the sample size. In this study, there are no specific numbers for each occupation. Because physical therapists make up the majority of the population, this result is largely representative of their situation. The population is skewed toward physical therapists, and the findings do not accurately reflect the situation of occupational and speech therapists. However, given that this is a survey of all members of a certain association for professional collaboration, it seems prudent to interpret the results as the status of EBP among rehabilitation professionals who are interested in and practice collaboration with other rehabilitation professions. The second limitation is that, due to the none‒very weak effect size of each model, the research findings that affect EBP implementation identified in this study may not be statistically significant factors when a more comprehensive model is developed. One possible explanation for the low effect size of the model is the overall low research achievements of the participants. When EBP becomes more popular among Japanese rehabilitation professionals, a comprehensive study will be required to determine what modifiable factors may be involved again in the future.

In conclusion, this study discovered that case studies and cross-sectional studies may improve EBP implementation among rehabilitation professionals in Japan. Research activities had no effect on attitudes toward EBP, and further research is needed to investigate the factors and strategies to improve attitudes toward EBP among rehabilitation professionals.

Funding and Conflict of interest

None.