Abstract
Purpose
A recent trial which examined the impact of a structured model of journal club (JC) demonstrated variability in evidence-based practice (EBP) outcomes across allied health disciplines. The aim of the current study was to determine if there are individual practitioner characteristics that could explain this variability and identify potential predictors of EBP outcomes.
Method
This exploratory study used the data obtained from the JC trial. The predictive value of practitioner-related variables including academic degree, previous exposure to EBP training, and previous research involvement was analyzed using univariate logistic regression models. The dose of intervention was also included in the exploratory analysis.
Results
The change in self-reported knowledge, evidence uptake, and attitude following participation in a JC was influenced by individual practitioner characteristics including their discipline, academic background, previous EBP training, previous research involvement, and JC attendance. Improvement in objective knowledge did not seem to be affected by any of these variables. Whether these individual characteristics have the ability to predict who will achieve less than, or greater than, 50% change in knowledge, attitude, and evidence uptake, is not known, except for academic background which predicted physiotherapists’ improvement in attitude.
Conclusion
Participation in a structured JC can lead to significant improvements in EBP knowledge irrespective of the characteristics of individual practitioners. The change in attitude and evidence uptake, however, may be influenced by individual characteristics which will therefore require careful consideration when designing EBP interventions. An EBP intervention is likely to be successful if a systematic assessment of the barriers at different levels (ie, individual, organizational, and contextual) informs the choice of evidence implementation strategy.
Keywords: allied health, evidence-based practice, evidence uptake, individual predictors, journal club
Introduction
The role of journal clubs in facilitating evidence-based practice and continuing education
There is an increasing body of evidence to demonstrate that participation in a journal club (JC) has become a popular approach to promoting evidence-based practice (EBP) across a range of health disciplines.1–7 For instance, the use of JCs has been reported in the medical literature as a means for improving reading habits1–3 and increasing doctors’ knowledge of biostatistics, research design, and critical appraisal.4–7 Nurses have also examined the impact of JCs and found promising results in terms of improving critical appraisal skills, promoting social networking among staff, and facilitating research utilization.8–10 JCs in allied health disciplines have more recently been reported in the literature.11–13 In a study by Bannigan and Bryar,14 JCs were found to have a significant role in overcoming barriers associated with implementing evidence into allied health practice. A study which involved occupational therapists suggested a positive change in professional practice and attitude following participation in a JC.15
JCs are reported to be relatively simple and inexpensive to set up, and are flexible and versatile as they can be developed across services, can be single- or multidisciplinary, and can be conducted face-to-face or electronically.14 As such, JCs have also been used as a venue for continuing education or professional development. Participation in continuing education programs is important for continued provision of safe, effective, and good quality services by allied health practitioners. It is therefore important for practitioners to participate in formal learning activities that provide them with current knowledge and information relevant to their practice.
The International Center for Allied Health Evidence JC model
In 2007, the International Center for Allied Health Evidence (iCAHE) (University of South Australia, Adelaide, SA, Australia) in collaboration with the Department of Health in South Australia commenced the organization of structured JCs across metropolitan and country allied health care sites in South Australia.
The iCAHE JC model is underpinned by the principles of adult learning or Andragogical theory.16,17 Prior to establishing this model, a systematic review was undertaken to identify important core elements of successful JCs.18 These included regular and anticipated meetings, mandatory attendance, clear short- and long-term purposes, appropriate meeting times and incentives, a trained leader to choose papers and lead discussion, circulating papers prior to the meeting, using the Internet for wider dissemination and data storage, using established critical appraisal processes, and summarizing JC findings. To ensure that this JC model was acceptable to users, informal opportunistic consultations were conducted with stakeholders representing allied health practitioners, administrators, funders, and researchers to identify local barriers and drivers for adopting EBP. Specifically structured processes were piloted in six JCs as the “iCAHE model” using a collaborative approach between researchers and allied health practitioners.
The iCAHE JC model aims to provide a practical and efficient collaborative vehicle for allied health practitioners to seek the current best evidence from research, and to apply it in clinical practice. This model of JC addresses barriers associated with access to, and critical analysis of, research evidence. It ensures that the tasks of searching, identifying, and appraising relevant literature, which are common barriers to EBP,19,20 are addressed by the involvement of researchers. Engaging in iCAHE JC provides a medium where allied health practitioners can increase their knowledge of research, share experiences, and discuss current practices with colleagues. It serves not only as a medium to educate allied health practitioners with the key processes involved in EBP, but it also addresses many of the barriers associated with evidence uptake.
The knowledge gap: what characteristics of allied health practitioners determine the success or failure of iCAHE JC?
A recent trial examined the impact of the iCAHE JC on the knowledge, attitude, and evidence uptake of different allied health professionals.21 This study found that only physiotherapists improved in all outcomes; speech pathologists and occupational therapists increased their knowledge but not attitude and evidence uptake; social workers and dieticians showed positive changes in knowledge and evidence uptake but not attitude (Table 1). Based on these data, the authors were interested in understanding if there were specific characteristics of participants that explained this variability in EBP outcomes across disciplines, following exposure to the same intervention. In a recent review of individual determinants related to evidence uptake in allied health, it was found that educational degree or academic qualification, previous EBP or research training, and involvement in research activities were significant predictors.22 Therefore in this analysis, the authors examined whether or not these variables were associated with the outcomes of the iCAHE JC. Our specific research questions were:
Table 1.
Summary of significant findings for all allied health disciplines21
| Allied health disciplines | EBP uptake | Attitude | Objective knowledge (AFT) | Self-reported knowledge |
|---|---|---|---|---|
| Speech pathology | x | x | ✓ | ✓ |
| Physiotherapy | ✓ | ✓ | ✓ | ✓ |
| Social work | ✓ | x | ✓ | ✓ |
| Occupational therapy | x | x | ✓ | ✓ |
| Dietetics/nutrition | ✓ | x | ✓ | ✓ |
Abbreviations: AFT, Adapted Fresno test; EBP, evidence-based practice.
Is there a difference in the mean percentage change in outcome measures in different clusters of participants (eg, undergraduate versus postgraduate, those with research involvement versus those without, etc)?
Are there important individual determinants or predictors of EBP outcomes (ie, knowledge, attitude, and evidence uptake) in allied health?
Continuing professional education programs such as JCs tend to be delivered on the basis of “one size fits all”, and thus allied health practitioners attending these sessions may, or may not, learn anything of value to their daily practice, or be influenced to put new knowledge into practice. Anecdotally, allied health professionals have repeatedly expressed their frustration regarding the inefficiencies of attending continuing education programs which may not be pitched at their level of need or capacity to learn. Therefore, the findings from this study can be used to identify allied health practitioners who are likely to benefit from a JC approach and hence optimize its outcomes. This will ensure provision of individually appropriate, efficient, and targeted education that optimizes learning outcomes, and ensures effective uptake of evidence into practice.
Methods
Ethics
This study was approved by the University of South Australia Human Research Ethics Committee and the Human Research Ethics Committee (Tasmania, Australia) Network. Participants provided written informed consent prior to participation in the study.
Study design and intervention
The data for this study were obtained from the trial which evaluated the impact of iCAHE JC on EBP outcomes across a range of allied health disciplines.21 The iCAHE JC trial utilized a pre–post study design without a control group. The intervention consisted of monthly iCAHE JC sessions for 6 months (Figure 1). Two questionnaires were administered at baseline and 6 months later (Adapted Fresno Test23 [AFT] and Evidence-based Practice Questionnaire [EBPQ]).24
Figure 1.
Summary of the steps involved in the intervention.
Abbreviations: iCAHE, International Center for Allied Health Evidence; JC, journal club; PECOT, population, exposure, comparison, outcome, timeframe; PICO, population, intervention, comparison, outcome.
Participants
Allied health therapists25 including physiotherapists, occupational therapists, speech pathologists, social workers, psychologists, nutritionists/dieticians, and podiatrists were invited to participate in JCs using the iCAHE JC model. Information about the sampling frame has been reported elsewhere. 21 Twelve groups (ie, JCs), consisting of 93 allied health practitioners, agreed to participate in the trial.
Measurements
The two outcome measures used in the trial were the AFT and the EBPQ. The AFT is an objective test of knowledge and skills in the major domains of EBP, such as formulating clinical questions, and searching for and critically appraising research evidence. It is a seven item instrument which yields a maximum possible score of 156.23 Clinical scenarios in the original version of the AFT are relevant only to physiotherapists and occupational therapists; therefore, new scenarios were developed for speech pathologists, social workers, and nutritionists/dieticians. Content validity of these newly developed scenarios was established through formal feedback from EBP experts in each discipline. Interrater reliability was acceptable. The EBPQ measures self-reported EBP uptake, attitude to, and perceived knowledge (self-reported knowledge) of EBP.24 It has 24 items which are organized into three subscales (self-reported EBP uptake, attitude, and knowledge). The items are scored on a scale of 1–7, with a higher score indicating a more positive outcome.
Individual determinants
The variables for analysis with respect to their predictive value were chosen based on the findings of a systematic review of the literature.22 All variables were practitioner-related and included the following: academic degree, previous exposure to EBP or research training, and previous research involvement. The number of JC sessions attended was also included as a potential determinant of outcome. The authors believed that the “dose” of intervention (ie, number of JC sessions) was important to consider as it may potentially impact on EBP outcomes. The possibility of a dose–response relationship between educational programs and learning is supported by the literature.26,27
Statistical methods
All analyses were performed using SAS (v 9.3; SAS Institute Inc, Cary, NC, USA). Outcome measures were analyzed descriptively (mean, standard deviation), and independent t-test and ANOVA models were used to determine if there was a difference in mean percentage change in scores for different subgroups of participants. The authors were concerned, however, that this conventional approach to subgroup analysis was poorly suited to detect heterogeneity in treatment benefit, particularly since the sample size was small (Hayward et al 2006).28 Hayward et al argued that conventional subgroup analysis often has quite limited statistical power.28 “Since there are usually multiple variables that merit subgroup comparisons, the risk of false positive findings due to multiple comparisons compounds the risk of false negative findings due to low statistical power.”28 Due to the constrained sample size in each allied health discipline involved in this trial, the authors treated the outcome variables as a dichotomous variable using the median as a cut-off score (≥50% of possible score).29 Univariate logistic regression models were then applied to determine if there were important predictors (ie, academic degree, EBP/research training, research involvement, JC sessions attended) of EBP outcomes (ie, AFT, attitude, and evidence uptake score). Associations were reported as odds ratios with confidence intervals. Where there were strong univariate associations, the important predictors were added in a stepwise fashion to multivariate predictive models, on an assumption of independence of action of each of the predictor variables.
Results
Sample characteristics
The data from 93 allied health practitioners who participated in the iCAHE JC trial, including ten speech pathologists (SP), 19 physiotherapists (PT), 16 social workers (SW), 36 occupational therapists (OT), and twelve dieticians/nutritionists (DN) were analyzed. The majority of participants worked in an acute setting, held undergraduate degrees in their discipline, and had more than 10 years of clinical experience.
Is there a difference in the mean percentage change in scores per variable (eg, undergraduate versus postgraduate, with EBP/research training versus without EBP/research training, etc) in different clusters of participants?
The disciplines that showed similar findings (as shown in Table 1) for each outcome were clustered and analyzed collectively. For example, for EBP uptake, the scores for PT, SW, and DN were analyzed together while scores for SPs and OTs were pooled. Comparison of mean percentage change for each subgroup based on pre-identified variables (ie, academic background, previous EBP or research training, previous research involvement, JC sessions) are presented in Tables 2–5.
Table 2.
Evidence uptake outcome in different clusters of allied health practitioners
| Variable | PT, SW, DN (N = 47) | ||
|---|---|---|---|
|
|
|||
| Pretest score Mean ± SD |
Posttest score Mean ± SD |
Mean percentage change (95% CI) | |
| Undergraduate (N = 20) | 19.7 ± 4.5 | 23.0 ± 5.4 | 20.2 (5.4–34.9) |
| Postgraduate (N = 27) | 20.5 ± 9.3 | 27.6 ± 7.7 | 69.2 (25.9–112.5) |
| P-value | 0.07 | ||
| No EBP/res training (N = 25) | 18.4 ± 6.3 | 25.3 ± 6.6 | 59.7 (17.5–101.8) |
| With EBP/res training (N = 22) | 22.0 ± 8.6 | 26.0 ± 7.8 | 35.5 (5.5–65.5) |
| P-value | 0.38 | ||
| No res involvement (N = 27) | 18.7 ± 6.8 | 25.0 ± 7.6 | 60.9 (17.5–104.3) |
| With res involvement (N = 18) | 22.1 ± 8.3 | 26.1 ± 6.6 | 30.6 (9.2–52.1) |
| P-value | 0.30 | ||
| 6 JC sessions (N = 5) | 25.2 ± 3.4 | 28.6 ± 5.1 | 13.5 (1.7–25.2) |
| 5 JC sessions (N = 21) | 17.2 ± 6.6 | 26.6 ± 7.4 | 82.4 (31.2–133.6) |
| 4 JC sessions (N = 10) | 22.8 ± 11.2 | 25.4 ± 7.4 | 38.1 (−6.0–82.2) |
| 2–3 JC sessions (N = 3) | 18.3 ± 2.5 | 24.3 ± 12.0 | 31.3 (−40.4–103.1) |
| P-value | 0.41 | ||
| SP, OT (N = 46) | |||
|
|
|||
|
Pretest score Mean ± SD |
Posttest score Mean ± SD |
Mean percentage change (95% CI) | |
|
| |||
| Undergraduate (N = 33) | 20.4 ± 7.8 | 21.9 ± 7.7 | 24.8 (−2.9–52.5) |
| Postgraduate (N = 13) | 27.9 ± 7.2 | 31.2 ± 7.9 | 15.4 (−0.2–31.0) |
| P-value | 0.67 | ||
| No EBP/res training (N = 23) | 23.4 ± 8.4 | 25.8 ± 9.8 | 14.3 (0.4–28.2) |
| With EBP/res training (N = 22) | 21.5 ± 8.3 | 23.1 ± 7.7 | 31.3 (−8.7–71.3) |
| P-value | 0.43 | ||
| No res involvement (N = 29) | 22.8 ± 7.6 | 23.3 ± 8.5 | 7.6 (−5.3–20.5) |
| With res involvement (N = 17) | 22.2 ± 9.6 | 26.6 ± 9.1 | 46.9 (−2.0–95.9) |
| P-value | 0.06 | ||
| 6 JC sessions (N = 11) | 24.7 ± 8.9 | 24.5 ± 10.3 | 0.6 (−13.5–14.8) |
| 5 JC sessions (N = 18) | 22.8 ± 9.3 | 29.1 ± 7.0 | 53.9 (9.3–98.5) |
| 4 JC sessions (N = 6) | 17.5 ± 7.0 | 22.3 ± 6.2 | 36.8 (4.0–69.6) |
| 3 JC sessions (N = 2) | 26.0 ± 8.5 | 14.5 ± 4.9 | −37.8 (−92.3–16.7) |
| 2 JC sessions (N = 3) | 23.7 ± 1.2 | 15.7 ± 10.0 | −34.9 (−78.9–9.2) |
| P-value | 0.11 | ||
Abbreviations: DN, dieticians/nutritionists; EBP, evidence-based practice; JC, journal club; OT, occupational therapists; PT, physiotherapists; res, research; SP, speech pathologists; SW, social workers.
Table 5.
Knowledge (self-reported knowledge) outcome in different clusters of allied health practitioners
| Variable | SP, PT, SW, OT, DN (N = 93) | ||
|---|---|---|---|
|
|
|||
| Pretest score Mean ± SD |
Posttest score Mean ± SD |
Mean percentage change (95% CI) | |
| Undergraduate (N = 53) | 54.6 ± 12.2 | 60.5 ± 12.2 | 13.5 (7.8–19.3) |
| Postgraduate (N = 40) | 58.4 ± 13.0 | 67.5 ± 9.2 | 19.8 (12.0–27.6) |
| P-value | 0.19 | ||
| No EBP/res training (N = 48) | 58.4 ± 13.0 | 67.5 ± 9.2 | 17.4 (10.1–24.6) |
| With EBP/res training (N = 44) | 57.6 ± 12.4 | 64.6 ± 10.7 | 15.3 (9.2–21.4) |
| P-value | 0.67 | ||
| No res involvement (N = 56) | 52.2 ± 11.9 | 59.9 ± 11.8 | 18.3 (11.4–25.2) |
| With res involvement (N = 35) | 62.3 ± 11.1 | 69.0 ± 8.4 | 13.1 (7.3–19.0) |
| P-value | 0.31 | ||
| 6 JC sessions (N = 16) | 61.4 ± 13.9 | 69.1 ± 11.8 | 15.8 (5.7–26.0) |
| 5 JC sessions (N = 39) | 54.7 ± 12.9 | 65.7 ± 9.6 | 24.7 (16.9–32.5) |
| 4 JC sessions (N = 16) | 56.2 ± 12.8 | 65.4 ± 8.4 | 20.0 (9.4–30.7) |
| 3 JC sessions (N = 3) | 59.3 ± 14.0 | 54.3 ± 6.3 | −6.4 (−22.9–10.0) |
| 2 JC sessions (N = 5) | 55.6 ± 7.1 | 54.6 ± 15.4 | −1.8 (−21.8–18.1) |
| P-value | 0.04 | ||
Abbreviations: DN, dieticians/nutritionists; EBP, evidence-based practice; JC, journal club; OT, occupational therapists; PT, physiotherapists; res, research; SP, speech pathologist; SW, social worker.
Evidence uptake outcomes
Three disciplines, namely PT, SW, and DN, showed positive changes in evidence uptake, and were therefore grouped together for analysis of this variable. For this group, no significant difference in mean percentage change was found between subgroups of practitioners (P > 0.05), as shown in Table 2. Although no significant difference was found across practitioners based on attendance, the confidence intervals indicated that only practitioners who completed five to six sessions demonstrated improvements in evidence uptake.
On the other hand, both SP and OT failed to improve on their baseline evidence uptake scores and hence these disciplines were clustered for analysis. Table 2 highlights that there was no significant difference in outcome between subgroups of practitioners (P > 0.05). However, the confidence intervals implied that those with no previous EBP or research training demonstrated improvements in evidence uptake compared to those who received EBP or research training. Similarly for JC attendance, the confidence intervals showed positive changes only for those who attended four to five sessions.
Attitude outcomes
None of the disciplines, except for PTs, showed significant improvement in attitude and therefore the SP, SW, OT, and DN were grouped together while the PTs were analyzed independently (Table 3). For the SP, SW, OT, and DN, there was no significant difference between practitioners based on academic background, previous EBP or research training, previous research involvement, and number of sessions attended. However, the confidence intervals revealed that practitioners with no previous EBP or research training, and those who were not previously involved in research, showed improvement in attitude score. With reference to attendance, no significant difference across practitioners was found; however, the confidence intervals showed that only practitioners who joined five sessions improved significantly in their attitude score.
Table 3.
Attitude outcome in different clusters of allied health practitioners
| Variable | PT (N = 19) | ||
|---|---|---|---|
|
|
|||
| Pretest score Mean ± SD |
Posttest score Mean ± SD |
Mean percentage change (95% CI) | |
| Undergraduate (N = 7) | 17.1 ± 2.7 | 18.4 ± 4.4 | 8.7 (−12.7–30.2) |
| Postgraduate (N = 12) | 20.2 ± 3.5 | 23.9 ± 3.1 | 20.0 (11.9–28.1) |
| P-value | 0.27 | ||
| No EBP/res training (N = 6) | 20.2 ± 2.2 | 23.8 ± 4.0 | 18.5 (3.3–33.6) |
| With EBP/res training (N = 13) | 18.5 ± 3.9 | 21.0 ± 4.5 | 14.7 (2.5–26.8) |
| P-value | 0.72 | ||
| No res involvement (N = 16) | 19.3 ± 3.3 | 21.4 ± 4.7 | 11.1 (4.0–18.1) |
| With res involvement (N = 3) | 18.0 ± 5.2 | 24.3 ± 2.1 | 41.4 (1.9–80.9) |
| P-value | 0.02 | ||
| 5 to 6 JC sessions (N = 10) | 19.0 ± 3.6 | 23.0 ± 3.7 | 23.3 (9.2–37.4) |
| 4 JC sessions (N = 4) | 18.5 ± 2.6 | 22.0 ± 5.5 | 17.8 (−0.9–36.4) |
| 3 JC sessions (N = 0) | NA | NA | NA |
| 2 JC sessions (N = 2) | 21.0 ± 4.2 | 21.0 ± 7.1 | −1.4 (−20.4–17.7) |
| P-value | 0.36 | ||
| SP, SW, OT, DN (N = 74) | |||
|
|
|||
|
Pretest score Mean ± SD |
Posttest score Mean ± SD |
Mean percentage change (95% CI) | |
|
| |||
| Undergraduate (N = 46) | 20.8 ± 2.9 | 21.4 ± 3.2 | 4.0 (−0.4–8.4) |
| Postgraduate (N = 28) | 21.5 ± 3.5 | 21.9 ± 3.7 | 2.6 (−2.5–7.7) |
| P-value | 0.68 | ||
| No EBP/res training (N = 42) | 20.8 ± 3.4 | 22.0 ± 3.4 | 6.4 (2.6–10.2) |
| With EBP/res training (N = 31) | 21.5 ± 2.7 | 21.2 ± 3.3 | −0.4 (−6.3–5.5) |
| P-value | 0.05 | ||
| No res involvement (N = 40) | 20.8 ± 3.1 | 21.7 ± 3.2 | 4.9 (1.0–8.8) |
| With res involvement (N = 32) | 21.5 ± 3.1 | 21.7 ± 3.6 | 1.8 (−3.4–6.9) |
| P-value | 0.33 | ||
| 6 JC sessions (N = 15) | 21.4 ± 3.5 | 22.3 ± 4.3 | 5.7 (−4.2–15.6) |
| 5 JC sessions (N = 30) | 20.9 ± 3.4 | 21.8 ± 2.8 | 6.1 (1.7–10.5) |
| 4 JC sessions (N = 12) | 21.7 ± 2.5 | 22.1 ± 3.1 | 3.3 (−6.9–13.5) |
| 3 JC sessions (N = 3) | 18.7 ± 3.5 | 15.7 ± 1.5 | −14.8 (−27.2 to –2.4) |
| 2 JC sessions (N = 3) | 20.0 ± 2.6 | 19.7 ± 5.1 | −2.4 (−22.1–17.2) |
| P-value | 0.25 | ||
Abbreviations: DN, dieticians/nutritionists; EBP, evidence-based practice; JC, journal club; NA, not applicable; OT, occupational therapists; PT, physiotherapists; res, research; SP, speech pathologists; SW, social workers.
For the PTs, while there was no significant difference between practitioners with undergraduate degrees and those who finished postgraduate degrees, the confidence intervals showed that only those with postgraduate degrees demonstrated a positive change in attitude. A significantly higher mean percentage change was found for PTs with previous research involvement compared to their counterparts. While there was no statistically significant difference found across practitioners based on attendance, the confidence intervals suggested that only those who attended five to six sessions showed improvements in attitude score.
Knowledge outcomes
All disciplines showed positive changes in knowledge as measured by the AFT, and therefore analysis involved all practitioners (Table 4). There was no significant difference in knowledge change between subgroups of practitioners. Similar results were observed for self-reported knowledge except for the number of sessions attended (Table 5). Only those who attended between four to six sessions showed improvements in self-reported knowledge.
Table 4.
Knowledge (Adapted Fresno Test) outcome in different clusters of allied health practitioners
| Variable | SP, PT, SW, OT, DN (N = 93) | ||
|---|---|---|---|
|
|
|||
| Pretest score Mean ± SD |
Posttest score Mean ± SD |
Mean percentage change (95% CI) | |
| Undergraduate (N = 53) | 25.1 ± 11.8 | 52.7 ± 23.9 | 150.4 (104.3–196.4) |
| Postgraduate (N = 40) | 28.1 ± 12.5 | 66.4 ± 21.0 | 208.4 (128.9–287.8) |
| P-value | 0.19 | ||
| No EBP/res training (N = 48) | 23.0 ± 12.0 | 54.6 ± 24.4 | 206.7 (132.1–281.4) |
| With EBP/res training (N = 44) | 30.4 ± 11.1 | 63.9 ± 21.0 | 145.0 (105.4–184.6) |
| P-value | 0.17 | ||
| No res involvement (N = 56) | 25.0 ± 13.4 | 58.1 ± 25.0 | 204.2 (137.8–270.6) |
| With res involvement (N = 35) | 29.2 ± 9.5 | 60.1 ± 21.8 | 128.2 (88.3–168.1) |
| P-value | 0.10 | ||
| 6 JC sessions (N = 16) | 30.2 ± 12.1 | 63.4 ± 23.7 | 144.1 (76.9–211.3) |
| 5 JC sessions (N = 39) | 23.8 ± 10.7 | 63.5 ± 17.2 | 246.4 (167.5–325.3) |
| 4 JC sessions (N = 16) | 29.5 ± 13.7 | 67.1 ± 19.8 | 196.7 (89.1–304.3) |
| 3 JC sessions (N = 3) | 36.1 ± 10.0 | 76.3 ± 6.0 | 127.5 (29.1–225.9) |
| 2 JC sessions (N = 5) | 26.3 ± 9.1 | 64.6 ± 23.1 | 168.8 (41.9–210.7) |
| P-value | 0.53 | ||
Abbreviations: DN, dieticians/nutritionists; EBP, evidence-based practice; JC, journal club; OT, occupational therapists; PT, physiotherapists; res, research; SP, speech pathologists; SW, social workers.
Are there important predictors of EBP outcomes (ie, knowledge, attitude, and evidence uptake) in allied health?
None of the variables tested (academic background, previous EBP, research training, previous research involvement, or numbers of sessions attended) were associated with evidence uptake outcomes (Table 6). Academic background was the only variable with significant impact on improvement in attitude for PTs. Physiotherapists with postgraduate degrees are more likely to demonstrate an increase in attitude score of equal or greater than 50%, as shown in Table 7. No significant relationships were found between knowledge outcome (both for AFT and self-reported knowledge) and any of the potential explanatory variables (Tables 8 and 9).
Table 6.
Potential predictors of evidence uptake following journal club exposure
| Variable | PT, SW, DN OR (95% CI) (N = 47) |
|---|---|
| Academic background | 2.18 (0.67–7.09) |
| Previous EBP/res training | 0.32 (0.09–1.06) |
| Previous res involvement | 1.35 (0.41–4.46) |
| Number of JC sessions attended (5–6 sessions versus 2–4 sessions) | 2.62 (0.67–10.35) |
|
SP, OT OR (95% CI) (N = 46) |
|
|
| |
| Academic background | 2.33 (0.63–8.64) |
| Previous EBP/res training | 1.08 (0.33–3.55) |
| Previous res involvement | 2.50 (0.73–8.60) |
| Number of JC sessions attended (5–6 sessions versus 2–4 sessions) | 1.63 (0.39–6.81) |
Note: *Median: 15.78.
Abbreviations: DN, dieticians; EBP, evidence based practice; JC, journal club; OR, odds ratio; OT, occupational therapists; PT, physiotherapists; res, research; SP, speech pathologists; SW, social workers.
Table 7.
Potential predictors of attitude change following journal club exposure
| Variable | SP, SW, OT, DN OR (95% CI) (N = 74) |
|---|---|
| Academic background | 1.56 (0.60–4.02) |
| Previous EBP/res training | 0.55 (0.21–1.43) |
| Previous res involvement | 0.84 (0.33–2.14) |
| Number of JC sessions attended (5–6 sessions versus 2–4 sessions) | 1.04 (0.35–3.12) |
|
PT OR (95% CI) (N = 19) |
|
|
| |
| Academic background | 12.50 (1.34–116.79) |
| Previous EBP/res training | 0.80 (0.10–6.10) |
| Previous res involvement | 1.20 (0.09–16.24) |
| Number of JC sessions attended (5–6 sessions versus 2–4 sessions) | 2.00 (0.20–19.91) |
Note: *Median: 4.76.
Abbreviations: DN, dieticians; EBP, evidence based practice; JC, journal club; OR, odds ratio; OT, occupational therapists; PT, physiotherapists; res, research; SP, speech pathologists; SW, social workers.
Table 8.
Potential predictors of knowledge outcome (Adapted Fresno Test) following journal club exposure
| Variable | PT, SW, DN, SP, OT OR (95% CI) (N = 93) |
|---|---|
| Academic background | 1.21 (0.53–2.75) |
| Previous EBP/res training | 1.18 (0.52–2.68) |
| Previous res involvement | 0.67 (0.28–1.57) |
| Number of JC sessions attended (5–6 sessions versus 2–4 sessions) | 1.09 (0.42–2.86) |
Note: *Median: 130.00
Abbreviations: DN, dieticians; EBP, evidence based practice; JC, journal club; OR, odds ratio; OT, occupational therapists; PT, physiotherapists; res, research; SP, speech pathologists; SW, social workers.
Table 9.
Potential predictors of knowledge outcome (selfreported knowledge) following journal club exposure
| Variable | PT, SW, DN, SP, OT OR (95% CI) (N = 93) |
|---|---|
| Academic background | 1.01 (0.44–2.31) |
| Previous EBP/res training | 1.41 (0.62–3.20) |
| Previous res involvement | 0.75 (0.32–1.75) |
| Number of JC sessions attended (5–6 sessions versus 2–4 sessions) | 1.77 (0.67–4.66) |
Note: *Median: 12.96.
Abbreviations: DN, dieticians/nutrionists; EBP, evidence-based practice; JC, journal club; OR, odds ratio; OT, occupational therapists; PT, physiotherapists; res, research; SP, speech pathologists; SW, social worker.
Discussion
Overall findings
Based on this analysis, evidence uptake, self-reported knowledge, and attitude change following exposure to iCAHE JC may be influenced by a number of factors, including academic background, previous EBP or research training, previous research involvement, number of JC sessions attended, and discipline. However, the effectiveness of iCAHE JC in improving objective knowledge was not influenced by any of these variables. While the results highlight an important issue in EBP training (ie, role of individual characteristics in influencing EBP uptake), the small sample size used in this study constrained confidence for generalization of these findings. In terms of whether these individual characteristics have the ability to determine who will achieve less than, or greater than, 50% change in knowledge, attitude, and evidence uptake outcome, no single variable was found to be strongly predictive, except for academic background which predicted improvement in attitude for PTs. Considering that there was only one significant predictor, it was not possible to proceed with constructing a multivariate predictive model.
General interpretation in the context of current evidence
The finding that an EBP training model such as iCAHE JC improves objective knowledge irrespective of the allied health discipline and individual characteristics of the practitioner is not surprising, as JC has been shown to facilitate learning of the key processes involved in EBP.15,30–32 As previously reported, the success of a JC relies on the inclusion of the following key ingredients: clearly set goals, mentoring, use of structured critical appraisal instrument, adhering to the principles of adult learning theory, support from researchers/academics, and regular meetings.18,32 An interesting finding is the impact of the number of sessions attended on knowledge gain. Comparison of different subgroups based on the number of sessions attended, although not significant, yielded a consistent finding. Improved scores for objective knowledge (as measured by the AFT) were observed regardless of the number of sessions attended. This means that participation in a JC, even for only two sessions, may improve knowledge. This has practical implications for those who work in busy departments where heavy clinical workloads may make it difficult for some allied health practitioners to attend professional development programs regularly. Busy practitioners who can attend at least two JC sessions will be able to use the knowledge gained from JC as a foundation to develop and build evidence into their clinical practice. However, based on the findings from this analysis, participation in more sessions can result in better acquisition of EBP competencies. The authors hypothesize that involvement in at least five JC sessions is required to expect a considerable change in self-reported knowledge, attitude, and evidence uptake. In a recent review which examined the effectiveness of JCs, it was reported that there are mediating factors which could influence its success, including the duration and frequency of exposure.32 A dose–response relationship where proficiency increased over time was proposed.32 The authors of the current study believe that one needs more exposure to JC to facilitate that attitudinal shift required to change evidence uptake behavior. It takes longer exposure to JC to foster the critical thinking necessary to change practice behavior. This kind of thinking was highlighted by Honey et al who argued that JCs may be used to “foster critical thinking about clinical practices that exist and generate creative thinking about how practices may be carried out differently.”33
The results of this study indicate that while individual characteristics are important ingredients in EBP, they are not in themselves predictors of outcomes following exposure to an EBP intervention such as JC. An individual health practitioner cannot be isolated from all other factors – organizational, social, political, and contextual – that affect the delivery of health care. After exploring the individual determinants of EBP, it is clear that in addition to an individual practitioner’s characteristics, the variability in EBP outcomes is a result of the interplay of these complex factors. In other words, the response of allied health practitioners to an EBP intervention is likely to be dependent on their characteristics, aspects of their practice setting, and the organizational context in which they deliver care. There is a wealth of literature which supports this framework and proposes that the assessment of barriers and facilitators at each of those levels is the essential step in facilitating EBP uptake.34–36 Grimshaw et al37 report that barriers may be classified into: knowledge management barriers (eg, access to research, lack of time and skills to analyze research), structural barriers (eg, financial disincentives), organizational barriers (eg, lack of facilities), peer group barriers (eg, local care not in line with desired practice), professional barriers (eg, lack of skills), and professional–patient interaction barriers (eg, communication and information processing issues). The Cochrane Effective Practice and Organization of Care group38 provides reviews of EBP interventions or strategies that can potentially address these barriers and facilitate practice behavior change. The choice of intervention/s to use should therefore be guided by the barriers identified at different levels as a comprehensive approach may be the key to closing the gap between research and clinical practice.
Limitations
The results of this exploratory analysis provide an initial step toward a better understanding of evidence implementation process in allied health. However, it has limitations which influence generalization of the results. The sample size was small and may not be reflective of all allied health practitioners in Australia; it may have also limited the ability to explore relationships between predictor variables and outcomes. Replication of the intervention in larger studies is required. In addition, the study focused only on individual characteristics as potential determinants and did not take into account other variables that play a role in the context of a practice setting (eg, culture, systems in place, resources available). The interaction between individual predictors was also not examined.
Implications on practice
The effectiveness of iCAHE JC in improving EBP knowledge and skills was not determined by any of the individual characteristics of the practitioners. Therefore, any allied health practitioner who aims to increase knowledge and skills relevant to EBP can participate in a structured JC such as the iCAHE JC and expect an improvement. The authors recommend the use of the iCAHE JC model for routine professional development as acquisition of EBP knowledge and skills creates a strong foundation for facilitating change in attitude and practice behavior. Although this is necessary, it may not be sufficient by itself as there may be other barriers relevant to the practice setting and organizational context that need to be addressed. It is therefore important to consider all the modifiable barriers and implement strategies which will effectively address the identified barriers and ultimately promote EBP.
Implications to research
A more in-depth study based on specific allied health settings (eg, palliative care, hand rehabilitation) could provide a better understanding of the role of organizational and contextual factors in the implementation of research evidence. Future research should build on the results of this study and work to integrate an educational strategy, such as iCAHE JC, with other context appropriate EBP interventions and then evaluate outcomes.
Conclusion
Participation in a structured JC such as iCAHE JC leads to significant improvements in EBP knowledge and skills regardless of the individual characteristics of the allied health practitioner and the number of sessions attended. However, change in evidence uptake and attitude may be influenced by an individual’s discipline, academic background, previous EBP or research training, previous research involvement, and JC attendance. Whether these individual characteristics have the ability to predict who will achieve less than, or greater than, 50% change in knowledge, attitude, and evidence uptake, is not known, except for academic background which predicted physiotherapists’ improvement in attitude. The results of this study suggest that the individual characteristics of practitioners be considered along with other organizational and contextual factors when designing EBP interventions. An EBP intervention is likely to be successful if a systematic assessment of the barriers at different levels informs the choice of evidence implementation strategy.
Footnotes
Disclosure
The authors report no conflicts of interest in this work.
References
- 1.Linzer M. The journal club and medical education: over one hundred years of unrecorded history. Postgrad Med J. 1987;63:475–478. doi: 10.1136/pgmj.63.740.475. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Linzer M, Brown JT, Frazier LM, DeLong ER, Siegel WC. Impact of a medical journal club on house-staff reading habits, knowledge, and critical appraisal skills: a randomized control trial. JAMA. 1988;260(17):2537–2541. [PubMed] [Google Scholar]
- 3.Khan KS, Dwarakanath LS, Pakkal M, Brace V, Awonuga A. Postgraduate journal club as a means of promoting evidence-based obstetrics and gynaecology. J Obstet Gynaecol. 1999;19(3):231–234. doi: 10.1080/01443619964968. [DOI] [PubMed] [Google Scholar]
- 4.Seelig C. Affecting residents’ literature reading attitudes, behaviours, and knowledge through a journal club intervention. J Gen Intern Med. 1991;6(4):330–334. doi: 10.1007/BF02597431. [DOI] [PubMed] [Google Scholar]
- 5.Spillane A, Crowe P. The role of the journal club in surgical training. Aust N Z J Surg. 1998;68(4):288–291. doi: 10.1111/j.1445-2197.1998.tb02085.x. [DOI] [PubMed] [Google Scholar]
- 6.MacRae H, Regehr G, McKenzie M, et al. Teaching practicing surgeons critical appraisal skills with an internet-based journal club: a randomized, controlled trial. Surgery. 2004;136(3):641–646. doi: 10.1016/j.surg.2004.02.003. [DOI] [PubMed] [Google Scholar]
- 7.Mukherjee R, Owen K, Hollins S. Evaluating qualitative papers in a multidisciplinary evidence-based journal club: a pilot study. Psychiatr Bull. 2006;30:31–34. [Google Scholar]
- 8.Kartes S, Kamel H. Geriatric journal club for nursing: a forum to enhance evidence-based nursing care in long-term settings. J Am Med Dir Assoc. 2003;4(5):264–267. doi: 10.1097/01.JAM.0000083381.66504.7C. [DOI] [PubMed] [Google Scholar]
- 9.Belanger L. The Vancouver General Hospital Acute Spine Journal Club: a success story. SCI Nurs. 2004;21(1):29–30. [PubMed] [Google Scholar]
- 10.Rich K. The journal club: a means to promote nursing research. J Vasc Nurs. 2006;24(1):27–28. doi: 10.1016/j.jvn.2005.12.001. [DOI] [PubMed] [Google Scholar]
- 11.Turner P, Mjolne I. Journal provision and the prevalence of journal clubs: a survey of physiotherapy departments in England and Australia. Phys Res Int. 2001;6(3):157–169. doi: 10.1002/pri.224. [DOI] [PubMed] [Google Scholar]
- 12.Field N, Agustin CB, Milinkovic D. Could a journal club be an effective education tool for radiation therapists? The Radiographer. 2004;51:123–127. [Google Scholar]
- 13.Sherratt C. The journal club: a method for occupational therapists to bridge the theory-practice gap. Br J Occup Ther. 2005;68(7):301–306. [Google Scholar]
- 14.Bannigan K, Bryar R. The importance of overcoming barriers to research utilization. Int J Ther Rehabil. 2002;9(7):270–273. [Google Scholar]
- 15.McQueen J, Miller C, Nivison C, Husband V. An investigation into the use of a journal club for evidence-based practice. Int J Ther Rehabil. 2006;13(7):311–316. [Google Scholar]
- 16.Knowles MS, Holton EF, III, Swanson RA. The Adult Learner. 5th ed. Texas: Gulf Publishing Company; 1998. [Google Scholar]
- 17.Merriam S, Caffarella R. Learning in Adulthood. Oxford: Jossey-Bass Publishers; 1991. [Google Scholar]
- 18.Deenadayalan Y, Grimmer-Somers K, Kumar S. How to run an effective journal club: a systematic review. J Eval Clin Pract. 2008;14:898–911. doi: 10.1111/j.1365-2753.2008.01050.x. [DOI] [PubMed] [Google Scholar]
- 19.Bennett S, Tooth L, McKenna K, et al. Perceptions of evidence-based practice: a survey of Australian occupational therapists. Aust Occup Ther J. 2003;50:13–22. [Google Scholar]
- 20.Jette DU, Bacon K, Batty C, et al. Evidence-based practice: beliefs, attitudes, knowledge, and behaviours of physical therapists. Phys Ther. 2003;83:786–805. [PubMed] [Google Scholar]
- 21.Lizarondo L, Grimmer-Somers K, Kumar S, Crockett A. Does journal club membership improve research evidence uptake in allied health: a pre-post study. BMC Res Notes. 2012;5:588. doi: 10.1186/1756-0500-5-588. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Lizarondo L, Grimmer-Somers K, Kumar S. A systematic review of the individual determinants of research evidence use in allied health. J Multidiscip Healthc. 2011;4:261–272. doi: 10.2147/JMDH.S23144. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.McCluskey A, Bishop B. The Adapted Fresno Test of competence in evidence-based practice. J Contin Educ Health. 2009;29(2):119–126. doi: 10.1002/chp.20021. [DOI] [PubMed] [Google Scholar]
- 24.Upton D, Upton P. Development of an evidence-based practice questionnaire for nurses. J Adv Nurs. 2006;53(4):454–458. doi: 10.1111/j.1365-2648.2006.03739.x. [DOI] [PubMed] [Google Scholar]
- 25.Turnbull C, Grimmer-Somers K, Kumar S, May E, Law D, Ashworth E. Allied, scientific and complementary health professionals: a new model for Australian allied health. Aust Health Rev. 2009;33(1):27–37. doi: 10.1071/ah090027. [DOI] [PubMed] [Google Scholar]
- 26.Eraut M. Learning to change and/or changing to learn. Learn Health Soc Care. 2004;3(3):111–117. [Google Scholar]
- 27.McGaghie WC, Issenberg SB, Petrusa ER, Scalese RJ. A critical review of simulation-based medical education: 2003–2009. Med Educ. 2010;44(1):50–63. doi: 10.1111/j.1365-2923.2009.03547.x. [DOI] [PubMed] [Google Scholar]
- 28.Hayward RA, Kent DM, Vijan S, Hofer T. Multivariable risk prediction can greatly enhance the statistical power of clinical trial subgroup analysis. BMC Med Res Methodol. 2006;6:18. doi: 10.1186/1471-2288-6-18. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Peters-Klimm F, Laux G, Campbell S, et al. Physician and patient predictors of evidence-based prescribing in heart failure: a multilevel study. PLoS One. 2012;7(2):e31082. doi: 10.1371/journal.pone.0031082. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Milinkovic D, Field N, Agustin CB. Evaluation of a journal club designed to enhance professional development of radiation therapists. Radiography. 2008;14(2):120–127. [Google Scholar]
- 31.Lizarondo L, Kumar S, Grimmer-Somers K. Exploring the impact of a structured model of journal club in allied health – a qualitative study. Creative Education. 2012;3:1094–1100. [Google Scholar]
- 32.Harris J, Kearley K, Heneghan C, et al. Are journal clubs effective in supporting evidence-based decision making? A systematic review. Med Teach. 2011;33(1):9–23. doi: 10.3109/0142159X.2011.530321. [DOI] [PubMed] [Google Scholar]
- 33.Honey CP, Baker JA. Exploring the impact of journal clubs: a systematic review. Nurse Educ Today. 2011;31(8):825–831. doi: 10.1016/j.nedt.2010.12.020. [DOI] [PubMed] [Google Scholar]
- 34.Grol R, Grimshaw J. From best evidence to best practice: effective implementation of change in patients’ care. Lancet. 2003;362:1225–1230. doi: 10.1016/S0140-6736(03)14546-1. [DOI] [PubMed] [Google Scholar]
- 35.Grol R, Wensing M. What drives change? Barriers to and incentives for achieving evidence-based practice. Med J Aust. 2004;180(Suppl 6):S57–S60. doi: 10.5694/j.1326-5377.2004.tb05948.x. [DOI] [PubMed] [Google Scholar]
- 36.Graham ID, Logan J, Harrison MB, et al. Lost in knowledge translation: time for a map? J Contin Educ Health Prof. 2006;26:13–24. doi: 10.1002/chp.47. [DOI] [PubMed] [Google Scholar]
- 37.Grimshaw J, Eccles MP, Lavis J, Hill S, Squires J. Knowledge translation of research findings. Implement Sci. 2012;7:50. doi: 10.1186/1748-5908-7-50. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Cochrane Effective Practice and Organisation of Care Group. [Accessed November 4, 2012.]. [webpage on the Internet]. Available at http://epoc.cochrane.org/epoc-reviews.