Abstract
Objective: Our objective was to compare the change in research informed knowledge of health professionals and their intended practice following exposure to research information delivered by either Twitter or Facebook.
Methods: This open label comparative design study randomized health professional clinicians to receive “practice points” on tendinopathy management via Twitter or Facebook. Evaluated outcomes included knowledge change and self-reported changes to clinical practice.
Results: Four hundred and ninety-four participants were randomized to 1 of 2 groups and 317 responders analyzed. Both groups demonstrated improvements in knowledge and reported changes to clinical practice. There was no statistical difference between groups for the outcomes of knowledge change (P = .728), changes to clinical practice (P = .11) or the increased use of research information (P = .89). Practice points were shared more by the Twitter group (P < .001); attrition was lower in the Facebook group (P < .001).
Conclusion: Research information delivered by either Twitter or Facebook can improve clinician knowledge and promote behavior change. No differences in these outcomes were observed between the Twitter and Facebook groups. Brief social media posts are as effective as longer posts for improving knowledge and promoting behavior change. Twitter may be more useful in publicizing information and Facebook for encouraging course completion.
Keywords: social media, evidence-based practice, communication, education, professional, computer-assisted instruction
INTRODUCTION
A significant gap remains between research generated healthcare knowledge and clinical practice.1–3 Social media can rapidly link researchers and clinicians from diverse geographical regions, disciplines, and areas of practice; making it an ideal medium for knowledge exchange and education. Approximately 25% of health professionals currently use social media for obtaining research information.4
Social media has been defined as a “collection of web-based technologies that share a user-focused approach to design and functionality, where users can actively participate in content creation and editing through open collaboration between members of communities of practice.”5 The use of social media in education may lead to positive learning experiences,5,6 increases in knowledge and skills,7–10 and changes to the clinical practices of health professionals.10,11 However, there is a need for studies to evaluate the relative effectiveness of different social media based applications.12
Two of the largest social media applications are Facebook (1.49 billion monthly active users) and Twitter (316 million monthly active users).13,14 Both sites promote user interaction and allow posting of text, videos, and weblinks; however, Twitter limits posts to 140 characters. Neither site charges access costs. The popularity and features of these sites indicate their potential application in communicating research information and, therefore, were chosen for investigation in this study.
OBJECTIVE
The primary objective of this study was to determine if research information delivered by Twitter or Facebook would result in greater changes in research informed knowledge and practices of health professionals. The secondary aim was to compare participant behavior and engagement with the two mediums.
METHODS
Design
An open label randomized comparative design was used, with a mixed methods approach to data collection and analysis. The Monash University Human Research Ethics committee (CF 14/1372 – 2014000640) approved the study.
Participants
Health professional clinicians of any discipline (e.g., medicine, physiotherapy, podiatry), geographical location, or level of expertise (including undergraduate students), were eligible to participate. Recruitment occurred via an email invitation distributed to clinical affiliates and departments of Monash University, Faculty of Medicine, Nursing and Health Sciences, Australia; Monash University Malaysia; Swami Vivekanand National Institute of Rehabilitation Training and Research, India; and the University of Southern California. Professional associations representing professions registered with the Australian Health Practitioner Regulation Agency15 were also invited to distribute the invitation to participate via email or their own social media sites.
Intervention
A short course, consisting of the same 8 “practice points” or key educational messages of 140 characters or less, on topics related to tendon management were delivered to each group via posts on Twitter16 and Facebook17 web pages. Each practice point was linked to supplementary information in the form of peer-reviewed journal articles or podcasts by clinical experts. The course was designed by educational, clinical, and research experts, and was identical except that the Facebook posts contained the practice point plus an additional 2–6 short written statements (1–2 sentences) that highlighted key concepts from the supplementary information. The practice points were delivered evenly over a 2 week period, to both groups at the same time points. The pages were not restricted access.
Procedure
Clinicians consented to participate by providing contact details through an online survey. Those who provided a valid email address were enrolled. Participants were stratified by role (student, clinician, or other) and randomized to receive the practice points via Twitter or Facebook. Participants received video and written instructions on obtaining a social media account and accessing the practice points from their allocated site. The instructions also encouraged interaction on the allocated site. Participants were sent three reminder emails at each data collection point to minimize attrition. The study was conducted between August and October 2014.
Outcomes
Data was obtained via an anonymous online survey completed 1 week before (baseline assessment) and after (post-intervention assessment) the short course. A password was used to match pre- and post-course data. Demographic details, information on tendon management experience, and current use of social media were obtained.
Outcomes were determined based on the Kirkpatrick hierarchical levels of evaluation 1–3.18 Participation and engagement data was also collected. A data collection summary can be found in Appendix 1.
Kirkpatrick Level 1: Participant Reactions
The Social Media Use and Perception Instrument (SMUPI), a questionnaire of 10 items with high internal consistency,19 measured attitudes towards using social media in continuing professional development.
Kirkpatrick Level 2: Knowledge
Sixteen multiple choice questions assessed knowledge (A–E responses) (Appendix 2). One question correlated with each “practice point” and one correlated with information from each piece of supplementary information. The questions in both assessments were identical, but question and response order were randomized to minimize score improvements based on pattern recognition. Participants were not given assessment answers until the conclusion of the study. Self-rated measures of tendon management confidence and knowledge were also obtained.
Kirkpatrick Level 3: Behavior Change
Participants were asked “has the education you have received via social media during this trial changed the way you practice, or intend to practice, with musculoskeletal clients?” and “has the education you have received during this trial increased your use of research evidence within your clinical practice?”
Participation was evaluated via the number of participants who connected with the social media pages and completed the assessments. Data on interaction was obtained through participant self-report and from the number of times posts were approved of (“liked” or “favorite”), shared or commented on.
Analysis
Mixed linear models were used to analyze the repeated measurements (pre- and post-exposure to the intervention) on the participants. The restricted maximum likelihood method (REML), as implemented in the GenStat statistical package,20 was used to fit the models, calculate predicted means and test, using F-tests, the main effects of group (Twitter vs Facebook) and time (pre vs post) as well as their 2-way interaction. Pairwise least significant difference tests of the group-by-time means were based on these analyses and conducted at the 5% significance level. Diagnostic plots of residuals were checked for assumptions on which these methods are based. Analyses of the 5-point Likert scale responses also used the restricted maximum likelihood method as is customary with large datasets.21 The analyses of binary response outcomes, measured post intervention, were based on logistic regression models, also fitted using GenStat. Discrete count data from Twitter and Facebook sites were analyzed using a variance-stabilizing transformation in an analysis of variance.
RESULTS
Five hundred clinicians consented to participate. Five were excluded due to an invalid email address, and one participant asked to be removed. Four hundred and ninety-four participants were randomized.
The attrition rates from randomization to baseline assessment were 48.2% for the Twitter group and 41.7% for the Facebook group; the difference was not significant [χ2 (1, n = 494) = 2.09, P = .148]. Attrition from baseline assessment to post intervention assessment was 32.8% for the Twitter group and 8.3% for the Facebook group; this difference was significant [χ2 (1, n = 494) = 17.37, P < .001]. Three hundred and seventeen responses were analyzed (140 Twitter, 177 Facebook). There were 99 baseline assessments, 45 post intervention assessments, and 173 matched baseline and post intervention assessments. A consort flow-chart is available in Figure 1.
Figure 1.
Consort flow chart showing attrition of study participants.
Demographics
Demographic data and data on tendon management experience and social media use was obtained from the baseline assessment and is presented in Table 1.
Table 1.
Participant demographics and participant characteristics
| N (%)a | N (%)a | |
|---|---|---|
| Baseline demographic data sets | 128 | 144 |
| Area of practice | ||
| Physiotherapy/physical therapy | 95 (74.2) | 98 (68.1) |
| Medicine | 18 (14.1) | 19 (13.2) |
| Osteopathy | 2 (1.6) | 3 (2.1) |
| Podiatry | 7 (5.5) | 11 (7.6) |
| Other | 4 (3.1) | 11 (7.6) |
| Not stated | 2 (1.6) | 2 (1.4) |
| Role | ||
| Undergraduate Student | 33 (25.8) | 36 (25.0) |
| Postgraduate Clinical Trainee | 9 (7.0) | 13 (9.0) |
| Clinician | 78 (60.9) | 78 (54.2) |
| Other | 8 (6.3) | 17 (11.8) |
| Not stated | 0 (0.0) | 0 (0.0) |
| Age | ||
| Under 18 | 0 (0.0) | 0 (0.0) |
| 18–24 | 28 (21.9) | 39 (27.1) |
| 25–34 | 59 (46.1) | 64 (44.4) |
| 35–44 | 31 (24.2) | 28 (19.4) |
| 45–54 | 8 (6.3) | 8 (5.6) |
| 55–64 | 2 (1.6) | 4 (2.8) |
| 65+ | 0 (0.0) | 1 (0.7) |
| Sex | ||
| Male | 79 (61.7) | 71 (49.3) |
| Female | 47 (36.7) | 71 (49.3) |
| Not stated | 2 (1.6) | 2 (1.4) |
| Country | ||
| Australia | 48 (37.5) | 59 (41.0) |
| India | 14 (10.9) | 14 (9.7) |
| Malaysia | 5 (3.9) | 6 (4.2) |
| UK | 29 (22.7) | 23 (16.0) |
| USA | 12 (9.4) | 17 (11.8) |
| Other | 19 (14.8) | 24 (16.7) |
| Not stated | 1 (0.8) | 1 (0.7) |
| Tendon management experience | ||
| Provide health care to clients with tendon disorders once a week or more | 61 (47.7) | 62 (43.1) |
| Social Media experience | ||
| Use Twitter | 75 (58.6) | 66 (45.8) |
| Use Facebook | 106 (82.8) | 130 (90.3) |
aPercent of group (Twitter or Facebook) that provided baseline data.
Kirkpatrick levels 1, 2 and 3
Following the intervention, (the short course consisting of practice points) there were statistically significant increases in SMUPI score, self-rated confidence, self-rated knowledge and multiple choice assessment score; but no statistically significant differences between the groups in their changes over time. Participants in both groups reported a change in practice/intended practice and increased use of research in practice/intended practice as a result of the intervention but there was no statistically significant difference between the groups. This is shown in Table 2.
Table 2.
Kirkpatrick level 1–3 outcomes
| Baseline measures Predicted Mean (n) | Post-Intervention measures Predicted Mean (n) | Difference (SED)b | P-value | |
|---|---|---|---|---|
| Kirkpatrick level 1 outcomes | ||||
| SMUPIa | ||||
| 40.34 (126) | 41.85 (86) | 1.51 (0.66) | .024 | |
| 39.53 (143) | 40.86 (127) | 1.33 (0.58) | .022 | |
| Difference (SED)b | −0.81 (0.82) | −0.99 (0.91) | ||
| P-value | .326 | .277 | .841d | |
| Kirkpatrick level 2 outcomes | ||||
| Self-rated confidence in tendon managementc | ||||
| 3.380 (128) | 3.784 (86) | 0.404 (0.083) | <.001 | |
| 3.216 (143) | 3.644 (131) | 0.428 (0.072) | <.001 | |
| Difference (SED)b | −0.164 (0.106) | −0.141 (0.116) | ||
| P-value | .124 | .227 | .830d | |
| Tendon management self-rated knowledgec | ||||
| 3.181 (127) | 3.727 (86) | 0.546 (0.082) | <.001 | |
| 3.027 (143) | 3.570 (131) | 0.543 (0.071) | <.001 | |
| Difference (SED)b | −0.154 (0.102) | −0.157 (0.112) | ||
| P-value | .135 | .163 | .975d | |
| Multiple choice assessment total score (max score 16) | ||||
| 7.649 (123) | 10.308 (80) | 2.659 (0.381) | <.001 | |
| 6.599 (136) | 9.435 (118) | 2.835 (0.331) | <.001 | |
| Difference (SED)b | −1.050 (0.469) | −0.874 (0.521) | ||
| P-value | .026 | .095 | .728d | |
| Assessment score for questions that addressed the practice points (max score 8) | ||||
| 4.155 (123) | 5.523 (80) | 1.368 (0.233) | <.001 | |
| 3.789 (136) | 5.431 (118) | 1.642 (0.203) | <.001 | |
| Difference (SED)b | −0.366 (0.259) | −0.093 (0.293) | ||
| P-value | .159 | .752 | .378d | |
| Assessment score for questions addressing the supplementary information (max score 8) | ||||
| 3.485 (123) | 4.819 (80) | 1.333 (0.211) | <.001 | |
| 2.848 (136) | 4.025 (118) | 1.177 (0.184) | <.001 | |
| Difference (SED)b | −0.637 (0.255) | −0.793 (0.211) | ||
| P-value | .013* | .006* | .578d | |
| Number reporting change (n) | % of group (95% CI) | P (between group differences) | ||
| Kirkpatrick level 3 outcomes | ||||
| Reported change in practice due to intervention | ||||
| 59 (77) | 77 (67-86) | .11 | ||
| 77 (117) | 66 (57-74) | |||
| Reported increased use of research in practice | ||||
| 55 (78) | 71 (60-81) | .89 | ||
| 80 (115) | 70 (61-78) | |||
aTotal of ten items, each measured on a 5 point Likert scale, whereby higher score = more favorable attitude.
bSED = Standard Error of the Difference.
cMeasured on a 5 point Likert scale 1 = very poor, 5 = very good.
dP-value is for the F-test of a two-way interaction.
*Statistically significant difference between groups.
The Twitter page developed 428 “followers” and the Facebook page received 155 “likes.” An estimated 10.0% (8/80) of the Twitter group and 7.8% (9/115) of the Facebook group reported interacting online. The difference between groups was not significant [χ2 (1, n = 195) = 0.28, P = 0.597)]. An estimated 42.6% (20/47) of the Twitter group and 34.8% (24/69) of the Facebook group reported lack of time as a reason for lack of interaction on the social media sites.
Statistically significant differences were found between groups for number of times information was shared (mean shares per post Twitter 10.40, Facebook 0.20, SED 3.030, P < .001) and approved of (“liked”/”favourite”) (mean Twitter 14.00, Facebook 8.00, SED 1.414, P = .005).
DISCUSSION
This study has demonstrated that research information delivered by either Twitter or Facebook can improve clinician knowledge and promote behavior change. No statistical differences in these outcomes were observed between the Facebook and Twitter groups. This research is consistent with previous literature that indicates that web based or social media programs are useful as learning tools,5,7,8,10,11 and can improve clinician knowledge and promote behavior change.10
This study has also found that the provision of extra information, beyond a 140 character message, did not impact on knowledge or behavior change. Short messages may be beneficial to busy healthcare workers as lack of time is often cited as a barrier to evidence based practice.1 However, trustworthiness of information gathered via social media is a key concern of clinicians.4 Our data indicates that brief messages, when obtained from a reputable source and linked to full sources of information may be acceptable to clinicians.
There were two interesting differences between the groups. There was greater overall attrition from the Twitter group. Site familiarity may be a factor, as more health professionals use Facebook than Twitter.4 In this study, over 80% of clinicians in each group use Facebook; <60% in each group use Twitter. The preference of clinicians to use Facebook over other social media sites for obtaining research information may also be a factor.4 Therefore, the use of Facebook may have encouraged online course completion. The Twitter page developed a far greater following than the Facebook page, and more participants in the Twitter group shared the received information within their own social networks. Twitter is particularly useful in publicizing information, and it appears this also applies to research information.
Social media promotes online social interactions, which may enhance learning22 and promote change through social influence.23 Interaction in this study was encouraged in the course instructions, and a tendon expert was available to answer questions. However, 10% or less of the participants in each group reported interacting online. Over 30% of participants in each group cited lack of time as a key barrier to interacting. Approximately 60% of clinicians are evidence “pragmatists” – those to whom validity of evidence is secondary to the daily demands of practice.2 Therefore, the interaction in this study may reflect everyday professional use of social media for accessing research evidence. Concerns about professional image may also influence online interactions.4 Herein lies the paradox of social media based learning communities; the openness and diversity which can enrich learning may also negatively impact upon the socio–emotional aspects of group formation which may be beneficial for collaborative learning.24
While significant improvements in knowledge occurred, the improvements were small (an increase in total assessment score of <3). A lack of time to read or listen to supplementary information may have influenced this result. The practice points may also have been lost among the large volumes of information that can appear on social media accounts, or may have been filtered out by the social media sites themselves.
There are several limitations to this study. Baseline measures were collected shortly after randomization had occurred, potentially resulting in chance bias. However, participant assessments were anonymous, therefore randomization after completion of baseline measures was not possible. There was no control group to assess the impact of a learning effect from the assessment or to see if the course was equally effective if delivered via email or text message. However, this study aimed to compare social media modalities and the benefits and limitations of each. Participants from the Twitter group had a statistically significant higher baseline assessment score for knowledge related to the supplementary information. There are a number of health professional information sharing sites on Twitter, and participants allocated to Twitter may be more inclined to participate if they had previous exposure to these sites. An error resulted in 5 participants from the Twitter group obtaining the course information for both Twitter and Facebook, however due to the small number of participants affected, this is unlikely to have impacted the results. Both Facebook and Twitter sites were publically available, and participants were not asked to keep group allocation or information confidential, meaning the groups may not have been mutually exclusive. However, participants were not informed of the alternate group, and the diversity of participants limits the potential impact of this confounding factor. The sites were open access; therefore people other than study participants may have interacted on the sites. The same assessment was used before and after the intervention however, question and answer order were randomized to limit any potential learning effects. The high attrition rates may have resulted in attrition bias,25 however, given that online courses often have dropout rates of ≥50%26,27 the attrition level is not abnormal for this type of education.
CONCLUSION
Evidence based “practice points” on tendinopathy management can increase clinician knowledge and influence changes in practice, whether delivered by Facebook or Twitter. No differences in these outcomes were observed between the Twitter and Facebook groups. Messages of 140 characters or less are as effective as longer posts in conveying research information.
Future research directions may include investigating social media interaction and the subsequent impact on learning and behavior change, and how perceived e-professionalism influences clinicians’ willingness to participate in social media based professional education. Social media may provide a low cost method of widespread distribution of information and an economic analysis of using social media to distribute research information would complement existing literature.
ACKNOWLEDGMENTS
The authors of this article have no formal relationship with either Twitter or Facebook.
Contributors
The authors listed contributed to the research design, data collection, analysis, write-up, and critical review of the final manuscript.
Funding
This work was supported by the Monash University Strategic Project Grant Scheme 2014, Grant Number: SPG-L 007
Competing interests
None.
REFERENCES
- 1. Glasziou P, Haynes B. The paths from research to improved health outcomes. Evid Based Nurs 2005;8(2):36–38. [DOI] [PubMed] [Google Scholar]
- 2. Scott A, Docking S, Vicenzino B, et al. Sports and exercise-related tendinopathies: a review of selected topical issues by participants of the second International Scientific Tendinopathy Symposium (ISTS) Vancouver 2012. Br J Sports Med 2013;47(9):536–544. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Thomas A, Menon A, Boruff J, et al. Applications of social constructivist learning theories in knowledge translation for healthcare professionals: a scoping review. Implement Sci 2014;9:54. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Tunnecliff J, Ilic D, Morgan P, et al. The acceptability among health researchers and clinicians of social media to translate research evidence to clinical practice: mixed-methods survey and interview study. J Med Internet Res 2015;17(5):e119. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Cheston CC, Flickinger TE, Chisolm MS. Social media use in medical education: a systematic review. Acad Med 2013;88(6):893–901. [DOI] [PubMed] [Google Scholar]
- 6. Cartledge P, Miller M, Phillips B. The use of social-networking sites in medical education. Med Teach 2013;35(10):847–857. [DOI] [PubMed] [Google Scholar]
- 7. Raupach T Muenscher C Anders S et al. . Web-based collaborative training of clincial reasoning: a randomized trail. Med Teach 2009;31:431–437. [DOI] [PubMed] [Google Scholar]
- 8. Carvas M, Imamura M, Hsing W. et al. An innovative method of global clinical reserach training using collaborative learning with Web 2.0 tools. Med Teach 2010;32(3):270. [DOI] [PubMed] [Google Scholar]
- 9. Junco R, Heiberger G, Loken E. The effect of Twitter on college student engagement and grades. J Comput Assisted Learn 2011;27:119–132. [Google Scholar]
- 10. Maloney S, Tunnecliff J, Clearihan L, et al. Translating evidence to practice via social media: a mixed methods study. J Med Internet Res 2015;17(10):e242. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. McGowan BS, Wasko M, Vartabedian BS. et al. Understanding the factors that influence the adoption and meaningful use of social media. J Med Internet Res 2012;14:117. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Moorhead SA, Hazlett DE, Harrison L, et al. A new dimension of health care: systematic review of the uses, benefits, and limitations of social media for health communication. J Med Internet Res 2013;15(4):e85. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. Statistica. Facebook: Monthly Active Users [Webpage] 2015. http://www.statista.com/statistics/264810/number-of-monthly-active-facebook-users-worldwide/. Accessed December 16, 2015. [Google Scholar]
- 14. Twitter Company About [Webpage] 2015. https://about.twitter.com/company. Accessed December 16, 2015. [Google Scholar]
- 15. Australian Health Practioner Registration Agency [Webpage] 2014. https://http://www.ahpra.gov.au/. Accessed December 2, 2014. [Google Scholar]
- 16. Monash University. CETSM@CETSM Twitter [Webpage] 2014. https://twitter.com/cetsm. Accessed January 16, 2015. [Google Scholar]
- 17. Monash University. Clinical Excellence Through Social Media Facebook [Webpage] 2014. https://www.facebook.com/Clinical-Excellence-Through-Social-Media-328092517357004/. Accessed September 16, 2014. [Google Scholar]
- 18. Hutchinson L. Evaluating and researching the effectiveness of educational interventions. BMJ 1999;318(7193):1267–1269. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19. Wang AT, Sandhu NP, Wittich CM. et al. Using social media to improve continuing medical education: a survey of course participants. Mayo Clinic Proc 2012;87(12):1162–1170. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20. VSN International GenStat Reference Manual (Release 17), Part 2 Directives Hemel Hempstead, UK: VSN International; 2014. [Google Scholar]
- 21. Lumley T, Diehr P, Emerson S, et al. The importance of the normality assumption in large public health data sets. Annu Rev Public Health 2002;23:151–169. [DOI] [PubMed] [Google Scholar]
- 22. Beaudoin MF. Learning or Lurking? Tracking the ‘Invisible' Online Student. Internet Higher Educ 2002;5(2):147–155. [Google Scholar]
- 23. Moulding N, Silagy CA, Weller DP. A framework for effective management of change in clinical practice: dissemination and implementation of clinical practice guidelines. Qual Health Care 1999;8(3):177–183. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24. Karel K, Kirschnerb PA, Jochemsb W. Identifying the pitfalls for social interaction in computer-supported collaborative learning environments: a review of the research. Comput Human Behav 2003;19:335–353. [Google Scholar]
- 25. Dumville JC, Torgerson DJ, Hewitt CE. Reporting attrition in randomised controlled trials. BMJ 2006;332(7547):969–971. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26. Nistor N, Neubauer K. From participation to dropout: quantitative participation patterns in online university courses. Comput Educ 2010;55(2):663–672. [Google Scholar]
- 27. Hoy M., MOOCs 101: an introduction to massive open online courses. Med Ref Serv Quart 2014;33(1):85–91. [DOI] [PubMed] [Google Scholar]