Using developmental research to design innovative knowledge translation technology for spinal cord injury in primary care: Actionable Nuggets™ on SkillScribe™

Abstract

Context/Objective

Actionable Nuggets™ for spinal cord injury (SCI) are a knowledge translation tool facilitating evidence-based primary care practice, originally developed in 2010 and refined in 2013. Evaluation results from these two phases of development have informed the design of SkillScribe™, an innovative electronic platform intended to offer reflective continuing medical education (CME) programming through mobile devices in order to support the key features of the Actionable Nuggets™ approach. This brief article describes the ongoing development of Actionable Nuggets™ for SCI on SkillScribe™ by: (1) summarizing the work to date on Actionable Nuggets™; (2) describing evaluation results of Actionable Nuggets™; (3) placing SkillScribe™ in the context of adult education.

Design

Developmental Research Design.

Setting

Canadian primary care.

Participants

Primary care physicians; specialist physicians.

Interventions

Twenty educational modules on SCI.

Outcome measures

Pre- and post-test knowledge survey, feedback and use statistics, impact assessment survey, qualitative analysis of evaluation data.

Results

In both hard copy and electronic form, physicians report that Actionable Nuggets™ are an acceptable and useful approach to providing CME for low-prevalence, high-impact conditions like SCI. The key elements of this tool are that they: offer evidence-based information in small, focused “nuggets”; position information where physicians most frequently seek it; offer information in a format that permits direct translation into action in primary care; allow time for reflection; attach practice tools; and offer CME credit.

Conclusion

Actionable Nuggets™ for SCI, delivered using a convenient and portable electronic medium, with time-released content and interactive testing has the potential to improve the primary care of patients with SCI.

Introduction

Continuing medical education (CME) for primary care physicians often focuses on issues that are common in primary care practice, for the purpose of disseminating current best practices into patient care.¹ While a generalist approach to the provision of CME has the potential to enhance patient care for the majority of patients in family medicine, the literature shows that physicians express anxiety about the provision of care for patients presenting with low-prevalence conditions associated with a high risk of morbidity.^2,3 Spinal cord injury (SCI) is one condition about which family physicians have expressed knowledge gaps.⁴ At a rough prevalence of 1.2/1000, the average Canadian family medicine caseload only has one or two patients at a time with SCI.^5–10 These patients are represented in the 5–6% of the typical family medicine caseload that consume approximately one-third of the practice's resources, due to multiple, complex chronic conditions.^11,12

The Canadian primary care environment is a complex and busy environment that is associated with many barriers to access to healthcare for people with disabilities.¹³ Knowledge translation is especially difficult to facilitate in the primary care environment, complicated by contextual factors including motivation, lack of time, and accessibility. Primary care physicians are unlikely to attend a live CME event to facilitate the care of only a few patients in their practice. Research demonstrates that primary care physicians are resistant to conventional approaches to CME supporting knowledge translation (such as research reports, web-based modules, or seminars), although college guidelines that insist on maintaining a level of CME for professional registration are associated with a small increase in knowledge transfer.^14–16 While practitioner engagement in the knowledge translation process is associated with greater compliance with best practices and more effective knowledge management,^4,17 CME providers report that it is challenging to provide learning opportunities that support physicians' learning needs about low-prevalence, high-impact conditions like SCI.^13–15 Innovative methods of CME that specifically consider both the CME preferences and the demands of the primary care environment are required in order to disseminate knowledge about best practices for complex chronic conditions like SCI.

By their own admission, many primary care physicians feel ill-equipped to address the healthcare needs of patients with SCI and the resultant disabilities.^18–21 Actionable Nuggets™ for SCI is a knowledge translation tool specifically designed to address the challenge of disseminating research findings on SCI to practicing family physicians in a manner that attracts their attention, and is readily translatable to their everyday practice. Originally launched in 2010, the prototype Actionable Nuggets™ for SCI consist of a series of 20 visually attractive postcards that are time-released through the conventional mail system.²² Participating physicians receive one per week in order to gain knowledge on the 20 most significant health problems and secondary complications that bring patients with SCI into the primary care environment. Fig. 1 depicts a sample Epidemiology Actionable Nugget™.

Figure 1 .

Example of an Actionable Nugget™ with key features indicated.

Actionable Nuggets™ provide evidence-based information in small, focused “nuggets” that permit direct translation into action in the primary care setting. Disseminated to three small samples of family physicians in Ontario, Newfoundland, and Australia, evaluation of the prototype was met with positive results²²; however, broad distribution of this innovative knowledge translation tool has been impeded by the cost associated with production and delivery through traditional mailing systems. Moreover, comments from participants offered a number of suggestions for revision and improvement of the tool.

Building on this feedback, the Actionable Nuggets™ project has evolved to embody a developmental research perspective²³ with the overarching goal of creating, refining, and launching an electronic version of the Actionable Nuggets™ approach that is grounded in the principles of adult education and can be accessed on mobile technology. This article will: (1) summarize the evolution of the Actionable Nuggets™ project to date; (2) present our analysis of participant feedback that has identified the key elements of a successful knowledge translation tool for primary care; and (3) place our emerging mobile platform SkillScribe™ in the context of adult education with a focus on knowledge translation for primary care.

Design

Actionable Nuggets™ project overview

This project has implemented a three-phase Type 1 Developmental Research approach in order to create context-specific knowledge pertaining to the learning needs of primary care physicians providing care to patients with SCI.^23,24 This methodology is designed to direct a thoughtful and systematic process of designing, developing, and evaluating innovative educational products and programs using instructional technology. In addition, it is seen as an effective manner of establishing new tools and techniques for learning based on methodical analysis of specific cases, and creates context-specific knowledge that can be used to inform subsequent program development. Developmental studies are often structured in phases that feature context analysis, study design, prototype development and evaluation, and ongoing prototype revision and re-evaluation.^23,24

This project has undergone three distinct phases:

Phase 1: Development and evaluation of the prototype tool, Actionable Nuggets™ for SCI^4,22 in order to answer the question “What do family physicians need to know to provide excellent, evidence-based care to their patients with spinal cord injuries?”. While the results of the scoping review have been described in detail in previous manuscripts, the application of these data in a knowledge translation tool have not been widely described and will be summarized in this article.

Phase 2: Widespread electronic distribution of Actionable Nuggets™, facilitated by the Canadian Medical Association (CMA). Content was revised and updated, however, the limited nature of the electronic online presentation did not offer participants access to some of the key elements associated with the Actionable Nuggets™ approach, such as timed release and portable access. This article will focus on the results of this phase of development, discussing how it has informed the ongoing development of Phase 3.

Phase 3 represents the ongoing design of an electronic application SkillScribe™, a platform that has been built to support all of the key elements of the Actionable Nuggets™ approach to facilitating knowledge translation in primary care. For the purpose of this article, the ongoing implementation of Phase 3 will be featured in the Discussion section as the future direction and application of previous phases.

Ethics approval for this study has been granted by the Queen's University Research Ethics Review Board.

Methods

It is common for Type 1 Developmental Research projects to also utilize multiple research methodologies, with a range of designs being used to inform the various phases of the project.

The York scoping review methodology was used to gather and organize the foundational literature (data) required to build the initial tool.^4,22 Scoping reviews are useful to identify sources and types of research evidence available, in order to provide an informational platform upon which to build more refined research endeavors.²⁵ They consists of six distinct steps: (i) identify the research question; (ii) identify relevant literature; (iii) select relevant literature; (iv) chart data; (v) collate, summarize, and report results; and (vi) consult with key stakeholders. The results of this review were analyzed by an expert panel consisting of primary care clinicians and researchers, knowledge translation expert researchers, and rehabilitation specialists. This process resulted in the development of the Actionable Nuggets™ tool, and was supported by funds obtained through the Rick Hanson Institute.

Following initial distribution of the prototype approach, updated content was delivered to physicians in a simple electronic format through the continuing education portal on the CMA website in both English and French. Descriptive analysis of website visits and qualitative analysis of practitioners' reflections were completed using the qualitative analysis software NVivo in order to organize feedback into themes and identify trends in the data.

Qualitative analysis of evaluative feedback was organized according to principles of adult education²⁶ and curriculum development for medical education²⁷ in order to identify the key educational principles embedded in the Actionable Nuggets™ tool. These elements were incorporated into the design of an electronic platform by the research team, with educational and technology expertise provided by the Queen's University Office of Continuing Professional Development and the Educational Technology Unit.

Results

Phase 1: prototype Actionable Nuggets™

The original scoping review resulted in 21 articles that offered evidence regarding primary care for patients with SCI.²² In consultation with the team of investigators representing both researchers and knowledge users, analysis of these data revealed 20 key topics that were most frequently seen in primary care for patients with SCI, forming the basis for the Actionable Nuggets™ (Table 1).

Table 1 .

Most prevalent issues for SCI patients in primary care

20 Key topics: Actionable Nuggets™ for SCI	Phase 1 results	Phase 2 results
Epidemiology of SCI	10	7
Accessibility of primary care office	1	0
Management of neurogenic bladder	173	6
Diagnosis of urinary tract infection (UTI)
Pharmacological management of UTI
Screening for bladder cancer
Assessment of neurogenic bowel	57	9
Management of neurogenic bowel
Screening for colorectal cancer
Periodic re-evaluation of bowel program
Assessment of pain	180	31
Management of neuropathic pain
Management of upper extremity pain
Autonomic dysreflexia	22	0
Screening for depression	26	9
Prevention of pressure ulcers	56	6
Treatment of pressure ulcers
Assessment of cardiovascular disease risk	35	7
Benefits of physical activity	66	5
Sexuality	35	6
Total	661	86

The prototype Actionable Nuggets™ for SCI were disseminated weekly over a period of 20 weeks to participating family physicians (N = 49). Participants were awarded CME credits for participation. All physicians received hardcopy postcards through regular mail, and most physicians also received electronic notification of the Nuggets, with the exception of two participants who did not wish to do so. Follow-up with participants at the conclusion of the mail-out showed that: self-evaluations of knowledge of SCI health issues and complications improved from fair to very good; attitudes of family physicians remain positive with regard to accommodating patients with SCI; practices changed to include more comprehensive assessment, increased awareness of accessibility, improved referral patterns, better attention to prevention and health promotion for SCI patients.²² Furthermore, 53% of participants reported that they had made changes to their practice as a result of the content of the tool; 75% of participants reported that they had applied information from the tool to their practice; and 33% of participants indicated that they had made a referral to another provider as a direct result of information provided in the tool.

Participants rated the SCI Nuggets as excellent on a variety of important dimensions, such as content, professionalism, relevance, and scientific rigor. However, on average participants declared that they had only read 16 of the 20 cards received in the mail, and it appeared that commitment to reading the cards was an all-or-nothing thing, as 65% of participants reviewed all 20 cards, 12% reviewed at least 15 cards, and the remaining 18% reviewed <5 cards. Participants offered a number of suggestions for revision and improvement of the tool, including: an electronic version that could be used on mobile devices; email prompts; correction of minor errors; removal of references pertaining to specific technical and pharmaceutical products; ensuring copyright dates in order to validate currency of information; and continuing to offer CME credits to participants subscribing to the tool.

Phase 2: tool refinement and dissemination

In March 2013 to March 2014, the CMA partnered with the Actionable Nuggets™ creators in order to facilitate widespread distribution of an electronic version of the tool through their online continuing education portal. Full scoping reviews were performed in all 20 topic areas in order to ensure the currency of the dataset, and resulted in the identification of an additional 86 new articles being added to the dataset (Table 1). Participant recommendations for revision, including the correction of minor errors, inclusion of copyright rates, and removal of product-specific reference were applied, and PDF versions of the updated tools were uploaded in order to instantly offer physicians access to the material. In addition, the presentation order of the Nuggets was changed, and an impact assessment and message forum were attached to each Nugget, representing a complete “module”. Physicians were offered CME credits in accordance with the number of Nugget modules completed (0.25 MainPro M1 credits per Nugget, for a total of 5 credits for the complete set); however, the majority of users (77%) who viewed Nuggets did not request CME credit.

Over the 1-year period, the Actionable Nuggets™ landing page on the CMA website was viewed a total of 7822 times, and 1038 users viewed at least one Actionable Nugget™. A total of 2145 Actionable Nugget™ modules (including the impact assessment questionnaires attached to each nugget) were completed by physicians, representing 241 users who requested CME credit. Eighty-nine percent (89%) of participants were family physicians, and the remaining 11% who completed the modules identified as specialist physicians.

Trends in individual completion of Nugget module topics are detailed in Table 2. Participants indicated that the content in 43% of the individual Nuggets was relevant to at least one patient in their practice, and 46% of the Nuggets were partially relevant to at least one patient in their practice. Only 10% of the content was rated as not relevant to individual practices. Thirty-eight percent of participants were extremely satisfied with the content in individual Nuggets, and a further 48% were “moderately” satisfied. Only 1% of participants were dissatisfied with the content presented in individual Nuggets. Table 3 details the self-rated impact of information on participants' practices.

Table 2 .

Trends in module completion in Phase 2

Nugget title and content	Number completed
Epidemiology of SCI	57
Screening for cardiovascular disease risk in SCI	32
Autonomic dysreflexia	31
Pharmacological management of neuropathic pain in SCI	26
Management of musculoskeletal pain	23
Diet and fluid management in neurogenic bowel	20
Screening for colorectal cancer in SCI patients	20
Assessment of pain in SCI patients	19
Management of cardiovascular risk in patients with SCI	18
Depression and SCI	17
Periodic re-evaluation of bowel management program	17
Annual assessment of neurogenic bowel	16
Monitoring of neurogenic bladder	15
Wheelchair accessibility of your practice	15
Prevention of skin breakdown	14
Recognizing urinary tract infections in patients with SCI	14
Sexuality in SCI	12
Treatment of skin breakdown	12
Screening for bladder cancer in SCI patients	11
Pharmacological management of UTI in SCI	8
Total	397

Table 3 .

Impact of information from Phase 2 on potential practice behaviors

Impact statement	Participants in agreement
My practice is (will be) changed and improved	15%
I learned something new	57%
I am motivated to learn more	36%
This information confirmed I did (am doing) the right thing	25%
I am reassured	19%
I am reminded of something I already knew	19%
I am dissatisfied	0.5%
There is a problem with the presentation of this information	0.2%
I disagree with the content of this information	0.2%
This information is potentially harmful	0%

Discussion

Actionable Nuggets™ is a knowledge translation tool designed to provide physicians with evidence-based best practices on low-prevalence, high-impact disorders seen in primary care practice. Between March 2013 and March 2014, electronic copies of Actionable Nuggets™ were made nationally available to members of the CMA in electronic format. The electronic version of this tool has been well-received by Canadian physicians from a range of disciplines including primary care; however, many aspects of this method of instant electronic distribution deviate from the original tool and the strengths that were originally associated with successful changes to practice following the educational intervention. Qualitative analysis of participant feedback through an adult education lens²⁶ has identified that the key to Actionable Nuggets™ is that they:

• • provide evidence-based information in small, focused “nuggets”;
• • position information at portals where physicians most frequently seek it;
• • offer information in a format that permits direct translation into action in the primary care setting;
• • allow time for reflection and application through timed release;
• • attach evidence-based tools for practice in an easily accessible format;
• • offer opportunities for CME credit.

In order to offer primary care physicians the effective portable electronic CME that they require to meet the primary care needs of patients with high-impact, low-prevalence disorders, an innovative platform is required. In response, our research team has engaged in the design of SkillScribe™, an electronic platform that has allowed us to offer physicians the essential elements of the Actionable Nuggets™ interactive approach through a mobile “app” that is conveniently available for use on personal electronic devices including computers, smartphones, and tablets. SkillScribe™ is an electronic platform that has the potential to replicate the strengths of Actionable Nuggets™ using a convenient electronic medium, time-released content, and interactive testing. Designed in response to administrative, educator, and learner needs, it is a user-friendly application that can offer specialized educational programming to a range of professionals and learners.

The design and development of both Actionable Nuggets™ and the SkillScribe™ mobile platform have been informed by the principles of adult learning.²⁷ Adult learners “ … must see a reason for learning something. Learning has to be applicable to their work or other responsibilities to be of value to them”.²⁸ Actionable Nuggets™ are designed to be easily consumable information applicable to primary care practitioners immediately, and the feedback that participants in both complete phases of this developmental research project have indicated that they are motivated to engage in the self-directed learning opportunities facilitated by this approach. This is especially apparent when considering that the overwhelming majority of Nuggets viewed by participants in Phase 2 were not submitted for CME credit.

The ability for the learner to access the information and complete each module on their own timeframe demonstrates respect to adult learner's life outside of the program and to engage in self-directed learning.²⁶ Adults “ … enter into the learning process with a goal in mind and generally take a leadership role in their learning”.²⁸ The ability to direct their own learning will help to engage adult learners and help them meet their self-directed learning objectives while engaging in reflective practice.

Future direction: Actionable Nuggets™ on SkillScribe™

When designing and refining SkillScribe™, our team of researchers and developers have been guided by Kern's six-step model of curriculum development for medical education²⁷ in order to envision an ideal platform for CME programming development and dissemination using the Actionable Nuggets™ content.

Step 1 (Problem identification): The content developers engage in reflection and conversation with one another as they design and refine the Actionable Nuggets™ content. Consultation and collaboration between educators are facilitated by the “virtual meeting place”, or the web interface of SkillScribe™.

Step 2 (Targeted needs assessment): Pre- and post-test questionnaires are embedded in programming in order to identify unperceived needs. In addition to operating as a focus group in its discussion and creation of the individual Nuggets, content experts and researchers moderate discussion boards in order to gauge the learning needs of physicians and inform the development of new topics.

Step 3 (Goals and objectives): Presenting each Nugget as key topics enables one concrete goal to be addressed at a time. SkillScribe™ also has the capacity for multiple choice questions to be embedded in each segment so that these can be measured. Otherwise, just clicking the button that reads “mark as complete” encourages some reflection on the content. Our team has noticed that when demonstrating the app to learners, they often tell us “I'll just mark this as complete for now, but will go back and review it again later”.

Step 4 (Educational strategies): SkillScribe™ is used to complement or supplement traditional approaches to CME, and is specifically pertinent to providing CME on conditions that represent a low percentage of patients but are associated with a high potential for morbidity. Thought goes into the order of Nugget topics, as the learner is required to engage in them consecutively.

Step 5 (Implementation): As one participant eloquently told us: “Don't tell me about something if you're not going to give it to me”, content developers have the option to embed links to external resources, add discussion boards, send reminders to facilitate reflection, and embed tools for use in practice. Moderators can monitor participation offsite, and the platform is easy to use and changes can easily be implemented.

Step 6 (Evaluation and feedback): As previously discussed, discussion boards and ease of implementing refinements help enable ongoing formative evaluation, a key element of developmental research designs.^23,24 Moreover, accreditation requirements for CME credit require that a quick evaluation be filled out following the intervention.

Conclusion

The SkillScribe™ platform is an innovative learning platform that meets the needs of today's learners and educators without compromising on content or delivery. By presenting highly researched data in small time-released “nuggets”, practitioners are able to absorb the information in a more concrete way. Using a mobile platform will allow users to have access to this information any time they need it, and to complete each learning module in a timeframe and location that work best for them. This recently launched, innovative electronic platform for self-directed learning represents a new approach to designing and offering CME. In summer 2014, Actionable Nuggets™ for SCI will be piloted through the SkillScribe™ electronic platform in order to provide effective mobile CME for specialized populations in primary care.

Disclaimer statements

Contributors Matt Simpson, Manager Education Services, Faculty of Health Sciences, Queen's University.

Funding None.

Conflicts of interest Authors DNN and LMA are also employees of the Office of Continuing Professional Development, Queen's University.

Ethics approval Ethics approval was obtained through the Queen's University Research Ethics Board