Abstract
Background
People with low back pain (LBP) in the community often do not receive evidence-based advice and management. Community pharmacists can play an important role in supporting people with LBP as pharmacists are easily accessible to provide first-line care. However, previous research suggests that pharmacists may not consistently deliver advice that is concordant with guideline recommendations and may demonstrate difficulty determining which patients require prompt medical review. A clinical decision support system (CDSS) may enhance first-line care of LBP, but none exists to support the community pharmacist–client consultation.
Objective
This study aimed to develop a CDSS to guide first-line care of LBP in the community pharmacy setting and to evaluate the pharmacist-reported usability and acceptance of the prototype system.
Methods
A cross-platform Web app for the Apple iPad was developed in conjunction with academic and clinical experts using an iterative user-centered design process during interface design, clinical reasoning, program development, and evaluation. The CDSS was evaluated via one-to-one user-testing with 5 community pharmacists (5 case vignettes each). Data were collected via video recording, screen capture, survey instrument (system usability scale), and direct observation.
Results
Pharmacists’ agreement with CDSS-generated self-care recommendations was 90% (18/20), with medicines recommendations was 100% (25/25), and with referral advice was 88% (22/25; total 70 recommendations). Pharmacists expressed uncertainty when screening for serious pathology in 40% (10/25) of cases. Pharmacists requested more direction from the CDSS in relation to automated prompts for user input and page navigation. Overall system usability was rated as excellent (mean score 92/100, SD 6.5; 90th percentile compared with similar systems), with acceptance rated as good to excellent.
Conclusions
A novel CDSS (high-fidelity prototype) to enhance pharmacist care of LBP was developed, underpinned by clinical practice guidelines and informed by a multidisciplinary team of experts. User-testing revealed a high level of usability and acceptance of the prototype system, with suggestions to improve interface prompts and information delivery. The small study sample limits the generalizability of the findings but offers important insights to inform the next stage of system development.
Keywords: low back pain; community pharmacy; decision support systems, clinical
Introduction
Background
Low back pain (LBP) is a major cause of disability worldwide [1], with almost 1 in 5 people reporting LBP at any one time [2]. People with LBP typically consult general practice, allied health, or community pharmacy for advice and management [3,4]. The role of community pharmacists has evolved from dispensing medication and providing medication advice, to include screening and management for a range of health conditions such as minor ailments and chronic health conditions [5-10]. In alignment with this expanding service model, there is interest for community pharmacy to play a greater role in the early management of back pain [11-13]. There are also potential economic benefits for using community pharmacy as an access point for a range of services, with lower patient and health system costs compared with other primary care models [13-15].
Evidence-Practice Gaps in Management of Low Back Pain
Current clinical practice guidelines for the management of LBP recommend first-line care that includes reassurance, advice to stay active and avoid bed rest, and discouraging diagnostic imaging such as plain radiographs unless serious pathology is suspected [3,16]. Despite these guideline recommendations, a substantial gap between evidence and practice still exists [17]. For example, Abdel Shaheed et al [18] reported that community pharmacists and their staff were able to deliver adequate advice on medication use for LBP, but their ability to provide advice on nonpharmacological management such as staying active, avoiding bed rest, and discouraging imaging was inconsistently delivered. The ability to identify presentations that required prompt medical review was also limited for some community pharmacists.
Support for the Community Pharmacist
Clinical decision support systems (CDSSs) are targeted electronic systems that link evidence-based recommendations with the clinical presentation of the individual to improve clinical decision making and support patient engagement with health decisions [19-24]. Recently, CDSSs have been implemented for management of noncancer pain in the primary care setting [25-29], but these do not transfer to the pharmacy setting because of differences in professional training and consultation environment. Pharmacists already have access to CDSSs (eg, management of infection and deprescribing) [30,31], but none exist to support the community pharmacist–client consultation for LBP. Therefore, a CDSS for the early management of LBP in community pharmacy is warranted [32].
The main objective of this study was to develop a CDSS for pharmacists to guide first-line care of LBP in the community pharmacy setting using a mobile data collection system (Apple iPad, Apple Inc). We also sought to evaluate the pharmacist-reported usability and acceptance of the high-fidelity prototype to inform the next stage of CDSS development.
Methods
Overview
This study describes the iterative development of a CDSS for the management of pharmacy clients with LBP in community pharmacy. The CDSS was developed by a multidisciplinary team that included two pharmacy academics, a human-computer interaction expert, and four content experts in LBP [33]. Team members were consulted during each stage of development. The CDSS (high-fidelity prototype) was evaluated via a small-scale usability study [34]. The study was approved by University of Sydney Human Research Ethics Committee (2017/027).
User-Centered Design Framework
Development of the CDSS was underpinned by the framework for user-centered design and evaluation of prototypes for clinical information systems (Figure 1) [35]. The framework describes the evolution of a CDSS based around low-cost usability testing methods before future evaluation with real clients in a clinical practice setting. During initial design of the CDSS, input was sought from a range of people involved with community pharmacy, including two pharmacy academic/educators, a community pharmacist, and an industry representative. This approach sought to uncover pharmacist training and procedural constraints that may impact pharmacist decision making for LBP [36], given that a pathway for the contemporary management of LBP specific to the community pharmacy setting does not exist.
Figure 1.
Clinical decision support system development based on prototyping and iterative testing (modified from Kushniruk et al). This study is represented by shading. CDSS: clinical decision support system.
Architectural Design, Analysis, and Refinement
Design goals were informed by Bates et al [37], Khorasani et al [38], and Zikos et al [39] who described features of a decision support system necessary to facilitate integration into clinical practice. The design goals of this CDSS were to (1) support pharmacists to offer simple, clear evidence-based advice to the pharmacy client who presents with LBP; (2) integrate with the pharmacist workflow (eg, consideration of medicines during decision making); (3) maximize time efficiency; and (4) provide a personalized report of recommendations for the pharmacy client.
The CDSS was designed in three components [22]: (1) knowledge base, (2) reasoning engine, and (3) interface (see Multimedia Appendix 1 for further explanation of design process). Briefly, the knowledge base included high level advice for the screening of serious pathology and early management of LBP [3,40-49]. The reasoning engine was coded from the knowledge base then refined using experts in LBP and community pharmacy to consider age, sex, results of screening questions, pain history, and up to three current medicines for LBP (Multimedia Appendix 2). Recommendations for the pharmacist are separated into key messaging for the pharmacy client, suggested medicine use, and referral options. The pharmacist progresses from a landing page (Figure 2), through to history, screening, and advice pages (Multimedia Appendix 3). Data input is via a touch interface (checkboxes, drop-down menus, and free-text input). The pharmacist can accept, modify, or reject advice generated by the CDSS. Finally, a custom letter for the pharmacy client is generated based on the pharmacist’s final recommendation (Multimedia Appendix 4).
Figure 2.
Clinical decision support system landing page showing clinical flow and scope of key messages. Tapping anywhere on this screen moves to the “clinical history” page.
The multidisciplinary team of experts were engaged at each stage of design, development, and internal testing. Decision trees were iteratively modified before coding decision logic and programming of the interface (Ionic Framework), then refined through multiple (>10) test cycles. Once the logic and interface were complete, each of the 408 unique decision combinations were checked for accuracy using the Web interface. Similarly, each of the clinical case vignettes (and CDSS-generated client handouts) were tested by the research team for language and accuracy before the interview phase.
Usability Testing
After the completion of the internal testing, the next phase of usability was based on the recommendations of Yen and Bakken [34], where the community pharmacist interacted with the CDSS through a series of 5 case vignettes (system-user-task).
Participants
In all, 5 practising community pharmacists (with 5-27 years of clinical experience), not involved in the initial development phase, from 5 different community pharmacies in the Sydney metropolitan area were each presented with 5 case vignettes during a one-to-one interview. Inclusion criteria required pharmacists to have experience with computer use within pharmacy (eg, computer-based dispensing systems) and be comfortable with tablet computer use (eg, internet browsing). Previous studies suggest that with 5 participants, up to 80% of usability issues can be identified (including up to 100% of major usability issues) when a system is designed for a specific group of users [50,51].
Interview Procedure and Training
Each pharmacist was presented with the same 5 case vignettes role-played by the lead researcher (AD). Clinical scenarios included presentations of both nonserious and serious causes of LBP in adult and elderly populations. Cases 1 and 2 involved nonserious LBP, case 3 involved suspicion of an osteoporotic compression fracture, case 4 presented with nonserious low back and leg pain below the knee, and case 5 presented with LBP and a recent history of cancer (Multimedia Appendix 5).
The interviews were held in a location convenient to the pharmacist, usually in the designated clinical consultation space within the pharmacy. The pharmacist was required to interact with both the “client” (researcher) and the CDSS on an iPad Air (iOS 12.4, Apple Inc). Before beginning the case vignettes, the pharmacist was trained in the operation of the CDSS via interface “walk through.” Training also included a brief summary of the evidence underpinning the CDSS, explanation of the pharmacist-client interview process, and how to accept or reject the decision support offered by the CDSS.
The interview was conducted using a think-aloud protocol and employed an active intervention approach [52]. That is, the pharmacist was allowed to ask questions of the researcher during interaction with the “client.” Active intervention by the researcher was triggered when the pharmacist was unable to progress through the CDSS, sought clarification when interacting with the interface, or had questions at completion of the case (eg, reflecting upon management decisions generated by the CDSS). All instances of active intervention were logged and evaluated.
Data Collection
Four modes of data collection were used during the interview: (1) think-aloud protocols [53,54] with active intervention approach [52], (2) video/audio recording and screen capture during interaction with the iPad [55,56], (3) direct interview questions at the completion of interaction with the CDSS [26,57], and (4) completion of a survey instrument (system usability scale) [58,59]. The survey instrument was completed by the pharmacist at the end of the interview and without the researcher present (Multimedia Appendix 5).
Evaluation
Evaluation of Usability Testing
Interviews were transcribed verbatim, then independently analyzed by two researchers with assistance by a third (AD, CS and AK) using a directed content analysis methodology, where key concepts from existing usability studies of health information technology methodology were used to inform initial coding categories [34]. Operational definitions for each category were determined based on the specific goals of the CDSS. Any redundant coding categories were collapsed. The analysis of pharmacist sentiment was categorized as “negative,” “neutral,” or “positive” in consultation with the research team [60]. Frequency of responses were tabulated first by category, then by sentiment (NVivo 12.5, QSR International). Interaction with the iPad was time-stamped to calculate duration spent on each page of the iPad, periods of pharmacist hesitation, and page navigation decisions. Responses to survey instruments were described, and a system usability scale was scored [61].
Level of Acceptance of Clinical Reasoning and Decision Support
At the completion of all case vignettes, each pharmacist was shown an overview of the clinical reasoning engine and then asked to reflect on the logic that informed the recommendation for each case. To quantify the level of acceptance for the core set of recommendations generated by the CDSS (self-care advice, medicines advice, and referral advice), the pharmacist’s acceptance (accept/not accept) was logged. Additional advice offered by the pharmacist relating to clinical management was entered in free-text fields on the iPad.
Results
Pharmacist Interview
All pharmacists completed 5 case vignettes on the Apple iPad. Pharmacists were exposed to cases in the same order. The total time taken to role-play all 5 case vignettes (excluding discussion on decision logic or system improvements) ranged from 14 min 35 seconds (Pharmacist #1) to 28 min 4 seconds (Pharmacist #2). Case vignettes that included nonserious LBP required less time (Cases 1 and 2: mean 3 min 40 seconds per case, SD 1 min 8 seconds) than cases that raised suspicion of serious causes of LBP (Cases 3-5: mean 4 min 46 seconds per case, SD 1 min 23 seconds).
Evaluation of Usability Testing
Coding Categories
A total of 162 statements during the 25 interactions between pharmacists and “clients” were logged. Nine coding categories were identified using directed content analysis (Ease-of-use, Consistency, Visibility, Navigation, Workflow, Content, Understandability, Clarity, and Acceptance). For final coding, the categories Navigation and Workflow were merged, and Understandability was defined under Clarity, which resulted in seven final categories. Statements were also coded by sentiment (negative, neutral, or positive). Table 1 describes each category, with statement frequency and representative examples. A total of 71 statements related to the CDSS interface, and 91 statements related to clinical information (content) provided by the CDSS.
Table 1.
Coding categories with statement frequency and representative examples.
| Coding category with subcategory | Sentiment frequency | Representative coded statements with sentiment | |||||||||
|
|
Negativea | Neutralb | Positivec | Totald |
|
||||||
| Interface | |||||||||||
|
|
Ease-of-use: commentary on the simplicity of operation of the CDSSe | 16 | 3 | 7 | 26 |
|
|||||
|
|
Consistency: commentary on the consistency of visual language or interaction model | 2 | 7 | 4 | 13 |
|
|||||
|
|
Visibility: commentary on the visibility of system capabilities and system status and navigational cues within the CDSS | 7 | 6 | 3 | 16 |
|
|||||
|
|
Navigation/workflow: observation and commentary on progression/sequence through the CDSS | 2 | 12 | 2 | 16 |
|
|||||
| Clinical information | |||||||||||
|
|
Content: commentary on what information is/is not provided by the CDSS | 6 | 9 | 12 | 27 |
|
|||||
|
|
Clarity: commentary on the clarity of the information provided by the CDSS | 1 | 9 | 5 | 15 |
|
|||||
|
|
Acceptance: commentary on the clinical value of the CDSS recommendations | 6 | 9 | 34 | 49 |
|
|||||
aNegative: negative sentiment.
bNeutral: neutral sentiment.
cPositive: positive sentiment.
dTotal: total sentiment count for subcategory.
eCDSS: clinical decision support system.
Pharmacists’ Statements Related to Interaction With the Interface
The categories Ease-of-use and Visibility together accounted for 59% (42/71) of statements about the interface with a positive to negative comment ratio of 0.4 (7:16) and 0.4 (3:7), respectively. The majority of statements with negative sentiment involved interaction with the screening page (10/27 statements, Figure 3). The remainder scored with negative sentiment included comments on layout (eg, button position inconsistent) or visibility issues (eg, text size too small).
Figure 3.
Clinical decision support system screening page for raising suspicion of a serious cause of low back pain. The “No” response is the default state.
For example, statements with negative sentiment reported during interaction with the screening page included:
I find this part a bit long. I’m always reading through it (risk of spinal inflammation) … maybe it’s just me. Maybe I should just read it properly.
Pharmacist #2
[reads from iPad] Leg pain with altered sensation or weakness. So, I guess I didn’t see that part... is there a reason that that’s here (points to the 2nd column) – Oh, because it’s not a clinical history, yep.
Pharmacist #3
It’s just like you are trying to focus on the patient, so you are trying to do two things at once. If the patient was happy for me to pause, ‘cause you feel a bit awkward, just processing this whilst the patient is in front of you.
Pharmacist #4
I think that with this (SCREENING PAGE) is probably the hardest screen here because, like some of the questions I knew, like all of the cases so far, I know that it’s not an infection because they say I fell and now I’ve got pain, so it seems like I probably of shouldn’t have asked the questions.
Pharmacist #5
Statements with positive sentiment for the interface referenced the simplicity of layout, navigation, and language used (5/16 statements). In addition, statements with positive sentiment were made regarding integration with the pharmacist’s workflow (5/16 statements). Pharmacists’ statements relating to the operation of the CDSS (eg, “so I just press here,” and “then it comes out of the printer?”) comprised the majority scored with neutral sentiment (17/28 statements). Queries related to the operation of the CDSS decreased in frequency as each interview progressed.
Pharmacists’ Statements Related to Clinical Information
The categories Content and Acceptance together accounted for 84% (76/91) of statements related to clinical information provided by the CDSS, with a positive to negative comment ratio of 2.0 (12:6) and 5.7 (34:6), respectively. The remainder of statements related to Clarity of the clinical information provided by the CDSS. Statements with negative sentiment for Content (6/27 statements) included request for items absent from history (eg, pharmacists wanted to record current level of pain, whether pregnant or breastfeeding, and history of ulcer). Statements with negative sentiment for Acceptance (6/49 statements) included disagreement with, or questioning of, CDSS-generated advice in the categories self-care, medicines, and referral advice. For example:
Yes, but we always need to do further investigations to find out… and she needs to see someone – like a specialist in this area to find out what is the reason – it is good to have an X-Ray.
Pharmacist #4; case 1: nonserious cause of LBP
In my practice, a typical customer that you have just described will usually be on some kind of blood pressure medication – usually – which is why we always tend to recommend paracetamol first.
Pharmacist #5; case 2: recommendation to begin NSAID therapy
But, from the other point of view, would that narrow the amount of medicine that we recommend? So, from the business-Pharm point of view, would that exclude a lot of products?
Pharmacist #1; general comment at end of interview
Statements with positive sentiment for Content (12/27 statements) included what pharmacists considered to be the right information displayed at the right time. For example:
This is what's really interesting, is what really gives this one the meaning – I like the logic behind it.
Pharmacist #1; reacting to decision for suspicion of fracture
Yes, we do need to know this.
Pharmacist #4; points to increased risk of cancer
OK, so it actually knows it’s sub-therapeutic when I put sub-therapeutic input. That’s very good. That’s very good.
Pharmacist #5
Statements with positive sentiment for Acceptance (34/49 statements) included commentary on the clinical value of information generated by the CDSS in areas of self-care, medicines advice, and referral advice. These statements broadly reflected agreement with advice generated by the CDSS. For example:
Most of the time it was very logical, it was very rational and logical… It leads us to the right decision.
Pharmacist #1
Definitely! Its prompting you to ask questions. I must admit some of those questions we probably don’t always ask, but we need to be asking...
Pharmacist #2
OK, so just add on (stay active) Yep. OK. Instead of just seeing the GP straight away. OK. Cool!
Pharmacist #3; during the selection of self-care advice
We excluded some diseases which is good. I found out that she is not taking enough medicine.
Pharmacist #4
Yes, so I would say: using a heat wrap will also help, and I would say take the Voltaren 2-3 times per day with food – yes, it says this already!
Pharmacist #5
Similar to pharmacists’ questions relating to the interface, comments/questions related to clarification of CDSS-generated advice were scored with neutral sentiment. This type of question also decreased in frequency as each pharmacist moved through the 5 cases.
System Usability Scale
The system usability scale [59,61] was administered to each participant at the completion of the interview without the researcher present. Individual usability scores ranged from 82.5 out of 100 to 100 out of 100, which were interpreted as good to excellent usability, respectively [59]. The overall usability score was rated as excellent (mean score 92 out of 100, SD 6.5; 90th percentile compared with similar systems).
Level of Acceptance of Clinical Reasoning and Decision Support
Across the 5 case vignettes, 70 recommendations were generated by the CDSS related to self-care advice, medicine advice, and referral advice. Pharmacists accepted 90% (18/20) of self-care recommendations, 100% (25/25) of medicines recommendations, and 88% (22/25) of referral recommendations. Of those accepted, pharmacists added to the advice for 8% (5/65) of the recommendations generated by the CDSS (eg, Figure 4).
Figure 4.
Advice page showing the pharmacist #1 acceptance of self-care advice, the augmentation of medicine advice by the pharmacist (“Also, ADD PHYSIO CREAM as required”), and the acceptance of referral advice (green arrows and green text box).
Discussion
Principal Findings
A CDSS was developed to enhance pharmacist care of LBP, underpinned by clinical practice guidelines and informed by a multidisciplinary team of experts that included consultation with community pharmacy. Community pharmacists rated the overall usability of the high-fidelity protype as good to excellent [59], despite expression of some negative sentiment in relation to guidance in screening for serious causes of LBP and interface inconsistency. There was a high level of acceptance for the advice generated by the CDSS for self-care, medicines, and referral, with pharmacists augmenting advice for a minority (5/65) of recommendations.
Usability
Pharmacists reported a high level of usability based around simple use of language, logical workflow, brief consultation time, ability to customize advice, and convenience of a customized handout for the client. A number of usability issues were raised with regard to interface including page layout, text size, and button placement, which will be considered in the next phase of the CDSS refinement. The screening page (Figure 3) received the majority of negative comments and may reflect nonintuitive interaction with the layout of the screening page and/or lack of familiarity with the screening questions used to raise suspicion of serious causes of LBP. Although education for pharmacists in Australia contains topics on symptom recognition for differential diagnosis [62] and interprofessional referral [63], pharmacists expressed interest for more training on this topic, which is consistent with recommendations of Abdel Shaheed et al [64].
Acceptance of the Clinical Support Provided by the Clinical Decision Support System
All pharmacists agreed that the information provided by the CDSS was applicable to the clinical scenarios presented and could potentially improve client-pharmacist encounters. One pharmacist disagreed with the messaging to avoid imaging and preferred to refer to medical care as a first option for nonserious LBP, but given the small sample, may not be representative of their peers. Pharmacists also commented that the CDSS helped them to ask more questions of the client with LBP and increased management options for LBP beyond their usual advice. However, it is unclear if the advice delivered by the CDSS in this setting would be superior to usual pharmacy care for LBP.
Guidance for Pain Management in Community Pharmacy
Pharmacists commented that they appreciated guidance provided by the CDSS in relation to management, particularly for options beyond medicines advice. This aligns with recommendations of Abdel Shaheed et al [65] and others [11] on the potential benefit of tools/guidelines to support pharmacists when managing clients with LBP. Pharmacists also reflected on the current general lack of guidance to manage pain within pharmacy compared with the promotion and availability of management tools for other health conditions [5,7,8,14,66]. This view is consistent with results from a recent study by Abdel Shaheed et al [65] who found that pharmacists were receptive to implementing a disease state management program for LBP. One area highlighted by pharmacists was the lack of operational knowledge in relation to screening clients for serious causes of LBP, which has been highlighted previously [18]. Abdel Shaheed et al also found that pharmacists had both the willingness and capacity to increase knowledge in this area [32,64]. One goal of a training module integrated into the next version of the CDSS would be to empower the pharmacist with the skillset to raise suspicion of potentially serious underlying pathology, then inform clients of options for prompt medical review [43,45].
Limitations
The small sample size may not be adequate to capture the full range of pharmacists’ views or usability issues thus limiting the generalizability of the results [67], particularly with regard to the level of acceptance. However, the sample size was appropriate for this stage of CDSS development [34,51]. That is, it was sufficient to identify major usability issues (eg, when screening for risk of serious disease), that the CDSS interface could be navigated with minimal training, and that decisions generated were logical and easy for the pharmacist to apply (in a simulated scenario). The method used to assess usability (think aloud with active intervention) may have enhanced task performance through researcher-induced bias [68] but allowed greater insight into the sections of the CDSS that required further development [52]. Another source of bias that may have enhanced task performance was the nonrandomized order of cases (case complexity was greater later in interview). This stage of CDSS development was to finalize design elements in the community pharmacy setting before testing with real clients [34]. In its current design, the CDSS does not integrate with existing electronic record systems in pharmacy, which would be necessary before advanced testing and would increase the chance of adoption by pharmacists [69]. One approach would be integration with existing disease state management systems [70], which was also suggested by pharmacists during testing.
Comparison With Prior Work
This CDSS is the first tool that the authors are aware of to assist community pharmacists in first-line care for people with LBP. Other electronic decision support systems have been targeted at the primary care setting for the management of LBP [25] and chronic pain [26,27]. This CDSS differs from existing systems in that it aims to empower the pharmacist to offer evidence-based first-line care beyond medicines advice, and stepped referral options to allied health, primary, or emergency care based on presentation or symptom progression. The opportunity to enhance the pharmacist-client interaction, identified as lacking in other systems [27,71], has been built into this CDSS by allowing the pharmacist to modify management advice then provide a customized handout for the client.
Future Direction
The next phase is to modify the CDSS with lessons learned from this usability study, then reevaluate during the next level of system development (integration with clients into the pharmacy setting) [34]. The CDSS will also be evaluated with respect to the credibility of advice and satisfaction with care from the perspective of clients with LBP. An education module on the evidence-based management of LBP could be delivered to pharmacists in conjunction with training for the CDSS, which would assist with knowledge of screening for pathology and give context to the guideline-based care options suggested by the CDSS. Future studies may establish if pharmacist training during the use of a CDSS within the clinical encounter improves both pharmacist and pharmacy client satisfaction with care.
Conclusions
Despite many years of clinical guidelines for the management of LBP, significant evidence-to-practice gaps remain. This CDSS has been designed to provide a unique opportunity for community pharmacists to provide simple evidence-based advice for clients who present with LBP. Importantly the CDSS offers key messages of reassurance, to remain active, to use medicines appropriately, and to avoid inappropriate imaging.
Acknowledgments
This study was supported by the 2017 Arthritis Australia Grant in Aid (Zimmer Biomet Australia Grant). The authors would like to acknowledge the support of the pharmacists who contributed during development of the CDSS or were participants in this study. The authors also thank professors Bart Koes, Ric Day, and Ian Harris who provided input during the development of the CDSS. A National Health and Medical Research Council Program Grant (ID APP1113532) and Research Fellowship (ID APP1103022) supports CM.
Abbreviations
- CDSS
clinical decision support system
- LBP
low back pain
Appendix
Clinical decision support system design methodology.
Clinical decision support system decision tree.
Clinical decision support system data entry screens.
Client handout example generated by clinical decision support system.
Interview guide.
Footnotes
Authors' Contributions: AD, CM, MH, CS, AM, CW and ZM designed the study. AD and AK were responsible for interface design and coding with revision from all authors. AD was responsible for the acquisition of data. AD, CS, and AK analyzed interview transcripts. AD wrote the first draft of the manuscript. All authors critically reviewed the manuscript and reviewed the final draft before submission.
Conflicts of Interest: None declared.
References
- 1.Smith E, Hoy DG, Cross M, Vos T, Naghavi M, Buchbinder R, Woolf AD, March L. The global burden of other musculoskeletal disorders: estimates from the Global Burden of Disease 2010 study. Ann Rheum Dis. 2014 Aug;73(8):1462–9. doi: 10.1136/annrheumdis-2013-204680. [DOI] [PubMed] [Google Scholar]
- 2.Hoy D, Bain C, Williams G, March L, Brooks P, Blyth F, Woolf A, Vos T, Buchbinder R. A systematic review of the global prevalence of low back pain. Arthritis Rheum. 2012 Jun;64(6):2028–37. doi: 10.1002/art.34347. doi: 10.1002/art.34347. [DOI] [PubMed] [Google Scholar]
- 3.Qaseem A, Wilt T, McLean R, Forciea M, Clinical Guidelines Committee of the American College of Physicians Noninvasive treatments for acute, subacute, and chronic low back pain: A clinical practice guideline from the American College of Physicians. Ann Intern Med. 2017 Apr 4;166(7):514–30. doi: 10.7326/M16-2367. [DOI] [PubMed] [Google Scholar]
- 4.Walker BF, Muller R, Grant WD. Low back pain in Australian adults. health provider utilization and care seeking. J Manipulative Physiol Ther. 2004 Jun;27(5):327–35. doi: 10.1016/j.jmpt.2004.04.006. [DOI] [PubMed] [Google Scholar]
- 5.Kaczorowski J, Chambers LW, Dolovich L, Paterson JM, Karwalajtys T, Gierman T, Farrell B, McDonough B, Thabane L, Tu K, Zagorski B, Goeree R, Levitt CA, Hogg W, Laryea S, Carter MA, Cross D, Sabaldt RJ. Improving cardiovascular health at population level: 39 community cluster randomised trial of Cardiovascular Health Awareness Program (CHAP) Br Med J. 2011 Feb 7;342:d442. doi: 10.1136/bmj.d442. http://europepmc.org/abstract/MED/21300712. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Giaccone M, Baratta F, Allais G, Brusa P. Prevention, education and information: the role of the community pharmacist in the management of headaches. Neurol Sci. 2014 May;35(Suppl 1):1–4. doi: 10.1007/s10072-014-1732-6. [DOI] [PubMed] [Google Scholar]
- 7.Ali M, Schifano F, Robinson P, Phillips G, Doherty L, Melnick P, Laming L, Sinclair A, Dhillon S. Impact of community pharmacy diabetes monitoring and education programme on diabetes management: a randomized controlled study. Diabet Med. 2012 Sep;29(9):e326–33. doi: 10.1111/j.1464-5491.2012.03725.x. [DOI] [PubMed] [Google Scholar]
- 8.Brown D, Portlock J, Rutter P. Review of services provided by pharmacies that promote healthy living. Int J Clin Pharm. 2012 Jun;34(3):399–409. doi: 10.1007/s11096-012-9634-2. [DOI] [PubMed] [Google Scholar]
- 9.Greer N, Bolduc J, Geurkink E, Rector T, Olson K, Koeller E, MacDonald R, Wilt TJ. Pharmacist-led chronic disease management: A systematic review of effectiveness and harms compared with usual care. Ann Intern Med. 2016 Jul 5;165(1):30–40. doi: 10.7326/M15-3058. [DOI] [PubMed] [Google Scholar]
- 10.Légat L, van Laere S, Nyssen M, Steurbaut S, Dupont AG, Cornu P. Clinical decision support systems for drug allergy checking: Systematic review. J Med Internet Res. 2018 Sep 7;20(9):e258. doi: 10.2196/jmir.8206. https://www.jmir.org/2018/9/e258/ [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Silcock J, Moffett JK, Edmondson H, Waddell G, Burton AK. Do community pharmacists have the attitudes and knowledge to support evidence based self-management of low back pain? BMC Musculoskelet Disord. 2007 Jan 31;8:10. doi: 10.1186/1471-2474-8-10. https://bmcmusculoskeletdisord.biomedcentral.com/articles/10.1186/1471-2474-8-10. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Mishriky J, Stupans I, Chan V. Expanding the role of Australian pharmacists in community pharmacies in chronic pain management - a narrative review. Pharm Pract (Granada) 2019;17(1):1410. doi: 10.18549/PharmPract.2019.1.1410. http://europepmc.org/abstract/MED/31015881. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Bennett MI, Bagnall A, Raine G, Closs SJ, Blenkinsopp A, Dickman A, Ellershaw J. Educational interventions by pharmacists to patients with chronic pain: systematic review and meta-analysis. Clin J Pain. 2011 Sep;27(7):623–30. doi: 10.1097/AJP.0b013e31821b6be4. [DOI] [PubMed] [Google Scholar]
- 14.Crealey GE, McElnay JC, Maguire TA, O'Neill C. Costs and effects associated with a community pharmacy-based smoking-cessation programme. Pharmacoeconomics. 1998 Sep;14(3):323–33. doi: 10.2165/00019053-199814030-00008. [DOI] [PubMed] [Google Scholar]
- 15.San-Juan-Rodriguez A, Newman T, Hernandez I, Swart E, Klein-Fedyshin M, Shrank W, Parekh N. Impact of community pharmacist-provided preventive services on clinical, utilization, and economic outcomes: An umbrella review. Prev Med. 2018 Oct;115:145–55. doi: 10.1016/j.ypmed.2018.08.029. [DOI] [PubMed] [Google Scholar]
- 16.National Institute for Health and Care Excellence (NICE) [2019-11-15]. Low Back Pain and Sciatica in Over 16s: Assessment and Management https://pathways.nice.org.uk/pathways/low-back-pain-and-sciatica#content=view-node%3Anodes-imaging. [PubMed]
- 17.Foster NE, Anema JR, Cherkin D, Chou R, Cohen SP, Gross DP, Ferreira PH, Fritz JM, Koes BW, Peul W, Turner JA, Maher CG, Lancet Low Back Pain Series Working Group Prevention and treatment of low back pain: evidence, challenges, and promising directions. Lancet. 2018 Jun 9;391(10137):2368–83. doi: 10.1016/S0140-6736(18)30489-6. [DOI] [PubMed] [Google Scholar]
- 18.Abdel Shaheed C, McFarlane B, Maher C, Williams K, Bergin J, Matthews A, McLachlan AJ. Investigating the primary care management of low back pain: a simulated patient study. J Pain. 2016 Jan;17(1):27–35. doi: 10.1016/j.jpain.2015.09.010. [DOI] [PubMed] [Google Scholar]
- 19.van der Weijden T, Boivin A, Burgers J, Schünemann HJ, Elwyn G. Clinical practice guidelines and patient decision aids. An inevitable relationship. J Clin Epidemiol. 2012 Jun;65(6):584–9. doi: 10.1016/j.jclinepi.2011.10.007. [DOI] [PubMed] [Google Scholar]
- 20.Stacey D, Légaré F, Lewis K, Barry M, Bennett C, Eden K, Holmes-Rovner M, Llewellyn-Thomas H, Lyddiatt A, Thomson R, Trevena L. Decision aids for people facing health treatment or screening decisions. Cochrane Database Syst Rev. 2017 Apr 12;4:CD001431. doi: 10.1002/14651858.CD001431.pub5. http://europepmc.org/abstract/MED/28402085. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Bright T, Wong A, Dhurjati R, Bristow E, Bastian L, Coeytaux RR, Samsa G, Hasselblad V, Williams JW, Musty MD, Wing L, Kendrick AS, Sanders GD, Lobach D. Effect of clinical decision-support systems: a systematic review. Ann Intern Med. 2012 Jul 3;157(1):29–43. doi: 10.7326/0003-4819-157-1-201207030-00450. [DOI] [PubMed] [Google Scholar]
- 22.Berner E, La Lande LT. Overview of clinical decision support systems. In: Berner E, editor. Clinical Decision Support Systems. Switzerland: Springer; 2016. pp. 1–17. [Google Scholar]
- 23.Bryan C, Boren S. The use and effectiveness of electronic clinical decision support tools in the ambulatory/primary care setting: a systematic review of the literature. Inform Prim Care. 2008;16(2):79–91. doi: 10.14236/jhi.v16i2.679. http://hijournal.bcs.org/index.php/jhi/article/view/679. [DOI] [PubMed] [Google Scholar]
- 24.Mickan S, Tilson J, Atherton H, Roberts N, Heneghan C. Evidence of effectiveness of health care professionals using handheld computers: a scoping review of systematic reviews. J Med Internet Res. 2013 Oct 28;15(10):e212. doi: 10.2196/jmir.2530. https://www.jmir.org/2013/10/e212/ [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Peiris D, Williams C, Holbrook R, Lindner R, Reeve J, Das A, Maher C. A web-based clinical decision support tool for primary health care management of back pain: development and mixed methods evaluation. JMIR Res Protoc. 2014 Apr 2;3(2):e17. doi: 10.2196/resprot.3071. https://www.researchprotocols.org/2014/2/e17/ [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Trafton J, Martins S, Michel M, Lewis E, Wang D, Combs A, Scates N, Tu S, Goldstein MK. Evaluation of the acceptability and usability of a decision support system to encourage safe and effective use of opioid therapy for chronic, noncancer pain by primary care providers. Pain Med. 2010 Apr;11(4):575–85. doi: 10.1111/j.1526-4637.2010.00818.x. [DOI] [PubMed] [Google Scholar]
- 27.Smith MY, DePue JD, Rini C. Computerized decision-support systems for chronic pain management in Primary Care. Pain Med. 2007;8(suppl 3):S155–66. doi: 10.1111/j.1526-4637.2007.00278.x. [DOI] [Google Scholar]
- 28.Guenter D, Abouzahra M, Schabort I, Radhakrishnan A, Nair K, Orr S, Langevin J, Taenzer P, Moulin DE. Design process and utilization of a novel clinical decision support system for neuropathic pain in primary care: Mixed methods observational study. JMIR Med Inform. 2019 Sep 30;7(3):e14141. doi: 10.2196/14141. https://medinform.jmir.org/2019/3/e14141/ [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Nicol A, Hurley R, Benzon H. Alternatives to opioids in the pharmacologic management of chronic pain syndromes: A narrative review of randomized, controlled, and blinded clinical trials. Anesth Analg. 2017 Nov;125(5):1682–703. doi: 10.1213/ANE.0000000000002426. http://europepmc.org/abstract/MED/29049114. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Curtain C, Peterson GM. Review of computerized clinical decision support in community pharmacy. J Clin Pharm Ther. 2014 Aug;39(4):343–8. doi: 10.1111/jcpt.12168. [DOI] [PubMed] [Google Scholar]
- 31.Monteiro L, Maricoto T, Solha I, Ribeiro-Vaz I, Martins C, Monteiro-Soares M. Reducing potentially inappropriate prescriptions for older patients using computerized decision support tools: Systematic review. J Med Internet Res. 2019 Nov 14;21(11):e15385. doi: 10.2196/15385. https://www.jmir.org/2019/11/e15385/ [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Abdel Shaheed C, Maher CG, Williams KA, McLachlan AJ. Pharmacists' views on implementing a disease state management program for low back pain. Aust J Prim Health. 2016;22(3):211–7. doi: 10.1071/PY14116. [DOI] [PubMed] [Google Scholar]
- 33.Mugisha A, Babic A, Wakholi P, Tylleskär T. High-fidelity prototyping for mobile electronic data collection forms through design and user evaluation. JMIR Hum Factors. 2019 Mar 22;6(1):e11852. doi: 10.2196/11852. https://humanfactors.jmir.org/2019/1/e11852/ [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Yen P, Bakken S. Review of health information technology usability study methodologies. J Am Med Inform Assoc. 2012;19(3):413–22. doi: 10.1136/amiajnl-2010-000020. http://europepmc.org/abstract/MED/21828224. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Kushniruk A, Monkman H, Borycki E, Kannry J. User-centered designevaluation of clinical information systems: a usability engineering perspective. In: Patel VL, Kannampallil TG, Kaufman DR, editors. Cognitive Informatics for Biomedicine: Human Computer Interaction in Healthcare. Switzerland: Springer International Publishing; 2015. pp. 141–62. [Google Scholar]
- 36.Karsh B. Agency for Healthcare Research and Quality. 2009. [2019-11-15]. Clinical Practice Improvement and Redesign: How Change in Workflow Can Be Supported by Clinical Decision Support https://digital.ahrq.gov/sites/default/files/docs/biblio/09-0054-EF-Updated_0.pdf. [PubMed]
- 37.Bates D, Kuperman G, Wang S, Gandhi T, Kittler A, Volk L, Spurr C, Khorasani R, Tanasijevic M, Middleton B. Ten commandments for effective clinical decision support: making the practice of evidence-based medicine a reality. J Am Med Inform Assoc. 2003;10(6):523–30. doi: 10.1197/jamia.M1370. http://europepmc.org/abstract/MED/12925543. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Khorasani R, Hentel K, Darer J, Langlotz C, Ip IK, Manaker S, Cardella J, Min R, Seltzer S. Ten commandments for effective clinical decision support for imaging: enabling evidence-based practice to improve quality and reduce waste. AJR Am J Roentgenol. 2014 Nov;203(5):945–51. doi: 10.2214/AJR.14.13134. [DOI] [PubMed] [Google Scholar]
- 39.Zikos D, DeLellis N. CDSS-RM: a clinical decision support system reference model. BMC Med Res Methodol. 2018 Nov 16;18(1):137. doi: 10.1186/s12874-018-0587-6. https://bmcmedresmethodol.biomedcentral.com/articles/10.1186/s12874-018-0587-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Abdel Shaheed C, Maher CG, Williams KA, McLachlan AJ. Interventions available over the counter and advice for acute low back pain: systematic review and meta-analysis. J Pain. 2014 Jan;15(1):2–15. doi: 10.1016/j.jpain.2013.09.016. [DOI] [PubMed] [Google Scholar]
- 41.Chou R, Deyo R, Friedly J, Skelly A, Hashimoto R, Weimer M, Fu R, Dana T, Kraegel P, Griffin J, Grusing S, Brodt ED. Nonpharmacologic therapies for low back pain: A systematic review for an American College of Physicians clinical practice guideline. Ann Intern Med. 2017 Apr 4;166(7):493–505. doi: 10.7326/M16-2459. [DOI] [PubMed] [Google Scholar]
- 42.Chou R, Deyo R, Friedly J, Skelly A, Weimer M, Fu R, Dana T, Kraegel P, Griffin J, Grusing S. Systemic pharmacologic therapies for low back pain: A systematic review for an American College of Physicians clinical practice guideline. Ann Intern Med. 2017 Apr 4;166(7):480–92. doi: 10.7326/M16-2458. [DOI] [PubMed] [Google Scholar]
- 43.Downie A, Williams C, Henschke N, Hancock M, Ostelo R, de Vet HC, Macaskill P, Irwig L, van Tulder MW, Koes BW, Maher CG. Red flags to screen for malignancy and fracture in patients with low back pain: systematic review. Br Med J. 2013 Dec 11;347:f7095. doi: 10.1136/bmj.f7095. http://www.bmj.com/cgi/pmidlookup?view=long&pmid=24335669. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44.Grossman JM, Gordon R, Ranganath VK, Deal C, Caplan L, Chen W, Curtis JR, Furst DE, McMahon M, Patkar NM, Volkmann E, Saag KG. American College of Rheumatology 2010 recommendations for the prevention and treatment of glucocorticoid-induced osteoporosis. Arthritis Care Res (Hoboken) 2010 Nov;62(11):1515–26. doi: 10.1002/acr.20295. doi: 10.1002/acr.20295. [DOI] [PubMed] [Google Scholar]
- 45.Maher C, Underwood M, Buchbinder R. Non-specific low back pain. Lancet. 2017 Feb 18;389(10070):736–47. doi: 10.1016/S0140-6736(16)30970-9. [DOI] [PubMed] [Google Scholar]
- 46.Mathieson H, Marzo-Ortega H. Axial spondyloarthritis: diagnosis and management. Prescriber. 2015;25(23-24):32–6. doi: 10.1002/psb.1290. [DOI] [Google Scholar]
- 47.Mathieson S, Kasch R, Maher CG, Pinto RZ, McLachlan AJ, Koes BW, Lin CC. Combination drug therapy for the management of low back pain and sciatica: Systematic review and meta-analysis. J Pain. 2019 Jan;20(1):1–15. doi: 10.1016/j.jpain.2018.06.005. [DOI] [PubMed] [Google Scholar]
- 48.McRae M, Hancock M. Adults attending private physiotherapy practices seek diagnosis, pain relief, improved function, education and prevention: a survey. J Physiother. 2017 Oct;63(4):250–6. doi: 10.1016/j.jphys.2017.08.002. https://linkinghub.elsevier.com/retrieve/pii/S1836-9553(17)30097-8. [DOI] [PubMed] [Google Scholar]
- 49.Verbeek J, Sengers M, Riemens L, Haafkens J. Patient expectations of treatment for back pain: a systematic review of qualitative and quantitative studies. Spine (Phila Pa 1976) 2004 Oct 15;29(20):2309–18. doi: 10.1097/01.brs.0000142007.38256.7f. [DOI] [PubMed] [Google Scholar]
- 50.Johnson CM, Johnson TR, Zhang J. A user-centered framework for redesigning health care interfaces. J Biomed Inform. 2005 Feb;38(1):75–87. doi: 10.1016/j.jbi.2004.11.005. https://linkinghub.elsevier.com/retrieve/pii/S1532-0464(04)00153-4. [DOI] [PubMed] [Google Scholar]
- 51.Nielsen J. Estimating the number of subjects needed for a thinking aloud test. Int J Hum Comput Stud. 1994;41(3):385–97. doi: 10.1006/ijhc.1994.1065. [DOI] [Google Scholar]
- 52.Alhadreti O, Mayhew P. To intervene or not to intervene: an investigation of three think-aloud protocols in usability testing. J Usability Stud. 2017;12(3):111–32. http://uxpajournal.org/wp-content/uploads/sites/8/pdf/JUS_Alhadreti_May2017.pdf. [Google Scholar]
- 53.Beuscart-Zéphir MC, Brender J, Beuscart R, Ménager-Depriester I. Cognitive evaluation: how to assess the usability of information technology in healthcare. Comput Methods Programs Biomed. 1997 Sep;54(1-2):19–28. doi: 10.1016/s0169-2607(97)00030-8. [DOI] [PubMed] [Google Scholar]
- 54.Jaspers M, Steen T, van den Bos C, Geenen M. The think aloud method: a guide to user interface design. Int J Med Inform. 2004 Nov;73(11-12):781–95. doi: 10.1016/j.ijmedinf.2004.08.003. [DOI] [PubMed] [Google Scholar]
- 55.Li AC, Kannry JL, Kushniruk A, Chrimes D, McGinn TG, Edonyabo D, Mann DM. Integrating usability testing and think-aloud protocol analysis with 'near-live' clinical simulations in evaluating clinical decision support. Int J Med Inform. 2012 Nov;81(11):761–72. doi: 10.1016/j.ijmedinf.2012.02.009. [DOI] [PubMed] [Google Scholar]
- 56.Kushniruk AW, Patel VL. Cognitive and usability engineering methods for the evaluation of clinical information systems. J Biomed Inform. 2004 Feb;37(1):56–76. doi: 10.1016/j.jbi.2004.01.003. https://linkinghub.elsevier.com/retrieve/pii/S1532046404000206. [DOI] [PubMed] [Google Scholar]
- 57.Bastien J. Usability testing: a review of some methodological and technical aspects of the method. Int J Med Inform. 2010 Apr;79(4):e18–23. doi: 10.1016/j.ijmedinf.2008.12.004. [DOI] [PubMed] [Google Scholar]
- 58.McLellan S, Muddimer A, Peres C. The effect of experience on system usability scale ratings. J Usability Stud. 2012;7(2):56–67. http://uxpajournal.org/wp-content/uploads/sites/8/pdf/JUS_McLellan_February_2012.pdf. [Google Scholar]
- 59.Bangor A, Kortum PT, Miller JT. An empirical evaluation of the System Usability Scale. Int J Hum Comput Interact. 2008;24(6):574–94. doi: 10.1080/10447310802205776. [DOI] [Google Scholar]
- 60.Hsieh H, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res. 2005 Nov;15(9):1277–88. doi: 10.1177/1049732305276687. [DOI] [PubMed] [Google Scholar]
- 61.Lewis JR. The System Usability Scale: past, present, and future. Int J Hum Comput Interact. 2018;34(7):577–90. doi: 10.1080/10447318.2018.1455307. [DOI] [Google Scholar]
- 62.Marriott JL, Nation RL, Roller L, Costelloe M, Galbraith K, Stewart P, Charman WN. Pharmacy education in the context of Australian practice. Am J Pharm Educ. 2008 Dec 15;72(6):131. doi: 10.5688/aj7206131. http://europepmc.org/abstract/MED/19325951. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 63.University of Sydney. 2019. [2019-11-15]. Unit of study: PHAR3826: Musculoskeletal, Dermatological and Senses https://sydney.edu.au/courses/units-of-study/2019/phar/phar3826.html.
- 64.Abdel Shaheed C, Maher CG, Mak W, Williams KA, McLachlan AJ. Knowledge and satisfaction of pharmacists attending an educational workshop on evidence-based management of low back pain. Aust J Prim Health. 2015;21(2):126–31. doi: 10.1071/PY14020. [DOI] [PubMed] [Google Scholar]
- 65.Abdel Shaheed C, Maher CG, Mak W, Williams KA, McLachlan AJ. The effects of educational interventions on pharmacists' knowledge, attitudes and beliefs towards low back pain. Int J Clin Pharm. 2015 Aug;37(4):616–25. doi: 10.1007/s11096-015-0112-5. [DOI] [PubMed] [Google Scholar]
- 66.Hanna A, White L, Yanamandram V. Patients' willingness to pay for diabetes disease state management services in Australian community pharmacies. Intl J of Pharm Health Mrkt. 2010;4(4):339–54. doi: 10.1108/17506121011095191. [DOI] [Google Scholar]
- 67.Faulkner L. Beyond the five-user assumption: benefits of increased sample sizes in usability testing. Behav Res Methods Instrum Comput. 2003 Aug;35(3):379–83. doi: 10.3758/bf03195514. [DOI] [PubMed] [Google Scholar]
- 68.Zhao T, McDonald S, Edwards H. The impact of two different think-aloud instructions in a usability test: a case of just following orders? Behav Inform Technol. 2014;33(2):163–83. doi: 10.1080/0144929X.2012.708786. [DOI] [Google Scholar]
- 69.Khairat S, Marc D, Crosby W, Al Sanousi A. Reasons for physicians not adopting clinical decision support systems: Critical analysis. JMIR Med Inform. 2018 Apr 18;6(2):e24. doi: 10.2196/medinform.8912. https://medinform.jmir.org/2018/2/e24/ [DOI] [PMC free article] [PubMed] [Google Scholar]
- 70.GuildCare: Pharmacy software for professional services. Sydney, Australia: GuildLink; 2019. [2019-11-17]. http://www.guildlink.com.au/guildcare/ [Google Scholar]
- 71.Kawamanto K, Flynn M, Kukhareva P, ElHalta D, Hess R, Gregory T, Walls C, Wigren AM, Borbolla D, Bray BE, Parsons MH, Clayson BL, Briley MS, Stipelman CH, Taylor D, King CS, Del Fiol G, Reese TJ, Weir CR, Taft T, Strong MB. A pragmatic guide to establishing clinical decision support governance and addressing decision support fatigue: a case study. AMIA Annu Symp Proc. 2018;2018:624–33. http://europepmc.org/abstract/MED/30815104. [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Clinical decision support system design methodology.
Clinical decision support system decision tree.
Clinical decision support system data entry screens.
Client handout example generated by clinical decision support system.
Interview guide.