Abstract
Purpose
Increasing demand for genomic testing coupled with genetics workforce shortages has placed unsustainable pressure on standard models of care. Digital tools can offer improved access, efficiency, and cost savings. We created a patient-facing digital health application to support genomic testing.
Methods
We developed the digital application through user-centered design, guided by an advisory board. We tested its usability and acceptability with patients, practitioners, and members of the general public using mixed methods; data were analyzed using qualitative description and descriptive statistics.
Results
The “Genetics Adviser” delivers pre-test education, counseling, and post-test return of results adaptable to any population, test platform, and setting. Usability testing with 25 patients, the general public, and genetics practitioners (15/25 female; mean age range 40-49 years) demonstrated enthusiasm about the application; users found it easy to navigate and comprehend. Acceptance testing with 19 patients and the public (13/19 female; mean age range 40-49 years) indicated high acceptability of the application and moderate knowledge of genomic sequencing after use.
Conclusion
The Genetics Adviser is a comprehensive, interactive, patient-centered application found to have high acceptability and usability for pre- and post-test genomic testing, counseling, and return of results adaptable for multiple testing platforms, populations, and settings.
Keywords: Digital health, Digital tool, Genetic counseling, Genetic testing, Genomic testing
Introduction
Patient-facing digital health tools are gaining traction in various medical disciplines. They can offer improved access, efficiency, and cost savings.1,2 Digital health tools can also aid patients with self-management and can help facilitate patient-provider communication. In clinical genetics, digital tools can help ameliorate human resource shortages and wait times for genetic counseling and testing that have resulted in delayed and inequitable access to genetics care.3 A recent systematic review found that digital tools can improve patients’ knowledge, psychosocial well-being, behavioral/management changes, family communication, decision making, or level of engagement.4 Digital tools have also been found to help increase the efficiency of clinician workflows by reducing the amount of time needed to spend with patients.
The integration of patient-facing digital tools in clinical genetics workflows has the potential to help improve the quality and efficiency of genetics care.5 Genetics education and counseling is the cornerstone of quality genetic service delivery and leads to improved patient outcomes.3 However, one of the limitations of genetics education and counseling is that it can be time consuming to deliver, resulting in lengthy clinical appointments that are, in part, responsible for the long wait times to receive genetics care.4 Genetics practitioners have used a variety of technical solutions, such as videos or web-based educational modules to augment genetics education.6 These patient-facing digital tools can deliver genetics education asynchronously before meeting with a clinician, giving patients the ability to access necessary genetics educational and counseling content in the location and the time of their preference. Asynchronously delivered digital genetics education and counseling may also give patients the opportunity to include trusted individuals in the process of considering whether genetic testing is right for them, something not always possible in a clinical visit. The use of digital tools in conjunction with clinical consults has been shown to enhance patient-centered care in genetic counseling sessions.7
Existing digital tools focused on genetics service delivery support one or more distinct stages of the genetic testing journey: pre-test education, assessment, and obtaining family history, as well as post-test education and follow-up, but few support the comprehensive genetic service pathway.4 Numerous computerized family health history-taking tools are available for various target populations.8,9 At the pre- and post-test counseling stages, there are several online decision aids that assist selection of a limited range of results from genome sequencing and comparatively fewer technologies that support the return of results.10, 11, 12, 13, 14 Existing digital tools often only focus on a specific type of result or a specific patient population and are supported by a limited evidence base.4
We aimed to develop and test the usability and acceptability of a novel comprehensive, patient-centered, and interactive digital health application that bridges pre- and post-test counseling with return of results and care planning, accommodates any genetic test type (exome, genome, and panels), provides a broad scope of results (primary and secondary findings), and can be used across various patient populations and settings (clinical, research, etc).
Materials and Methods
Setting
The main study site was St. Michael’s Hospital, Unity Health Toronto, Toronto, Ontario, Canada. Ethics approval was obtained from the Unity Health Toronto research ethics board. The study was conducted virtually via Zoom to adhere to COVID-19 public health measures. All participants provided written consent.
Study design
We applied user-centered design principles to codevelop the Genetics Adviser.15,16 In keeping with user-centered design, we assembled an advisory board of end users to inform the design and content of the application. The advisory board of end users consisted of patients with genetic testing experience, health care practitioners including genetic counselors and medical geneticists, along with experts in digital application design. The role of the advisory board was to provide feedback on the platform design and content throughout the development process and to review and approve the final version of the platform. Overall, the development of the Genetics Adviser consisted of 4 phases (Figure 1).
Figure 1.
Stages of Genetics Adviser development and testing. RCT, randomized controlled trial.
Phase 1: Needs assessment
To guide early development of the Genetics Adviser, we identified user needs and gaps generated from qualitative interviews conducted as part of the Genomics ADvISER randomized controlled trial6 (RCT; trial registration number NCT03244202). We used secondary descriptive analysis of these qualitative interviews to examine participant feedback on the Genomics ADvISER prototype, as well as their preferences and needs for a comprehensive genetics digital health application.6 Thirty-one individuals were purposefully sampled for semistructured interviews from a sample of 133 participants from the Genomics ADvISER RCT, who had previously had traditional genetic counseling and cancer genetic testing. Characteristics of the interview participants can be found from the Genomics ADvISER RCT (RCT; trial registration number NCT03244202)6,17; briefly, participants were mostly female (28/31) and about half of them were over the age of 50 years (16/31).
Phase 2: Prototype development
Following user-centered design principles, we used end-user perspectives from the needs assessment phase of the study along with feedback form our advisory board to guide the development of a prototype. We outlined the content and hierarchy of the application to optimize structure and flow of information. Where appropriate, the application was designed in accordance with the International Patient Decision Aids Standards (IPDAS 74 item checklist) to ensure content clarity and ease of use.16 Once the content and application hierarchy were finalized, digital wireframes (a digital interactive prototype) were created using Adobe Indesign. The advisory board reviewed the digital wireframes and provided feedback that was incorporated into the wireframe prototype before usability testing.
Phase 3: Usability testing
Method
We usability tested the digital wireframe prototype of the Genetics Adviser with end users to validate its functions and to capture feedback on the design, content, and navigation. We had a target population of approximately 5 to 7 end users per round, a sufficient size as research has shown that up to 80% of usability issues can be identified with 5 to 8 participants.18 After each round, the wireframe prototype of the Genetics Adviser was refined for the subsequent round of testing. After usability testing with patients, a convenience sample of health care practitioners reviewed and provided feedback on the revised wireframe prototype. Revisions were incorporated into the final version of the Genetics Adviser.
Population and sampling
Individuals for usability testing were recruited between October 2020 and January 2021 from members of the general public and prior participants of the Genomics ADvISER RCT6 (trial registration number NCT03244202). General public participants were recruited from the volunteer sections of either the Kijiji or Craigslist websites. The participants from the RCT were sampled purposively to reflect diversity in gender, previous experience with genetic testing, and age (overall trial sample described elsewhere6) and were predominantly White/European (74%), female (90%), and ≥50 years old (60%). Practitioners were purposively recruited from hospital and research networks by specialty type. Participants were contacted through email. Participants were eligible if they had English proficiency (reading and speaking), were able to use a computer, and were over the age of 18 years.
Data collection
We used the “think aloud” method to elicit reactions from participants while they used the Genetics Adviser. When conducting the think-aloud session, participants were asked to imagine that they are receiving genetic testing for a health condition and were asked to use the Genetics Adviser to learn about, choose, and receive results. Participants used the Genetics Adviser from beginning all the way through to return of results, where they viewed a sample results report along with management recommendations. Participants were also asked to reflect on the application’s content clarity and navigability. The study coordinator (M.C.) noted responses and observations, and the sessions were audio taped and transcribed. After completing the think-aloud exercise, open-ended, semistructured questions were posed to elicit participants’ thoughts on the Genetics Adviser. Modifications to the Genetics Adviser that arose were communicated to the development team following each round of usability testing to inform the next iteration of the tool. Interviews were coded and cataloged by application section, item, and issue. We conducted walk-throughs with practitioners to further validate the content and functionality of the second to last iteration of the application. Practitioner feedback was then incorporated into the third iteration.
Phase 4: Acceptability testing
Method
Acceptability testing was conducted on the final version of the Genetics Adviser using validated measures and survey questions that evaluated components of the Genetics Adviser content and presentation, including amount of information, content clarity, and length. After using the Genetics Adviser, participants completed a satisfaction with decision-making process questionnaire (responses ranging from not at all to a great deal), knowledge of genomic sequencing benefits and limitations (responses ranging from strongly disagree to strongly agree), perceived utility for the application (yes or no), and overall acceptability adapted for assessment of a digital application for genetic testing.19, 20, 21 Knowledge was measured using (1) an established 11-item questionnaire consisting of 2 subscales assessing benefits (Cronbach’s alpha 0.70) and limitations (Cronbach’s alpha 0.80) of GS.19 Satisfaction with decision was measured using the Satisfaction with Decision Scale (Cronbach’s alpha 0.86)20 Semistructured qualitative interviews were used to elicit feedback on their experience.
Population and sampling
Acceptability testing occurred between May 2021 and July 2021 and followed the same strategy for patient and general public recruitment outlined under usability testing. Participants were eligible if they had English proficiency (reading and speaking), were able to use a computer, and were over the age of 18. Participants were sampled purposively to reflect diversity in gender, ethnicity, previous experience with genetic testing, and age. Practitioners were not a part of acceptability testing.
Analysis
Quantitative analysis
Descriptive statistics were used to describe participant demographics (age, sex, education, etc) using Microsoft Excel. We similarly summarized acceptability evaluations of the tool’s presentation, length, clarity of information, and likelihood to recommend the application to other patients using descriptive statistics. Mean scores for knowledge and satisfaction with decision making, were compared using t tests. Knowledge of sequencing benefits and sequencing limitations scores were assessed by summing the number of correct responses to the questions and compared adjusting for baseline score using analysis of covariance (ANCOVA).
Qualitative analysis
Qualitative description was used to analyze data from the think-aloud processes and semistructured qualitative interviews.22 In keeping with qualitative methodology, data collection and analysis were conducted concurrently, and the interview guide was modified accordingly. The analysis team met to compare emergent and existing findings and modified the analysis and sampling strategy based on these findings. This process was completed through peer debriefing and team discussions.
Results
Phase 1: Needs assessment
Learnings from our advisory board meetings, as well as from interviews conducted with participants in the previous RCT of the Genomics ADvISER, uncovered common concerns, preferences, and priorities for digital tools in clinical genetic testing.6,17 Patients and clinicians wanted a more comprehensive digital health application that extended beyond a decision aid to encompass return of results. Patients also remarked feeling untethered from the health care system during the waiting period before receiving results. Because patients felt like they were in the dark while waiting for genetic test results, a check-in module was designed to provide support during this crucial waiting period. They desired access to supports in the form of resources and check-ins. Clinicians on our advisory board echoed this need for more touchpoints with patients between pre-test counseling and return of results. Additionally, practitioners in the advisory group raised the need for a platform that could be applicable to different tests and clinical needs. Advisory board comments and suggestions were consistent with patient input and additionally suggested that content be clear and at an appropriate reading level, terms be defined, and consideration be taken for the emotional needs of patients.
Phase 2: Prototype development
The Genetics Adviser prototype hierarchy and content were developed using preexisting content and user flow of the Genomics ADvISER decision aid,6 patient feedback from the needs assessment, and with the input of the study advisory board. Overall, the content of the prototype covered all types of sequencing results that patients may receive—including primary and secondary, as well as incidental results. Basic concepts from videos were summarized using a combination of icons and text. An FAQ section was added for participants seeking out more information. Similarly, rollover definitions were inserted throughout so that participants could access simple definitions without having to return to previous pages. Vignettes of patient journeys were incorporated to convey information in a more personal but balanced manner, reflecting real-world scenarios (eg, negative results). The genetic testing process beyond pretest education was demystified for patients by clearly outlining next steps and the inclusion of check points and resources to support patients while they wait for results. The prototype was designed for reading comprehension at an eighth-grade level. As a part of the content creation, educational videos were scripted and animated, which would later be integrated into the Genetics Adviser. All content was developed with and reviewed by certified GCs (R.K. and S.S.).
Once the content and application hierarchies were finalized, digital wireframes (a digital interactive prototype of the Genetics Adviser) were created. Wireframes were produced for 3 sequential modules: pretest, check-in, and results disclosure. The pretest module comprised 3 sections: learn, explore, and choose. To accommodate different learning preferences, the platform uses multiple mediums to deliver educational content, using a combination of video, text, and graphic imagery responsive to different learning styles. The advisory board reviewed the digital wireframes and content and provided feedback that was incorporated into the final wireframe prototype before usability testing.
Phase 3: Usability testing
Usability testing of the wireframes (and videos) occurred in 3 rounds with the 19 patients, with 5 to 7 participants in each round of testing. The majority of the patients were female (13/19) and had completed some form of post-secondary education (13/19). The sample population was relatively diverse across age, employment status, country of origin, and ethnicity (Table 1). Three of the 19 patients had traditional genetic testing and counseling for a personal or family history cancer. Five practitioners were recruited at this stage, 3 hospital health care providers and 2 managers involved in hospital and university settings. Experience among the 5 practitioners ranged from 10 to 20 years of practice (Table 2).
Table 1.
Demographic characteristics of participants in usability and acceptability testing
| Characteristic | Usability Sample | Acceptability Sample |
|---|---|---|
| Age | (n = 19) | (n = 19) |
| 19–29 | 7 | 4 |
| 30–39 | 4 | 2 |
| 40–49 | 1 | 2 |
| 50–59 | 3 | 5 |
| 60–69 | 3 | 5 |
| >70 | 1 | 1 |
| Gender | (n = 19) | (n = 19) |
| Male | 5 | 6 |
| Female | 13 | 13 |
| Nonbinary | 1 | |
| Education (highest level completed) | (n = 19) | (n = 19) |
| High school | 3 | 2 |
| College/trade school | 2 | 2 |
| Bachelor’s degree | 8 | 9 |
| Postgraduate degree | 6 | 5 |
| Employment status | (n = 19) | (n = 19) |
| Full-time | 6 | 9 |
| Part-time | 7 | 3 |
| Retired | 1 | 5 |
| Self-employed | 2 | 2 |
| Work in home/housework | 1 | |
| Student | 2 | |
| Country of birth | (n = 19) | (n = 19) |
| Canada | 11 | 12 |
| Outside Canada | 8 | 7 |
| Ethnicity | (n = 19) | (n = 19) |
| White/European | 8 | 8 |
| Black-Caribbean region | 2 | |
| Black-Africa | 1 | |
| Asian-South East | 7 | 1 |
| Asian-East | 3 | |
| Asian-South | 1 | |
| West Indian | 1 | |
| Middle Eastern | 3 | |
| Armenian | 1 | |
| Prefer not to answer | 2 | |
| Personal experience with genetic testing | (n = 19) | (n = 19) |
| Yes | 3 | 9 |
| No | 16 | 10 |
Table 2.
Demographic characteristics of health care providers in usability testing
| Characteristic | Health Care Provider (n = 3) | Manager (n = 2) |
|---|---|---|
| Setting | ||
| Hospital | 3 | 1 |
| University | 0 | 1 |
| Years of practice | ||
| 10 | 1 | 1 |
| 15 | 2 | 0 |
| 20 | 0 | 1 |
Overall, participants found the Genetics Adviser easy to use and intuitive and perceived that their ability to understand genetics and testing had improved. Participants were generally able to navigate through all components of the application without difficulty. Usability issues were raised around overall appearance, preferences for navigation, and content clarity and appropriateness. These items converged on task clarification, time management, framing risk, and general refinements. These revisions and the final version of the Genetics Adviser are described below.
Task clarification
Initially, in the “learn” step of the pre-test module, categories of results were presented in a comparison table. Many participants found it to be confusing; therefore, it was omitted. Participants valued an upfront explanation for each section of the application as to the purpose of each task. Introductory pages and statements were added to explain how each task built upon each other to prepare and inform their decisions about genetic testing. For example, a page was added to introduce the learn and explore sections explaining their purposes and utility.
Time management
In the first round of testing, participants expressed a desire to know the length of each task and suggested including a progress bar to delineate progression through the Genetics Adviser. In response, the average time to complete the modules was added to the landing page and the length of videos was noted where applicable. A progress bar was also added to indicate where in the platform the user is located. Relatedly, some participants wanted to freely navigate the application and use it in a nonlinear way and skip over sections. Clarification was provided regarding the intentional sequencing and that once a section was completed, it could be returned to for review.
Framing risk
Choice of language when delivering sensitive information was discussed by participants. Some participants found the language to be too cautionary, especially when discussing the possibility of emotional distress associated with genetic testing. Participants expressed that this type of language served to elevate their concerns rather than providing a sense of preparation. Language was edited throughout the Genetics Adviser to convey a more balanced tone, especially with respect to the cautionary statements where possible. Originally the platform content stated that genetic testing causes an emotional response, and this sentiment was highlighted throughout the platform. Participants felt this was unnecessary; therefore, the emotional response content was scaled back to one section of the pre-test journey and one section of the post-test module.
Other refinements
Other revisions made to the platform based on usability testing were derived from participant suggestions to improve the content and interface. For example, a summary of the steps involved in the testing pathway preceding more detailed descriptions and a privacy disclaimer were included. Video and text content were better aligned to allow users to review either of the 2. The FAQs were modified to include more detailed information on privacy and insurance implications. Other edits were made to clarify names of categories of results, define key terms, and simplify language.
Final version of Genetics Adviser
After usability testing was completed and participant feedback was incorporated into the application, the final version of the application was reviewed and approved of by the study advisory board. The application was then programmed. The final version of the Genetics Adviser is an interactive web-based digital application that can be adapted to serve a range of patient populations (cancer, prenatal, pediatric, etc) for various genetic tests (gene panels and exome or genome sequencing), and a spectrum of result types (primary and secondary findings) across a variety of settings (clinical, research, laboratories, and direct to consumer). The Genetics Adviser comprises 3 modules—pre-test, check-in, and return of results. The pre-test module consists of multimodal approaches to deliver educational content on genetic testing and all results from sequencing. After Learn is the Explore module, which provides an FAQ section, followed by interactive exercises to prepare users and explore their preferences and values. The check-in module automatically prompts users to review their testing choices, reviews educational content, and ends with interactive activities to prepare for results. The return of results module provides an overview of the results and recommended care planning. The results reports incorporate any standard report the clinician or laboratory uses along with a patient-friendly results summary. The content of the platform is written to be test and health condition agnostic, enabling much of the information provided to be applicable to most testing contexts. It has an eighth-grade reading level overall. The platform uses an adaptive design, enabling it to be used on a variety of internet-enabled devices and operating systems. The application uses an administrative access portal that allows clinicians and administrative staff to manage individual users and to modify platform content and flow.
Phase 4: Acceptability testing
Acceptability testing of the final version of the Genetics Adviser was conducted with 19 patients and members of the general public. Many participants were female (14/19), born in Canada (12/19), and had completed postsecondary education (17/19). Half of participants had experience with genetic testing (10/19) and included cancer testing, carrier screening, and direct-to-consumer testing. The sample was relatively diverse across age, country of birth, and ethnicity (Table 1).
After using the Genetics Adviser, the mean score for satisfaction with decision making was 4.26 on a scale of 1 to 5 across all domains indicating high satisfaction with understanding the decision and the options, preparation for decision making, and future dialog with a doctor. Level of knowledge was moderate (sequencing limitations mean = 7.95 [SD 1.84]; sequencing benefits mean 7.42 [SD 1.95] out of max score of 10). When evaluating acceptability of the application, all participants found the content across all modules to be excellent or good (19/19). Most participants found the length of the presentation (19/19) and amount of information (18/19) to be just right, and all stated it was clear and balanced. All participants responded that they found the tool helpful (19/19) and would recommend it to patients (19/19). It took on average 27 minutes for participants to navigate through the application (Table 3).
Table 3.
Acceptability results from usability testing (n = 19)
| General Information (n = 19) | |
| Excellent | 15 |
| Good | 4 |
| Fair | 0 |
| Poor | 0 |
| Risks and benefits (n = 19) | |
| Excellent | 13 |
| Good | 5 |
| Fair | 1 |
| Poor | 0 |
| Questions and answers (n = 19) | |
| Excellent | 10 |
| Good | 9 |
| Fair | 0 |
| Poor | 0 |
| Disease risk categories (n = 19) | |
| Excellent | 13 |
| Good | 6 |
| Fair | 0 |
| Poor | 0 |
| The length of presentation in this digital tool was (n = 19) | |
| Just right | 19 |
| Too short | 0 |
| Too long | 0 |
| The amount of information in this digital tool was (n = 19) | |
| Just right | 18 |
| Too little information | 0 |
| Too much information | 1 |
| I found the information presented (n = 19) | |
| Balanced | 19 |
| Slanted toward choosing certain categories | 0 |
| How clear is the information in this decision aid? Select the best answer (n = 19) | |
| Everything was clear | 11 |
| Most things were clear | 8 |
| Some things were unclear | 0 |
| Most things were unclear | 0 |
| Do you think we included enough information to help a patient decide on which results to learn? (n = 19) | |
| Yes | 18 |
| No | 0 |
| Don’t know | 1 |
| Would you recommend this decision aid to patients? Select the best answer (n = 19) | |
| I would definitely recommend it | 16 |
| I would probably recommend it | 3 |
| I would probably not recommend it | 0 |
| I would definitely not recommend it | 0 |
Participants were asked to please rate each section, by indicating “poor,” “fair,” “good,” or “excellent” to show what they felt about the way the information was presented.
Interviews with participants confirmed findings from acceptability testing. Participants described the application as easy to navigate and appreciated its visual design. Overall, they found the tool user-friendly and the content to be informative and easy to comprehend. Participants commented that they would have liked to use the Genetics Adviser as part of their past genetic testing experiences and that they could see its utility for other patients. Participants noted some items for future development including tailored information regarding insurance policy and privacy concerns specific to particular jurisdictions. Others raised access and equity issues as to how those with limited digital literacy or internet connectivity may not benefit from this application. Conversely, some participants noted that it could enhance equity in rural settings and regions with limited access to health care.
Discussion
New solutions are needed to improve the quality and efficiency of genetic services considering significant workforce shortages and growing demand for genetic testing.3 Using a user-centered design approach, we created the “Genetics Adviser,” a new, interactive, and patient-centered digital health application codesigned to deliver genetics care and optimizing the patient journey from pretest education and counseling, waiting period, and post-test return of results. Through usability and acceptability testing, we found that participants were enthusiastic about the platform and found it easy to navigate and comprehend. The Genetics Adviser was well received by patients and clinicians and was associated with high acceptability, usability, and relevance. Results suggest that the Genetics Adviser can provide a continuity of care in 1 digital platform.
The process we undertook to create the Genetics Adviser represents a possible model of how to develop and evaluate patient-facing tools that are responsive to end-user needs. Unlike digital technologies developed without the involvement of the end users, user-centered design and the inclusion of end users (consumer, patients, and provider communities) throughout the development process ensures that the design elements and workflows most valued by end users are incorporated into digitals tools. The Genetics Adviser also represents an innovation on how digital tools can be integrated in genetics care. Digital tools focused on genetics service delivery are often limited to specific health conditions, patient populations, result types, and discrete stages of the genetic testing journey.8, 9, 10, 11, 12, 13, 14 In contrast, the Genetics Adviser provides comprehensive continuity of care from pretest counseling and support during the waiting period through to return of results and management recommendations or care planning. Second, the Genetics Adviser is scalable to accommodate any number and type of results and is adaptable to the needs of various patient population (cancer, prenatal, pediatric, etc), test modality (gene panels, exome sequencing, or GS), and setting (clinical, research, laboratories, and direct to consumer). Finally, it is designed to adapt to existing genetics care delivery workflows, obviating the need to modify existing workflows to accommodate the digital platform. This has the potential to reduce barriers to adoption and implementation.
Current models of genetics care require practitioners to attend to patient intake and deliver patient education and support in person, via telephone or virtually. Patient intake could be completed by digital tools—along with basic education and counseling—enabling the provider to spend time on more personalized care during a clinical consult. Comprehensive digital tools such as the Genetics Adviser, designed to support rather than replace genetics clinicians, may facilitate more streamlined workflows, allowing providers to practice at the top of their scope.23 When compared with standard pre-test genetic counseling, studies have shown that participants that used digital tools spent 10 to 40 minutes less with clinicians while still preserving positive patient outcomes.6,24,25 In fact, the original Genomics ADvISER prototype demonstrated service efficiencies via significantly reduced counseling times, without compromising patient-centered care.6 Likewise, the new Genetics Adviser could translate to downstream health care cost savings, reduced wait times, and improved accessibility and uptake of genetics services.26 Future research agendas should evaluate service efficiencies and impacts across a range of diverse populations (ancestry, ethnicity, age, sex, gender, socioeconomic status, language, and cultural differences), using clinically accepted benchmarks or standard evaluative metrics.4,27 Furthermore, real-world data on the effectiveness and implementation of digital platforms such as the Genetics Adviser are needed to understand barriers to implementation and how these tools may play a role in care across a variety of care contexts, clinical and cultural settings, and institutional settings.
Limitations in this study arise from characteristics of the study participants. Participants were recruited from the general population through online advertisements, which may introduce self-selection bias if they have higher level of exposure to or interest in genetics. However, a large majority of participants at usability testing and half of those at the acceptability phase did not have prior experience with genetic testing, a strength to ensure its usability with the general public. Online recruitment may also skew the sample toward those with higher overall digital literacy affecting how ease of use is perceived. We currently have an RCT underway to evaluate the clinical and cost-effectiveness of the Genetics Adviser in which we will formally evaluate the impact of digital literacy on the user experience and effectiveness of the tool.28
Our study demonstrated the high usability and acceptability of the Genetics Adviser. The Genetics Adviser is one of the first digital applications that integrates the genetics service pathway from the pretest waiting period to the post-test return of results and management recommendations, providing continuity of care. Furthermore, the Genetics Adviser can be easily modified to suit any test type, population, and laboratory or clinical setting. Implementation of this application may create efficiencies in genetic service delivery and reduce health care costs, while enhancing quality and patient-centered care.
Data Availability
The data that support the findings of this study are available from the corresponding author Y.B. upon request.
Conflict of Interest
Yvonne Bombard and Marc Clausen are co–founders of Genetics Adviser. All other authors declare no conflicts of interest.
Acknowledgments
The authors thank the research participants and clinicians who partnered on the development and testing of the Genetics Adviser. Y.B. holds the Canada Research Chair in Genomics Health Services Research and Policy. Portions of the data in this manuscript were presented at 2022 ACMG Annual Clinical Genetics Meeting, March 22-26, Nashville, Tennessee, and the 2022 Virtual ELSI Congress Meeting, May 31 to June 3, 2022.
Funding
The study was supported by a grant awarded to Y.B. from the Canadian Institutes of Health Research (#165963) and the McLaughlin Centre (#MC-2018-04).
Author Information
Conceptualization: Y.B., M.C., R.H.K., N.N.B., J.C.C., A.S., K.E.T., K.A.S., H.F.; Data Curation: M.C., S.K., D.H., E.S., S.S., R.H.K., C.M., E.R., J.S., E.A.-W., S.P., M.A., T.G., S.R.A., C.E., A.E., J.L.-E., H.F.; Formal Analysis: M.C., S.K., D.H., R.K., S.S., E.R., J.S., E.A.-W.; Funding Acquisition: Y.B.; Methodology: Y.B., M.C., E.S.; Project Administration: M.C., S.K., R.K., S.S., D.H.; Resources: R.K., S.S., E.G., S.P., J.C.C., E.S., S.S.-E., G.F., C.S.; Writing-original draft: M.C., S.K., D.H., Y.B.; Writing-review and editing: M.C., S.K., D.H., R.K., S.S., E.R., C.M., J.S., E.A.-W., N.N.B., G.F., E.G., J.L.-E., A.S., S.S.-E., C.S., S.R.A., M.A., T.G., S.P., K.E.T., J.C.C., A.E., C.E., R.H.K., H.F., K.A.S., E.S., Y.B.
Ethics Declaration
The main study site was St. Michael’s Hospital in Toronto, Ontario. Ethics approval was obtained from the St. Michael’s Hospital Research Ethics Board. The study was conducted virtually via Zoom to adhere to COVID-19 public health measures. All participants provided written consent.
Footnotes
The Article Publishing Charge (APC) for this article was paid by Yvonne Bombard.
References
- 1.Borg K., Boulet M., Smith L., Bragge P. Digital inclusion and health communication: a rapid review of literature. Health Commun. 2019;34(11):1320–1328. doi: 10.1080/10410236.2018.1485077. [DOI] [PubMed] [Google Scholar]
- 2.Arsenijevic J., Tummers L., Bosma N. Adherence to electronic health tools among vulnerable groups: systematic literature review and meta-analysis. J Med Internet Res. 2020;22(2) doi: 10.2196/11613. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Dragojlovic N., Borle K., Kopac N., et al. The composition and capacity of the clinical genetics workforce in high-income countries: a scoping review. Genet Med. 2020;22(9):1437–1449. doi: 10.1038/s41436-020-0825-2. [DOI] [PubMed] [Google Scholar]
- 4.Lee W., Shickh S., Assamad D., et al. Patient-facing digital tools for delivering genetic services: a systematic review. J Med Genet. 2023;60(1):1–10. doi: 10.1136/jmg-2022-108653. [DOI] [PubMed] [Google Scholar]
- 5.Bombard Y., Hayeems R.Z. How digital tools can advance quality and equity in genomic medicine. Nat Rev Genet. 2020;21(9):505–506. doi: 10.1038/s41576-020-0260-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Bombard Y., Clausen M., Shickh S., et al. Effectiveness of the Genomics ADvISER decision aid for the selection of secondary findings from genomic sequencing: a randomized clinical trial. Genet Med. 2020;22(4):727–735. doi: 10.1038/s41436-019-0702-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Shickh S., Rafferty S.A., Clausen M., et al. The role of digital tools in the delivery of genomic medicine: enhancing patient-centered care. Genet Med. 2021;23(6):1086–1094. doi: 10.1038/s41436-021-01112-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Welch B.M., Wiley K., Pflieger L., et al. Review and comparison of electronic patient-facing family health history tools. J Genet Couns. 2018;27(2):381–391. doi: 10.1007/s10897-018-0235-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Wang C., Paasche-Orlow M.K., Bowen D.J., et al. Utility of a virtual counselor (VICKY) to collect family health histories among vulnerable patient populations: a randomized controlled trial. Patient Educ Couns. 2021;104(5):979–988. doi: 10.1016/j.pec.2021.02.034. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Lewis M.A., Paquin R.S., Roche M.I., et al. Supporting parental decisions about genomic sequencing for newborn screening: the NC NEXUS decision aid. Pediatrics. 2016;137(suppl 1):S16–S23. doi: 10.1542/peds.2015-3731E. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Freed A.S., Gruß I., McMullen C.K., et al. A decision aid for additional findings in genomic sequencing: development and pilot testing. Patient Educ Couns. 2021;104(5):960–968. doi: 10.1016/j.pec.2020.10.038. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Birch P., Adam S., Bansback N., et al. DECIDE: a decision support tool to facilitate parents’ choices regarding genome-wide sequencing. J Genet Couns. 2016;25(6):1298–1308. doi: 10.1007/s10897-016-9971-8. [DOI] [PubMed] [Google Scholar]
- 13.Tabor H.K., Jamal S.M., Yu J.H., et al. My46: a web-based tool for self-guided management of genomic test results in research and clinical settings. Genet Med. 2017;19(4):467–475. doi: 10.1038/gim.2016.133. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Suckiel S.A., Odgis J.A., Gallagher K.M., et al. GUÍA: a digital platform to facilitate result disclosure in genetic counseling. Genet Med. 2021;23(5):942–949. doi: 10.1038/s41436-020-01063-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Holmes-Rovner M. International Patient Decision Aid Standards (IPDAS): beyond decision aids to usual design of patient education materials. Health Expect. 2007;10(2):103–107. doi: 10.1111/j.1369-7625.2007.00445.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Légaré F., O’Connor A.M., Graham I.D., Wells G.A., Tremblay S. Impact of the Ottawa Decision Support Framework on the agreement and the difference between patients’ and physicians’ decisional conflict. Med Decis Making. 2006;26(4):373–390. doi: 10.1177/0272989X06290492. [DOI] [PubMed] [Google Scholar]
- 17.Mighton C., Carlsson L., Clausen M., et al. Development of patient “profiles” to tailor counseling for incidental genomic sequencing results. Eur J Hum Genet. 2019;27(7):1008–1017. doi: 10.1038/s41431-019-0352-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Macefield R. How to specify the participant group size for usability studies: a practitioner’s guide. J Usability Stud. 2009;5(1):34–45. [Google Scholar]
- 19.Kaphingst K.A., Facio F.M., Cheng M.R., et al. Effects of informed consent for individual genome sequencing on relevant knowledge. Clin Genet. 2012;82(5):408–415. doi: 10.1111/j.1399-0004.2012.01909.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Holmes-Rovner M., Kroll J., Schmitt N., et al. Patient satisfaction with health care decisions: the satisfaction with decision scale. Med Decis Making. 1996;16(1):58–64. doi: 10.1177/0272989X9601600114. [DOI] [PubMed] [Google Scholar]
- 21.User manual-acceptability. The Ottawa Hospital Research Institute. Published 1996. http://www.ohri.ca/decisionaid
- 22.Sandelowski M. Vol. 23. John Wiley & Sons; 2000. (Focus on Research Methods Whatever Happened to Qualitative Description?). [DOI] [PubMed] [Google Scholar]
- 23.Stoll K., Kubendran S., Cohen S.A. The past, present and future of service delivery in genetic counseling: keeping up in the era of precision medicine. Am J Med Genet C Semin Med Genet. 2018;178(1):24–37. doi: 10.1002/ajmg.c.31602. [DOI] [PubMed] [Google Scholar]
- 24.Green M.J., Peterson S.K., Baker M.W., et al. Effect of a computer-based decision aid on knowledge, perceptions, and intentions about genetic testing for breast cancer susceptibility: a randomized controlled trial. JAMA. 2004;292(4):442–452. doi: 10.1001/jama.292.4.442. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Griffith J.M., Sorenson J.R., Bowling J.M., Jennings-Grant T. Assessment of an interactive computer-based patient prenatal genetic screening and testing education tool. Health Educ Behav. 2005;32(5):613–626. doi: 10.1177/1090198105278747. [DOI] [PubMed] [Google Scholar]
- 26.Raspa M., Moultrie R., Toth D., Haque S.N. Barriers and facilitators to genetic service delivery models: scoping review. Interact J Med Res. 2021;10(1) doi: 10.2196/23523. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Assamad D., Majeed S., Aguda V., et al. Digital health tools in genomics: advancing diversity, equity, and inclusion. Public Health Genomics. 2023;26(1):194–200. doi: 10.1159/000534804. [DOI] [PubMed] [Google Scholar]
- 28.Shickh S., Hirjikaka D., Clausen M., et al. Genetics Adviser: a protocol for a mixed-methods randomised controlled trial evaluating a digital platform for genetics service delivery. BMJ Open. 2022;12(4) doi: 10.1136/bmjopen-2022-060899. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Availability Statement
The data that support the findings of this study are available from the corresponding author Y.B. upon request.