Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2022 Apr 1.
Published in final edited form as: J Genet Couns. 2020 Oct 24;30(2):598–605. doi: 10.1002/jgc4.1348

Evaluating co-created patient-facing materials to increase understanding of genetic test results

Andrew A Dwyer 1,2, Margaret G Au 3, Neil Smith 4, Lacey Plummer 2, Margaret F Lippincott 2, Ravikumar Balasubramanian 2, Stephanie B Seminara 2
PMCID: PMC8284837  NIHMSID: NIHMS1718736  PMID: 33098367

Abstract

Patients often have difficulty understanding genetic test reports. Technical language and jargon can impede comprehension and limit patients using results to act on findings. One potential way to improve patient understanding of genetic test reports is to provide patient-facing materials. This study aimed to examine understandability and actionability of co-created patient-facing materials for genetic test results in a research context. We combined interprofessional perspectives and patient engagement to co-create patient-facing materials for patients undergoing research genetic testing for congenital hypogonadotropic hypogonadism (Kallmann syndrome). The iterative development process was guided by principles of health literacy and human-centered design (i.e. design thinking). Readability was assessed using eight validated algorithms. Patients and parents evaluated materials using a web-based survey. The gold standard Patient Education Materials Assessment Tool for print materials (PEMAT-P) was employed to measure understandability (content, style, use of numbers, organization, design, use of visual aids) and actionability. PEMAT-P scores >80% were considered high quality. Results were analyzed descriptively and correlations performed to identify relationships between education/health literacy and PEMAT-P ratings. A consensus score of eight algorithms indicated the materials were an 8th-9th grade reading level. Our findings are consistent with the U.S. Department of Health and Human Services ‘average difficulty’ classification (i.e. 7th-9th grade). In total, 61 patients/parents evaluated the materials. ‘Visual Aids’ received the lowest mean PEMAT-P rating (89%). All other parameters scored 90-97%. PEMAT-P scores did not differ according to educational attainment (less than college vs. college or more, p=0.28). Participants with adequate health literacy were more likely to approve of the ‘organization’ of information (p<0.05). Respondents with low health literacy had more favorable views of ‘visual aids’ (p<0.01). Involving patients in a co-creation process can produce high quality patient-facing materials that are easier to understand.

Keywords: genetic testing, health literacy, patient centered care, patient education, reading comprehension

INTRODUCTION

The 2018 report from the National Academy of Sciences, Engineering and Medicine notes that growing genomic health disparities are preventing the uptake of genetic testing and the implementation of precision medicine (NASEM, 2018). Factors contributing to disparities are multifaceted yet limited health literacy and numeracy pose significant barriers for uptake of genetic testing and implementation of precision medicine (Crawford, Cooke Bailey, & Briggs, 2019; Haga et al., 2013; J. R. Williams et al., 2018). Literacy is defined as the ability to understand, use, and respond appropriately to written texts (Institute of Education Sciences, 2017). Health literacy is defined as is defined as the degree to which individuals have the capacity to obtain, process, and understand basic health information and services needed to make appropriate health decisions (NASEM, 2018). A 2016 report from the Organization for Economic Co-operation and Development (OECD) identifies 1 in 5 people globally have limited literacy/numeracy ((OECD), 2016). According to the 2017 Program for the International Assessment of Adult Competencies survey, 19% of American adults have level 1, or lower, literacy skills (i.e. basic) and 29% have level 1, or lower, numeracy skills (Institute of Education Sciences, 2017). However, a recent evaluation of genetic testing reports from 26 laboratories revealed that results are written at a graduate school level (Davis, Hamby Erby, Fiallos, Martin, & Wassman, 2019). Moreover, even highly educated individuals (i.e. college graduates) have numeracy gaps that impede estimation of genetic risk (Bergman, Goodson, & Goltz, 2017).

One viable approach to bridge lay public literacy/numeracy gaps is to use so-called ‘universal precautions’ to enhance readability and understandability of printed materials (DeWalt et al., 2011). The Agency for Healthcare Research and Quality (AHRQ) has developed gold standard tools for evaluating patient education materials (Shoemaker, Wolf, & Brach, 2014). Patient-facing materials (i.e. materials written in plain language for patients) have been proposed as a means to engage patients and promote understanding of genetic concepts (Haga et al., 2013; J. L. Williams et al., 2018). The term genetic literacy has been defined as genetics knowledge as it relates to and affects the lives of individuals (Bowling et al., 2008). The concept of genetic literacy has been used to examine the training of healthcare professionals (Allen, McBride, Balcazar, & Kaphingst, 2016; Kaye & Korf, 2013; Regan, Engler, Coleman, Daack-Hirsch, & Calzone, 2019) as well as in the lay public (Hurle et al., 2013; Lea, Kaphingst, Bowen, Lipkus, & Hadley, 2011; Peterson et al., 2018). A recent systematic review on the emerging topic of patient-facing materials in genetics/genomics suggests there is limited knowledge on how to improve patient-facing materials and best practices have yet to be delineated (Wynn et al., 2018). Work to date using patient-facing materials has largely focused on diagnostic settings i.e. CLIA certified labs performing genetic testing (J. L. Williams et al., 2018). There is a paucity of data on understandability and actionability of patient-facing materials.

We have previously used a community engagement approach to match expert clinicians (who have deep knowledge of disease pathophysiology, diagnosis and treatment) with expert patients (who understand what it is like to live with a disease) to create patient-facing educational materials in clinical settings (COST action BM1105. et al., 2017). The resulting co-created patient education materials received high patient ratings for both understandability and actionability (i.e. knowing what to do with the information). Thus, using co-creation, plain language principles and evaluation using validated measures has shown to be an effective approach for developing patient education materials in clinical setting (COST action BM1105. et al., 2017). However, there is a paucity of evidence on this approach related to genetics and the use of patient-facing materials for test results in research settings has yet to be examined. Moreover, it is also not known if insights from other fields such as design thinking (the cognitive, strategic and practical processes for designing products) can inform genetic literacy interventions.

This study aims to combine interprofessional perspectives, patient engagement and co-creation practices (i.e. design thinking) to develop patient-facing materials to engage research participants and support them in understanding genetic test results delivered in a research setting. We use the rare disease research paradigm as an exemplar case to examine patient ratings of understandability and actionability of co-created patient-facing materials.

METHODS

Approval to conduct this human subjects research was obtained by the Boston College institutional review board. All procedures followed were in accordance with the ethical standards of the responsible committee on human experimentation (institutional and national) and with the Helsinki Declaration of 1975, as revised in 2000. Electronic, opt-in informed consent was obtained from all participants prior to being included in the study.

The co-creation process involved an interprofessional team of stakeholders spanning several disciplines (medicine, nursing, genetic counseling, molecular genetics) and patients. The development process was guided by principles of health literacy (DeWalt et al., 2011) and drew on a design thinking process. Design thinking is sometimes referred to as human- or user-centered design and has been widely used to involve stakeholders (including the target population) in developing and launching technology products (Rousseau et al., 2019). Design thinking is a five-step process involving: empathizing (understanding the perspectives end-users), defining the needs of end-users, ideating (brainstorming a range of approaches to address needs), prototyping drafts of a proposed interventions (that are rapidly adapted and refined in an iterative process with stakeholders) and testing the intervention or product.

Patient-facing materials were created for patients and families with a rare genetic endocrine disorder -congenital hypogonadotropic hypogonadism/Kallmann syndrome (CHH/KS). The condition is caused by isolated deficiency of gonadotropin releasing hormone (GnRH) that clinically manifests as absent/incomplete puberty and infertility and presents with a range of associated phenotypes that occur at variable rates (Maione et al., 2018). Neither intellectual nor learning disabilities are associated with CHH/KS. In addition to being clinically heterogeneous, CHH/KS is also genetically heterogeneous. More than 30 genes have been identified to contribute to the etiopathogenesis of the condition - accounting for approximately half of cases. Further, the genetic architecture of CHH/KS is complex including autosomal recessive, autosomal dominant, X-linked, digenic and oligogenic forms (Maione et al., 2018). We aimed to develop patient-facing materials (“Understanding Your Genetic Test Results”) to help patients and families understand research genetic testing and appreciate how they can use the information for making health decisions. The patient-facing materials provided a description of its purpose, responses to patient-identified questions and links to useful information and support. Patient priority questions included: “how did the researchers study my sample and interpret the results?”, “what are the strengths and limitations of my test results?”, “how might these results affect my care?” and “what should I do with my results?”.

Participants

Patients diagnosed with CHH/KS and parents of adolescents/young adults with CHH/KS evaluated patient-facing materials. Adult participants (18 years and older) provided electronic opt-in consent prior to participation.

Instrumentation

Prior to patient evaluation, the final version of the patient-facing materials was assessed for readability. Given that there is no accept “gold standard” algorithm to assess readability, we utilized multiple validated algorithms (n=8). Briefly, the algorithms calculate the number of difficult words/sentences to provide an estimated age range and grade reading level. We employed the Flesch Reading Ease Formula, Gunning Fox Index, Flesch Kincaid Grade Level, Coleman Liau Index, Simple Measure of Gobbledygook index (SMOG), Automated Readability Index, and the Linsear Write Formula as previously reported (COST action BM1105. et al., 2017). Patients/parents evaluting the patient-facing materials provided sociodemographic information, a measure of health literacy (Chew et al., 2008) and completed the Patient Education Materials Assessment Tool for print materials (PEMAT-P). The PEMAT-P is a 17 item validated instrument developed by the U.S. Department of Health & Human Services Agency for Health Research & Quality to evaluate print/audiovisual educational materials (Shoemaker et al., 2014). After reviewing the materials, participants select agree/disagree/not-applicable to items assessing six domains relating to understandability (ability to process key messages - i.e. content, style, use of numbers, organization, design and use of visual aids) and a domain on actionably (ability to identify steps one can take in response to presented information). The Patient Group Leader followed the PEMAT-P User Guide to review items and removed questions that were not applicable to the patient-facing materials. As such, the “not applicable” option was not given as an option. Cumulative scores are expressed as a percentage (total score/possible total X 100). Psychometric evaluation demonstrates strong internal consistency, good reliability, and evidence of construct validity (Shoemaker et al., 2014).

Procedures

To evaluate patient-facing materials, we partnered with a patient organization and used a web-based approach to reach geographically dispersed rare disease patients (Dwyer, Quinton, Morin, & Pitteloud, 2014). After providing electronic opt-in consent, participants viewed a pdf of the patient-facing materials and completed a Qualtrics™ survey (i.e. sociodemographic information, health literacy and PEMAT-P).

Data analysis

Some participants consented to participate and provided sociodemographic information - yet did not complete the evaluation of the patient-facing materials. Characteristics of participants completing the survey were compared to those who did not complete the survey using Student’s T-tests, Mann-Whitney rank sum test and Chi square as appropriate. Pearson correlation coefficients were calculated to identify relationships between education level and health literacy. PEMAT-P results are descriptively reported and PEMAT-P score ≥80% on a given parameter was deemed to represent high quality. Student’s T-tests were used to compare PEMAT-P scores between groups according to educational attainment (i.e. less than college education vs. college education or greater) and health literacy (i.e. low vs. adequate) respectively. As an exploratory measure, we employed one-way ANOVA with Tukey’s HSD post hoc test to examine relationships between educational attainment (i.e. elementary, high school/vocational, college, post-graduate) and specific PEMAT-P dimensions. A p value <0.05 was considered statistically significant.

RESULTS

Material development and readability

The design thinking process involving key stakeholders generated an initial draft (a 3-page patient-facing document) that subsequently underwent three iterative revisions. The Flesch Reading Ease scores range from 0-100 with lower scores indicating easier readability. Patient-facing materials were rated 61.4/100 (equivalent to U.S. 8-9th grade level, i.e. approximately 13-15 years-old). Other algorithm scores ranged from 7th to 11th grade level (Linsear Write Formula = grade 7.1, Automated Readability Index = grade 7.5, Flesch-Kincaid Grade level = 7.7, SMOG = grade 8.5, Coleman-Liau Index = grade 11, Gunning Fog = grade 11.9). The readability consensus of the eight algorithms rated the patient-facing materials at a “standard/average” reading level - compatible with U.S. 8th-9th grade readers (i.e. 13-15 years-old).

Participant characteristics

The online survey had 103 “hits” from patients and parents. Fifteen individuals (15/103, 15%) were “one click” participants who opened the survey then closed it without completing a question. Roughly one quarter of participants (27/103, 26%) provided some responses (i.e. sociodemographics) without completing the entire survey. In total, more than half of participants (61/103 (59%) completed the online evaluation of co-created patient-facing materials (Table 1). Self-reported sex of respondents (64% male) closely corresponded with the observed male predominance of CHH/KS (i.e. 3.6 males for each female diagnosed) (Maione et al., 2018). Overall, participants were well educated. Only 10/61 (15%) reported having less than a college education. Similarly, the majority of participants (53/61, 87%) exhibited adequate health literacy. Comparing those who completed the survey with participants who did not (n=27), revealed similar sex ratio (M:F), educational attainment and age at diagnosis. However, participants who completed the survey were older (44.9±12.4 vs. 38.0±10.9 years, p<0.05) and were more likely to have adequate health literacy (87% vs. 67%, p<0.05).

Table 1.

Participant characteristics (n=61)

Participants n (%)
 parents 4 (7%)
 patients 57 (93%)
 age (yrs) at diagnosis (mean ± SD) 20.7 ± 7.2
Sex
 male 39 (64%)
 female 22 (36%)
Age (yrs)
 range 20-74
 mean ± SD 44.9 ± 12.4
 median 21
Education
 less than high school 2 (3.3%)
 high school/vocational 6 (9.8%)
 college/university 35 (57.4%)
 post-gradutate 18 (29.5%)
Health literacy
 adequate 53 (86.9%)
 inadequate 8 (13.1%)
Open in a new tab

Understandability and readability

After reviewing a pdf document of the patient-facing materials, participants completed the PEMAT-P evaluation. Participants had positive views of the overall content understandability and actionability (Figure 1). Approval ratings were high across aspects of understandability (content, organization, visual aids, use of numbers, style and design) (range: 89-97%). Similarly, participants gave the patient-facing materials overall high marks for actionability (90%) indicating that they understood how they could use the information in relation to their health. Notably, all individual items received scores well above the pre-determined 80% threshold for high quality materials (Table 2).

Figure 1.

Figure 1.

Open in a new tab

Patient Education Materials Assessment Tool for printed materials (PEMAT-P) ratings of understandability and actionability (n=95). The PEMAT-P includes 17 items within seven domains. Six domains relate to understandability and the other is actionability. Approval ratings are depicted by the shaded circle areas and percent approval is noted in the center.

Table 2.

PEMAT Understandability and actionability of co-created materials (n = 61)

PEMAT topic/category % agree
Content
Makes its purpose completely evident. 95.1%
Does not include information/content that distracts from its purpose. 90.2%
Word choice & style
Uses common, everyday language. 86.9%
Medical terms are used only to familiarize the audience with the terms. 93.4%
Medical terms are defined (when used). 95.1%
Use of numbers
Numbers appearing are clear and easy to understand. 91.8%
Does not expect the user to perform calculations. 96.7%
Organization
“Chunks” information into short sections. 95.1%
Sections have informative headers. 88.5%
Presents information in a logical sequence. 98.4%
Provides a summary. 88.5%
Layout & design
Uses visual cues to draw attention to key points. 96.7%
Visual aids*
Uses visual aids whenever they could make content more easily understood. 90.2%
Visual aids reinforce rather than distract from the content. 86.9%
Visual aids have clear titles or captions. 90.2%
Uses illustrations and photographs that are clear and uncluttered. 86.9%
Actionability*
Clearly identifies at least one action the user can take. 90.2%
Addresses the user directly when describing actions. 90.2%
Breaks down any action into manageable, explicit steps. 88.5%
Uses visual aids whenever they could make it easier to act on the instructions. 85.2%
Open in a new tab
*

Three respondents did not complete ratings for Visual Aids and Actionability

Educational attainment among respondents was not correlated with health literacy (p=0.08). To assess the role of educational attainment, we collapsed participants into two groups (less than college vs. college education or greater). PEMAT-P scores did not differ between education groups (p=0.28). To assess the role of health literacy we collapsed the respondents into two groups (adequate vs. low health literacy) (Chew et al., 2008). Respondents with adequate health literacy rated the ‘organization’ of materials more favorably (p<0.05). In contrast, respondents with low health literacy gave more favorable ratings to ‘visual aids’ (p<0.01). As an exploratory step, we examined PEMAT-P scores in relation to individual educational attainment. Those respondents with only an elementary school-level education had significantly lower PEMAT-P ratings across all domains. However, these results should be interpreted with great caution, as there were few participants in this group.

DISCUSSION

Herein we describe the development process used to develop patient-facing genetics information and results from patient evaluation of the co-created materials. The aim of this project was to produce patient-facing materials to engage research participants, help them better understand their genetic test results and appreciate how results could be used for health decisions. Our interprofessional team (including patients) drew on health literacy principles and used a design thinking (Roberts, Fisher, Trowbridge, & Bent, 2016) to develop innovative patient-centered materials to overcome literacy and numeracy barriers and support patient activation and agency (i.e. actionability). The co-created patient-facing materials were readable (8th-9th grade reading level) and received high marks for understandability and actionability (PEMAT-P scores ranging from 85-96%).

Parallel to the growth of genomic healthcare is the challenge of mounting health disparities (NASEM, 2018). Limited literacy and numeracy skills contribute significantly to genomic healthcare disparities (Crawford et al., 2019; Haga et al., 2013; J. R. Williams et al., 2018). We observed the lowest approval rating for visual aids (89%) suggesting that optimal modes for visually conveying genetic concepts (i.e. sensitivity, specificity, risk) merit further investigation. This is a salient point as even college educated individuals have limited genetic numeracy skills (Bergman et al., 2017). Further, lay public familiarity and understanding of genetic terms and concepts remains challenging despite efforts to simplify language (NASEM, 2018). We also found that greater educational attainment was associated with higher ratings of organization of the material and that respondents with less education gave higher ratings for visual aids. Together, these data underscore the challenge of making genetics/genomics universally accessible given the specialized, complex nature of genetic terminology (NASEM, 2018).

Study Limitations

This study has several limitations. First, the sample size is relatively small. However, given the rarity of CHH/KS (approximately 1 in 48,000) our prospectively recruited sample is quite sizeable - highlighting the importance of partnering with patient organizations in rare disease research (Dwyer et al., 2014). Despite overall high ratings, one participant was a relative outlier and gave no positive approval ratings for any PEMAT-P items. So while the study findings are encouraging, more work can be done to effectively communicate with all patients. We provided participants with instructions for completing the survey based on the PEMAT-P User Guide. Survey respondents did not read the PEMAT-P User Guide and this could have affected the results. A limitation is that we used a validated, brief measure of health literacy rather than lengthier versions with greater sensitivity and specificity for identifying inadequate health literacy (Chew et al., 2008; Wallace, Rogers, Roskos, Holiday, & Weiss, 2006). Another limitation is that participants were highly educated and most had adequate health literacy thus limiting generalizability. Patients with low health literacy may struggle to understand the material due to the 8th-9th grade reading level. Indeed, it is worthwhile to note that those participants who did not complete the survey were younger and more likely to have low health literacy. Patients and families involved in patient support groups may not be representative of the broader patient population. We did not formally assess patient comprehension, so we are unable to know how well patients grasped the information presented. Similarly, we are unable to make any statement regarding whether or not the materials affect informed decision-making.

Practice Implications

Existing data on genomic health disparities (NASEM, 2018) point to a need for new models and approaches to care to help patients understand genetic/genomic concepts. Comprehension is one essential aspect for empowering patients to use information to make high quality decisions about their health that are both informed and aligned with values and preferences (Stoll, Kubendran, & Cohen, 2018). Using design thinking and patient engagement produced high quality materials that were understandable and acceptable to patients and parents. Thus, co-creating patient-facing materials may be one way to address gaps in genetic literacy/numeracy and help surmount genomic health disparities.

Research Recommendations

Currently, there is relatively little evidence on patient-facing materials in genomics (J. L. Williams et al., 2018). Future work may examine the best approaches for developing patient-facing materials and optimal modes for communicating complex genetic information like sensitivity, specificity and risk. While our results are promising, patient facing materials generated by a similar approach should be examined in a broader dataset (and other disease states – i.e. more prevalent, more lethal) to validate the current study findings. Future work could involve a design specialist in the interprofessional team and utilize a pre-post design to compare patient-facing materials created without design thinking and then following the iterative design process.

CONCLUSIONS

In summary, we provide empirical evidence supporting the notion that involving patients in a co-creating process can help produce high quality patient-facing materials. Design thinking (i.e. human-centered design) has received relatively limited attention in developing health interventions (Rousseau et al., 2019). Results from the current study support the utility of interprofessional teams using design thinking (i.e. human-centered design) in developing patient-centered interventions. Findings from the present study suggest that patient-human-centered design appears to parallel person-centered care. This study may be applicable to wide-ranging genetic conditions and could serve as a roadmap for others interested in using patient engagement and co-creation to develop more patient-centered approaches to genetic testing (Hiort et al., 2019).

Supplementary Material

supinfo

ACKNOWLEDGEMENTS

We wish to recognize the generous participation of patients and families in this study. This work was supported by the Harvard Reproductive Sciences Center (grant P50 HD028138, to S.B.S) from the Eunice Kennedy Shriver National Institute of Child Health and Development. Dr. Melanie Myers served as Action Editor on the manuscript review process and publication decision.

Footnotes

CONFLICT OF INTEREST STATEMENTS

  • Andrew A. Dwyer declares that he has no conflict of interest.
  • Margaret G. Au declares that she has no conflict of interest.
  • Neil Smith declares that he has no conflict of interest.
  • Lacey Plummer declares that she has no conflict of interest.
  • Margaret F. Lippincott declares that she has no conflict of interest.
  • Ravikumar Balasubramanian declares that he has no conflict of interest.
  • Stephanie B. Seminara declares that she has no conflict of interest.

HUMAN STUDIES AND INFORMED CONSENT

This study conforms with principles of the Declaration of Helsinki and the US Federal Policy for the Protection of Human Subjects. All participants provided opt-in electronic consent prior to participation. The study was reviewed and approved by the Boston College IRB (BC-IRB). As some investigators involved in this study are employed by the Massachusetts General Hospital (Partners Healthcare), the Partners Human Research Committee (PHRC) reviewed the protocol and ceded authority to the BC-IRB. A reliance agreement between PHRC and BC-IRB has been fully executed and both institutions are in receipt of the signed documentation. The respective protocol numbers for the approved study are #2018p000620 (PHRC) and #18.081.01 (BC-IRB).

ANIMAL STUDIES

Not applicable

DATA SHARING AND DATA ACCESSIBILITY

De-identified data and the associated research findings will be made readily available upon request for research purposes to qualified individuals within the scientific community once the primary aims of the study have been published. To facilitate this, we will use the Boston College Dataverse, an open access data repository for researchers affiliated with Boston College and hosted by the Harvard Dataverse Network. The repository consists of datasets produced by the Boston College community, available for archiving, public access and re-use. Each dataset includes citation information and a Digital Object Identifier (DOI), facilitating attribution, usage tracking and linking of data to research publications.

ONLINE SUPPLEMENTARY INFORMATION

PDF of co-created patient facing materials.

REFERENCES

  1. Allen CG, McBride CM, Balcazar HG, & Kaphingst KA (2016). Community Health Workers: An Untapped Resource to Promote Genomic Literacy. J Health Commun, 21(sup2), 25–29. doi: 10.1080/10810730.2016.1196272 [DOI] [PubMed] [Google Scholar]
  2. Bergman MW, Goodson P, & Goltz HH (2017). Exploring Genetic Numeracy Skills in a Sample of U.S. University Students. Front Public Health, 5, 229. doi: 10.3389/fpubh.2017.00229 [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. COST Action BM1105, C. A., Badiu C, Bonomi M, Borshchevsky I, Cools M, Craen M, … Dwyer AA. (2017). Developing and evaluating rare disease educational materials co-created by expert clinicians and patients: the paradigm of congenital hypogonadotropic hypogonadism. Orphanet J Rare Dis, 12(1), 57. doi: 10.1186/s13023-017-0608-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Bowling BV, Acra EE, Wang L, Myers MF, Dean GE, Markle GC, … Huether CA. (2008). Development and evaluation of a genetics literacy assessment instrument for undergraduates. Genetics, 178(1), 15–22. doi: 10.1534/genetics.107.079533 [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Chew LD, Griffin JM, Partin MR, Noorbaloochi S, Grill JP, Snyder A, … Vanryn M. (2008). Validation of screening questions for limited health literacy in a large VA outpatient population. J Gen Intern Med, 23(5), 561–566. doi: 10.1007/s11606-008-0520-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Crawford DC, Cooke Bailey JN, & Briggs FBS (2019). Mind the gap: resources required to receive, process and interpret research-returned whole genome data. Hum Genet, 138(7), 691–701. doi: 10.1007/s00439-019-02033-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Davis KW, Hamby Erby L, Fiallos K, Martin M, & Wassman ER (2019). A comparison of genomic laboratory reports and observations that may enhance their clinical utility for providers and patients. Mol Genet Genomic Med, 7(7), e00551. doi: 10.1002/mgg3.551 [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. DeWalt DA, Broucksou KA, Hawk V, Brach C, Hink A, Rudd R, & Callahan L (2011). Developing and testing the health literacy universal precautions toolkit. Nurs Outlook, 59(2), 85–94. doi: 10.1016/j.outlook.2010.12.002 [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Dwyer AA, Quinton R, Morin D, & Pitteloud N (2014). Identifying the unmet health needs of patients with congenital hypogonadotropic hypogonadism using a web-based needs assessment: implications for online interventions and peer-to-peer support. Orphanet J Rare Dis, 9, 83. doi: 10.1186/1750-1172-9-83 [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Haga SB, Barry WT, Mills R, Ginsburg GS, Svetkey L, Sullivan J, & Willard HF (2013). Public knowledge of and attitudes toward genetics and genetic testing. Genet Test Mol Biomarkers, 17(4), 327–335. doi: 10.1089/gtmb.2012.0350 [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Han A, & Carayannopoulos AG (2020). Readability of Patient Education Materials in Physical Medicine and Rehabilitation (PM&R): A Comparative Cross-Sectional Study. PM R, 12(4), 368–373. doi: 10.1002/pmrj.12230 [DOI] [PubMed] [Google Scholar]
  12. Hiort O, Cools M, Springer A, McElreavey K, Greenfield A, Wudy SA, … Gn, R. H. N. a. w. a. t. E. R. N. f. R. E. C. (2019). Addressing gaps in care of people with conditions affecting sex development and maturation. Nat Rev Endocrinol, 15(10), 615–622. doi: 10.1038/s41574-019-0238-y [DOI] [PubMed] [Google Scholar]
  13. Hurle B, Citrin T, Jenkins JF, Kaphingst KA, Lamb N, Roseman JE, & Bonham VL (2013). What does it mean to be genomically literate?: National Human Genome Research Institute Meeting Report. Genet Med, 15(8), 658–663. doi: 10.1038/gim.2013.14 [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Institute of Education Sciences, National Center for Education Statistics, U.S. Department of Education. (2017). Highlights of the 2017 U.S. PIAAC Results Web Report (NCES 2020-777). Retrieved from https://nces.ed.gov/surveys/piaac/current_results.asp. [Google Scholar]
  15. Kaye C, & Korf B (2013). Genetic literacy and competency. Pediatrics, 132(Suppl 3), S224–230. doi: 10.1542/peds.2013-1032G [DOI] [PubMed] [Google Scholar]
  16. Lea DH, Kaphingst KA, Bowen D, Lipkus I, & Hadley DW (2011). Communicating genetic and genomic information: health literacy and numeracy considerations. Public Health Genomics, 14(4–5), 279–289. doi: 10.1159/000294191 [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Maione L, Dwyer AA, Francou B, Guiochon-Mantel A, Binart N, Bouligand J, & Young J (2018). GENETICS IN ENDOCRINOLOGY: Genetic counseling for congenital hypogonadotropic hypogonadism and Kallmann syndrome: new challenges in the era of oligogenism and next-generation sequencing. Eur J Endocrinol, 178(3), R55–R80. doi: 10.1530/EJE-17-0749 [DOI] [PubMed] [Google Scholar]
  18. National Academies of Sciences, Engineering and Medicine (NASEM). (2018). Understanding Disparities in Access to Genomic Medicine: Proceedings of a Workshop. Washington, D.C.: National Academies Press. [PubMed] [Google Scholar]
  19. Organization for Economic Cooperation and Development (OECD). (2016). Skills Matter: Further Results from the Survey of Adult Skills, OECD Skills Studies, OECD Publishing, Paris, 10.1787/9789264258051-en. [DOI] [Google Scholar]
  20. Peterson EB, Chou WS, Gaysynsky A, Krakow M, Elrick A, Khoury MJ, & Kaphingst KA (2018). Communication of cancer-related genetic and genomic information: A landscape analysis of reviews. Transl Behav Med, 8(1), 59–70. doi: 10.1093/tbm/ibx063 [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Regan M, Engler MB, Coleman B, Daack-Hirsch S, & Calzone KA (2019). Establishing the Genomic Knowledge Matrix for Nursing Science. J Nurs Scholarsh, 51(1), 50–57. doi: 10.1111/jnu.12427 [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Roberts JP, Fisher TR, Trowbridge MJ, & Bent C (2016). A design thinking framework for healthcare management and innovation. Healthc (Amst), 4(1), 11–14. doi: 10.1016/j.hjdsi.2015.12.002 [DOI] [PubMed] [Google Scholar]
  23. Rousseau N, Turner KM, Duncan E, O'Cathain A, Croot L, Yardley L, & Hoddinott P (2019). Attending to design when developing complex health interventions: A qualitative interview study with intervention developers and associated stakeholders. PLoS One, 14(10), e0223615. doi: 10.1371/journal.pone.0223615 [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Shoemaker SJ, Wolf MS, & Brach C (2014). Development of the Patient Education Materials Assessment Tool (PEMAT): a new measure of understandability and actionability for print and audiovisual patient information. Patient Educ Couns, 96(3), 395–403. doi: 10.1016/j.pec.2014.05.027 [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Stoll K, Kubendran S, & Cohen SA (2018). The past, present and future of service delivery in genetic counseling: Keeping up in the era of precision medicine. Am J Med Genet C Semin Med Genet, 178(1), 24–37. doi: 10.1002/ajmg.c.31602 [DOI] [PubMed] [Google Scholar]
  26. Wallace LS, Rogers ES, Roskos SE, Holiday DB, & Weiss BD (2006). Brief report: screening items to identify patients with limited health literacy skills. J Gen Intern Med, 21(8), 874–877. doi: 10.1111/j.1525-1497.2006.00532.x [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Williams JL, Rahm AK, Zallen DT, Stuckey H, Fultz K, Fan AL, … Williams MS. (2018). Impact of a Patient-Facing Enhanced Genomic Results Report to Improve Understanding, Engagement, and Communication. J Genet Couns, 27(2), 358–369. doi: 10.1007/s10897-017-0176-6 [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Williams JR, Yeh VM, Bruce MA, Szetela C, Ukoli F, Wilkins CH, & Kripalani S (2018). Precision Medicine: Familiarity, Perceived Health Drivers, and Genetic Testing Considerations Across Health Literacy Levels in a Diverse Sample. J Genet Couns. doi: 10.1007/s10897-018-0291-z [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Wynn RM, Adams KT, Kowalski RL, Shivega WG, Ratwani RM, & Miller KE (2018). The Patient in Precision Medicine: A Systematic Review Examining Evaluations of Patient-Facing Materials. J Healthc Eng, 2018, 9541621. doi: 10.1155/2018/9541621 [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

supinfo
NIHMS1718736-supplement-supinfo.pdf (238.1KB, pdf)

RESOURCES