The NYCKidSeq randomized controlled trial: Impact of GUÍA digitally enhanced genetic results disclosure in diverse families

Summary

Digital solutions are needed to support rapid increases in the application of genetic/genomic tests (GTs) in diverse clinical settings and patient populations. We developed GUÍA, a bilingual digital application that facilitates disclosure of GT results. The NYCKidSeq randomized controlled trial enrolled diverse children with neurologic, cardiac, and immunologic conditions who underwent GTs. The trial evaluated GUÍA’s impact on understanding the GT results by randomizing families to results disclosure genetic counseling with GUÍA (intervention) or standard of care (SOC). Parents/legal guardians (participants) completed surveys at baseline, post-results disclosure, and 6 months later. Survey measures assessed the primary study outcomes of participants' perceived understanding of and confidence in explaining their child’s GT results and the secondary outcome of objective understanding. The analysis included 551 diverse participants, 270 in the GUÍA arm and 281 in SOC. Participants in the GUÍA arm had significantly higher perceived understanding post-results (OR = 2.8, CI[1.004, 7.617], p = 0.049) and maintained higher objective understanding over time (OR = 1.1, CI[1.004, 1.127], p = 0.038) compared to SOC. There was no impact on perceived confidence. Hispanic/Latino(a) individuals in the GUÍA arm maintained higher perceived understanding (OR = 3.9, CI[1.603, 9.254], p = 0.003), confidence (OR = 2.7, CI[1.021, 7.277], p = 0.046), and objective understanding (OR = 1.1, CI[1.009, 1.212], p = 0.032) compared to SOC. This trial demonstrates that GUÍA positively impacts understanding of GT results in diverse parents of children with suspected genetic conditions and builds a case for utilizing GUÍA to deliver complex results. Continued development and evaluation of digital applications in diverse populations are critical for equitably scaling GT offerings in specialty clinics.

Graphical abstract

GUÍA, a digital application designed to enhance the delivery of genetic/genomic test results, was evaluated through the NYCKidSeq randomized controlled trial. Findings from the trial demonstrate that GUÍA positively impacts perceived and objective understanding of genetic results in diverse, multilingual parents/guardians of children with suspected genetic conditions who underwent testing.

Introduction

Rapid advancements in genome sequencing over the last two decades have resulted in an increasing number of patients receiving results from genetic and genomic tests (GTs) in various clinical settings. This growing availability of GTs, coupled with technological innovation in digital technology and artificial intelligence (AI), has resulted in an increasing convergence between healthcare and technology toward enhancing and scaling genomic medicine. A vital component of genomic medicine is the communication of GT results to patients and their families, which is usually carried out by genetic counselors via in-person or telehealth counseling. This type of effective, patient-centered communication is essential for patients to understand their results, and across medical research, studies have found that understanding test results can positively impact adherence to medical follow up, overall satisfaction, and psychological adaptation.^1,2 While the positive impact of genetic counseling on patients’ understanding of their GT results is well established,³ relying solely on current practices as genome medicine becomes more prevalent in health systems is likely impractical given an insufficient genetics workforce and rapidly expanding patient populations receiving GT results.⁴

Digital solutions are emerging as valuable tools for expanding access and supporting the GT process.^5,6 For example, decision support tools use web- or app-based interfaces to help patients and families decide whether or not to undergo GT,^7,8 make decisions about medical management related to their genomic risk,⁹ and choose whether to receive secondary findings from genome sequencing.¹⁰ Patients can access self-guided educational material and modules,^11,12,13 which help to streamline pre-test counseling. Chatbots are also helpful in facilitating the genetic counseling and testing process by providing an AI-driven mechanism for collecting family history, delivering education, performing risk assessments, and helping patients share results with family members.^14,15,16,17 Typically, digital genetic counseling tools address the more routine aspects of the pre-test counseling workflow, such as family history collection and education, since these are often simpler to standardize. Far fewer applications are focused on communicating GT results (post-test counseling).¹⁸ Communicating GT results to patients can be challenging due to the complexity of the results, the degree of technical language used in clinical GT reports, the potential for misinterpretation, and the range of result types from benign to uncertain to pathogenic.^19,20 Digital tools could potentially improve the communication of complex genetic results with features such as digital post-test educational modules, personalization, and visualization of results and medical recommendations. Additionally, digital tools could be designed to support patients with low health literacy or limited English proficiency to understand and potentially act on highly technical GT information.

To investigate whether digital tools designed for results disclosure could improve the understanding of GT results, we created a web-based application called GUÍA (Genomic Understanding, Information, and Awareness application), which has been previously described.²¹ GUÍA, which means “guide” in Spanish, is designed to facilitate the communication of GT results to patients and families, with a particular focus on supporting diverse and multilingual families. GUÍA encompasses relevant aspects of a results disclosure genetic counseling session, including genetic education, primary (related to indication for testing) and secondary (unrelated to the indication for testing) results disclosure, clinical implications, inheritance and family implications, medical recommendations, and patient resources. In GUÍA’s development, we aimed to enhance understanding and reduce information overload by (1) minimizing unnecessary text, including text written in the active voice and at the most accessible reading level; (2) incorporating textbook-inspired illustrations to explain complex concepts; (3) adopting a modular design to break up content and allow individuals to control how they access information; and (4) making the application accessible in English and Spanish. Community stakeholders provided input during development, which was critical to ensuring the application was culturally appropriate and resonated with the target population. Healthcare providers can customize the content for each patient by inputting the genetic results and other pertinent information into the software. This customization allows relevant clinical information from the genetic test report to be displayed in Spanish, which is notable considering clinical reports are typically only available in English. Additionally, the Spanish content was carefully translated to ensure comprehension across a range of Spanish dialects.

Here, we report the findings from NYCKidSeq, a randomized controlled trial (RCT) jointly funded by the National Human Genome Research Institute and the National Institute on Minority Health and Health Disparities, and one of seven national clinical sites that are part of the Clinical Sequencing Evidence-Generating Research (CSER) consortium.²² The NYCKidSeq RCT was designed to compare digitally enhanced (using GUÍA) results disclosure to traditional results disclosure to diverse pediatric patients and their families recruited from two large health systems in New York City.²³ The objectives of the study were to assess the impact of GUÍA on parents’ perceived understanding of and confidence in explaining their child’s GT results (primary outcomes) and on objective understanding of the GT results (secondary outcome). Evaluating the effectiveness of GUÍA in NYCKidSeq sheds light on the acceptability and utility of using this digital application to support the return of GT results to diverse families of pediatric patients.

Material and methods

Study design

The NYCKidSeq study design, described previously in Odgis et al.,²³ is an RCT (Clinicaltrials.gov: NCT03738098) in which pediatric children received GTs and were randomized to one of two study arms to receive either digitally enhanced (using GUÍA) results disclosure genetic counseling or traditional genetic counseling (standard of care) (see Figure 1). The Institutional Review Boards at the Icahn School of Medicine at Mount Sinai (IRB# 17–00780) and the Albert Einstein College of Medicine (IRB# 2016–6883) approved the study.

Figure 1.

Consort diagram for the NYCKidSeq randomized controlled trial

A total of n = 1,017 children were assessed for eligibility in NYCKidSeq. Of n = 888 who were eligible for enrollment, n = 38 were assigned to the Lead-In Phase and n = 622 were included in the RCT with n = 312 randomized to the GUÍA arm and n = 310 to the standard of care (SOC) arm. Note, ^∗one participant in the GUÍA arm did not complete the baseline survey. Excluding the Lead-In Phase (n = 38), n = 613 provided a sample and enrolled in the RCT: n = 308 in the GUÍA arm and n = 305 in SOC. In the GUÍA arm, n = 281 completed the ROR1 survey (∼3 months from enrollment) and n = 255 completed the ROR2 survey (∼9 months from enrollment and ∼6 months post-disclosure). In the SOC arm, n = 294 completed the ROR1 survey and n = 268 completed the ROR2 survey. The final analytic sample was n = 551 (n = 270 in the GUÍA arm, n = 281 in the SOC arm) for ROR1 and n = 487 (n = 238 in the GUÍA arm, n = 249 in the SOC arm) for ROR2. RCT, randomized controlled trial; LTFU, lost to follow-up; SOC, standard of care; GC, genetic counseling; GS, genome sequencing; TGP, targeted gene panel; ROR1, results disclosure time point; ROR2, 6-month post-disclosure time point.

Study population

The study recruited children with suspected genetic conditions from two healthcare systems in New York City, Mount Sinai Health System (MS) and Albert Einstein College of Medicine/Montefiore Medical Center (EM). For each child, at least one available English- or Spanish-speaking parent or legal guardian (participant) was also recruited to complete study surveys. Children were referred to the study by their healthcare providers at MS or EM. Eligibility criteria included being ≤21 years of age at the time of enrollment and having an undiagnosed neurologic (epilepsy/seizure disorder [epilepsy] or intellectual developmental disability [IDD]/global developmental delay), cardiac (congenital heart disease, cardiomyopathy, or cardiac arrhythmia), or immunologic (features of primary immunodeficiency) phenotype suspected of having a genetic etiology. Those with a previously uninformative GT or who had undergone genetic counseling were eligible after three months had passed. Children were excluded if they had a molecular diagnosis for their primary phenotype, an obvious genetic diagnosis based on clinical features, or a bone marrow transplant. We employed a study-wide definition of diversity based on self-reported non-White/European ancestry and/or living in a Health Resources & Services Administration (HRSA) medically underserved area (https://data.hrsa.gov).²³

Recruitment, enrollment, and study visits

Participants and children were recruited and enrolled in the NYCKidSeq study and completed three study visits: baseline enrollment and survey, results disclosure and survey (ROR1), and 6-month post-disclosure survey (ROR2; see Figure 1). The study commenced on January 31, 2019 and was completed on April 28, 2022. Per the CSER consortium guidelines, we initially targeted the enrollment of 1,100 children and their families to the study. However, due to institutionally required changes in clinical research prompted by the COVID-19 pandemic, we amended the enrollment target to 650. Children referred to the study (n = 1,017) were assessed for eligibility, and parents/legal guardians of those eligible (n = 888) were approached for recruitment. Of these, 660 expressed interest in participating while 228 were either lost to follow up or declined participation. The first 38 consented children and participants were assigned to the study’s Lead-in Phase, and data collected throughout their study visits was used to improve and refine the surveys and GUÍA.²¹ Data from the Lead-in Phase is not used in any downstream analysis. The subsequent 622 children and participants were consented to the RCT and randomized to either the control or intervention arm using the randomization module in REDCap applied by a study staff member. The stratified randomization scheme balanced primary phenotype (neurologic, cardiac, immunologic) and institution (MS, EM) across each arm. All 622 participants completed the baseline consent and 621 completed the baseline survey, 615 completed the baseline pre-test visit, and blood and/or saliva samples were collected from 613 children and their biological parent(s), if available. These 613 children and participants were designated as being successfully enrolled in the study.

Prior to the GTs, five participants withdrew from the study. For the remaining 608 children from MS (n = 386) or EM (n = 222), clinical genome sequencing (GS) and targeted gene panel (TGP) testing were performed at the New York Genome Center and Sema4, respectively. TGPs consisted of a neurodevelopmental panel, immunodeficiency panel, and cardiovascular panel, and one or multiple panel(s) were run based on the child’s phenotype. All genetic testing was New York State approved and Clinical Laboratory Improvement Amendments (CLIA)-certified, and variant interpretation and reporting were performed independently at each of the two laboratories. Study genetic counselors (GCs) reviewed both diagnostic GT report(s) to determine a case-level (clinical) interpretation of the results. Each child was assigned a case-level interpretation of positive, likely positive, uncertain, or negative. More details on diagnostic GT and case-level interpretation in NYCKidSeq are described in Abul-Husn et al.²⁴

Twenty-four participants were lost to follow up prior to results disclosure. We disclosed results to 584 participants; 575 completed the ROR1 survey, and 523 completed the ROR2 survey. Overall, 85.3% (523/613) of participants enrolled in the RCT completed the study. Families received an incentive of up to $80 for completing the study visits. We obtained written informed consent from children >18 years who were cognitively able, from parents/legal guardians of children <18 years or who were unable to consent themselves, and from parents/legal guardians providing samples and/or completing the study surveys. Further details on recruitment, enrollment, and inclusion/exclusion criteria are described in Odgis et al.²³

Genetic counseling in the standard of care (control) and GUÍA (intervention) arms

Since standard of care (SOC) genetic counseling is not well defined,²⁵ we modeled SOC genetic counseling on clinical pediatric genetics settings for this study. Pre-test counseling, which included education, informed consent prior to the GTs, and family history assessments, was conducted the same way for both arms. The components of the post-test results disclosure session included reviewing the purpose of the GTs, disclosure of the GT results, clinical implications of the results, including medical and support referrals when appropriate, disclosure of any secondary findings, family implications, and the provision of relevant resources. GCs across both study arms did not speak Spanish and used a medical interpreter in disclosure sessions with Spanish-speaking participants (19.7%; 115/584). Pre- and post-test genetic counseling visits occurred in person until the COVID-19 pandemic in March 2020, which prompted a rapid transition to telehealth. In total, 27.4% (n = 160) of results were returned in person and 72.6% (n = 424) were returned via telehealth. GCs in the SOC arm utilized visual aids at their discretion, which were displayed to the family via screen sharing during telehealth visits. The GCs provided families with a copy of the child’s GT results, and families could be given additional resources, educational handouts, and/or results summary materials at the GC’s discretion.

In the intervention arm, GCs mimicked SOC and utilized GUÍA for all aspects of the results disclosure. Prior to the visit, GCs entered the child’s results and pertinent result-related information into GUÍA’s purpose-built, web-based GC form, which contained fields for primary and secondary results that included variants and genes, inheritance pattern, description of the condition, affected organ and/or systems, and hyperlinks to resources. The form also contained fields for report personalization, including child’s name, pronouns, and preferred language. GUÍA’s patient-facing, personalized web pages were then generated, containing digital education modules, the child’s results, family implications, and details on medical recommendations and next steps. Prior to the result disclosure visit, the GC sent a copy of the GUÍA report to the referring provider for their review and input. Participants and GCs worked together to navigate the different sections of the GUÍA display during the session, which was shown on an iPad/computer monitor for in-person counseling or via screen-sharing for telehealth visits. GCs displayed the Spanish version of GUÍA at their discretion (35.1%; 100/285). After the session, a PDF or printout of the full content of GUÍA was provided to the family as well as a copy of the child’s GT results. For both arms, GCs documented the results disclosure visit and uploaded the GT results to the child’s electronic health record (EHR). A PDF of the GUÍA report was also uploaded to the EHR for those in the intervention arm. All visit documentation was routed to the referring provider.

Outcome measures and surveys

Outcomes, demographics, and other participant and child characteristics were collected using surveys administered by trained study staff at baseline, ROR1, and ROR2. The primary outcomes of the RCT are participants’ perceived understanding of their child’s GT results and their perceived confidence in explaining the results to others in the intervention arm vs. the control arm at ROR1 and ROR2. We assessed the primary outcomes using two questions developed specifically for this study. The first asked participants to rate their understanding of the GT results on a scale of 1 (“very little or none of it”) to 5 (“almost all or all of it”), and the second asked them to rate their confidence in explaining their child’s genetic test results to someone else on a scale of 1 (“not confident at all”) to 5 (“completely confident”) (see Table S1).

The secondary outcome is participants’ objective understanding of their child’s GT results in the intervention arm vs. the control arm at ROR1 and ROR2. Objective understanding was measured using four questions asked of participants, which were developed specifically for this study. After completing the results disclosure visit, GCs also answered these questions (see Table S1). We calculated the agreement between GCs and participants by matching the participant’s responses to the GC’s. We did not evaluate the additional secondary outcomes of participants’ understanding of the actionability of the results and adherence to medical follow-up recommendations due to inconsistencies in the administration of the survey questions and differences in participants’ interpretation and responses to the survey questions (see supplemental methods). Table S1 contains all outcome measures and survey time points.

We collected information on a wide range of characteristics of participants and children, including age, biological sex, education, type of insurance, household income, and health literacy, using survey instruments that were developed specifically for the study or adapted from previously validated scales, many of which were harmonized across the CSER consortium.^23,26 Participant race and ethnicity were collected using the CSER harmonized self-reported measure that originated from the 2020 Census recommendation.²⁶ This single-item measure combined race and ethnicity, allowing selection from nine choices: seven racial and/or ethnic groups, prefer not to answer, or unknown/none of these describe me. For this analysis, we defined population groups as follows: Hispanic/Latino(a) (H/L) ethnicity was prioritized such that participants who selected H/L were re-categorized as H/L regardless of any other race designation made. Participants that selected more than one race were re-categorized into “More than one race.” All other population groups remained if they were the only selection made by the participant. We also collected information on cohort characteristics and activities during study visits, including the language the survey was administered in (language), health system, mode of result disclosure, and the case-level interpretation of the results (see Table S2). Study staff input all survey measurements into a REDCap database.

Analysis

The primary analytic population consists of randomized participants who completed the baseline survey, attended the results disclosure ROR1 visit, and completed the ROR1 survey within four weeks of results disclosure (n = 551). An ROR2 analytic sample used the same criteria above, plus completing the ROR2 survey and not receiving an amended result that changed the clinical interpretation between ROR1 and ROR2 surveys (n = 487; see supplemental methods). We used descriptive statistics to explore outcome frequencies of baseline, ROR1, and ROR2 survey data. We assessed equivalency between the two arms using the chi-square or Fisher’s exact tests for categorical variables and the Wilcoxon rank-sum or t tests for continuous variables. The following covariates were included in all models: parent age, education level, language, child’s insurance, and case-level interpretation. Additionally, GC and health system covariates were associated with the primary outcome of perceived understanding. As these variables were found to be highly correlated, only GC was included as a covariate in the models for the primary outcomes. Health system was used instead of GC as a covariate in the analysis of the secondary outcome since GC responses were used to analyze the survey measure.

We used ordinal logistic regression to evaluate between arm differences for perceived understanding and confidence (primary outcomes) and logistic regression for objective understanding (secondary outcome). We generated an objective understanding summary score by summing the number of questions with an agreement between the GC and participant for the four binary objective understanding questions. The summary score ranged from 0 to 4, with a higher number indicating better objective understanding.

We performed longitudinal (repeated measure) analysis using ROR1 and ROR2 data to calculate the cumulative odds ratios via generalized estimating equations. We analyzed the primary outcomes using alternating logistic regression with a fully exchangeable working correlation structure. For the secondary outcome, we used a binomial distribution with a logit link for the individual objective understanding questions and a Poisson distribution with a log link for the summary score. We repeated the longitudinal analysis, stratified by health literacy levels (adequate [n = 351] and inadequate/marginal [n = 200]), case-level interpretation (positive/likely positive [n = 91], uncertain [n = 317], and negative [n = 143]), the three largest neurological phenotypes (IDD [n = 175], epilepsy [n = 149], and both [n = 177]), the three largest population groups (H/L [n = 246], Black/African American [AA; n = 84], and White/European American [EA; n = 130]), poverty level (at or below 200% of the New York City federal poverty level [n = 230] and above [n = 258]) and interpreter use (with interpreter [n = 107] and without interpreter [n = 444]). The H/L group (n = 246) was further stratified by the use of an interpreter at results disclosure (with interpreter [n = 104] and without interpreter [n = 142]) and by health literacy level (adequate [n = 138] and inadequate/marginal [n = 108]). All statistical analyses were performed using SAS software version 9.4 (SAS, Cary, North Carolina).

Results

Participant characteristics

Of the n = 608 enrolled families who underwent GTs, n = 551 parents/legal guardians (participants) completed both the baseline and ROR1 survey, attended the results disclosure visit, and were thus included in the analysis. Table 1 displays the characteristics of all n = 551 participants, with n = 281 in the SOC arm and n = 270 in the GUÍA arm. Participants were asked which racial and/or ethnic category(ies) best described them, and responses were transformed into population groups, prioritizing H/L ethnicity, as described in the materials and methods section. Population groups were only available for biological parents (n = 531), where 46.3% self-reported as H/L, 24.5% as EA, 15.8% as AA, 7.2% Asian, 1.5% Middle Eastern/North African/Mediterranean, 0.4% American Indian/Native American/Alaska Native, 1.3% more than one population, 0.7% other, 1.1% unknown/none of these describe me, and 1.1% preferred not to answer. Participants’ mean age was 41.1 years (range 20.9–81.5), 88.6% were mothers, and 62.3% reported having less than a college degree. 53.9% of participants lived in a medically underserved area, 41.8% were at or below 200% of the New York City federal poverty level, and 64.8% of children had public insurance. Additionally, 36.3% of participants had inadequate or marginal health literacy, and 19.4% used a medical interpreter during results disclosure. Most children (90.9%) had a primary neurological phenotype. 12.9% of children received positive results, 3.6% received likely positive results, 25.9% negative, and 57.5% uncertain. There was equivalence across study arms for all baseline covariates (Table 1) except case-level interpretation (p = 0.003); the SOC arm included more children with positive and negative results, while there were more children with uncertain results in the GUÍA arm. All analytical models included case-level interpretation as a covariate to control for this variance.

Table 1.

Participant and child characteristics for the total sample and by study arm

Participant characteristic, N (%)	Total sample (n = 551)	Standard of care (n = 281)	GUÍA (n = 270)	p valuea
Age (mean, SD, range)b	41.1 (9.1) 20.9–81.5	41.3 (8.8) 20.9–69.3	40.9 (9.3) 23.2–81.5	0.64
Relationship to child
Mother	488 (88.6)	249 (88.6)	239 (88.9)	1.00
Father	43 (7.8)	22 (7.8)	21 (7.8)
Legal Guardian	20 (3.6)	10 (3.6)	10 (3.3)
Health system
Mount Sinai Health System	364 (66.1)	185 (65.8)	179 (66.3)	0.90
Montefiore Medical Center	187 (33.9)	96 (34.2)	91 (33.7)
Previous genetics testing	222 (40.3)	117 (41.6)	105 (38.9)	0.23d
Population groups (N=531)c
American Indian/Native American/Alaska Native	2 (0.4)	2 (0.7)	0 (0.0)	0.60e
Asian	38 (7.2)	18 (6.6)	20 (7.7)
Black/African American	84 (15.8)	40 (14.8)	44 (16.9)
Hispanic/Latino(a)	246 (46.3)	132 (48.7)	114 (43.8)
Middle Eastern/North African/Mediterranean	8 (1.5)	3 (1.1)	5 (1.9)
White/European American	130 (24.5)	66 (24.4)	64 (24.6)
More than one population	7 (1.3)	3 (1.1)	4 (1.5)
Other	4 (0.7)	1 (0.4)	3 (1.2)
Prefer not to answer	6 (1.1)	5 (1.8)	1 (0.4)
Unknown/None of these fully describe me	6 (1.1)	1 (0.4)	5 (1.9)
Survey conducted in Spanish	121 (22.0)	64 (22.8)	57 (21.1)	0.64
Interpreter present at result disclosure	107 (19.4)	56 (19.9)	51 (18.9)	0.76
Mode of result disclosure
In person	155 (28.1)	80 (28.5)	75 (27.8)	0.93
Telehealth	396 (71.9)	201 (71.5)	195 (72.2)
Education level (N=549)
< High school graduate	100 (18.2)	59 (21.2)	41 (15.2)	0.26
HS grad, GED, technical school, associate degree	242 (44.1)	114 (40.9)	128 (47.1)
College graduate	114 (20.8)	58 (20.8)	56 (20.7)
>College graduate	93 (16.9)	48 (17.2)	45 (16.7)
Public insurance (Medicaid)	357 (64.8)	187 (65.6)	170 (63.0)	0.38
MUA (residence in a HRSA defined “medically underserved area”)	297 (53.9)	155 (55.2)	142 (52.6)	0.55
200% below NYC federal poverty levelf
No	258 (46.8)	133 (47.3)	125 (46.3)	0.70
Yes	230 (41.8)	119 (42.4)	111 (41.1)
Brief health literacy score
Inadequate	88 (16.0)	53 (18.9)	35 (13.0)	0.16
Marginal	112 (20.3)	54 (19.2)	58 (21.5)
Adequate	351 (63.7)	174 (61.9)	177 (65.5)
Child characteristic, N (%)
Age in years (mean, SD, range)	9.7 (5.8) 0.1–22.0	9.7 (5.7) 0.1–22.0	9.8 (5.8) 0.3–21.8	0.97
Sex assigned at birth
Female	202 (36.7)	102 (36.3)	100 (37.0)	0.86
Male	349 (63.3)	179 (63.7)	170 (63.0)
Primary phenotype
Cardiac	21 (3.8)	14 (5.0)	7 (2.6)	0.34
Immunologic	29 (5.3)	15 (5.3)	14 (5.2)
Neurologic	501 (90.9)	252 (89.7)	249 (92.2)
Neurologic phenotype category
Epilepsy	149 (29.7)	71 (28.2)	78 (31.3)	0.60
IDD	175 (34.9)	93 (36.9)	82 (32.9)
Both IDD and epilepsy	177 (35.3)	88 (34.9)	89 (35.7)
Case-level interpretation of genetic test result
Positive	71 (12.9)	42 (14.9)	29 (10.7)	0.003
Likely positive	20 (3.6)	10 (3.6)	10 (3.7)
Uncertain	317 (57.5)	141 (50.2)	176 (65.2)
Negative	143 (25.9)	68 (31.3)	55 (20.4)

SD, standard deviation; HS, high school; GED, general education development; MUAP, medically underserved area; HRSA, Health Resources & Services Administration; IDD, intellectual developmental disability.

^ap value: from t test for age of parent/legal guardian and from chi-square for the categorical covariates unless otherwise indicated.

^bAge: available for n = 280 in the SOC and n = 269 in the GUÍA arms.

^cPopulation group: collected race and ethnicity for biological parents only; n = 531 in the total sample, n = 271 in SOC and n = 260 in the GUÍA arms.

^dp value: from Fisher’s Exact Test.

^ep value from chi-square derived from the 3 largest population groups (Black/African American, Hispanic/Latino(a), White/European American).

^fMissing values for n = 63 due to lack of information on household size or accurate household income range.

Participants’ overall understanding of genetic test results

We used three measures to evaluate participants’ understanding of their child’s GT results: perceived understanding, perceived confidence in explaining the results to others, and objective understanding. Overall, most participants (58.8%, n = 324) selected the highest level of perceived understanding (level 5 “almost all or all of it”) following results disclosure (ROR1), and 45.6% (n = 196) selected this level approximately 6 months after results disclosure (ROR2). Additionally, at ROR1, 45.6% (n = 251) of participants reported the highest level of confidence in their ability to explain the GT results to others, and 33.7% (n = 163) selected this level at ROR2. For objective understanding, there was over 69% agreement between the participant and GC for each question at ROR1 and ROR2. The mean objective understanding summary score (range 0–4) for the cohort was 2.9 (standard deviation [SD] = 1.2) at ROR1 and 2.8 (SD = 1.3) at ROR2. We evaluated which covariates of those we measured impacted overall understanding of GT results. Education level, insurance type, population group, GC, health system, case-level interpretation, and survey language were significantly associated with one or both primary and secondary outcomes (see Table 2) and were included as covariates in downstream analyses.

Table 2.

Association of covariates to perceived understanding, confidence, and objective understanding at ROR1

Covariate	Perceived understanding			Perceived confidence			Objective understandinga
	n	OR (95% CI) or Wald chi-squareb	p valuec	n	OR (95% CI) or Wald chi-squareb	p valuec	n	OR (95% CI) or Wald chi-squareb	p valuec
Health system: Mount Sinai vs. Montefiore Medical Center	542	1.03 (1.03, 2.06)	0.036	551	1.20 (0.87, 1.67)	0.263	551	1.03 (0.92, 1.14)	0.645
Genetic counselord: n = 8 categories	542	16.89	0.018	551	27.69	0.149	551	5.89	0.553
Agee	540	1.00 (0.98, 1.02)	0.973	549	1.01 (0.99, 1.02)	0.549	549	0.99 (0.99, 1.00)	0.341
Education level: n = 3 categories	540	13.52	0.004	549	19.74	<0.001	549	4.07	0.254
Survey language: English vs. Spanish	542	1.03 (0.69, 1.54)	0.897	551	0.93 (0.64, 1.35)	0.705	551	1.13 (1.00, 1.28)	0.048
Population groupf n = 3 categories	454	20.88	<0.001	460	8.42	0.015	460	11.32	0.004
Insurance type: Medicaid vs. Non-Medicaid	542	0.61 (0.43, 0.88)	0.007	551	0.60 (0.43, 0.83)	0.002	551	0.86 (0.78, 0.96)	0.004
Case-level interpretation of resultsg: n = 4 categories	542	3.73	0.292	551	5.47	0.141	551	30.29	<0.001

OR, odds ratio; CI, confidence interval; ROR1, results disclosure time point.

^aThe objective understanding summary score was used for the analysis.

^bWald chi-square was run for categorical variables with 3 or more categories.

^cItalics indicate p values that fall below the significance threshold of 0.05.

^dEight genetic counselors provided result disclosure genetic counseling to participants.

^eParticipant age was collected at baseline.

^fPopulation groups included the three largest population groups (Black or African American, Hispanic/Latino(a), White or European American).

^gCase-level interpretation of results included positive, likely positive, uncertain, or negative categories.

Impact of GUÍA on understanding

The primary study outcome was the impact of the GUÍA intervention on participants’ perceived understanding of and confidence in explaining the GT results up to four weeks (ROR1) and approximately 6 months (ROR2) after receiving results. We first assessed the impact of GUÍA on participants’ perceived understanding at the two time points (ROR1 and ROR2) separately. In a fully adjusted model for perceived understanding at ROR1, participants in the GUÍA arm were more likely to choose the highest level of understanding (level 5) than the lower levels compared to those in the SOC arm (odds ratio [OR] = 2.8, 95% confidence interval [CI] [1.004,7.617], p = 0.049). No significant differences were identified between the arms for perceived understanding at ROR2. Additionally, there were no significant differences between the arms for perceived confidence at ROR1 or ROR2, comparing the highest level (level 5) to the lower levels. The study’s secondary outcome was GUÍA’s ability to improve participants’ objective understanding of the GT result. The objective understanding summary score was not significantly associated with the study arms at ROR1 or ROR2.

We next assessed the impact of GUÍA on participants’ understanding over time by conducting a repeated measure analysis across the ROR1 and ROR2 time points. We did not observe a significant difference between the arms for perceived understanding and confidence (Figure 2). However, participants in the GUÍA arm maintained a higher objective understanding summary score than those in SOC (OR = 1.1, CI[1.004, 1.127], p = 0.038; Figure 2). The results from this analysis did not change when education level was removed from the model, indicating that education level is not driving the observed difference in objective understanding. Together, these findings suggest that GUÍA was effective in helping participants maintain their objective understanding of their child’s genomic test results over time; however, across the two time points, it did not impact their perceived understanding of the GT results or their confidence in explaining their child’s results to others.

Figure 2.

Forest plots displaying results of the longitudinal analyses using repeated measures for the entire cohort (ALL) and stratified by sociodemographic and clinical characteristics

The stratified analyses were conducted with six characteristics: population group, health literacy, case-level result interpretation, neurological phenotype, poverty level, and interpreter use at results disclosure. The separate panels show differences in perceived understanding, perceived confidence, and objective understanding between GUÍA and SOC arms for ALL randomized participants and stratified by characteristic subgroups. We analyzed perceived understanding and confidence using alternating logistic regression, and we used Poisson regression to analyze the objective understanding summary score. The following covariates were included in all models: parent age, education level, language, child’s insurance, and case-level interpretation. In addition, genetic counselor was added as a covariate for perceived understanding and confidence, and health system was added as a covariate for objective understanding. The p value is shown for every characteritic subgroup, and the bars around the odds ratio point designate the 95% confidence interval. SOC, standard of care; IDD, intellectual developmental disability.

Effectiveness of GUÍA across participant characteristics

To obtain additional insight into how GUÍA impacts understanding, we performed a stratified analysis of understanding over time, considering six socioeconomic and clinical characteristics: health literacy level, poverty level, interpreter use, population group, case-level interpretation, and neurological phenotype. GUÍA was associated with a positive impact on understanding for four specific subgroups (see Figure 2). Participants in the GUÍA arm whose children had a neurological phenotype of IDD had higher perceived understanding than those in SOC (OR = 3.2, CI[1.4, 7.1], p = 0.004). Participants whose children received uncertain results maintained a higher objective understanding summary score in the GUÍA arm compared to SOC (OR = 1.1, CI[1.005, 1.201], p = 0.039); however, we did not detect a significant difference for those with positive and negative results. Participants in the GUÍA arm who did not use an interpreter maintained a higher objective understanding summary score (OR = 1.1, CI[1.007, 1.143], p=0.030). Finally, we observed that H/L participants in the GUÍA arm maintained significantly higher perceived understanding (OR = 3.9, CI[1.603, 9.254], p = 0.003) and confidence (OR = 2.7, CI[1.021,7.277], p = 0.046; see Figure 2), and maintained a higher objective understanding summary score (OR = 1.1, CI[1.009, 1.212], p = 0.032; Figure 2) compared to SOC. There were no significant differences between the arms for AA and EA population groups for all three measures of understanding.

To further investigate the impact of GUÍA on understanding GT results in the H/L population, we stratified this population by use of a medical interpreter and health literacy level. H/L participants in the GUÍA arm who did not use an interpreter during results disclosure maintained higher perceived understanding and confidence (OR = 12.6, CI[3.9, 40.5], p < 0.001 and OR = 5.4, CI[1.7, 17.8], p = 0.005, respectively) and a higher objective understanding summary score (OR = 1.2, CI[1.035, 1.316], p = 0.012) (Figure 3). However, for H/L participants who used an interpreter, no significant associations with understanding were identified between the arms. H/L participants with adequate health literacy had higher perceived understanding and confidence in the GUÍA arm (OR = 7.2, CI[2.3, 22.5], p = 0.001 and OR = 6.0, CI[1.4, 26.4], p = 0.018, respectively; Figure 3). However, there were no significant differences in objective understanding between the arms for H/L participants with inadequate or marginal health literacy. These findings demonstrate that while GUÍA positively impacted H/L participants’ understanding of the results overall, it did not provide a significant benefit for the sub-groups of H/L participants who used a Spanish-speaking interpreter during the results disclosure session or had inadequate or marginal health literacy.

Figure 3.

Forest plots displaying results of the longitudinal analyses using repeated measures of the H/L population by interpreter use and health literacy level

The panels display differences in perceived understanding, perceived confidence, and objective understanding between GUÍA and SOC arms for H/L participants stratified by medical interpreter use at result disclosure and health literacy level. We analyzed perceived understanding and confidence using alternating logistic regression and Poisson regression to analyze the objective understanding summary score. The following covariates were included in all models: parent age, education level, language, child’s insurance, and case-level interpretation. In addition, genetic counselor was added as a covariate for perceived understanding and confidence, and health system was added as a covariate for objective understanding. The p value is shown for every characteritic subgroup, and the bars around the odds ratio point designate the 95% confidence interval. SOC, standard of care; H/L, Hispanic/Latino(a).

Discussion

We evaluated the impact of a digital application called GUÍA, designed to facilitate the communication of GT results, on understanding of those results in parents of children with suspected genetic conditions. Families in this study represent diverse, multilingual, and predominantly medically underserved communities in New York City. We showed that GUÍA positively impacted parents’ understanding of their child’s GT results, demonstrating a 2.8-fold increase in perceived understanding immediately following results disclosure and modestly improving their objective understanding by 1.1-fold over time. Stratified analyses provided a more nuanced discernment of GUÍA’s impact, demonstrating that GUÍA significantly increased understanding for parents whose children had a neurological phenotype of IDD and for parents whose children received uncertain GT results. We also demonstrated that the effectiveness of GUÍA in improving understanding was most significant in H/L populations, where we observed a 3- to 4-fold increase in the primary understanding measures over time.

Helping patients and their families understand their GT results is an essential aspect of genomic medicine and an important function of results disclosure genetic counseling.²⁷ Previous work has shown that patients and families can struggle to understand GT results,²⁸ and that results can be misunderstood and misinterpreted by patients, families, and providers, especially when the results are uncertain or have a complicated clinical interpretation.^19,29 This is a concern since the level of understanding people have about their results can influence downstream decision-making about medical care, family, and life planning. All these factors make understanding a critical outcome to consider when evaluating novel approaches and interventions for delivering GT results. However, understanding is a complex, multifaceted construct that is challenging to measure, and there are currently no validated or accepted survey instruments designed to assess parents’ understanding of their child’s GT results. For these reasons, we developed three measures of understanding to gain a more thorough and nuanced evaluation of parents’ understanding of the GT results. We demonstrated that GUÍA specifically improved objective understanding of uncertain results, typically the most challenging to communicate due to the complex nature of the results, limited available data on the implications of the finding, and ambiguity surrounding the clinical significance and subsequent steps in medical management.^20,30,31 We also note a significant impact on understanding in parents of children with IDD, a multifactorial disorder where the underlying etiology can be difficult to determine, suggesting GUÍA may have increased parents’ understanding of results in the context of heightened clinical complexity. In this way, we are directly measuring the effect of a digital tool on one of the essential aspects of genetic counseling encounters, which in turn may help to guide the development of digitally enhanced solutions for scaling genomic medicine.

Almost 80% of parents included in this study were from underrepresented and medically underserved populations. Because GUÍA was specifically designed with diverse communities in mind,²¹ we evaluated its utility in different population subgroups. A key finding was that GUÍA was most helpful in increasing understanding outcomes for the H/L population in our study. These results suggest that GUÍA may be a useful and beneficial tool for counseling individuals of H/L descent who currently represent almost 19% of the US population (https://minorityhealth.hhs.gov/) yet for whom research has demonstrated significant unmet needs in genomic medicine.³² Subsequent effort should draw on insights gained through this study to expand GUÍA to other population groups. To do so, similar approaches could be utilized for culturally tailoring applications, such as applying knowledge gained from qualitative interviews conducted with diverse patients and their families, adapting the application based on community stakeholders’ feedback, and incorporating additional languages.

Interestingly, GUÍA’s effectiveness in the H/L population was attenuated when a medical interpreter was used for result disclosure. Although there are cited challenges with using a medical interpreter, such as the possibility for interpreters conveying inaccurate information or lacking knowledge of the correct translations of genetic terminology,^33,34 we had designed GUÍA to address these challenges by displaying both English and Spanish text on the GUÍA interface. It is possible that the reading level of the Spanish text was still too high or that the participants could not process the written text while listening to an interpreter. Nevertheless, this approach was inadequate in overcoming obstacles associated with a language discordance between families and GCs. Addressing this barrier will be critical in the future for ensuring GUÍA is effective in linguistically diverse populations where there is already a lack of high-quality genetic counseling resources³⁵ and where only 1% of the genetic counseling profession speaks a language other than English (www.nsgc.org/policy-research-and-publications/professional-status-survey).

There are limitations to this study that should be considered. The NYCKidSeq study was ongoing during the COVID-19 pandemic, which impacted the clinical environment and changed how research was conducted. Consequently, the study shifted from in-person to telehealth genetic counseling, leading to the predominant use of telehealth for results disclosure (71.5% in SOC, 72.2% in GUÍA). This shift may have affected parents’ experiences and the outcomes in ways not measured in this study and could have disproportionally impacted one of the study arms. We did not investigate associations between the number of times participants accessed their GUÍA report and their level of understanding, which is an area for additional research. Most children had a neurologic phenotype, limiting our insight into how GUÍA will perform in other clinical contexts. We were interested in whether GUÍA affected parents’ understanding of and adherence to medical recommendations. However, these outcomes were difficult to assess given the complexity of the clinical care for enrolled children and the limitations of the CSER harmonized survey instruments to capture the full spectrum of medical care. Increasing diversity among study participants helps advance our understanding of the clinical utility of genomic medicine in all populations and ensures that genetic research is broadly applicable. However, if there are differences between groups, stratification analysis to uncover those differences reduces statistical power (although reduced environmental variability within a sub-group may also increase power). For example, stratifying by population group uncovered a significant signal in H/L (the largest group), but no significant signal in EA or AA, which could be due to insufficient power in those groups. Also, we excluded several population groups (e.g., Asian) from the stratified analysis due to low sample size. In general, broadening diversity in research studies teaches both about generalizable and specific effects; however, the latter may need to be followed up with further, well-powered studies.

Conclusions

The NYCKidSeq RCT demonstrated that GUÍA digitally enhanced results disclosure positively impacted parents’/legal guardians’ perceived and objective understanding of their child’s GT results, most significantly in H/L populations. These findings illustrate that digital tools can be developed for diverse families with sick children and applied in clinically complex contexts to improve understanding. Future research should explore participants’ experiences with GUÍA, such as their satisfaction and perceptions of the application’s value and could also expand the functionalities and use cases for GUÍA. For instance, one avenue for additional research is evaluating GUÍA as a tool for non-genetics providers who increasingly offer GTs in their clinics and often encounter challenges communicating genomic information outside their expertise.³⁶ GUÍA could also be expanded for use in other clinical contexts or throughout the entire testing process. As advances in deep learning and generative AI increase, burgeoning opportunities exist to continue to augment genetic counseling by providing more interactive and personalized interfaces, chatbot-based communications, and making genomic information more approachable for patients and providers. Research frameworks, such as the one developed for this study, will be vital to assess the effectiveness of digital tools for enhancing the communication of genomic information in myriad clinical or public health settings. In the future, advancing digital technology could help scale genomic medicine and empower patients’ downstream medical decision-making.

Data and code availability

There are restrictions on the availability of datasets due to participant-level sharing permissions. A subset of the dataset for those who gave permission to share with external researchers may be available from the corresponding authors upon request. The datasets supporting the current study have not been deposited in a public repository because the primary and secondary outcome measures were not part of the publicly shared CSER harmonized measures but may be requested from the corresponding authors.

Published: November 24, 2023