Impact of previous genetic counseling and objective numeracy on accurate interpretation of a pharmacogenetics test report

Abstract

Introduction:

Pharmacogenetic (PGx) testing can be useful for providing information about a patient’s drug response by increasing drug efficacy and decreasing the incidence of adverse drug events. While PGx tests were previously only offered to patients under healthcare provider supervision, they are now available as direct to consumer (DTC) tests. This study aimed to assess how accurately individuals from the general population were able to interpret a sample PGx test report, and if accuracy differed based on individuals’ numeracy or prior genetic counseling (GC).

Methods:

We surveyed 293 individuals from the general population, ascertained through ResearchMatch. The survey included questions about PGx test interpretation, numeracy, and genetic literacy.

Results:

In our cohort, numeracy level impacted PGx result interpretation, with those of high numeracy performing statistically significantly better on both the table format and graphical format (p-value= 0.002 and p-value <0.001, respectively) and genetics knowledge questions (p-value <0.001) than those with low/average numeracy. Additionally, previous GC did not impact test interpretation or genetic knowledge, but the number of individuals with prior GC was small (n=26).

Discussion/Conclusion:

We found that numeracy had a significant impact on correct interpretation of PGx test reports. Because many individuals in the United States have low numeracy levels, it is extremely important that patients do not make their own medication management decision based on the test results, and that they consult with their physicians about their PGx testing. The importance of consultation and discussion with providers about results should be emphasized on the test report.

Introduction

Pharmacogenetic (PGx) testing is a type of genetic testing that analyzes single nucleotide variants (SNVs) and copy number variants (CNVs) in genes important in drug metabolism, with the intent of identifying patient-specific variants in drug-metabolizing enzymes. Ideally this leads to the use of more appropriate drugs, thus increasing drug efficacy and decreasing the rate of adverse drug events in individual patients. PGx tests were previously only offered to patients under healthcare supervision, but they are now available as direct to consumer (DTC) tests. With DTC tests, individuals can send in a saliva sample and get results directly at their home, without the need to obtain the approval or prior involvement of a healthcare provider. While some think that DTC testing may be beneficial for reasons such as decreasing healthcare provider paternalism [1], improving patient self-efficacy [2], and convenience [3], there are other who fear that those who undergo DTC testing may be unable to properly interpret their own test results [4]. This could lead to confusion about risk and, further, opponents of DTC testing worry that patients may change their medical management without physician consultation. Studies have shown that those in the general public tend to interpret DTC testing reports differently than healthcare providers [5]. Some of these differences in interpretation have been hypothesized to be due to differences in numeracy (the ability to comprehend, use, and attach meaning to numbers) [6]. Numeracy has significant impact on patient navigation of healthcare information due to its importance in following health regimens and evaluating risks and benefits of healthcare options [7]. In some cases, higher levels of education have been shown to independently correlate with higher numeracy [7], yet other studies show that even highly educated people have difficulty with relatively simple numeracy skills [8]. Along with numeracy, health literacy (the capacity to obtain, process, and understand basic health information and services needed to make appropriate health decisions) [6] and genetic literacy (having the skills to comprehend clinician-provided information about genetics) may also influence the observed differences in test report interpretation [9]. Genetic counseling may also impact understanding of test results. During a genetic counseling session, a genetic counselor interprets their patient’s personal and family history to assess for risk, educates about issues related to heredity and a patient’s own personal risk, and facilitates informaed decision making [10]. It has been shown that genetic counseling can increase genetic knowledge [11] and can also lead to more accurate disease risk perceptions [12].

The increased awareness of DTC testing [13] and the 2018 FDA approval of PGx testing through the DTC company 23&Me [14] will likely increase uptake of PGx testing through 23&Me and other DTC companies. The goal of this study was to assess how accurately individuals from the general population were able to interpret a sample PGx test report, as result misinterpretation has potential significant clinical implications. Secondly, we aimed to identify factors, such as numeracy levels and previous genetic counseling, that correlated with test interpretation. While there are a number of studies that evaluate numeracy and genetic testing in the context of risk perception, there are very few studies that assess numeracy and interpretation of test reports.

The primary objective of this study was to determine if having genetic counseling on a non-PGx issue prior to this study enabled individuals to more accurately interpret a PGx test report. We also assessed if prior genetic counseling was associated with higher genetic knowledge. A secondary objective was to determine if the level of objective numeracy impacted an individual’s ability to interpret a PGx test report.

We hypothesized that participants who had previous genetic counseling (and potentially, a higher genetic knowledge score) would be able to interpret a PGx test report with more accuracy. We also hypothesized that those with average or high objective numeracy would be able to interpret a PGx test report with more accuracy compared to those with low objective numeracy.

Materials and Methods

Study Participants

This study used the ResearchMatch database [15] to recruit participants. Individuals within the database who met the inclusion criteria (over the age of 18 and English speaking) were sent a recruitment email by ResearchMatch that detailed the study, research question, inclusion criteria for participation, and the estimated time needed to complete the survey. The study team sent interested individuals an email with a link to take the survey, which was done through Qualtrics [16]. In total, e-mails were sent to 496 individuals who initially expressed an interest in participating in the study to ResearchMatch. A copy of the survey can be found in the supplementary materials.

Participant responses were collected over an approximately 3-month time period, from August 2, 2019 to October 22, 2019. The survey was closed on October 22, 2019 at which time had we had 293 participants who completed the survey. Survey completion was defined by participants answering 100% of the survey questions that applied to them. Upon completion of the survey, participants could choose to opt into a drawing for one of five $20 Amazon gift cards. The original goal sample size of 300 was chosen to ensure appropriate power to detect a difference in accuracy of interpretation of a PGx test (above vs. below the median score) between individuals with low numeracy (estimated to be 20% of the sample) and those with medium or high numeracy (estimated to be 80% of the sample based on previous studies)[6]. With 300 individuals and assuming 55% of those high or medium numeracy would be above the median while only 35% of those with low numeracy would be, we would have 80% power with a two-sided alpha of 0.05.

Survey Design

Qualtrics survey questions were designed to assess participants’ interpretation of two hypothetical PGx test reports as well as their genetic knowledge and objective numeracy (Supplemental Material 1). The Qualtrics survey was developed by the research team for this study and, therefore, is not a validated tool. The survey featured two hypothetical test reports: one with test results displayed in the form of a table and the other with test results displayed in graph format. The hypothetical reports were designed based on actual test reports that are provided to individuals having PGx testing through a DTC or genetic testing company, which was done to reflect a test report that someone who was having PGx testing may receive. Participants were asked true/false interpretation questions such as, “This patient is at an increased risk to develop a severe side effect from this drug” and “The patient should share these results with their doctor.” Next, participants were asked to express whether the table or the graph report was easier to comprehend and why. Both reports featured the HLA-B*57:01 variant, which has a strong association with Abacavir hypersensitivity [17]. For this reason, the HLA-B*57:01 variant is included on many DTC PGx tests. Questions to assess participants’ exposure to DTC testing, genetic counseling, genetic knowledge, and numeracy were also asked. To evaluate participants’ genetic knowledge, questions from a 2016 validated survey on genetic knowledge by Fitzgerald-Butt, et al., were used [18]. Participants’ numeracy was measured using questions adapted from the validated Rasch abbreviated numeracy scale questionnaire [19]. Numeracy was measured using 10 free-response questions, and participants were categorized into low, average, or high numeracy based on the number of questions answered correctly (0–2 low, 3–6 average, and 7+ high). Due to an error with a numeracy question on an initial version of the survey, 8 individuals were unable to be correctly categorized based on numeracy and were excluded from numeracy-related analyses.

Data analysis

Means and proportions were used to describe characteristics of the sample and the number correct on test questions (basic PGx report, graphical PGx report, and genetic knowledge). Comparisons of mean scores between groups were done using linear regression allowing for robust standard errors. Data were analyzed using Stata (version 15) and no corrections were made for multiple comparisons. P-values of <0.05 were considered significant.

Results

Demographic Characteristics of Study Population

A link to an online survey was sent to 496 individuals from Research Match who indicated interest in participating. Of the 341 individuals who started the survey, 293 (86%) individuals completed the survey. Demographic characteristics of the participants are described in Table 1. The majority of respondents were female (78.2%) and Caucasian (80.6%). Of the 293 participants in this study, 26 (8.9%) had received genetic counseling in the past, 266 (90.8%) had not received genetic counseling in the past, and one participant did not disclose if they had or had not received past genetic counseling.

Table 1.

Participant demographic information

	Total n (%)	Received GC n (%)	No GC n (%)*
Gender
Female	229 (78.2)	23 (88.5)	206 (77.4)
Male	62 (21.2)	3 (11.5)	58 (21.8)
Other	2 (0.7)	0 (0)	2 (0.8)
Total	293	26	266
Age
18–29	114 (38.9)	8 (30.8)	106 (39.9)
30–39	64 (21.8)	7 (26.9)	57 (21.4)
40–49	37 (12.6)	5 (19.2)	32 (12.0)
50–59	37 (12.6)	1 (3.9)	35 (13.2)
60–69	25 (8.5)	4 (15.4)	21 (7.9)
70+	16 (5.5)	1 (3.9)	15 (5.6)
Education
Bachelor’s	87 (29.7)	9 (34.6)	77 (29.0)
High school graduate	20 (6.8)	2 (7.7)	18 (6.8)
Less than high school	3 (1.0)		3 (1.1)
Master’s	63 (21.5)	8 (30.8)	55 (20.7)
Prefer not to answer	1 (0.3)	0 (0)	1 (0.4)
Professional/Doctorate	23 (7.9)	1 (3.9)	22 (8.3)
Some college	96 (32.8)	6 (23.08)	90 (33.8)
Race
African American/Black	9 (3.1)	1 (3.9)	8 (3.0)
Asian	9 (3.1)	0 (0)	9 (3.4)
Caucasian	236 (80.6)	19 (73.1)	216 (81.2)
Hispanic/Latino	21 (7.2)	4 (15.4)	17 (6.4)
Other	16 (5.5)	1 (3.9)	15 (5.6)
Prefer not to answer	2 (0.7)	1 (3.9)	1 (0.4)

n, number; GC, genetic counseling; %, percent

^*one individual did not answer question about counseling

Impact of genetic counseling on interpretation of PGx results

For the PGx interpretation questions, there was no statistically significant difference in the average number of questions answered correctly between the groups with previous genetic counseling and without previous genetic counseling for both the table (7.3 versus 7.5) and graph (2.5 versus 2.4) representations (p=0.35, p= 0.45 respectively). When comparing the graph and table formats, there was a higher percentage of correct answers for participants answering questions from the table representation compared to the graphical representation. In general, the cohort performed very well in interpreting the PGx test result that was presented in table format. In fact, with the exception of one question, over 90% of participants answered all questions correctly related to the PGx test in the table format. The lowest performing question, with 75% of participants answering it correctly was question 4: “The patient should immediately choose to stop taking this drug before talking to their doctor.” A list of all questions for the graph and table representations and the percentage of those who answered correctly can be found in supplementary materials (Supplemental Tables 1 and 2). The cohort as a whole did not perform as well with the interpretation of the graphical representation of the PGx test report. The lowest performing question, with only 62.5% of participants providing the correct answer was question 6: “These results will not affect their continuation of Abacavir.” Interestingly, when asked which representation was easiest to understand, approximately 44% said the graph was easier compared to 35.8% who stated that the table was easier, even though participants performed better with the table representation. The difference in the average number of correct answers for the genetic knowledge section was not statistically significant between the group with prior genetic counseling and the group without (15.9 versus 15.2; p=0.11).

Impact of numeracy on PGx test interpretation

To determine the impact of numeracy level on PGx test results, individuals completed 10 free-response questions from the Rasch abbreviated numeracy scale questionnaire [19].

Forty-three of 285 (15.0%) individuals were categorized into low numeracy, 147 (51.6%) were average numeracy, and 95 (33.3%) were high numeracy (Table 2). Numeracy level impacted PGx result interpretation. The differences in the average number of correct answers for the table form of the PGx test, the graph form of the PGx test, and genetic knowledge were all statistically different based on numeracy level (Table 2). Those with high numeracy averaged more correct answers for the table representation of the PGx report (n=7.7 out of 8), than either those with low (n=7.3) or average numeracy (n=7.4). Similarly, the average number of correct answers for the graph representation of the PGx report for those with high numeracy (n=6.2 out of 7) was higher than the average number correct for low (n=3.8) or average numeracy (n=5.0).

Table 2.

Number and percentage of those who answered PGx report and genetic knowledge questions correctly by numeracy level

	Average correct answers (SD)
Numeracy Level	Table PGx report (8 questions)	Graph PGx report (7 questions)	Genetic knowledge (18 questions)
Low (n=43, 15%)	7.3 (0.99)	3.8 (2.5)	13.7 (2.8)
Average (n=147, 51.6%)	7.4 (0.95)	5.0 (2.5)	14.9 (2.7)
High (n=95, 33.3%)	7.7 (0.58)	6.2 (1.6)	15.2 (2.5)
p-value	0.002	<0.001	<0.001

n, number of participants, %, percent, sd, standard deviation

Discussion/Conclusion

PGx testing has been gaining popularity and exposure over the past years. Over 100 drugs now include pharmacogenetics information on their labels [20]. Additionally, as of 2016, the Food and Drug Administration (FDA) has approved or cleared the use of 12 DNA-based PGx tests as well as 12 companion tests [20]. Currently, there are an array of companies that offer PGx testing. The rise in DTC testing means that individuals in the general population may be interpreting their own DTC test results without the consultation of a doctor or a genetics professional. There is a growing fear by some health care professionals that those who undergo DTC testing may be unable to accurately interpret their test results [4].

This study found that the difference in the number of PGx interpretation questions that were answered correctly was not different between individuals with or without prior genetic counseling. However, we found that numeracy impacted result interpretation. There was a statistically significant difference in the number of interpretation questions correctly answered by individuals with high numeracy versus individuals with average or low numeracy (Table 2).

We found similar findings for the genetic knowledge portion of the survey. The difference in the number of genetics knowledge questions that were answered correctly was not significantly different between individuals with or without prior genetic counseling. However, the average number of questions answered correctly was statistically different between those with high numeracy and those with average and low numeracy (Table 2). Although not directly tested in this study, our data suggests that numeracy had more of an impact on both accurate interpretation of the PGx test reports and genetic knowledge than did prior genetic counseling.

Another finding of the study was that, when asked which test results were easier to understand between the table and the graph, 35.8% said table, while 44.4% said graph. (The remaining participants said they were equally easy to understand). While participants thought that the graphical representation of the PGx test results were easier to understand, they actually performed better in understanding the table representation.

Additionally, the lowest performing interpretation questions with both the table (“The patient should immediately choose to stop taking this drug before talking to their doctor”) and the graph form (“These results will not affect their continuation of Abacavir”) show the importance of sharing DTC PGx testing results with healthcare providers. Having patients make immediate decisions about their medication use without consultation with a healthcare professional could result in serious harms depending on the medication and the indication for use.

Overally, participant answers from this study indicate that it is important that materials for DTC results are provided in an array of representations to ensure patient understanding. DTC companies need to also consider that those consuming their products may be of low numeracy and low genetic knowledge [21]. Thus, testing materials need to be written in such a way that they are understood across all numeracy and genetic knowledge levels. Additionally, in the PGx results provided to patients, DTC testing companies should emphasize that patients should not make their own medical management decisions based off the provided test results, and they should consult with a healthcare provider to discuss their implications.

This study has several strengths. While there have been a number of studies that have evaluated numeracy and genetic testing in the context of risk perception, there are a very limited number of studies that have studied numeracy in the context of understanding test reports. Our findings that numeracy is an indicator of patient ability to accurately interpret test reports can aide PGx DTC companies in designing the materials that accompany their tests. Because nearly 300 study participants (n=293) completed the survey, we were well powered to determine whether or not those from the general public are able to interpret a PGx test, and if correct interpretation has correlation with objective numeracy.

This study has a number of limitations. First, sections of the Qualtrics survey that specifically assessed patient interpretation of PGx test reports were not from validated survey tools. Thus, these questions have not been tested in another cohort or assessed for dependability. Additionally, the vast majority of participants self-identified as Caucasian and as female. The cohort that was surveyed was also highly educated compared to the general population. Less than 8% of participants disclosed that their highest level of educational attainment was high school or less than high school. This means that approximately 92% of those surveyed had at least some level of college education. The ResearchMatch database was used in anticipation of reaching a diverse study population, but study participants were not representative of the general population. Based on the literature, we would expect that those with less education would have a lower objective numeracy score [7]. Had our cohort been less educated, we assume that they would not have done as well on the PGx interpretation questions given that those with higher numeracy performed better than those with average or low numeracy. Additionally, we presume that, had the cohort been less educated, they would not have performed as well on the genetic knowledge questions, since those scores were statistically significantly lower for those with lower numeracy and those without at least a bachelor’s degree.

Another limitation to this study is that the number of people who had previous genetic counseling was less than 9% of the total which means that we were underpowered to assess the impact of genetic counseling on PGx test interpretation. It is also possible that the amount of time that has passed since prior genetic counseling could affect patient interpretation of the PGx test report. Due to the limited number of those surveyed who have had prior genetic counseling (n=26), and the limited number of people who disclosed how many years ago that counseling took place (n=17), we could not determine if differences in time since counseling impacted understanding. In addition, some of the individuals who had genetic counseling may not have had genetic testing as part of their genetic counseling. Furthermore, we did not survey participants on their reason for counseling (e.g. prenatal testing versus assessment for Mendelian disease risk versus genome-wide associate study) which could have impacted genetic literacy. It is also important to note that, even amongst those who have had prior genetic counseling (especially for a Mendelian condition), the counseling received may or may not translate to PGx. All of the above factors could have influenced the apparent lack of effect that prior genetic counseling had on the accurate interpretation of the PGx test report. Due to the fact that those with previous genetic counseling were not specifically recruited, any results relating to the impact of genetic counseling on accurate interpretation of a PGx test should be taken with caution. The under recruitment of individuals who had prior genetic counseling is a bias in the study design. As such, results of this study pertaining to previous genetic counseling should be interpreted with this limitation in mind.

Future studies should evaluate PGx test interpretation of demographic groups who were underrepresented in this cohort (males, those with lower levels of education, and individuals of non-European ancestry). Additionally, in order to fully understand if previous genetic counseling impacts genetic knowledge or the ability to interpret a PGx test report, future studies should target individuals who had previous genetic counseling and ask more details about the counseling experience, such as time since counseling occurred, the reason for the genetic counseling appointment and whether the individual had genetic testing.

In conclusion, we found that numeracy plays a significant role in PGx test interpretation. As such, DTC and other genetic testing companies should develop test reports that are understandable across a wide range of numeracy levels.