Abstract
Background
Collection of patient-reported outcomes measures (PROs) may augment clinical data and inform health research, improving care, yet approaches to sustaining interest among patient cohorts in research participation are needed. One approach may involve returning aggregate research results (ARRs), which may help patients contextualize personal experiences, prompt conversations with providers or family, and encourage information seeking. This model has been demonstrated for Web-based patient-centered registries. Studies with clinical cohorts may further elucidate the model, its impacts on interest in research participation and planned actions, and potential for participants to experience this as helpful or harmful—gap areas.
Objective
We sought to investigate the impacts of returning ARRs comprising summaries of PROs and clinical metrics to parents of children with rheumatic disease, assessing interest in future research participation among parents who viewed ARRs and plans for acting on returned information. Further, we sought to investigate reactions to viewing ARRs and how these reactions impacted planned actions.
Methods
Clinical and PRO data were obtained about children in a national clinical disease registry, summarized, and processed into annotated infographics, comprising ARRs for children’s parents. Parents who viewed ARRs (n=111) were surveyed about the information’s perceived value and their reactions. Reaction patterns were summarized using principal components analysis (PCA), and associations among reaction patterns and interest in research participation and planned actions were estimated using multivariate logistic regression.
Results
Parental endorsement of the value of ARRs for understanding their child’s condition and making care decisions was high (across 10 topics for which ARRs were shared, 42.2%-77.3% of the parents reported information was “very valuable”). Most (58/111, 52.3%) parents reported being more interested in participating in research after viewing ARRs, with the remainder reporting that their interest levels were unchanged. Reactions to viewing ARRs reflected experiencing validation/affirmation and information burden based on PCA. Reactions were not associated with child demographic or clinical characteristics and PROs, except that parents from households with less education reported greater information burden than those from more educated households (P=.007). In adjusted models, parents with higher validation/affirmation scores had increased odds of reporting heightened interest in research participation (adjusted odds ratio [AOR] 1.97, 95% CI 1.18-3.30), while higher information burden scores were associated with decreased odds of planned discussions with their child (AOR 0.59, 95% CI 0.36-0.95) and increased odds of planned discussions with providers (AOR 1.75, 95% CI 1.02-3.00).
Conclusions
Returning ARRs may foster a “virtuous cycle” of research engagement, especially where ARRs are experienced favorably and affect plans to share and discuss ARRs in support of a child’s chronic disease care and treatment. Reactions to ARRs vary with education level, underscoring the need for attention to equity for this model.
Keywords: aggregate research results, decision making, juvenile idiopathic arthritis, patient engagement, patient-reported outcome measures, rheumatic disease
Introduction
Growing evidence supports the importance of engaging patients in research to share information about their disease and treatment experiences and health-related quality of life [1,2]. Where shared data flow into health care systems and clinical epidemiologic studies, better symptom management, improved treatments, and better outcomes result [3]. In support of engaging patients in sharing high-quality data, considerable investment has been made in developing standardized patient-reported outcomes measures (PROs) that characterize aspects of patients’ physical, mental, and social health [3-6]. PROs enable scientific rigor and help capture the patient voice, consistent with the paradigm shifting efforts to advance patient-centered outcomes research [7-10]. Yet, open questions remain about how to motivate ongoing patient engagement in research, and this question is central to ambitious efforts to activate vast cohorts in donating PROs and clinical data [11].
One possible approach is to operationalize a process whereby patients donate health data that are subsequently processed and returned in aggregate. Here, the return of aggregate research results (ARRs) about the cohort is hypothesized to motivate a virtuous cycle of data donation that can help grow the evidence base to advance more acceptable, effective therapies [12]. Viewing ARRs may be motivating for research participation if the returned information helps patients appraise personal experiences of disease and treatment [13] and informs conversations with health care providers and family members, factors that are relevant to health care decision making [14]. These actions, which reflect an engaged and activated patient and socially embedded nature of health care decision making, are central to models of chronic illness care [15-17]. These factors are also consistent with survey reports about what motivates sharing of personal health information [18-20] and are reflected in the appeal of Web-based patient-centered health information repositories [17]. Additional empirical testing of this “closed-loop” model is essential for ascertaining whether it fosters interest in research participation among clinical cohorts and to better understand its potential for being experienced as helpful or harmful—gap areas that are central to ensuring equipoise.
Returning ARRs may be informative and reassuring for some research participants, providing a normalizing context around experiences; ARRs may also be overwhelming and disquieting for others, including if ARRs show evidence of problems experienced by others with the same condition. Other factors, including the level of education and health literacy, might also affect acceptability. Concern about the balance of benefit or harm experienced when viewing ARRs may be especially acute for conditions that are rare, have treatments that rest on an immature evidence base or incompletely ameliorate symptoms or health-related quality of life, and pose risks for side effects [21]. Arguably, motivating ongoing research engagement for such conditions is especially important because gaps in knowledge might be filled, driving improvements in therapies at the system level and decision making at the patient or family level.
Pediatric-onset rheumatic disease (RD) is a trenchant case for examining these issues. Among youth, RD is rare but rising in incidence [22], with affected children facing significant hurdles regarding health-related quality of life due to the chronic relapsing nature of RD, unpredictable disease course, and difficult treatments [1-3]. Lack of a mature evidence base guiding RD care makes maintaining patient engagement in research especially important for gathering information about disease and treatment experiences to improve care and outcomes. For example, in a prior study focused on youth with RD, we found that treatment-related problems for standard RD therapies were common and contributed to poor health-related quality of life even after controlling for patient clinical characteristics and ameliorative effects of these treatments on symptoms, such as pain [23].
This study aims to investigate impacts of returning ARRs on interest in participating in future research within a larger project focused on investigating pediatric-onset RD. We engaged parents of children with RD in donating PROs about their child’s health and treatment experiences and then returned cohort-level summaries of clinical measures and PROs to parents, testing whether viewing these ARRs increased parents’ interest in future research participation and their intentions to discuss ARRs with others or seek further information. Such discussions might encourage shared decision making, consistent with theory [6,24]. We hypothesized that the receipt of ARRs would be highly motivating for future research engagement and that reactions to and planned actions arising from viewing ARRs would vary with participants’ experiences of benefit and burden.
Methods
Study Design and Setting
We obtained data from The Learning Cohort (TLC) study [23], which surveyed consented parents of youth with a pediatric-onset RD enrolled in the Childhood Arthritis and Rheumatology Research Alliance (CARRA) Registry [25] during subspecialty care visits. Surveys included PROs to capture aspects of disease or treatment experience and well-being [23]. PRO survey measures were programmed in the Research Electronic Data Capture system [25], from whence they could flow into the registry using a modular, ontology-based, federated informatics infrastructure constructed from open-source software; this infrastructure provides research investigators full ownership and access to their contributed data while supporting permissioned and robust data-sharing across federated sites [26]. In total, 4 CARRA Registry sites participated in this study, with Institutional Review Board approval from each. All participants provided informed consent. Details of the TLC study are published elsewhere [23].
Toward the close of the study period, the research team aggregated PRO data from the full TLC cohort (N=202 dyads) and clinical metrics from the CARRA Registry. These results were summarized by the research team into a curated, annotated set of ARRs, reflecting areas of leading concern to parents of registry-involved youth based on an initial survey of their information needs conducted when they enrolled [23]. ARRs comprised 68 unique slides (including titles and section headers) delivered on a tablet computer at a routine visit and were delivered as static infographics (visuals, figures, and charts). Topics covered study methods, clinical and treatment characteristics of children in the cohort, including medications used, patterns of health-related quality of life, experiences of pain and morning stiffness, and treatment problems (Figure 1). All materials were pretested, including the ARR slides, and the survey that was to be administered to parents to elicit reactions to returned data. The process was iterative to address all concerns. Pretests for accessibility were conducted with 5 parent volunteers and a representative of a family-based disease advocacy group. Pretests for accuracy and safety were conducted with 6 pediatric rheumatologists and 1 pediatric emergency room physician/ informatician.
Figure 1.
Example content from aggregate research results returned to parents.
From August 2015 to February 2016, in lieu of collecting additional PROs during clinic visits, an approximately 50% convenience sample of participating parents viewed ARRs (119 were approached, 115 consented; 96.6% consent rate), after which they completed a survey about their reactions to these materials; 111 parents provided complete data on their reactions to ARRs.
Measures
Demographic, Clinical and Health Characteristics
Parents reported their child’s age; sex, race/ethnicity; diagnosis (juvenile idiopathic arthritis, systemic lupus erythematosus, or mixed connective tissue disease); overall health status; health-related quality of life [27]; pain interference [28]; morning stiffness; experience of serious side effects from a medication; methotrexate intolerance [29]; and highest education attained in the family. Disease duration was obtained from the CARRA Registry. Time in cohort was calculated as the number of days from the enrollment date (initial PRO collection date) to the final PRO collection date. Sample mode and mean imputation were used for 9 participants with incomplete data on demographic or clinical characteristics.
Perceived Value of and Reactions to the Return of Aggregate Research Results
Novel measures were developed and used to assess parents’ reactions to the return of ARRs. Perceived value of the return of ARRs was ascertained with the question, “Overall, how valuable might this summary information be when understanding and making decisions about your child’s condition and care?” asked for each of 10 topics that were shared in ARRs. Responses were given using a 3-point Likert scale (very valuable, somewhat valuable, and not valuable) and reported as frequencies. Sample size for each item ranged from 106 to 111 due to participant nonresponse on select items. Parents’ reactions to seeing ARRs were determined by the extent to which they agreed or disagreed with the following 6 statements, each rated on a 5-point Likert scale (strongly agree, agree, undecided, disagree, strongly disagree): “Seeing summary information about the experiences of other study participants was comforting because it made me feel like my experiences are shared and validated as real;” “Overall, seeing summary information about other patients’ experiences help me understand my child’s experience;” “Reviewing this type of information is within my comfort zone;” “Overall, this type of information raises more questions than it answers;” “This type of information requires more knowledge or expertise than I have to understand it;” and “I prefer to let my rheumatologist digest this type of information.” The mean of nonmissing items within the scale was used to impute missing response for 5 participants missing 1 or 2 (of 6) items.
Engagement Outcomes
Parents were asked to report their interest in participating in research studies after seeing ARRs; options included more interested, less interested, and not any more or less interested. As one respondent endorsed “less interested,” this response was combined with “not any more or less interested” to create a dichotomous variable. Parents were asked to report their planned actions after reviewing the summary information provided in ARRs by selecting all that applied from the following list: (1) discuss contents of the slide-deck with my child’s health care provider; (2) discuss contents of the slide-deck with my child; (3) explore different medications; (4) look up information about something I saw in this slide-deck; and (5) do something else.
Statistical Analyses
All analyses were conducted using SAS 9.4 software (SAS Institute, Inc, Cary, North Carolina). Summary statistics were generated to describe sample characteristics; differences in demographic and clinical characteristics between parents who did (n=111) or did not (n=91) receive ARRs were evaluated using t tests, Kruskal-Wallis, or chi-square (χ²) tests, as appropriate. Principal components analysis (PCA) was conducted to investigate the commonalities between ARR reactions and generate summary variables based on individual reaction measures. All factors with eigenvalues >1 were retained consistent with the standard practice [30,31], leading to a 2-factor solution, and an orthogonal rotation was applied to generate 2 uncorrelated scales (hereafter referred to as “validation/affirmation” and “information burden”).
Validation/affirmation reflects reactions to ARRs that reflect experiences of recognizing the legitimacy of personal experiences, greater insight into their child’s condition, and level of comfort with ARRs. Information burden reflects reactions to ARRs that reflect experiences akin to being over one’s head or uncomfortable with information in ARRs owing to perceived lack of personal expertise, preferences for their provider to digest the information and uncertainty. Pearson correlations, t tests, and analysis of variance (as appropriate) were used to examine bivariate relationships between each of the 2 ARR reaction constructs and demographic and clinical characteristics. Subsequently, multivariate logistic regression was used to model engagement outcomes as predicted by the 2 ARR reaction constructs; both unadjusted (controlling only for both factors simultaneously) as well as models adjusting for child age, race/ethnicity, and parent education were used.
Results
Sample Characteristics
Children in this cohort were 12.0 years of age on average (SD 3.6) and predominantly female (n=161, 79.7%), diagnosed with juvenile idiopathic arthritis (n=187, 92.6%), white/non-Hispanic individuals (n=152, 75.2%), and had parents with any college education (n=144, 71.3%); the average disease duration was 7.7 (SD 3.5) years. No differences were observed in demographic or clinical characteristics between those who did or did not receive ARRs (Table 1).
Table 1.
The parent-reported sample characteristics by receipt of aggregate research results (ARRs).
| Characteristic | Total (N=202) | Received ARRs (n=111) | Did not receive ARRs (n=91) | P value | ||
| Demographics | ||||||
| Child age (years), mean (SD) | 12.0 (3.6) | 11.7 (3.6) | 12.3 (3.7) | .29 | ||
| Child sex, n (%) | .87 | |||||
| Female | 161 (79.7) | 88 (79.3) | 73 (80.2) | —a | ||
| Male | 41 (20.3) | 23 (20.7) | 18 (19.8) | — | ||
| Child race or ethnicity, n (%) | .63 | |||||
| White, non-Hispanic | 152 (75.2) | 85 (76.6) | 67 (73.6) | — | ||
| Non-white or Hispanic | 50 (24.8) | 26 (23.4) | 24 (26.4) | — | ||
| Highest level of parent education, n (%) | .79 | |||||
| High school graduate or less | 58 (28.7) | 31 (27.9) | 27 (29.7) | — | ||
| Any college | 144 (71.3) | 80 (72.1) | 64 (70.3) | — | ||
| Clinical characteristics | ||||||
| Diagnosis, n (%) | .50 | |||||
| Juvenile idiopathic arthritis | 187 (92.6) | 104 (93.7) | 83 (91.2) | — | ||
| Systematic lupus erythematosus or mixed connective tissues disease | 15 (7.4) | 7 (6.3) | 8 (8.8) | — | ||
| Disease duration (in years), mean (SD) | 7.7 (3.5) | 7.7 (3.5) | 7.7 (3.6) | .99 | ||
| Methotrexate use and intolerance, n (%) | .65 | |||||
| No methotrexate use | 103 (51.0) | 59 (53.2) | 44 (48.4) | — | ||
| Use with no intolerance | 60 (29.7) | 30 (27.0) | 30 (33.0) | — | ||
| Methotrexate intolerance | 39 (19.3) | 22 (19.8) | 17 (18.7) | — | ||
| Overall healthb, mean (SD) | 8.1 (2.0) | 8.2 (1.8) | 8.0 (2.3) | .45 | ||
| Typical morning stiffness in the past 2 weeks (minutes)c | .30 | |||||
| >15 | 36 (17.8) | 17 (15.3) | 19 (20.9) | — | ||
| ≤15 | 166 (82.2) | 94 (84.7) | 72 (79.1) | — | ||
| Lifetime serious medication side effects | .64 | |||||
| One or more | 52 (25.7) | 30 (27.0) | 22 (24.2) | — | ||
| None | 150 (74.3) | 81 (73.0) | 69 (75.8) | — | ||
| Pediatric Quality of Life Inventory 4.0 scores, mean (SD) | ||||||
| Total score | 75.7 (18.3) | 74.9 (18.4) | 76.7 (18.4) | .50 | ||
| Psychosocial score | 76.1 (18.0) | 75.2 (18.2) | 77.1 (17.8) | .46 | ||
| Physical score | 75.1 (22.6) | 74.4 (22.3) | 75.9 (23.1) | .65 | ||
| Patient-Reported Outcomes Measurement Information System Pain Interferencee, mean (SD) | 50.5 (10.9) | 51.2 (11.0) | 49.7 (10.9) | .35 | ||
| Time in cohort (days), mean (SD) | 95.8 (100.9) | 105.1 (100.7) | 84.3 (100.4) | .14 | ||
aNot applicable.
bParents’ rating of their child’s overall health from 1 to 10, where higher scores indicate better health.
cParents’ report of the number of minutes of morning stiffness their child experiences on a typical day over the past 2 weeks.
dThe possible range of scores is from 0 to 100, with higher score indicating better quality of life.
eRaw pain interference scores were transformed into a “t score” for each participant. The t score rescales the raw score into a standardized score with a mean of 50, SD of 10, and the possible range of 38-78, with higher score indicating more pain interference.
Perceived Value of and Reactions to the Return of Aggregate Research Results
The proportion of parents finding the ARR topics to be “very valuable” ranged from 42.2% (for experiences of morning stiffness) to 77.3% (for medication problems; Figure 2).
Figure 2.
Parents’ ratings of the value of each patient-reported topic presented in aggregate research results for understanding and making decisions regarding their child’s condition and care.
PCA identified a 2-factor solution with high loadings for all 6 ARR reaction items (Figure 3).
Figure 3.
Percentages and factor loadings of response items regarding reactions to the return of aggregate research results.
The validation/affirmation construct was defined by 3 items reflecting having one’s experience validated (factor loading=0.873), improved understanding of their child’s condition (factor loading=0.865), and feeling that the ARR materials are within their comfort zone (factor loading=0.564). The information burden construct was defined by reports of requiring more knowledge to understand ARRs (factor loading=0.856), preferring a physician to “digest” ARRs (factor loading=0.717), and viewing ARRs raising more questions than were answered (factor loading=0.682). Parents from households with less education reported greater information burden than those from more educated households (P=.007; Table 2); factors were not significantly associated with other demographic or clinical characteristics.
Table 2.
Reactions to aggregate research results (ARRs) by demographic and clinical characteristics (n=111).
| Characteristic | Factor 1: Validation/affirmationa | P value | Factor 2: Information burdenb | P value | ||||||
| Demographic characteristics | ||||||||||
| Child age (years), correlation coefficient | 0.12 | .21 | 0.00 | .98 | ||||||
| Child sex, mean (SD) | ||||||||||
| Female | –0.05 (1.02) | .29 | 0.07 (1.02) | .15 | ||||||
| Male | 0.20 (0.92) | —c | –0.27 (0.89) | — | ||||||
| Child race or ethnicity, mean (SD) | ||||||||||
| White, non-Hispanic | –0.01 (1.05) | .92 | –0.06 (0.96) | .24 | ||||||
| Non-white or Hispanic | 0.02 (0.81) | — | 0.20 (1.11) | — | ||||||
| Highest level of parent education, mean (SD) | ||||||||||
| High school graduate or less | 0.07 (0.89) | .67 | 0.41 (1.10) | .007 | ||||||
| Any college | –0.03 (1.04) | — | –0.16 (0.92) | — | ||||||
| Clinical characteristics | ||||||||||
| Diagnosis, mean (SD) | ||||||||||
| Juvenile idiopathic arthritis | –0.02 (1.02) | .52 | 0.01 (0.99) | .69 | ||||||
| Systematic lupus erythematosus or mixed connective tissues disease | 0.24 (0.70) | — | –0.15 (1.15) | — | ||||||
| Disease duration (years), correlation coefficient | 0.11 | .27 | 0.05 | .57 | ||||||
| Methotrexate use and intolerance, mean (SD) | ||||||||||
| No methotrexate use | –0.01 (0.96) | .27 | 0.07 (0.97) | .54 | ||||||
| Use with no intolerance | 0.20 (0.82) | — | –0.17 (1.01) | — | ||||||
| Methotrexate intolerance | –0.25 (1.28) | — | 0.04 (1.08) | — | ||||||
| Overall healthd, correlation coefficient | −0.07 | .48 | −0.03 | .73 | ||||||
| Typical morning stiffness in the past 2 weeks (minutes)e, mean (SD) | ||||||||||
| >15 | 0.11 (1.05) | .61 | 0.05 (1.02) | .83 | ||||||
| ≤15 | –0.02 (1.00) | — | –0.01 (1.00) | — | ||||||
| Lifetime serious medication side effects, mean (SD) | ||||||||||
| One or more | 0.27 (0.95) | .09 | –0.25 (0.78) | .10 | ||||||
| None | –0.10 (1.01) | — | 0.09 (1.06) | — | ||||||
| Pediatric Quality of Life Inventory 4.0 scoresf, correlation coefficient | ||||||||||
| Total | –0.09 | .34 | –0.11 | .24 | ||||||
| Psychosocial Score | –0.11 | .25 | –0.10 | .29 | ||||||
| Physical Score | –0.05 | .61 | –0.11 | .26 | ||||||
| Patient-Reported Outcomes Measurement Information System Pain Interferenceg, correlation coefficient | 0.06 | .54 | –0.02 | .82 | ||||||
| Time in Cohort (days), correlation coefficient | –0.02 | .80 | 0.08 | .43 | ||||||
aReflects the extent to which parents feel their experience is validated or affirmed when viewing ARRs; higher scores indicate greater agreement.
bReflects the extent to which parents experienced information burden when viewing; higher scores indicate greater agreement.
cNot applicable.
dParent’s rating of their child’s overall health from 1 to 10, where higher scores indicate better health.
eParent’s report of the number of minutes of morning stiffness their child experiences on a typical day over the past 2 weeks.
fThe possible range of scores is from 0 to 100, with higher score indicating better quality of life.
gRaw pain interference scores were transformed into a “t score” for each participant. The t score rescales the raw score into a standardized score with a mean of 50, SD of 10, and the possible range of 38-78. A higher score indicates more pain interference.
Engagement Outcomes
The majority of the parents (58/111, 52.3%) reported that after seeing ARR, they were more interested in participating in research. Higher validation/affirmation scores were associated with nearly a 2-fold increase in the odds of reporting more interest in research participation (adjusted odds ratio [AOR]: 1.97, 95% CI 1.18-3.30; Table 3). Nearly one-third (35/111, 31.5%) of the parents reported that they would discuss the contents of ARRs with their child, nearly one-fifth (20/111, 18.0%) reported they would look up information, 15.3% (17/111) said that they would discuss ARR contents with their child’s health care provider, and approximately one-tenth (11/111, 9.9%) reported they would explore different medications. Parents who reported greater information burden were less likely to report wanting to discuss the contents of ARRs with their child (AOR 0.59, 95% CI 0.36-0.95) but were more likely to report wanting to discuss the contents of ARRs with their child’s health care provider (AOR 1.75, 95%CI 1.02-3.00).
Table 3.
Associations between reactions to aggregate research results and engagement outcomes.
| Outcomea | Outcome prevalence, n (%) | Unadjusted models, OR (95% CI) | Adjusted models, OR (95% CI) | ||
| More interest in participating in research | |||||
| Validation/affirmation | 58 (52.3) | 1.97 (1.21-3.18) | 1.97 (1.18-3.30) | ||
| Information burden | 1.33 (0.89-1.98) | 1.36 (0.89-2.09) | |||
| Planned actions | |||||
| Discuss with child | |||||
| Validation/affirmation | 35 (31.5) | 1.22 (0.79-1.87) | 1.18 (0.75-1.86) | ||
| Information burden | 0.69 (0.45-1.07) | 0.59 (0.36-0.95) | |||
| Look up information | |||||
| Validation/affirmation | 20 (18.0) | 0.93 (0.58-1.47) | 0.87 (0.54-1.41) | ||
| Information burden | 0.68 (0.39-1.17) | 0.64 (0.36-1.15) | |||
| Discuss with providers | |||||
| Validation/affirmation | 17 (15.3) | 0.69 (0.43-1.11) | 0.69 (0.42-1.13) | ||
| Information burden | 1.63 (0.98-2.71) | 1.75 (1.02-3.00) | |||
| Explore different medications | |||||
| Validation/affirmation | 11 (9.9) | 1.03 (0.54-1.98) | 1.06 (0.53-2.10) | ||
| Information burden | 1.15 (0.63-2.11) | 1.31 (0.69-2.52) | |||
aFrequency and unadjusted prevalence of engagement outcomes among those who received aggregate research results (n=111). Planned action prevalence did not sum to 100% as participants could endorse multiple actions. “Unadjusted” models controlled for validation/affirmation and information burden scales only. Adjusted models controlled for the child’s age, race/ethnicity, and highest education attained in the family in addition to both validation/affirmation and information burden scales.
Discussion
This study provides evidence that returning ARRs to study participants who have donated data is highly motivating for ongoing research participation. By investigating effects of sharing ARRs on interest in research participation and planned actions stemming from this experience, findings extend the understanding of benefits of patient engagement in health care research [32]. More than half of the parents in the cohort exposed to ARRs reported increased interest in research participation as a function of receiving ARRs—good news for participatory research models, including those predicated on engaging cohorts in donating data. Moreover, parents endorsed plans for proactive engagement in their child’s treatment and outcomes after receiving ARRs, with specific plans varying as a function of validation/affirmation and information burden. Overall, reactions to ARRs aligned with hopes for fostering motivation for research participation and activation in the health care process [33,34].
As hypothesized, the experience of viewing ARRs was not “one size fits all”—participants’ responses to the model reflected both experiences of validation/affirmation, wherein parents gained value from contextualizing disease or treatment experiences through viewing cohort-level data, and information burden, wherein aggregate results may be cognitively overwhelming. Yet, as greater information burden was associated with plans to discuss results with a provider, even parent participants who may have felt overwhelmed by ARRs might benefit if they are stimulated to talk with health care providers to understand findings and discuss any implications for their child. Such activities could foster improved patient-provider partnership and shared decision making. Notably, experiencing greater information burden was evident among participants who reported lower levels of educational attainment. As such, achieving goals of optimizing and equalizing health benefits and reducing the potential for disparities stemming from this model may require additional support around interpreting and processing ARRs [34]. Nationally, low levels of health literacy and numeracy indicate that large segments of the US population may face barriers to understanding health data, underscoring the importance of attending to these issues [35,36].
This investigation was undertaken with a cohort situated in a well-defined national multisite disease registry, whose members’ diagnoses were clinically confirmed—significant strengths. Still, several limitations merit mention. First, findings reflect the experiences of engaged parents of registry participants who viewed a specific set of ARRs and may not generalize to other populations that may differ in conditions, concerns, and experiences (including the history of benefit or harm from research and care); the ARR contents are also specific to the population. Second, this study leveraged the visit structure of a registry-engaged clinical cohort, layering data collection and return onto the natural visit cycle of this cohort. Additional research using a randomized trial design would inform comparisons of engagement outcomes for participants who did and did not receive ARRs over time. Third, participants shared structured PROs, some validated, others novel; items describing motivation to participate in research and experiences of receiving ARRs are not validated but capture important patient-centered dimensions of research experience. Fourth, parent proxy reports of child clinical characteristics and PROs and parent reactions to the return of ARRs may differ from those of the child [9,37], and results from this study should not be construed as reflecting child (ie, patient) reactions to the return of ARRs or reactions of another parent or guardian. Relatedly, excepting the measure of the highest parent education attained, demographic characteristics describe the child not parents. Congruence between child and parent experiences of this model and further investigation of effects of parent demographics on outcomes may further inform this work and merit future study. Lastly, self-reported data are subject to recall and social desirability biases; however, the use of structured and validated measures and electronic data collection help protect against known validity threats.
In sum, viewing ARRs increased motivation for research participation among a majority of study participants and shows promise for driving greater patient activation and engagement. Results of this study are encouraging in light of national plans for engaging volunteers in donating personally generated data, including PROs, to drive precision medicine and comparative effectiveness research [7,11,38,39]. To the extent that these efforts utilize a closed loop approach in which ARRs are returned to a learning and sharing cohort, they may thrive. Protecting against the potential for unintentionally worsening health disparities is vital as results show that participants from households with less parent education were more likely to experience information burden from viewing ARRs. Should this lead to the differential engagement or attrition of less educated participants, biases in study findings could be introduced and the ultimate fairness and beneficence of the model undermined.
Acknowledgments
This study was funded by grants from the National Library of Medicine award #R01LM011185; the National Institute of Arthritis and Musculoskeletal and Skin Diseases award #U19AR069522 and #R21AR070944; and the Maternal and Child Health Bureau (Title V, Social Security Act), Health Resources and Services Administration award #T71MC00009. The funding bodies were not involved in the design of the study or collection, analysis, and interpretation of data, nor in writing, review or approval of the manuscript.
Abbreviations
- AOR
adjusted odds ratio
- ARR
aggregate research result
- CARRA
Childhood Arthritis and Rheumatology Research Alliance
- PCA
principal components analysis
- PRO
patient-reported outcome measure
- RD
rheumatic disease
- TLC
The Learning Cohort
Footnotes
Conflicts of Interest: None declared.
References
- 1.Manary M, Boulding W, Staelin R, Glickman S. The patient experience and health outcomes. N Engl J Med. 2013 Jan 17;368(3):201–3. doi: 10.1056/NEJMp1211775. [DOI] [PubMed] [Google Scholar]
- 2.Washington A Eugene, Lipstein Steven H. The Patient-Centered Outcomes Research Institute--promoting better information, decisions, and health. N Engl J Med. 2011 Oct 13;365(15):e31. doi: 10.1056/NEJMp1109407. [DOI] [PubMed] [Google Scholar]
- 3.Basch E. Patient-Reported Outcomes - Harnessing Patients' Voices to Improve Clinical Care. N Engl J Med. 2017 Jan 12;376(2):105–108. doi: 10.1056/NEJMp1611252. [DOI] [PubMed] [Google Scholar]
- 4.Concannon T, Fuster M, Saunders T, Patel K, Wong J, Leslie L, Lau J. A systematic review of stakeholder engagement in comparative effectiveness and patient-centered outcomes research. J Gen Intern Med. 2014 Dec;29(12):1692–701. doi: 10.1007/s11606-014-2878-x. http://europepmc.org/abstract/MED/24893581. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Asch D, Rosin R. Engineering Social Incentives for Health. N Engl J Med. 2016 Dec 29;375(26):2511–3. doi: 10.1056/NEJMp1603978. [DOI] [PubMed] [Google Scholar]
- 6.Lavallee DC, Chenok KE, Love RM, Petersen C, Holve E, Segal CD, Franklin PD. Incorporating Patient-Reported Outcomes Into Health Care To Engage Patients And Enhance Care. Health Aff (Millwood) 2016 Apr;35(4):575–82. doi: 10.1377/hlthaff.2015.1362. [DOI] [PubMed] [Google Scholar]
- 7.Ahmed S, Berzon RA, Revicki DA, Lenderking WR, Moinpour CM, Basch E, Reeve BB, Wu AW, International Society for Quality of Life Research The use of patient-reported outcomes (PRO) within comparative effectiveness research: implications for clinical practice and health care policy. Med Care. 2012 Dec;50(12):1060–70. doi: 10.1097/MLR.0b013e318268aaff. [DOI] [PubMed] [Google Scholar]
- 8.Levine DR, Mandrell BN, Sykes A, Pritchard M, Gibson D, Symons HJ, Wendler D, Baker JN. Patients' and Parents' Needs, Attitudes, and Perceptions About Early Palliative Care Integration in Pediatric Oncology. JAMA Oncol. 2017 Sep 01;3(9):1214–1220. doi: 10.1001/jamaoncol.2017.0368. http://europepmc.org/abstract/MED/28278329. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Brudvik C, Moutte S, Baste V, Morken T. A comparison of pain assessment by physicians, parents and children in an outpatient setting. Emerg Med J. 2017 Mar;34(3):138–144. doi: 10.1136/emermed-2016-205825. http://emj.bmj.com/cgi/pmidlookup?view=long&pmid=27797872. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Selby JV, Forsythe Laura, Sox Harold C. Stakeholder-Driven Comparative Effectiveness Research: An Update From PCORI. JAMA. 2015 Dec 01;314(21):2235–6. doi: 10.1001/jama.2015.15139. [DOI] [PubMed] [Google Scholar]
- 11.Collins Francis S, Varmus Harold. A new initiative on precision medicine. N Engl J Med. 2015 Feb 26;372(9):793–5. doi: 10.1056/NEJMp1500523. http://europepmc.org/abstract/MED/25635347. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Califf R, Robb M, Bindman A, Briggs J, Collins F, Conway P, Coster T, Cunningham F, De Lew Nancy, DeSalvo K, Dymek C, Dzau V, Fleurence R, Frank R, Gaziano J, Kaufmann P, Lauer M, Marks P, McGinnis J, Richards C, Selby JV, Shulkin David J, Shuren Jeffrey, Slavitt Andrew M, Smith Scott R, Washington B Vindell, White P Jon, Woodcock Janet, Woodson Jonathan, Sherman Rachel E. Transforming Evidence Generation to Support Health and Health Care Decisions. N Engl J Med. 2016 Dec 15;375(24):2395–2400. doi: 10.1056/NEJMsb1610128. [DOI] [PubMed] [Google Scholar]
- 13.Oerlemans S, Arts L, Horevoorts N, van de Poll-Franse Lonneke V. “Am I normal?” The Wishes of Patients With Lymphoma to Compare Their Patient-Reported Outcomes With Those of Their Peers. J Med Internet Res. 2017 Aug 15;19(8):e288. doi: 10.2196/jmir.7079. http://www.jmir.org/2017/8/e288/ [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Dwamena F, Holmes-Rovner M, Gaulden C, Jorgenson S, Sadigh G, Sikorskii A, Lewin S, Smith R, Coffey J, Olomu A. Interventions for providers to promote a patient-centred approach in clinical consultations. Cochrane Database Syst Rev. 2012 Dec 12;12:CD003267. doi: 10.1002/14651858.CD003267.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Wagner EH, Austin BT, Davis C, Hindmarsh M, Schaefer J, Bonomi A. Improving chronic illness care: translating evidence into action. Health Aff (Millwood) 2001;20(6):64–78. doi: 10.1377/hlthaff.20.6.64. [DOI] [PubMed] [Google Scholar]
- 16.Gee P, Greenwood D, Paterniti D, Ward D, Miller L. The eHealth Enhanced Chronic Care Model: a theory derivation approach. J Med Internet Res. 2015 Apr 01;17(4):e86. doi: 10.2196/jmir.4067. http://www.jmir.org/2015/4/e86/ [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Bradley M, Braverman J, Harrington M, Wicks P. Patients' motivations and interest in research: characteristics of volunteers for patient-led projects on PatientsLikeMe. Res Involv Engagem. 2016;2:33. doi: 10.1186/s40900-016-0047-6. https://researchinvolvement.biomedcentral.com/articles/10.1186/s40900-016-0047-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Sanderson S, Linderman M, Suckiel S, Diaz G, Zinberg R, Ferryman K, Wasserstein M, Kasarskis A, Schadt E. Motivations, concerns and preferences of personal genome sequencing research participants: Baseline findings from the HealthSeq project. Eur J Hum Genet. 2016 Jan;24(1):14–20. doi: 10.1038/ejhg.2015.118. doi: 10.1038/ejhg.2015.118. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Weitzman ER, Kelemen S, Kaci L, Mandl KD. Willingness to share personal health record data for care improvement and public health: a survey of experienced personal health record users. BMC Med Inform Decis Mak. 2012 May 22;12:39. doi: 10.1186/1472-6947-12-39. https://bmcmedinformdecismak.biomedcentral.com/articles/10.1186/1472-6947-12-39. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Weitzman E, Kaci L, Mandl K. J Med Internet Res. Sharing medical data for health research: The early personal health record experience. J Med Internet Res Internet Journal of Medical Internet Research; 2010. May 25, Sharing medical data for health research: the early personal health record experience; p. e14. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Politi Mary C, Lewis Carmen L, Frosch Dominick L. Supporting shared decisions when clinical evidence is low. Med Care Res Rev. 2013 Feb;70(1 Suppl):113S–128S. doi: 10.1177/1077558712458456. [DOI] [PubMed] [Google Scholar]
- 22.Helmick C, Felson D, Lawrence R, Gabriel S, Hirsch R, Kwoh C, Liang M, Kremers H, Mayes M, Merkel P, Pillemer S, Reveille J, Stone J, National Arthritis Data Workgroup Estimates of the prevalence of arthritis and other rheumatic conditions in the United States. Part I. Arthritis Rheum. 2008 Jan;58(1):15–25. doi: 10.1002/art.23177. doi: 10.1002/art.23177. [DOI] [PubMed] [Google Scholar]
- 23.Weitzman E, Wisk L, Salimian P, Magane K, Dedeoglu F, Hersh A, Kimura Y, Mandl K, Ringold S, Natter M. Adding patient-reported outcomes to a multisite registry to quantify quality of life and experiences of disease and treatment for youth with juvenile idiopathic arthritis. J Patient Rep Outcomes. 2018;2:1–11. doi: 10.1186/s41687-017-0025-2. http://europepmc.org/abstract/MED/29645010. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Wagner E. Chronic disease management: what will it take to improve care for chronic illness? Eff Clin Pract. 1998;1(1):2–4. [PubMed] [Google Scholar]
- 25.Ringold S, Beukelman T, Nigrovic P, Kimura Y, CARRA Registry Site Principal Investigators Race, ethnicity, and disease outcomes in juvenile idiopathic arthritis: a cross-sectional analysis of the Childhood Arthritis and Rheumatology Research Alliance (CARRA) Registry. J Rheumatol. 2013 Jun;40(6):936–42. doi: 10.3899/jrheum.121147. [DOI] [PubMed] [Google Scholar]
- 26.Natter M, Quan J, Ortiz D, Bousvaros A, Ilowite N, Inman C, Marsolo K, McMurry A, Sandborg C, Schanberg L, Wallace C, Warren R, Weber G, Mandl K. An i2b2-based, generalizable, open source, self-scaling chronic disease registry. J Am Med Inform Assoc. 2013 Jan 01;20(1):172–9. doi: 10.1136/amiajnl-2012-001042. http://europepmc.org/abstract/MED/22733975. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Varni J, Seid M, Rode C. The PedsQL: measurement model for the pediatric quality of life inventory. Med Care. 1999 Feb;37(2):126–39. doi: 10.1097/00005650-199902000-00003. [DOI] [PubMed] [Google Scholar]
- 28.Amtmann D, Cook K, Jensen M, Chen W, Choi S, Revicki D, Cella D, Rothrock N, Keefe F, Callahan L, Lai Jin-Shei. Development of a PROMIS item bank to measure pain interference. Pain. 2010 Jul;150(1):173–82. doi: 10.1016/j.pain.2010.04.025. http://europepmc.org/abstract/MED/20554116. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Bulatović Maja, Heijstek M, Verkaaik M, van Dijkhuizen E H Pieter, Armbrust W, Hoppenreijs E, Kamphuis S, Kuis W, Egberts T, Sinnema G, Rademaker C, Wulffraat N. High prevalence of methotrexate intolerance in juvenile idiopathic arthritis: development and validation of a methotrexate intolerance severity score. Arthritis Rheum. 2011 Jul;63(7):2007–13. doi: 10.1002/art.30367. doi: 10.1002/art.30367. [DOI] [PubMed] [Google Scholar]
- 30.Jolliffe IT. Principal Component Analysis, Second Edition. New York, USA: Springer-Verlag New York; 2002. [Google Scholar]
- 31.Dunteman G. Principal components analysis. Beverly Hills: Sage Publications; 1989. pp. 65–69. [Google Scholar]
- 32.Domecq J, Prutsky G, Elraiyah T, Wang Z, Nabhan M, Shippee N, Brito J, Boehmer K, Hasan R, Firwana B, Erwin P, Eton D, Sloan J, Montori V, Asi N, Dabrh A, Murad M. Patient engagement in research: a systematic review. BMC Health Serv Res. 2014 Feb 26;14:89. doi: 10.1186/1472-6963-14-89. https://bmchealthservres.biomedcentral.com/articles/10.1186/1472-6963-14-89. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Hibbard J, Greene J, Overton V. Patients with lower activation associated with higher costs; delivery systems should know their patients' 'scores'. Health Aff (Millwood) 2013 Feb;32(2):216–22. doi: 10.1377/hlthaff.2012.1064. [DOI] [PubMed] [Google Scholar]
- 34.Hibbard J. Patient Activation and Health Literacy: What's the Difference? How Do Each Contribute to Health Outcomes. Stud Health Technol Inform. 2017;240:251–262. [PubMed] [Google Scholar]
- 35.Keim-Malpass J, Letzkus L, Kennedy C. Parent/caregiver health literacy among children with special health care needs: a systematic review of the literature. BMC Pediatr. 2015 Aug 05;15:92. doi: 10.1186/s12887-015-0412-x. https://bmcpediatr.biomedcentral.com/articles/10.1186/s12887-015-0412-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Friis K, Lasgaard M, Osborne R, Maindal H. Gaps in understanding health and engagement with healthcare providers across common long-term conditions: a population survey of health literacy in 29,473 Danish citizens. BMJ Open. 2016 Jan 14;6(1):e009627. doi: 10.1136/bmjopen-2015-009627. http://bmjopen.bmj.com/cgi/pmidlookup?view=long&pmid=26769783. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Rescorla L, Ginzburg S, Achenbach T, Ivanova M, Almqvist F, Begovac I, Bilenberg N, Bird H, Chahed M, Dobrean A, Döpfner Manfred, Erol N, Hannesdottir H, Kanbayashi Y, Lambert M, Leung P, Minaei A, Novik T, Oh K, Petot D, Petot J, Pomalima R, Rudan V, Sawyer M, Simsek Z, Steinhausen H, Valverde J, Ende Jan van der, Weintraub S, Metzke C, Wolanczyk T, Zhang E, Zukauskiene R, Verhulst F. Cross-informant agreement between parent-reported and adolescent self-reported problems in 25 societies. J Clin Child Adolesc Psychol. 2013;42(2):262–73. doi: 10.1080/15374416.2012.717870. [DOI] [PubMed] [Google Scholar]
- 38.Devine E, Alfonso-Cristancho R, Devlin A, Edwards T, Farrokhi E, Kessler L, Lavallee D, Patrick D, Sullivan S, Tarczy-Hornoch P, Yanez N, Flum D, CERTAIN Collaborative A model for incorporating patient and stakeholder voices in a learning health care network: Washington State's Comparative Effectiveness Research Translation Network. J Clin Epidemiol. 2013 Aug;66(8 Suppl):S122–9. doi: 10.1016/j.jclinepi.2013.04.007. http://europepmc.org/abstract/MED/23849146. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Beskow LM, Burke W, Fullerton SM, Sharp RR. Offering aggregate results to participants in genomic research: opportunities and challenges. Genet Med. 2012 Apr;14(4):490–6. doi: 10.1038/gim.2011.62. http://europepmc.org/abstract/MED/22261761. [DOI] [PMC free article] [PubMed] [Google Scholar]