Predictive Analytics and the Return of “Research” Information to Participants

Abstract

The World Health Organization (WHO) estimates older adults aged 60+ will double by 2050 with 80% living in low to moderate income countries. As remote research studies supported by digital devices increase separation between researchers and participants, it is important to maintain participant trust. Research participants have expressed an interest in accessing both group and individual level results, which are not readily available. To bridge this gap, we engaged residents of a local continuing care senior housing community (CCSHC) to co-design documents used to convey information about study results. The process informed the refinement of informational materials for communicating scientific research that the CCSHC community considers accessible and meaningful.

1. Introduction

Globally, the number of adults aged 65 and older is currently one in 11 (9%). By 2050, that number is expect to grow to one in 6 (16%), with the absolute number set to double from 0.73 billion to 1.55 billion [1]. With the increasing population of older adults, research involving those over 65 is also increasing where technology plays an increasingly important role. Often referred to as ‘digital’ or ‘mobile’ health research, sensor technologies may be used to monitor and/or intervene with older adults with chronic diseases [2]. Digital health research is increasing across all demographics including hard to reach and vulnerable subpopulations [3]. Researchers can now obtain continuous participant data in real-time and over long durations with minimal involvement from research participants, creating unprecedented data that can answer important research questions. Moreover, digital tools are changing the research landscape with respect to how participants are involved in research. The Precision Medicine Initiative, also known as the All of Us Research Program (AoURP), is a case in point. In this study, one million people will contribute to a research database to help researchers observe how genes, behavior and environment influence human health [4]. Each participant will contribute psychological, physical, lifestyle and environmental data and will be expected to not only contribute at study onset but also to remain engaged for longitudinal data collection. This ambitious research platform has not only advanced the dialogue about participant involvement in the design of research but also in determining what will motivate participant ongoing engagement – both short and long-term.

Questions about participant engagement has prompted discussion about whether and how to return study results to participants. While sharing results may seems intuitive, for many valid reasons it is not a common practice due to clinical relevance, premature diagnoses, and feasibility issues [5]. However, researchers are increasingly interested in how to influence participant engagement to increase retention in longitudinal studies. Concurrently, participant demands for greater transparency has been on the rise. In 2018, the National Academies published a consensus report of recommendation for the return of research results drawing attention to the potential benefits, harms, and costs [6]. Related research has documented the benefits of greater transparency but, also has recognized that research data may not be accurate nor make sense to the individual [7].

During prior research, participants expressed a desire for accessible and personalized feedback at both the group and individual level [8]. The study reported here builds on this interest by studying how to communicate study information to research participants. This study is part of a larger longitudinal, observational study of physical, cognitive, and mental health in participants residing independently at a continuing care senior housing community (CCSHC) [9].

2. Methods

The study purpose was to obtain participant input on strategies to communicate group- and individual-level research information. The UC San Diego Institutional Review Board approved this research.

2.1. Study 1: Return of Group-Level Research Information

Study 1 used a qualitative approach to obtain feedback regarding communication of group-level research results. Participants for this co-design session were recruited by sharing study information with retirement community residents. Those interested in a 60–90 min focus group session met in November 2019. The lead researcher (CN) facilitated the group discussion with an undergraduate research assistant (SW) managing audio recording and notetaking. Informed consent was obtained prior to commencing the session.

To identify how to share group level research results, we identified two peer-reviewed publications reporting data collected under the parent study in Healthcare and the American Journal of Geriatric Psychiatry (AAGP) [8, 9]. In addition, we obtained documents reporting on each publication in the form of an article in Forbes Magazine and a press release created by the university science communications. From publications and public facing documents two infographic materials were created with a goal of conveying study questions, approach, analysis, study results and conclusions (see Fig. 1 (c) and 2 (b)). Participants were asked to review and comment on these three formats for communicating group-level study information.

Fig. 1.

(a) is the peer-reviewed publication in Healthcare, (b) is a Forbes Magazine article based on it, and (c) is the infographic summary.

Fig. 2.

(a) is the press release and (b) is the infographic summarizing about research published in the AAGP.

2.2. Study 2: Return of Individual-Level Research Information

Study 2 was designed to communicate individual-level wearable sensor data to participants, with two separate phases.

Phase 1:

Development of the Return of Information Protocol. The Fitbit Charge 3 wearable fitness sensor was selected for use in the parent study to assess sleep, heart rate, and physical activity in free-living environments [10, 11]. The Fitabase platform [12] was used to monitor the devices and data collection. Participants were recruited through community-wide talks that described the study and devices. Research staff taught participants how to operate the device and navigate the Fitbit app. Participants without smartphones were visited by staff weekly to download device data. Participants were instructed to wear the device continuously to allow for 24-h monitoring. After 2–4 weeks, the device was retrieved, and data was migrated from Fitabase to a database. Researchers developed feedback reports to create understandable visual representations of the individual-level data, based on the research group’s input.

Phase 2:

Providing Individual-Level Return of Value and Evaluating User Experience. Researchers met with participants in a one-on-one setting to share individual-level data reports. The research staff explained the findings, monitoring the participants’ understanding of the data and encouraging them to contact their clinical providers for concerns. Participants were asked to rate using the device and app, and availability of staff help [on a 5-point Likert scale (“Strongly Agree” to “Strongly Disagree”)], as well as what they enjoyed most and least about the study.

3. Results

3.1. Study 1: Return of Group-Level Research Information

A total of six residents (three male and three female) participated. Several participants were relatively new to the retirement community and a few had contributed to our earlier research in 2018 that prompted our interest in communicating research information.

Open Access Journal.

Participants displayed apprehension about understanding the material due to unfamiliar vocabulary and phrases. While the material “seem[ed] very extensive,” it looked “very intimidating.” Participants preferred a more concise summary “to say this study is about such and such, and this is what we found out.” While most preferred fewer details in favor of clarity, one participant liked reading academic papers for deeper and more comprehensive insight into the research.

Forbes Magazine Article.

Participants were more receptive to this “public-friendly” version of the studies due to their more concise and understandable language. However, participants saw this method of synthesizing the research study as too generalized, commenting that it “[doesn’t] tell me where I stand.” It also lacked specific research details seen in the research paper and was not written with research participants in mind.

Infographic.

The group indicated it was “nicer” and that it used accessible terminology. Particularly appreciated was the use of charts and iconography, as well as the contrasting color palette of “dark with bright colors,” to differentiate blocks of information.

When asked how it could be improved, participants mentioned the terminology used still posed challenges to understanding the material. Participants suggested conveying the results in simpler language, saying that “even here there [are] words that are not familiar.” The physical size was also a source of frustration, with some pointing out that a smaller font size makes colored text blocks harder to read. “For this audience and age group, I think it can be a reasonable size and make it larger, split it up.”

Taking into account comments across from both infographic examples, new designs were developed with a focus on emphasizing the visual characteristics by enhancing contrast and segmentation between sections. Given the participant desire for larger size of the content, text and graphical content were resized and reorganized across three pages. Complexity of terms and phrases was also considered, resulting in a rewrite with simpler terms or with extra footnotes to explain a technical phrase (Fig. 3).

Fig. 3.

Improved multi-page infographics

3.2. Study 2: Return of Individual-Level Research Information

Phase 1.

Seven participants (six women, one man) wore the device for 2–4 weeks, with usage ranging from wearing/charging it to active self-monitoring daily activity levels.

The information available to participants from the device and app was limited to the current day’s steps, calories and heart rate. The Fitbit app displayed weekly activity and sleep parameters but lacked day-to-day trends over multiple weeks and summary data. The app data had limited explanations, requiring paid subscription services to analyze restlessness and lacking definitions of terms like Sleep Score. The app and device interfaces were not geared toward older adults: small font and non-intuitive navigation. With these limitations in mind, the plan to return information considered which data to include and how to best display the data. The preliminary feedback reports were based on Fitabase output (unique in format and structure from the app and device data) from research team members who wore the device for two weeks. Eight half-page graphs and charts were created to display longitudinal information on sleep efficiency, bedtimes, restlessness, wake after sleep onset, daily steps, daily distance travelled, daily steps vs. distance travelled, and activity level. The graphs were presented to the research team and streamlined to remove measures that were potentially confusing, unclearly depicted, or lacked clinical utility, resulting in four charts.

The final feedback report prepared for each participant displayed longitudinal trends of daily bedtimes, total sleep time (and sleep efficiency), step count, and resting heart rate. Study staff met with participants to explain the reports. Based on their suggestions, the report was altered to maximize readability with large font 2: (16 point), one graph per page, and use of bright, contrasting colors. A short description under each graph explained the meaning of each measure and provided context to interpret the data in comparison to other older adults (Fig. 4).

Fig. 4.

Example of the Fitbit feedback report given to participants.

Phase 2.

Participants reported high levels of satisfaction with their experiences during the study overall. During feedback sessions, participants expressed interest in personalized results and appreciated the visualizations. While all participants agreed or strongly agreed that the device was comfortable, three noted itchy wrists or skin irritation, and one noted an oversized wristband. The device rated as “easy to sync” in the neutral to strongly agree range. The Fitbit app/device was rated easy to use from neutral to strongly agree. On average, participants agreed that the research staff was available for help. Five participants noted that they most enjoyed “learning about health info,” “usefulness of data feedback,” and “data tracking was interesting.”

4. Discussion

The literature on return of research results has focused on the return of individual-level information. In fact, Institutional Review Boards (IRB) have typically discouraged the disclosure of participant results with some exceptions – incidental or secondary findings. For example, in research involving brain imaging, the research team and IRB must determine the medical significance of the observation and how to communicate it to the participant. The ethics literature on reporting incidental or secondary findings has focused on brain imaging [13, 14] and genetic sequencing [15–17]. The research reported here was conducted in response to a shifting research ecosystem where participants are more engaged in the process as partners and are interested in what is being learned about them as a group and as individuals.

While the return of group-level information gets little attention, it is increasingly an expectation of participants who have contributed their time and data to the research. When vetting the materials depicting group-level results, participants were eager to learn about how their participation was making a difference to the scientific community. How the information is presented influences both access to and usefulness of the communications. Peer-reviewed publications are written in scientific language and in many cases, require a fee to access. By creating and evaluating our initial steps of making a peer-reviewed publication accessible to those who contributed, we are demonstrating our respect for participants and our value of their partnership.

With respect to the return of individual-level information, participants found information useful and engaging. All expressed interest in learning more about their health patterns, a key factor in motivating their participation. To return information in an understandable and meaningful way, shared findings should be limited in scope and the user’s experience should be continually assessed. While the Fitbit study would be easiest to deploy within a tech-savvy subset of the population (i.e., those with smartphones and prior experience with wearable sensors), it is paramount that such studies recruit people with less experience with technology to prevent bias.

Despite limitations such as a small sample size and lack of long-term followup, we were able to develop prototypes for communicating both group and individual-level information in an accessible, aesthetically pleasing, and potentially meaningful manner.

5. Conclusion

Involving participants in the process of designing research and reporting study results is an important step toward authentic engagement of participants as partners. In this study, a small sample of participants contributed to shaping how we communicate the return of individual and group-level research results. Future research is needed to assess the extent to which accessible communications of research results influence participant motivation to engage in longitudinal research over the short and long-term.