Assessing Participant-Centered Outcomes to Improve Clinical Research

The value of patient-centered outcome measures for improving the care and satisfaction of patients is now well established, and the U.S. health care system — prodded by such developments as the newly established Patient-Centered Outcomes Research Institute (PCORI) — has begun incorporating patient feedback into quality-improvement efforts.¹ In contrast, although intense interest has been expressed in the ethical conduct of clinical studies, research participants' perspectives on their research experiences, such as whether the informed consent process properly and completely prepared them for a research study, are virtually never systematically collected. Indeed, though one might imagine that survey research could be accomplished as an extension of the requisite data gathering for clinical trials, we are unaware of any validated surveys that obtain empirical data on research participants' experiences and perspectives in an effort to evaluate the effectiveness of current practices as the basis for improving processes. Such information may be especially important in the expanding areas of genetic research where there are strong disagreements among investigators, bioethicists and other research professionals, such as in the reporting of incidental findings identified by next generation DNA sequencing.

To begin to address this deficiency in the clinical research-improvement process, we used qualitative and quantitative methods to develop and validate a standardized Research Participant Perception Survey based on themes derived from focus-group discussions involving research participants and research professionals. We deployed the survey to 18, 890 research participants at 15 U.S.-based clinical research centers supported by the National Institutes of Health (NIH) — 13 Clinical and Translational Science Award (CTSA) sites, 1 General Clinical Research Center site, and the NIH Clinical Center (see Supplementary Appendix, available with the full text of this article at NEJM.org).^2,3

A total of 4961 surveys (29% of those delivered) were returned from participants of diverse ethnic and racial backgrounds (85% white, 12% black, 5% Hispanic, 3% Asian), of whom 61% were female (in the centers that provided data on sex) and 37% were healthy volunteers. The demographic distribution of responders approximated that of the sample population of the participating centers. The survey fielding met standards for tests of face value and content validity, survey and item completion, and psychometric analysis (validation).³ Response rates varied among sites, from 18% to 74%, depending largely on whether or not sites permitted full implementation of survey mailing, or restricted the population to which surveys could be sent. Responses to questions about participants' overall experience were similar at sites with high and low response rates.

The table summarizes the responses to selected questions. In aggregate, 73% of participants rated their overall research experience very highly (9 or 10 on a 10-point scale). Similarly, 66% said they would “definitely” recommend research participation to friends or family members, and 31% said they would “probably” do so. Participants were more likely to rate their overall experiences very highly when they trusted the investigators and nurses; felt that investigators and nurses treated them with respect, listened to them, gave them understandable answers to their questions; and could meet with the principal investigator as much as they wanted.

One of our aims was to assess whether the informed consent process ensured that participants understood the details of the research and their role in the study and that their consent was given voluntarily. A majority of participants indicated that they did not feel pressure from research staff to join the study (94%); believed that the consent form covered the study's risks (81%), included study details (80%), and was understandable (78%); and said they had sufficient time to evaluate whether to participate (79%). Two thirds (67%) indicated that the informed consent process completely prepared them, and an additional 25% said it “mostly prepared” them, for participation.

One striking finding was that most participants wanted to receive information about the results of the study. A small minority (23%) of participants reported having received a general summary of research results. Of those who did not receive a summary, 85% indicated that they would have liked to receive one. When asked to rate items that “would be important in a future study,” 72% of respondents rated as “very important” having a “summary of the overall research results shared with me.”

Trust also emerged as an important theme. Overall, 86% of respondents said they trusted the research team completely. Of these, most felt that they were treated with courtesy and respect (99%), were treated as valued partners (79%), and were listened to carefully by investigators (93%) and research coordinators (95%). Analyses of racial and ethnic subgroups revealed that white participants had a higher level of trust for the research team — 88% giving the highest rating — than did the four nonwhite groups, in which 78 to 82% gave the highest trust rating. This difference in trust levels raises important questions that will require more data to answer.

Since we conducted our survey primarily at major academic medical centers that conduct clinical research funded through three NIH programs, the results may not be generalizable beyond this cohort. However, the 61 institutions receiving CTSAs, which are broadly distributed throughout the United States, represent an important segment of U.S. clinical research and offer an infrastructure for identifying successful practices and testing their broad implementation. Our 29% response rate also limits generalizability, although it closely matches response rates nationally for the Consumer Assessment of Healthcare Providers and Systems (CAHPS) patient care survey³, and we did not observe major differences in responses between sites with high response rates and those with low rates.

Since the 1970s, human research subjects in the United States have been protected through a “prospective regulatory approach” involving review of studies by institutional review boards (IRBs). IRB review relies primarily on process indicators, such as properly produced and executed informed consent forms, rather than measurement of outcomes such as whether participants believed the consent process provided the information they needed to make an informed choice. Klitzman and Appelbaum recently recommended modifying current methods to include retrospective analysis based on “objective, validated questionnaires” to assess “how well subjects understood the study or whether they were distressed by the research procedures.”⁴ We believe that our survey contributes to this important goal. For example, knowing that one third of participants didn't feel completely prepared by the consent process should be an impetus for identifying deficiencies, designing interventions to improve the consent process, and assessing the effects of interventions on outcomes by analyzing future surveys.

We also designed our survey to provide other participant-centered outcomes data on the clinical research enterprise, such as: participants' motivations to join a study and complete participation, outcomes addressing classical patient-centered dimensions of care measured in hospital surveys such as respect for patient preferences, the quality of information, education and communication, and aspects of the continuity of care tailored to the research context.³ The recent Institute of Medicine report reviewing the CTSA program emphasized the need for a “learning health care system,”⁵ and we believe our survey could be an important component of a “learning clinical research system.” Broad participation by CTSA-funded and other institutions in using and refining this questionnaire would provide both robust benchmarking data and opportunities for identifying and disseminating best practices.

Our findings provide encouraging news but also reveal opportunities for improvement. We were reassured that most participants have positive perceptions of the research experience and would recommend participation to others. Conversely, the data on informed consent processes underscore the need for performance-improvement activities that are driven by these data and then assessed by means of future data.

Research Participants' Responses to Selected Questions

Use the scale to rate your overall experience in the research study, where 0 is the worst possible experience and 10 is the best possible experience. 9 or 10 73% 8 or less 27%

Would you recommend joining a research study to your family and friends? Definitely 66% Probably 31% Other responses 3%

During your discussions about the research study, did you feel pressure from the research staff to join the study? Never 94% Sometimes, Usually, or Always (combined) 6%

Were the risks of joining the study included in the informed consent form? Always 81% Usually, Sometimes, or Never (combined) 19%

Were the details of the study details included in the informed consent form? Yes 80% No 20%

After the study was explained to you, did you have enough time to think about your decision before signing the informed consent form? Yes 79% No 21%

Was the consent form written in a way that you could understand? Always 78% Usually, Sometimes, or Never (combined) 22%

Did the informed consent form prepare you for what to expect in the research study? Completely 67% Mostly 25% Somewhat or Not at all (combined) 8%