Comparison of Approaches for Notification and Authorization in Pragmatic Clinical Research Evaluating Commonly Used Medical Practices

Abstract

Background

For pragmatic clinical research comparing commonly used treatments, questions exist about if and how to notify participants about it and secure their authorization for participation.

Objective

To determine how patients react when they seek clinical care and encounter one of several different pragmatic clinical research studies.

Research Design

In an online survey using a between-subjects experimental design, respondents read and responded to one of 24 hypothetical research scenarios reflecting different types of studies and approaches to notification and authorization (eg, general notification, oral consent, written consent).

Subjects

English-speaking US adults 18 years and older.

Measures

Willingness to participate in the hypothetical study, acceptability of the notification and authorization approach, understanding of the study, perceptions of benefit/harm, trust, and perception of amount of study information received.

Results

Willingness to participate did not differ by notification and authorization approach. Some (21% to 36%) of the patients randomized to general notification with an explicit opt-out provision were not aware they would be enrolled by default. Acceptability was greatest for and similar among notification and authorization approaches that actively engaged the patient (eg, oral or written consent) and lower for approaches with less engagement (e.g., general notification). Problems of understanding were found among 20% to 55% of respondents, depending on the particular scenario. Most respondents (77% to 94%) felt that participation in the hypothetical study posed no risks of harm to their health or privacy.

Conclusions

Current attitudes about notification and authorization approaches and difficulties understanding pragmatic clinical research pose significant challenges for pragmatic research. Data from this study provide a starting point to developing solutions to these surprisingly complex issues.

Introduction

Substantial efforts are now being directed at improving the evidence base of health care decision making through research that compares commonly used medical practices conducted in the context of usual care settings. Such “pragmatic” clinical research includes both observational and randomized designs such as comparative effectiveness research (CER). The interventions evaluated vary widely, including preventive measures, diagnostic algorithms, individual treatments, organizational changes, and educational programs. By design, this research aims to entail less risk and burden than traditional clinical trials. As such, “questions arise about the appropriate mechanisms to protect the rights, interests, and welfare of research participants, including whether to adopt all or some of those mechanisms used to protect those enrolled in traditional clinical trials”(p. 71).¹ In particular, there are marked uncertainties concerning if, when, and how to notify participants about a specific research study and secure their authorization for participation (eg, general notification with an option to opt out, oral consent, written informed consent).

Several empirical studies have addressed people’s preferences for and reactions to different approaches to notification and authorization for pragmatic clinical research.^2–8 Although some of this research is reviewed elsewhere,¹ much of it involved providing extensive education about CER/pragmatic trials prior to assessing participants’ attitudes toward it. In addition, participants in these studies were typically asked to compare different notification and authorization approaches. These studies have generated important insights, yet it is unclear how people may react when they seek clinical care and encounter any given type of pragmatic clinical research study with a particular approach to notification and authorization.

To address this gap in understanding, we conducted a large, national experiment in which participants were randomly assigned to 1 of 24 hypothetical research scenarios with a specified notification and authorization approach, thereby having greater verisimilitude to what might be encountered in actual clinical settings. This approach also allowed us to compare the effects of different approaches to notification and authorization (in terms of willingness to participate in the research, perceived acceptability of the approach, understanding, trust, and other outcomes) and to examine differences among approaches for a variety of types of pragmatic research studies.

Methods

Sample

Sample selection and online administration of the survey was managed by GfK (formerly KnowledgeNetworks). During the field period between March 9 and 26, 2016, 4879 English-speaking potential respondents 18 years and older were randomly sampled from GfK’s Web-enabled KnowledgePanel of 55,000 US adults. KnowledgePanel members are recruited using probability selection algorithms, which enable results from the panel to statistically represent the US population with a consistently higher degree of accuracy than results obtainable from volunteer opt-in panels.⁹ Participating households that lack Internet access are provided a Web-enabled computer and free Internet service so they can also participate in online panels. Potential respondents were emailed an invitation to participate in the study. Nonresponders received an email reminder after 3 days. Respondents who completed the survey received the equivalent of $5 for their time.

Design Overview

We used a between-subjects design in which respondents were randomly assigned to read and respond to a description of 1 of 24 hypothetical research scenarios, based on 6 hypothetical studies. The 6 hypothetical studies were constructed to reflect some (but not all) combinations of design (observational, individual randomization, and cluster randomization), level of clinical risk (lower versus higher stakes), and type of intervention (individual pharmacotherapy or devices used at an institution). Not all combinations were created because we sought to create realistic studies. Within each of these 6 studies, we created multiple scenarios with differing approaches to notification and authorization . The 6 studies did not have the same approaches to notification and authorization, because we sought to compare only those approaches that were plausible for a specific type of study. For the 4 studies that compared individual pharmacotherapies, the possible notification and authorization approaches included (a) general notification by a poster (with the option to opt out); (b) oral consent; (c) oral consent and an information sheet; (d) a conversation with a doctor and written consent describing only the study risks; or (e) a conversation with a doctor and written consent describing both the clinical and study risks. For the 2 studies that compared different devices used at an institutional level, the possible notification and authorization approaches included (a) post-study notification, in which survey participants learned of the study after it was completed via a newsletter in the mail; (b) general notification with a poster that included an opt-out provision notifying patients that to opt out of the study they would need to seek care elsewhere; and (c) general notification with a poster with 2 opt-out provisions (patients could opt out of having their medical records included in the study but still receive the intervention assigned to that institution, or they could opt out of the study completely by seeking care elsewhere). Appendix 1 summarizes the design and describes the scenarios in greater detail.

Materials and Methods

The study scenarios were drafted based on prior focus group findings⁶ and input from the principal investigators involved with the NIH Health Care Systems Research Collaboratory, a national effort to conduct a series of major pragmatic clinical trials (https://www.nihcollaboratory.org/Pages/default.aspx), who reviewed the scenarios for clinical relevance and authenticity. The study team created posters, information sheets, and consent forms corresponding to notification and authorization approaches to accompany the hypothetical research scenarios, using samples of actual study documents as models. To ensure that the fictional study documents were realistic reflections of what an institutional review board (IRB) might approve, we enlisted 6 IRB members from the Collaboratory sites to review sets of study documents for 2 to 3 study scenarios. Each set of documents was reviewed by at least 1 IRB chair and 1 other IRB member from different institutions.

Study consent and study execution were conducted online via GfK. After agreeing to participate, respondents read the description of the research scenario to which they were randomly assigned, including any documents (eg, written consent form). They then answered a series of questions described below. The institutional review boards of [REDACTED] approved this research.

Measures

The study team drafted closed-ended questions that were asked about each research scenario: willingness to participate in the hypothetical research study (yes/no); acceptability of the approach to notification and authorization (acceptable/unacceptable); 3 questions testing understanding that (a) the interventions being compared are commonly used, (b) the intervention is chosen randomly, and (c) nothing extra is required of participants beyond their clinical care (all correct/incorrect); perceptions regarding participation’s effect on (a) how they would be treated by the doctor, (b) their health, and (c) how well their health information would be protected (all better/same/worse); how the notification and authorization approach affected their trust in (a) the doctor and (b) the place where they received care (both increased/no effect/decreased); and the perceived amount of information they received (too much/about right/too little). For most of these items, open text fields were provided to allow people to explain their answers. The survey also contained additional questions about respondents’ attitudes toward research and healthcare in general, which will be analyzed and reported separately. All items created by the study team were evaluated and revised through 5 rounds of cognitive interviews (n = 31).

After the survey data were collected, prebanked information previously collected by GfK on panel members’ demographic characteristics and backgrounds was added to the data set.

Analysis

Respondents who sped through the experiment (less than 2 minutes to read the scenario and answer all questions) and/or were missing greater than one-third of the responses were excluded from all analyses. We summarized respondents’ sociodemographic characteristics using standard descriptive statistics and compared them across experimental groups using 1-way analysis of variance and chi-square tests. Within each of the 6 hypothetical research studies, we compared binary outcomes among the notification and authorization approaches using chi-square tests and described using 95% CIs. We compared ordinal categorical responses (eg, “worse,” “same,” or “better”) among notification and authorization approaches within study using Mantel-Haenszel tests. We interpreted a statistically significant (P < .05) difference of 15 percentile points or more between notification and authorization approaches within a given study type as relevant for policy and practice. All statistical analyses were conducted using SAS version 9.4 (SAS Institute, Cary, NC). Respondents’ brief open-ended explanations of their survey responses were assigned qualitative content codes using an inductive strategy of code generation and revision.

Results

Of the 4879 US adults sampled from GfK’s Knowledge Panel, 2994 completed the experiment. After excluding 39 respondents who sped through the experiment and/or were missing more than one-third of the responses, the final analyzable sample was 2955 (61.4% completion rate). The sample was diverse with respect to age, education, race/ethnicity, and geographic region (Table 1). Sample sizes for the scenarios differed due to changes in the study design after the contract with GfK was finalized, resulting in larger sample sizes for scenarios 19 through 24 (average N = 216) relative to the first 18 scenarios (average N = 92). There were no statistically significant differences among the 24 scenarios with respect to the sociodemographic characteristics assessed (all P values > .05, without adjustment for multiple comparisons).

Table 1.

Characteristics of the Study Sample

Characteristic	Overall (n = 2995)
Age, No. (%)
18–29 y	460 (15.6)
30–44 y	676 (22.9)
45–59 y	861 (29.1)
≥ 60 y	958 (32.4)
Education level, No. (%)
Less than high school	227 (7.7)
High school	849 (28.7)
Some college	828 (28.0)
Bachelor’s degree or higher	1051 (35.6)
Race/ethnicity, No. (%)
Black, non-Hispanic	268 (9.1)
Hispanic	299 (10.1)
Two or more races, non-Hispanic	86 (2.9)
White, non-Hispanic	2169 (73.4)
Other, non-Hispanic	133 (4.5)
Sex, No. (%)
Female	1473 (49.8)
Male	1482 (50.2)
Current relationship status, No. (%)
Married	1714 (58.0)
Living with partner	138 (4.7)
Separated	49 (1.7)
Divorced	312 (10.6)
Widowed	156 (5.3)
Never married	586 (19.8)
Metropolitan statistical area status, No. (%)
Metro	2526 (85.5)
Nonmetro	429 (14.5)
Region - based on state of residence, No. (%)
Northeast	568 (19.2)
Midwest	692 (23.4)
South	1021 (34.6)
West	674 (22.8)
Current employment status, No. (%)
Working - as a paid employee	1534 (51.9)
Working - self-employed	206 (7.0)
Not working - on temporary layoff from a job	22 (0.7)
Not working - looking for work	127 (4.3)
Not working - retired	617 (20.9)
Not working - disabled	200 (6.8)
Not working - other	249 (8.4)
Household Internet access, No. (%)
No	479 (16.2)
Yes	2476 (83.8)

Willingness to Participate in the Hypothetical Study

Figure 1 shows the percent of respondents who would be willing to be included in the hypothetical research study to which they were assigned. Those willing to be included ranged from 52% to 72% when general notification was used, 51% to 55% for oral consent, 49% to 58% for oral consent with an information sheet, and 53% to 59% for written consent. Willingness of participation did not vary by notification and authorization approach in any of the 6 study types (p > .06 for all comparisons).

Figure 1.

Percent of Respondents (with 95% confidence intervals) Willing to be Included in the Hypothetical Research Study to Which They Were Assigned.

(POST = Post-study notification; GN = General notification; GN-OO = General notification with the option to exclude their medical records from the study; O = Oral consent; O+I = Oral consent and a one- page study information sheet; WC (MR) = Written consent for medical record review; WC (I) = Written consent detailing the incremental risks; WC (C) = Written consent detailing incremental and clinical risks; see Appendix A for more details.)

Based on respondents’ qualitative explanations for unwillingness to participate, some believed study participation would pose additional risks, which were associated with perceptions that one or both treatments were experimental, that the organization and/or researchers conducting the study may not be reputable, that physicians may have conflicting interests, and—most frequently—that privacy and confidentiality of medical records could be compromised.

Of those respondents assigned to scenarios with general notification, 21% to 36% indicated that they were not aware that they would be automatically enrolled in the study unless they told someone they did not want to participate.

Acceptability of the Notification and Authorization Approach

As depicted in Figure 2, acceptability of notification and authorization approaches differed for all 6 study types (p ≤ .001). Acceptability was greatest for and similar among the notification and authorization approaches that involved actively engaging the patient (oral, oral consent and an information sheet, and written consent; range, 77%–93%). In contrast, approaches with less engagement (post-study notification and general notification) were much less acceptable (range, 31%–57%). Qualitative analyses of reasons for indicating an notification and authorization approach was specifically unacceptable revealed some differences that varied according to notification and authorization approach. For post-study notification and general notification, the most frequent reasons included a concern that patients may not be able to see or read a general notification that was posted, the desire to have a conversation about the study, the desire to be explicitly asked their permission, the desire for notification prior to the study (for those receiving no post-study notification), and the burden of the initiative required to opt-out. For the relatively small number of participants who found the more active approaches unacceptable, reasons included wanting more time to make the decision, the potential for “coercion” by the doctor, and legal liability for the institution or the patient.

Figure 2.

Percent of Respondents (with 95% Confidence Interval) Who Rated Their Approach to Notification and Authorization as Acceptable.

(POST = Post-study notification; GN = General notification; GN-OO = General notification with the option to exclude their medical records from the study; O = Oral consent; O+I = Oral consent and a one- page study information sheet; WC (MR) = Written consent for medical record review; WC (I) = Written consent detailing the incremental risks; WC (C) = Written consent detailing incremental and clinical risks; see Appendix A for more details.)

Understanding

Overall, respondents demonstrated greater understanding of the idea that the interventions being tested are commonly used (average 80% correct) relative to understanding of random assignment to treatments (average 45% correct) and that no extra things would need to be done for research (average 59% correct; Table 2). For “commonly used” and “random assignment,” there were some differences among the notification and authorization approaches for some scenarios, but there were no consistent differences across scenarios. Those who indicated that there would be “extra things” they would have to do as part of the research usually cited additional clinic visits and tests compared to usual clinical care.

Table 2.

Percent of Respondents Who Answered Correctly to Understanding Items by Scenario

Scenarioa	Commonly Usedb		Random Assignmentc		Extra Thingsd
	Percent (95% CI)	pe	Percent (95% CI)	pe	Percent (95% CI)	pe
Medical Records: Blood Clot	77 (73–81)	0.01	24 (20–28)	0.31	57 (52–62)	0.31
GN	85 (78–92)		29 (20–38)		60 (51–70)
O	77 (69–85)		18 (11–26)		53 (43–63)
O+I	80 (72–88)		22 (13–30)		51 (41–61)
WC-MR	66 (57–75)		25 (17–33)		62 (53–71)
Medical Records: UTI	80 (76–84)	0.08	23 (19–27)	0.38	56 (51–60)	0.67
GN	72 (64–81)		28 (19–36)		53 (43–62)
O	86 (79–92)		25 (17–33)		61 (51–70)
O + I	83 (76–91)		20 (12–27)		56 (46–66)
MC - MR	80 (72–88)		19 (11–27)		54 (44–63)
Individual Randomization: Blood Clot	83 (79–86)	0.22	58 (54–63)	<0.001	51 (47–56)	0.92
GN	78 (68–87)		61 (50–72)		49 (38–60)
O	86 (79–93)		72 (64–81)		53 (43–62)
O + I	84 (75–93)		67 (56–78)		56 (44–67)
WC(C)	87 (81–94)		48 (39–58)		51 (41–60)
WC(I)	76 (66–86)		39 (28–51)		49 (37–61)
Individual Randomization: UTI	82 (78–85)	0.03	57 (52–61)	0.03	59 (54–64)	0.49
GN	74 (65–83)		64 (54–74)		53 (43–63)
O	76 (67–85)		54 (44–65)		59 (49–69)
O + I	87 (80–95)		67 (56–77)		65 (55–76)
WC(C)	88 (81–95)		43 (32–55)		55 (44–66)
WC(I)	86 (79–94)		54 (43–65)		63 (52–73)
Cluster Randomization: Surgical Rods	82 (79–85)	0.21	46 (42–50)	0.13	60 (55–65)	0.32
GN	85 (80–90)		51 (45–58)		62 (56–69)
NO/POST	80 (75–86)		42 (35–49)
OPT-Out	79 (73–85)		45 (38–52)		58 (51–64)
Cluster Randomization: Needles	78 (75–81)	0.99	55 (51–59)	0.27	72 (68–76)	0.73
GN	78 (72–84)		59 (53–66)		71 (65–77)
NO/POST	78 (72–83)		54 (48–61)
OPT-Out	78 (73–84)		52 (45–58)		73 (67–78)

^aPOST, post-study notification; GN, general notification; GN-OO, general notification with the option to exclude their medical records from the study; O, oral consent; O+I, oral consent and a 1-page study information sheet; WC (MR), written consent for medical record review; WC (I), written consent detailing the incremental risks; WC (C), written consent detailing incremental and clinical risks. See Appendix A for more information.

^b“In this imaginary research study, are both of the [medicines/needles/rods] commonly used to treat [specify condition]?” (Responses: Yes, No)

^c“If you were a participant in the research study, how would your [medicine/needle/rod] be chosen?” [Responses: Doctor chooses, Doctor and I choose together, Randomly (like flipping a coin), Unsure]

^d“Besides being examined by the doctor, undergoing any tests necessary for your clinical care, having the study explained to you, receiving an information sheet, and following your prescribed treatment plan, do you recall any extra things you would have to do as part of the study?” (Responses: Yes, No, Unsure)

^eChi-square P value testing the null hypothesis that the number of correct understanding responses is the same for every method of notification and authorization within scenario.

Perception of Personal Risks and Benefits

Respondents were asked how being a participant in the hypothetical research study would affect (a) how they were treated by the doctor, (b) their health, and (c) how well their health information would be protected. For all three, responses did not differ by notification and authorization approach (p ≥ .05 for all). Most respondents believed the treatment by the doctor would be the same (79%) or better (16%), with 6% believing it would be worse than if they did not participate in the study. A similar pattern was observed for the effect of participation on their health: same (86%), better (8%), worse (6%). In contrast, while the majority of respondents believed that participation in the study would lead to the same (66%) or better (11%) protection of their health information, a larger portion (23%) thought protection of health information would be worse.

Trust

Participants’ reports of how the notification and authorization approach affected their trust in (a) the doctor and (b) the place where they received care differed by approach for all 6 hypothetical research studies (p ≥ 0.001 for all). Because the pattern of findings was almost identical for trust in the doctor and in the place where they received care, Figure 3 shows only results for trust in the doctor. The majority of respondents in all groups felt that the notification and authorization approach either had no effect or increased their trust, with the exception of the 2 groups in which there was no notification prior to enrollment. In these 2 cases, at least half of respondents said that no prior notification decreased their trust. Across all groups, general notification was associated with greater distrust than other more active forms of notification and authorization.

Figure 3.

Respondents’ Perceptions of How Approach to Notification and Authorization Affected Their Trust in The Doctor.

(POST = Post-study notification; GN = General notification; GN-OO = General notification with the option to exclude their medical records from the study; O = Oral consent; O+I = Oral consent and a one- page study information sheet; WC (MR) = Written consent for medical record review; WC (I) = Written consent detailing the incremental risks; WC (C) = Written consent detailing incremental and clinical risks; see Appendix A for more details.)

Amount of Information

Perceptions about the amount of information provided varied significantly (p < .05) by notification and authorization approach for all but one (p = .11) of the 6 study types (Figure 4). At least half of all respondents in each experimental group judged the amount to be “about right.” While the differences among the notification and authorization approaches were not entirely consistent across study types, the number of respondents who felt the information was “too little” was greater for the more passive notification and authorization approaches (no prenotification or general notification) relative to the more active ones. Across all groups, no more than 7% of respondents believed there was “too much” information.

Figure 4.

Respondents’ Perceptions of the Amount of Study-Related Information Provided.

(POST = Post-study notification; GN = General notification; GN-OO = General notification with the option to exclude their medical records from the study; O = Oral consent; O+I = Oral consent and a one- page study information sheet; WC (MR) = Written consent for medical record review; WC (I) = Written consent detailing the incremental risks; WC (C) = Written consent detailing incremental and clinical risks; see Appendix A for more details.)

Conclusions

In this large experiment with a nationally representative sample, we elicited information about the acceptability of several types of pragmatic clinical research and attitudes toward a range of notification and authorization approaches that are commonly used for this type of research. Our findings begin to fill important gaps that exist in the literature about this research by providing data based on a more realistic portrayal of the research and the notification and authorization approaches used for it than have been studied previously. These data are critical to informing ethics and policy debates that must be resolved with the expansion of pragmatic research efforts. The calls for data to drive health care decision making provide a strong case for such research, but it is also essential to ensure that it is done in an ethically acceptable manner and does not outpace public attitudes and other interests.

Although written informed consent is an expectation for most traditional clinical research, proponents of pragmatic research designs aimed at providing data needed to inform health care decision making have raised substantial concerns about the feasibility of conducting pragmatic research if written informed consent is required. These concerns include the possibility that obtaining written consent will undermine trial integrity due to a potential consent bias. We found that, across all the research scenarios in our study (with the exception of the post-study notification groups), 51% to 72% were willing to participate. Willingness to participate was not significantly different according to whether the research involved higher or lower risks. Moreover, willingness to participate was similar for those who were randomly assigned to general notification, oral consent, or written consent. However, it is important to underscore that willingness to participate was not universal, with 28% to 49% declining to participate depending upon the particular scenario being considered. These data suggest that there could be nontrivial consent bias in pragmatic research, but that this bias would not depend on the specific approach to notification and authorization taken.

Our findings also suggest that conducting pragmatic clinical research without some type of notification and perhaps consent is currently not acceptable to the majority of the public and is associated with lower trust in physicians and the places where care is provided. One of the concerns participants raised about general notification was born out by our data. For those participants who were randomly assigned to consider a scenario that involved general notification, 21% to 36% was unaware that they were in research. This is troubling given that in this study, the general notification was explicitly displayed on the screen for the participant, which may not be the case in a general hospital or clinic setting where such information may be simply posted or given in a packet of other materials, thereby escaping notice. More broadly, the lower acceptance of no notification or general notification may reflect a set of beliefs—correct or not—about the relative risks, benefits, and burdens of pragmatic research, as well as perceptions about patients’ rights. Future work is needed to understand how the acceptability of different notification and authorization approaches evolve as a result of broader education about pragmatic research.

Despite providing information about the research in a variety of ways with materials that were carefully developed and reviewed, across all scenarios, understanding of the nature of randomization was low, as was understanding that there were not incremental patient burdens associated with participating in pragmatic research. Similar types of misunderstandings have been documented in prior work⁶ and highlight the challenges in helping people to understand research that so closely resembles regular medical care. It was also noteworthy and somewhat paradoxical that better understanding was not demonstrated consistently by participants who were randomly assigned to a scenario that included a written informed consent process. Thus, at least with respect to understanding, other active approaches to notification and consent may be equally or even more effective than written consent.

Our data also address the issue in research comparing commonly used interventions concerning whether the known risks of the interventions should be included in the research consent document or whether the document need only include the incremental risks of participation in the research.^10–12 In our study, we found that the addition of the clinical risks of the interventions to the consent document did not affect any of the outcomes studied—including perceptions of risks and benefits—relative to a document that did not disclose clinical risks. However, inclusion of the clinical risks did increase the amount of risk information to be read from two paragraphs to three.

In summary, these data can help to inform the selection of notification and authorization approaches for pragmatic research in several ways. First, providing the opportunity to opt out of or refuse participation in pragmatic research may lead to nontrivial consent bias that is consistent across approaches for notification and authorization. Second, most of the public currently view less active approaches to notification and authorization, including no notification and general notification, as unacceptable for some types of pragmatic research. Third, when using written consent, it seems that descriptions of background clinical risks may be omitted without substantially affecting potential participants’ understanding, while helping to diminish the burdens associated with consent, including the length of consent documents. Fourth, active alternatives to written consent, such as oral consent, may also not be expected to compromise consent quality. Accordingly, efforts to develop, field and test them would be welcome.

Our findings should be interpreted in the context of several limitations. First, the scenarios we used were hypothetical, so it is unclear how well the same reactions would be found in real clinical encounters. Nevertheless, with this in mind we attempted to maximize generalizability by selecting realistic hypothetical scenarios that could be easily imagined by the general public. Second, while we used survey weights to minimize the impact of selection bias, it is still possible that those who agreed to participate in our study are different in some way from those who declined. Third, our notification and authorization approaches were artificial in several respects. The general notification (eg, poster in the waiting room) was explicitly displayed on the screen for the participant, which likely led to an overestimate of people’s awareness and understanding of general notification. Other notification and authorization approaches, such as oral consent and written consent, could not include the interpersonal element of consent discussions. Related to this, participants were unable to have their questions answered about the studies before having to make a decision about whether they would participate. Future work is needed to examine people’s reactions to different notification and authorization approaches as they occur in real pragmatic studies.