Abstract
Background:
The University of California, San Diego Brief Assessment of Capacity to Consent (UBACC) is a tool to assess the capacity of participants to consent in psychiatric research. However, little is known about the psychometric properties in low and middle-income countries. This study aimed to examine the psychometric properties of the UBACC.
Methods:
We examined the reliability, latent factor structure, and item response of the first attempt of the UBACC items in a sample of 32,208 adults (16,467 individuals with psychosis and 15,741 controls) in Ethiopia, Kenya, South Africa, and Uganda; exploring these properties in the full sample and stratified by country, diagnostic status, sex, and ethnolinguistic language groups.
Results:
Exploratory factor analysis (EFA) suggested a two-factor model for the overall sample. However, a three-factor model was more appropriate when examining the latent structure across country, language, and sex. Confirmatory factor analyses (CFA) revealed an adequately fitting three-factor model for the full sample and across country, sex, and language. A two-factor model, however, was more appropriate for English and Amharic languages. Across all groups, the internal consistency of the UBACC was low, indicating below-threshold reliability (Cronbach’s α (95 % CI = 0.58 (0.57–0.59). Using a multidimensional item-response theory framework for the full sample revealed that UBACC item 8, measuring understanding of the benefits of study participation, was the most discriminating item. Many of the other items had below-threshold discriminating characteristics.
Conclusion:
EFA and CFA converged towards a two and three-dimensional structure for the UBACC, in line with the developers of the original scale. The differences in properties between populations and language groups, low internal consistency, and below-threshold item functioning suggest that investigations into the cultural and linguistic nuances are still warranted. Understanding the utility of consent tools, such as the UBACC, in underrepresented populations will be a part of the larger process which ensures that research participants are adequately protected.
Keywords: UBACC, Psychometric properties, Informed consent, Genetic studies, Africa
1. Introduction
Voluntary informed consent in research participation is essential in ethical research practice (Faden et al., 1986). It aims to ensure that individuals participating in studies do so willingly, with comprehensive knowledge about the implications of their involvement before engaging in the research. The University of California, San Diego Brief Assessment of Capacity to Consent (UBACC) is a tool used to assess the capacity of participants to consent in psychiatric research [1]. Compared to other tools, the UBACC offers the benefit of being quick to administer, does not require specific training to administer, can easily be tailored to the specific research protocol [1] and the iterative nature has been shown to improve understanding (DuBois et al., 2011). Despite being highly recommended for use in vulnerable populations, the psychometric properties of the UBACC have not been assessed in low-middle income countries (LMIC) where English is not the primary language. Given the variations in the cultural expression of mental health disorders across the globe (Kleinman, 1988); we cannot assume that the latent structure and expression of disorders would be the same in all populations.
The UBACC was developed to assess whether adults with schizophrenia can make an informed decision to participate in a research study [1]. It measures three domains of capacity to consent, forming three factors explaining more than 56 % of the variance. These domains include the ability of the individual to fully comprehend the information being disclosed (understanding), apply the relevant information to their own situation (appreciation), and engage in consequential and comparative reasoning (reasoning).
Using a small outpatient sample of elderly European adults with Schizophrenia (n = 127) and healthy controls (n = 30), Jeste and colleagues found a strong agreement between the UBACC and the established MacArthur Competency Assessment Tool for Clinical Researchers (MacCAT-CR), a preferred tool that was widely used at the time for assessing capacity to consent [2]. The UBACC was also reported to have good concurrent validity, sensitivity and specificity, as rated by two psychiatrists. Further, items that measured the same general construct produced similar scores (internal consistency), and scores did not vary significantly across those who administered the UBACC (inter-rater reliability) [1].
Other small studies in high-income English-speaking countries consisting of individuals with Alzheimer’s disease or mild cognitive impairment and healthy controls ([3], n = 132; [4], n = 61) also showed strong concurrent validity and acceptable internal consistency of the UBACC and identified a negative correlation with an unrelated tool (discriminant validity) [3,4]. However, the items loaded onto only one factor related to ‘understanding’ [3]. UBACC scores were also significantly lower for cases than controls, particularly those with Alzheimer’s disease, and were associated with age but not sex or educational levels [4]. While these findings were significant, the study was limited by its small sample size, and the capacity to consent was compared against evaluations from only one physician.
Only one study has explored the utility of the UBACC in an LMIC setting and in a different language. Although, the psychometric properties of the UBACC were not examined in this study, several points for consideration were raised. The UBACC was translated into the Xhosa language and assessed in a South African study of 528 predominantly male (90 %) participants with schizophrenia and matched controls in South Africa [5]. The study demonstrated that the iterative learning approach improved UBACC scores for each subsequent attempt. Therefore, the authors recommended the UBACC as a tool to improve the understanding of research protocols in LMIC settings involving participants with little formal education and those with severe mental conditions. Diagnosis of schizophrenia and low education levels strongly predicted a poor understanding of the study protocol; a weak association was also seen between older participants (between 40 and 59 years) and lower UBACC scores. The study recruiter, who administered the UBACC, emerged as the most significant predictor of participant understanding. There were some context-specific difficulties, such as challenges in translating certain concepts, such as genetics, into Xhosa no direct Xhosa translation is available, particularly for the more complex genomics terminology. This further highlights the importance of language.
Although a few studies have investigated the psychometric properties in high-income European and American English-speaking populations, no independent studies have evaluated the psychometric properties (validity and reliability) in LMIC African countries, where English is typically not the first language. Scale reliability and validity are important measures to assess whether the tool consistently measures the trait it was developed to measure, particularly when using it in new populations where cultural nuances and language differences may affect item responses[6]. To better understand how informed consent is assessed using the UBACC in LMICs, such as those in Africa, there is a need to evaluate the ability of the scale to measure the underlying latent construct of interest (latent structure) and the ability of the scale items to differentially predict being high on the latent construct being tested (item response).
Tools, such as the UBACC, to ascertain individuals’ capacity to consent are encouraged by clinicians and bioethicists [7] particularly in vulnerable populations. However, the psychometric properties of the UBACC have only been evaluated in high-income countries, in the English language, with small sample sizes. This study aimed to test the psychometric properties of the UBACC for use in diverse African populations. This was done by investigating the scale reliability, factor structure, and item response [8]. This approach may provide valuable insights into the performance of the UBACC within the African context, ensuring these populations are adequately considered in future research endeavours.
2. Methods
2.1. Participants
Participants in this study were from the NeuroGAP-Psychosis project, a case-control study aiming to expand knowledge of the environmental and genetic risk factors for psychosis in previously represented African populations [9]. Participants were recruited from general and specialised health facilities, including inpatient facilities in Ethiopia and South Africa, between 2018 and 2022. Collections beginning in February 2018 in Uganda up to June 2022, March 2018 to February 2022 in the two sites in Kenya, April 2018 to June 2022 in South Africa and June 2018 to June 2022 in Ethiopia. The final analysis cohort included 32,208 adult participants (Table 1): 6834 from South Africa, 8152 from Kenya, 8999 from Uganda, and 8215 from Ethiopia. Cases (n = 16,467) were participants that had a psychotic disorder diagnosis (i. e., schizophrenia, schizoaffective disorder, psychotic disorder not otherwise specified, and bipolar disorder), and the controls (n = 15,741) were participants that did not have a diagnosis of a psychotic disorder. Males (n = 17,945) made up 56 % of the cohort. Written informed consent was obtained from all participants.
Table 1.
Distribution of demographics in the NeuroGAP-Psychosis cohort.
| Demographic | How the demographic is described | Cases | Controls | Total | unadj P-value |
|---|---|---|---|---|---|
|
| |||||
| Sex | Males (n, % of total) | 9596 (58 %) | 8349 (53 %) | 17,945 (56 %) | p < 1e-3 |
| Age (yrs.) | Range | 18–83 | 18–84 | p < 1e-3 | |
| Mean (SD) | 36.9 (12) | 36.3 (12) | |||
| Country | South Africa (n, %) | 3624 (22 %) | 3213 (20 %) | 6837 (21 %) | p < 1e-3 |
| Kenya (n, %) | 4351 (26 %) | 3803 (24 %) | 8154 (25 %) | ||
| Uganda (n, %) | 4504 (27 %) | 4498 (29 %) | 9002 (28 %) | ||
| Ethiopia (n, %) | 3988 (24 %) | 4227 (27 %) | 8215 (25 %) | ||
| Education | None (n, %) | 430 (3 %) | 424 (3 %) | 854 (3 %) | p < 2e-16 |
| Some primary (n, %) | 3014 (18 %) | 2103 (13 %) | 5117 (16 %) | ||
| Complete primary (n, %) | 2099 (13 %) | 1471 (9 %) | 3570 (11 %) | ||
| Some secondary (n, %) | 4705 (29 %) | 3779 (24 %) | 8484 (26 %) | ||
| Complete secondary (n, %) | 2623 (16 %) | 2747 (18 %) | 5370 (17 %) | ||
| Some tertiary (n, %) | 1524 (9 %) | 1898 (12 %) | 3422 (11 %) | ||
| Complete tertiary (n, %) | 2069 (13 %) | 3314 (21 %) | 5383 (17 %) | ||
| Language | English (n, %) | 4715 (29 %) | 5351 (34 %) | 10,066 (31 %) | p < 2e-16 |
| Afrikaans (n, %) | 295 (2 %) | 192 (1 %) | 487 (2 %) | ||
| Amharic (n, %) | 3795 (23 %) | 4227 (27 %) | 8022 (25 %) | ||
| Niger-Congo (n, %) | 6301 (38 %) | 4678 (30 %) | 10,979 (34 %) | ||
| Oromo (n, %) | 193 (1 %) | 0 (0 %) | 193 (1 %) | ||
| Nilo-Saharan (n, %) | 1165 (7 %) | 1288 (8 %) | 2453 (8 %) | ||
| UBACC total | Range | 0–2 | 1–20 | p < 1e-3 | |
| Mean (SD) | 12 (4) | 14 (3) | |||
| Total | 16,467 | 15,741 | 32,208 | ||
Note: Percentages are calculated relative to each column; UBACC, San Diego Brief Assessment of Capacity to Consent; yrs., years; SD, standard deviation; unadj, unadjusted.
2.2. Measures
Demographics.
A demographics questionnaire, designed by the NeuroGAP–Psychosis team, was used to collect information on the education, sex at birth, country of origin, and language of each participant (Table 1).
UBACC.
The UBACC [1] is a 10-item tool that assesses three of the four domains of decisional capacity (Box 1; Supplementary File 1 Appendix 1). These domains included four items assessing understanding (items 1, 3, 7 and 8), five assessing appreciation (items 4–6,10) and one assessing reasoning (item 2). Items 1, 2, 6, 7 and 8 were scored on a scale of 0–2, indicating whether neither one nor both concepts were comprehended. Items 3,5, 9 and 10 required a response of ‘yes’ or ‘no’, scored as 1 or 0. Item 4 was reverse scored, with 0 indicating ‘no’ and 1 indicating ‘yes’. We reverse-coded this item in the analysis dataset to ensure the item aligned with the scoring of the 9 other items. Total scores ranged from 0 to 20. Per the original author’s findings, a score above 15 was deemed sufficient for informed consent. The UBACC was translated and back-translated into various languages and was administered by trained research assistants in the participant’s preferred language. The non-English versions of the UBACC were translated and reviewed in each language by native speakers of each language for accuracy before participant enrolment. English versions of the UBACC were offered in all four countries, and local languages were offered by country as follows: Kiswahili in Kenya; isi-Xhosa or Afrikaans in South Africa; Amharic in Ethiopia, and Luganda, Acholi, Lugbara, or Runyankole in Uganda. As there were not adequate sample sizes for each interview language to be adequately compared in the four countries, apart from English and Afrikaans, we consolidated the remaining languages into their larger language families based on the Ethnologue (Eberhard et al., (eds.). 2023) ethnological database. These family groups are as follows: Niger-Congo (including Kigiriyama, Kiswahili, Luganda, isiXhosa, Runyankole), Nilo-Saharan (including Lugbara and Acholi-Luo) and Oromo (Cushitic subgroup in Afro-Asiatic language family) and Amharic (Semitic subgroup in the Afro-Asiatic language family).
2.3. Procedure
Upon initial recruitment, the voluntary nature of participation was explained. If the participant agreed to continue, an information sheet with a summary of the research objectives and processes and a consent form written in the preferred language was read to them, and each section was explained. The UBACC was then administered by trained research staff who were fluent in English and the specific language requested by the participant. If the participant did not obtain a score of 20, information from the sections the participant answered incorrectly was explained again, and only those questions in the UBACC were readministered up to three additional times. The participant was excluded from the study if a score of at least 14.5 was not attained after the fourth attempt. Therefore, all of the participants included in this study received a score of either 20 in one of the trials or at least 14,5 in the fourth trial. For the analysis of the psychometric properties, only the data from the first attempt were analysed, as this was the only trial in which all of the questions were answered in a single attempt.
Written consent was obtained from all study participants after passing the UBACC and attaining a score of 14.5 and above. The NeuroGAP–Psychosis study was approved by the Institutional Review Board at Harvard T.H Chan School of Public Health (#IRB17–0822) in the USA and by the following ethics committees: Addis Ababa University College of Health Sciences (#014/17/Psy) and the Ministry of Science and Technology Ethics Committee (#3.10/14/2018) in Ethiopia; Moi Teaching and Referral Hospital Ethics Committee (#IREC/2016/145), Kenya National Council of Science and Technology (#NACOSTI/P/17/56302/19576), KEMRI Centre Scientific Committee (CSC# KEMRI/CGMRC/CSC/070/2016) and KEMRI Scientific and Ethics Review Unit (SERU#KEMRI/SERU/CGMR-C/070/3575) in Kenya; The University of Cape Town (#466/2016) and Walter Sisulu University (#051/2016) Ethics Committees in South Africa; The Makerere University School of Medicine (SOMREC #REC REF 2016–057), and the Uganda National Council for Science and Technology Ethics Committee (UNCST #HS14ES) in Uganda.
2.4. Statistical analysis
The analyses were conducted on the full dataset and by data subsets determined by case status (i.e., case or control), sex, and interview language group (described above). All statistical analyses were conducted using R (version 3.6) [10].
Exploratory Factor Analysis (EFA).
Since the UBACC has not been widely used in LMIC, we first examined the underlying latent factor structure using EFA. The Kaiser-Meyer-Olkin measure of sampling adequacy [11] and Bartlett’s test of sphericity (Bartlett, 1950) were used to determine whether the UBACC was suitable for factor analysis. To determine how many factors (i.e., latent constructs) to retain, we examined the number of factors with eigenvalues greater than 1.0 [11] in combination with the corresponding scree plot [12] and parallel analysis [13]. Eigenvalues represent how much information is contained in a factor, with higher eigenvalues indicating that the factor will have more helpful information about the construct [14]. We use ‘oblimin’ rotation in the EFA as this would allow for our factors to be correlated. We ran EFA on a random split of half the dataset, and the interpretability of the grouping of the factors was based on our theoretical knowledge of the latent constructs.
Confirmatory Factor Analysis (CFA).
To complement our EFA and evaluate the previously reported factor structure of the UBACC, we conducted CFA without rotation [15]. The factors were derived from the initial three-factor solution proposed by Jeste and colleagues in the development of the UBACC, who noted three factors: understanding (items 1, 3, 7 and 8), appreciation (items 4, 5, 6, 9 and 10) and reasoning (item 2). We also tested the one- and two-factor models, as previously reported ([3]; Duron et al.,2013), as well as the two-factor solution that emerged from the EFA. We used a standardised coefficient loadings cut-off of 0.30 [16] to assess for items that were non-salient or had significant cross-loadings. An acceptable model fit was defined if the root mean squared error of approximation (RMSEA) and standardised root mean square residual (SRMR) were both < 0.06 and if the Tucker–Lewis index (TLI) and comparative fit index (CFI) were both > 0.90 [17,18]. Factor loadings represent how much of a participant’s response to the question is due to the underlying latent construct being measured; therefore, a low factor loading is an indication that an item may have little variance with the other items in the same factor and inclusion of these items in the scoring may introduce bias in the results. Therefore, non-salient items were dropped from the EFA and CFA, and the models were reassessed to evaluate any improvement in the model fit.
Reliability.
The internal consistency of the UBACC and its two main domains, understanding and appreciation, was computed using Cronbach’s alpha (α) and McDonald’s Omega (ω) (with a coefficient > 0.80 deemed acceptable [19]. The analyses were conducted with all items and repeated using salient items only (items with factor loadings <0.30). Item-total correlations were also computed to identify any items that did not correlate well with the overall scale.
Item Response Theory.
Item response theory (IRT) is helpful in evaluating the performance of tools by determining the difficulty of each item [13]. Since the UBACC is not a unidimensional construct, we used multidimensional IRT to assess the relationship between the latent trait (decisional capacity) and item responses by evaluating the item/category response curves. A graded response model and a maximum likelihood approach were used to fit the IRT models. The RMSEA was used to test the model’s goodness of fit and to compute item parameters, which generated slope discrimination and location parameters [20]. The slope parameters are a measure of how well an item differentiates between respondents with different levels of the latent trait (decisional capacity); larger values denote better differentiation. We also plotted item characteristics curves (ICCs) and item information curves to visualise item difficulty and item discrimination. Item difficulty represents the level of the latent construct (e.g., decisional capacity) where 50 % of respondents endorse an item. An item of a higher level of difficulty required a higher level of the latent construct. Item discrimination determines the rate at which the probability of endorsing an item changes for differing ability levels.
3. Results
3.1. Exploratory factor analysis
The Kaiser-Meyer-Olkin measure of sampling adequacy was 0.77, which surpassed the 0.60 threshold of adequacy, and Bartlett’s test of sphericity was statistically significant, suggesting that there are likely underlying factors that show item correlation. We identified two eigenvalues greater than 1.00 (2.39, 1.12) with a third eigenvalue at 1.00, barely meeting the threshold (the scree plot is available in Supplementary File 1, Fig. 1). Parallel analysis also suggested that we retain two factors (the scree plot is available in Supplementary File 1, Fig. 2). Items 1 (Study purpose), 2 (Why participate), 6 (Study activities), 7 (Risks) and 8 (Benefits) were loaded onto factor 1 (factor loadings ranged between 0.14 and 0.62), and items 3 (Research or treatment), 4 (Voluntariness), 5 (If you withdraw) and 9 (No benefits) were loaded onto factor 2 (factor loadings ranged between 0.14 and 0.42). Item 10 (Broad consent) is loaded equally on both factors. Based on interpretations by Jeste and colleagues, factor one likely relates to understanding and reasoning, while factor two relates to appreciation as domains of decisional capacity [1]. The EFA further indicated that there were three factors in the UBACC structure from each country and language (Supplementary Table 2), with factor loadings ranging from −0.01 to 0.62.
Fig. 1.
Three-factor structure with factor loadings of the UBACC.
Note: Und, understanding; App, appreciation; Rsn, reason.
Fig. 2.
Item characteristic curves in the UBACC indicate that most items are easy to answer.
3.2. Confirmatory factor analysis
We conducted CFA to determine whether the two-factor solution from the EFA, a one-factor model, or a three-factor model (from Jeste et al.) provided the best fit for the whole cohort (overall fit for the three-factor model in Table 2 and other models and item loadings in Supplemental Tables 1, 3, 4, 5). The UBACC was designed to measure decisional capacity, and the theoretical assumption is that the questions in the UBACC measure three domains/factors of decisional capacity. The three-factor model provided the best fit across the whole cohort: diagnostic status, language, and sex. The results below, therefore, focus primarily on the three-factor model results. After the removal of non-salient items, there was a slight improvement in the fit statistics across all groups (Supplementary Table 1 and 3; Supplementary Files 1, Figs. 3 and 4).
Table 2.
Goodness-of-fit measures for three-factor CFA models.
| Cohort grouping | RMSEA | SRMR | TLI | CFI | X2 | df | P-value |
|---|---|---|---|---|---|---|---|
|
| |||||||
| Overall | 0.038 | 0.031 | 0.945 | 0.960 | 1561.2 | 33 | 0.001 |
| South Africa | 0.054 | 0.048 | 0.914 | 0.937 | 695.897 | 33 | 0.001 |
| Kenya | 0.037 | 0.030 | 0.880 | 0.912 | 392.025 | 33 | 0.001 |
| Uganda | 0.030 | 0.024 | 0.964 | 0.973 | 291.982 | 33 | 0.001 |
| Ethiopia | 0.029 | 0.034 | 0,0.61 | 0.971 | 256.254 | 33 | 0.001 |
| Male | 0.039 | 0.033 | 0.947 | 0.961 | 930.934 | 45 | 0.001 |
| Female | 0.037 | 0.031 | 0.941 | 0.957 | 689.813 | 33 | 0.001 |
| Case | 0.042 | 0.034 | 0.927 | 0.947 | 994.987 | 33 | 0.001 |
| Control | 0.037 | 0.032 | 0.936 | 0.953 | 756.217 | 33 | 0.001 |
| English | 0.042 | 0.034 | 0.887 | 0.917 | 618.455 | 33 | 0.001 |
| Afrikaans | 0.042 | 0.069 | 0.939 | 0.955 | 683.358 | 33 | 0.001 |
| Amharic | 0.029 | 0.035 | 0.960 | 0.970 | 261.047 | 33 | 0.001 |
| Niger-Congo | 0.040 | 0.032 | 0.906 | 0.931 | 603.783 | 33 | 0.001 |
| Oromo | 0.023 | 0.059 | 0.954 | 0.966 | 143,937 | 33 | 0.001 |
| Nilo-Saharan | 0.045 | 0.037 | 0.919 | 0.940 | 194.437 | 33, | 0.001 |
Note: RMSEA, root mean squared error of approximation; SRMR, standardised root mean square residual; TLI, Tucker–Lewis Index; CFI, Comparative Fit Index; df, degrees of freedom; p, p-value.
Fig. 3.
Item information curves of the UBACC denoting little information about the decisional capacity.
3.2.1. Overall sample
The three-factor solution from Jeste et al. reached good fit levels (RMSEA = 0.038; SRMR = 0.031; CFI = 0.960; TLI = 0.945) (Fig. 1 and Table 2). In the overall sample, four items (items 3 (Research or treatment), 4 (Voluntariness), 5 (If you withdraw), and 10 (Broad consent) were non-salient, i.e., the responses did not reach the 0.30-factor loading cut-off for standardised coefficients. The final model included items 1, 2, 6, 7, 8 and 9.
3.2.2. Diagnostic differences
The three-factor model, when evaluated according to the diagnostic group (i.e., cases vs. controls), showed good fit statistics (RMSEA <0.042, SRMR<0.034, CFI > 0.936, TLI > 0.953) (Table 2). Items 3 (Research or Treatment), 4 (Voluntariness), and 10 (Broad consent) were non-salient in both cases and controls, but item 5 (If you withdraw) was non-salient in cases only. The final models included: 1) cases-only, comprised of items 1, 2, 6, 7, 8 and 9; and 2) controls-only, comprised of items 1, 2, 5, 6, 7, 8 and 9.
3.2.3. Country differences
When we analysed each country separately, each of the three-factor models still had above-adequate fit levels (RMSEA<0.054, SRMR<0.048; CFI > 0.912; TLI > 0.880) (Table 2). The responses to items 4 (Voluntariness) and 5 (If you withdraw) were non-salient in all four countries. In addition, responses to items 3 (Research or treatment) and 10 (Broad consent) were non-salient in Kenya, Uganda and Ethiopia, item 9 (No benefits) in Kenya only, and item 6 (Study activities) in Ethiopia only. The final country models included the following items: South Africa – items 1, 2, 3, 6, 7, 8, 9, 10; Kenya – items 1, 2, 6, 7, 8, 9; Uganda – items 1, 2, 6, 7, 8, 9; Ethiopia – items 1, 2, 6, 8, 9.
3.2.4. Sex differences
The three-factor model amongst both males and females showed good fit statistics (RMSEA <0.039, SRMR<0.033, CFI > 0.95, TLI > 0.941) therefore indicating the factor structure is invariant by sex. In both groups, items 3 (Research or treatment), 4 (Voluntariness), 5 (If you withdraw) and 10 (Broad consent) were non-salient. The final model included items 1, 2, 6, 7, 8 and 9 for both males and females.
3.2.5. Language differences
The latent structure of the UBACC items across each language group was very similar for the one-, two- and three-factor models. The two-factor model was best fitting for English and Oromo languages (RMSEA <0.041, SRMR<0.035, CFI > 0.917, TLI >0.92), and the three-factor model had the best fit for Afrikaans and Amharic languages (RMSEA <0.047, SRMR<0.072, CFI > 0.943, TLI >0.924). Both the Niger-Congo and Nilo-Saharan language families showed no difference in fit measures between the two and three-factor models (RMSEA <0.044, SRMR<0.037, CFI > 0.93, TLI > 0.908). None of the languages had the same pattern of saliency across the items (summarised in Supplementary File 1, Fig. 4). Except for Nilo-Saharan languages, items 4 (Voluntariness) and 10 (Broad consent) were non-salient across all languages. Item 3 (Research or treatment) was also non-salient across all languages, except for Afrikaans and Oromo languages. Item 5 was non-salient across all languages except for Oromo languages. Item 9 was non-salient across all languages except Amharic and Niger-Congo languages. Item 6 was only non-salient in Afrikaans and Amharic languages, and item 7 was only non-salient in Oromo. The final items in the language and language families categories were as follows: Afrikaans – items 1, 2, 3, 7, 8; Amharic – items 1, 2, 7, 8, 9; English – items 1, 2, 6, 7, 8; Niger-Congo – items 1, 2, 6, 7, 8, 9; Nilo-Saharan – items 1, 2, 4, 6, 7, 8, 10; Oromo – items 1, 2, 3, 5, 6, 8.
3.3. Reliability
The internal consistency, as measured by Cronbach’s alpha (α) and McDonald’s Omega (ω), of the first pass of the UBACC in this study across all domains for the overall sample (α 0.65 (0.64–0.65) ω 0.67 (0.66–0.67)) was below the threshold, and when stratified by country, sex, language group, and diagnostic status, was still below the acceptable threshold of 0.70 apart from reliability coefficients for the South Africa dataset which approached the acceptable threshold (α 0.68 (0.67–0.69) ω 0.70 (0.69–0.71)) (Table 3). When we removed the non-salient items (items 2 (Why participate), 4 (Voluntariness), 5 (If you withdraw) and 10 (Broad consent), as determined with the factor analysis of the whole cohort, and re-evaluated internal consistency, the reliability coefficients remain somewhat the same. However, this still did not meet the acceptable threshold (Supplementary Table 6a). We also evaluated the internal consistency of the four non-salient items we omitted and observed that Cronbach’s α and McDonald’s ω are poor in the overall sample and across country, sex, language group and diagnostic status, ranging from 0.00 to 0.13. When we examine the corrected item-total correlations (Supplementary Table 6b), we see that most of the items have a moderate to strong correlation with the overall scale. Item 3 (Research or treatment), 4 (Voluntariness), 5 (If you withdraw), and 10 (Broad consent) have low correlation coefficients (below 0.30), and these same items have low dropped-item-total correlation coefficients as well; this indicates that these items are not particularly consistent with the overall scale.
Table 3.
Tests of Reliability.
| Participant groups | All UBACC items |
Understanding |
Appreciation |
|||
|---|---|---|---|---|---|---|
| Cronbach’s α (95 % CI) | McDonald’s ω (95 % CI) | Cronbach’s α (95 % CI) | McDonald’s ω (95 % CI) | Cronbach’s α (95 % CI) | McDonald’s ω (95 % CI) | |
|
| ||||||
| Overall | 0.65 (0.64–0.65) | 0.67 (0.66–0.67) | 0.48 (0.47–0.48) | 0.51 (0.50–0.52) | 0.41 (0.30–0.41) | 0.42 (0.41–0.43) |
| South Africa | 0.68 (0.67–0.69) | 0.70 (0.69–0.71) | 0.52 (0.50–0.54) | 0.56 (0.55–0.58) | 0.45 (0.43–0.47) | 0.47 (0.44–0.49) |
| Kenya | 0.50 (0.49–0.52) | 0.52 (0.50–54) | 0.29 (0.26–0.31) | 0.31 (0.28–0.34) | 0.20 (0.17–0.23) | 0.22 (0.19–0.24) |
| Uganda | 0.63 (0.62–0.64) | 0.66 (0.65–0.67) | 0.53 (0.52–0.55) | 0.57 (0.55–0.58) | 0.33 (0.31–0.36) | 0.35 (0.34–0.38) |
| Ethiopia | 0.60– (0.59–0.62) | 0.66 (0.65–0.67) | 0.45 (0.43–0.47) | 0.51 (0.50–0.54) | 0.17 (0.15–0.20) | 0.18 (0.13–0.24) |
| Male | 0.65 (0.65–0.66) | 0.68 (0.67–0.69) | 0.48 (0.47–0.49) | 0.52 (0.51–0.53) | 0.42 (0.41–0.43) | 0.43 (0.42–0.44) |
| Female | 0.63 (0.62–0.64) | 0.66 (0.65–0.66) | 0.47 (0.45–0.49) | 0.50 (0.49–0.52) | 0.39 (0.37–0.40) | 0.40 (0.38–0.42) |
| Case | 0.63 (0.62–0.64) | 0.66 (0.65–0.66) | 0.47 (0.46–0.48) | 0.50 (0.49–0.52) | 0.40 (0.38–0.41) | 0.41 (0.39–0.42) |
| Control | 0.62 (0.61–0.63) | 0.64 (0.63–0.65) | 0.43 (0.41–0.44) | 0.47 (0.45–0.49) | 0.39 (0.37–0.41) | 0.40 (0.38–0.42) |
| English | 0.57 (0.56–0.58) | 0.59 (0.58–0.60) | 0.43 (0.41–0.45) | 0.46 (0.45–0.48) | 0.27 (0.24–0.29) | 0.28 (0.24–0.31) |
| Afrikaans | 0.65 (0.62–0.69) | 0.69 (0.65–0.73) | 0.56 (0.48–0.61) | 0.58 (0.52–0.65) | 0.19 (0.07–0.29) | 0.18 (0.09–0.33) |
| Amharic | 0.61 (0.59–0.62) | 0.66 (0.65–0.68) | 0.46 (0.44–0.47) | 0.51 (0.49–0.54) | 0.18 (0.15–0.21) | 0.18 (0.12–0.23) |
| Niger–Congo | 0.58 (0.57–0.59) | 0.60 (0.59–0.61) | 0.40 (0.38–0.41) | 0.43 (0.41–0.45) | 0.33 (0.31–0.34) | 0.33 (0.31–0.35) |
| Oromo | 0.43 (0.31–0.54) | 0.50 (0.35–1.00) | 0.38 (0.20–0.49) | 0.55 (0.34–1.00) | 0.35 (0.30–0.47) | 0.51 (0.20–1.00) |
| Nilo–Saharan | 0.63 (0.60–0.65) | 0.65 (0.63–0.66) | 0.49 (0.45–0.52) | 0.54 (0.51–0.57) | 0.43 (0.39–0.45) | 0.46 (0.43–0.49) |
3.4. IRT analyses
Given that the UBACC is a multidimensional instrument, we carried out a multidimensional IRT model approach to complement classical theory approaches to psychometric analysis. IRT models describe the interaction between a participant’s response to an item and the latent construct being measured by an instrument. An RMSEA value of 0.049 (95 %CI, 0.049–0.051) and an SRMR value of 0.037 indicated that the UBACC item response model reached adequate fit levels. Using the multidimensional IRT (mirt) implementation in R for dichotomous (two response) and polytomous (three response) IRT models, we found that the degree of fit (RMSEA) for each item was below 0.06, indicating a good fit (Supplementary Table 7). We then generated IRT parameters for each item (Supplementary Table 7). The values of the discrimination slope (a-parameter) ranged from low (a-parameter −0.62) for item 4 (Do you have to be in this study?) to moderate (a-parameter) 1.67) for item 8 (Please describe some of the possible benefits of this study?), suggesting that a few of the UBACC items discriminate respondents well along the latent trait (i.e., decisional capacity). The item difficulty parameter estimates range from −2.92 (item 5- If you withdraw from the study, will you still receive regular treatment?) to −0.44 (item 7- Please describe some of the risks or discomforts that you may experience), indicating moderate value ranges. A larger difficulty parameter indicates a higher difficulty of an item. The range of difficulty parameters of the UBACC items indicates that they may be varied in terms of their predictive quality in relation to the latent construct/ability level, with moderate to high difficulty items. We examined the probability of responding to specific options in an item’s response scale by using item characteristic curves (ICCs) (Fig. 2). ICCs illustrate the slope of the latent trait, meaning that individuals with a higher level of the latent construct (i.e., decisional capacity) have a higher chance of endorsing/passing the item. We evaluated how well each item contributed to the precision of score estimation by using item information curves (Fig. 3), in which the steepness of the slope indicates how much information the item provides about the latent trait. Items 4 (Do you have to be in this study?) and 10 (If you agree to be in this study, is it possible that your DNA and health information will be used for other research studies?) had the shallowest slope, meaning that they provided the least statistical information about the latent trait, while items 2 (What makes you want to consider participating in this study?) and 8 (Please describe some of the possible benefits of this study?) had the steepest slope, meaning that they provided the most statistical information about the latent trait.
4. Discussion
This study aimed to examine the psychometric properties of the first attempt of the UBACC when applied to participants from four African countries in a large case-control study of the genetic architecture of psychosis. The internal consistency of the UBACC was low in the overall sample and, when compared across the four countries, sex, language groups and case status, thereby pointing to poor reliability, which may be because the UBACC has 10 items as a scale. CFA showed that a three-factor solution, as found in the original validation of the UBACC by Jeste et al. [1], was comparable to our factor solution and had a good fit in the overall sample, at the country level, according to sex, language, and diagnostic status. Item response analysis revealed that item 4 (Voluntariness) had the lowest item information and moderate difficulty across all groups. These findings suggest that the UBACC, in its current form, would benefit from measures to improve its reliable use in similar settings. The structural validity of the UBACC is acceptable in the model fit statistics from the results of the CFA.
In the original development and assessment of the UBACC by Jeste and colleagues, the first trial had good internal consistency (Cronbach α = 0.77 for cases and 0.76 for controls). This study was conducted amongst adults with schizophrenia and schizoaffective disorder (n = 127) and healthy controls (n = 30) to assess capacity to consent for a hypothetical clinical trial study. In another study of older adults with mild cognitive impairment (n = 49) and healthy controls (12) who provided consent for a study investigating the association between blood metabolites and cognitive ability, the first trial of the UBACC also had an acceptable internal consistency (Cronbach α = 0.747) [4]. These findings suggest that the UBACC is a reliable tool for evaluating the capacity to consent in high-income populations and research settings. In contrast to these previous studies, the two main domains, understanding and appreciation, for the full sample and according to country, sex, and diagnostic status, had below threshold reliability coefficients. After the retention of only the salient items, we saw a slight improvement in internal consistency coefficients, but they still did not meet the acceptable threshold of 0.70. The differences between this study and the previous studies may be attributable to multiple factors, for example: 1) the substantial increase in sample size may provide more power to adequately assess the performance of the UBACC, 2) the research designs and participant diagnoses in each study vary, and the complexities may influence participants’ understanding, 3) research concepts may not be equally understood by participants across the studies due to the cultural, literacy and linguistic differences.
One of the common suggestions for improving reliability is the lengthening of instruments [21]. The UBACC has four items related to the understanding domain, the appreciation domain has five items, and the reasoning domain only has 1 item. It is possible that the addition of questions specific to the reasoning domain would improve internal consistency for the first trial of any future study. It is expected that cultural experiences and norms shape how participants understand aspects of research [22,23]. Further engagement with community members may be helpful to ensure that all of the information in the consent forms and the questions in the UBACC are culturally relevant, which will further strengthen the internal consistency of the scale.
In the original UBACC validation study, three factors were identified with eigenvalues greater than 1, explaining more than 56 % of the variance [1]. Consistent with our findings, another study used UBACC in a sample of older adults with Alzheimer’s Disease and mild cognitive impairment [4] and found that the UBACC loaded onto only one factor, which likely relates to ‘understanding’. The three-factor model was, however, also a good fit in the overall NeuroGAP-Psychosis sample and across the four different countries, sex, language, and diagnostic status.
By using multidimensional IRT, we found that item 8 (‘Please describe some of the possible benefits of this study?’) was the best-performing item with a discrimination slope estimate of 1.67, whereas item 4 (‘Do you have to be in this study?’) was the least discriminating with a slope estimate of −0.62. Many of the other items in the NeuroGAP-P study had poor discrimination properties, meaning that many of the items did not differentiate between high and low decisional capacity. The high information slopes for items 2 and 3, as measured by ICCs, suggest that these items provided the most statistical information about the latent trait. In comparison, items 4 and 10 had the lowest slope. Findings from the IRT analysis indicate that scores from the UBACC had strong item discrimination and item difficulty. These measures provide strong motivation to re-examine certain items and retain or add others. These measures provide strong motivation to re-examine and potentially rephrase certain items, such as item 4 (‘Do you have to be in this study?’) and retain others, such as item 8 (‘Please describe some of the possible benefits of this study?’) and item 2 (‘What makes you want to consider participating in this study?’).
It is interesting to note the overall poor performance of item 4 (Voluntariness) on the first trial of the UBACC, not only in the IRT analyses but this item was also non-salient in many of the factor structure models. A potential explanation for this, as shared anecdotally by the study staff, is that the potential study participants’ first interpretation of this question was not related to voluntariness, as we might expect, but was influenced by different motivations to participate. Items 3 (Research of treatment), 5 (If you withdraw), and 9 (No benefits) also performed poorly and were non-salient in many of the factor structure models. A review of informed consent between developed and developing countries indicated that understanding of the research process was highly variable between participants and that complex biomedical terms were poorly understood in both research settings, irrespective of apparent differences in resources and education [24]. However, in LMICs, the participants responded that they were less likely to refuse or withdraw from research because they were concerned about the consequences, e. g., reduced access to medical care. Thus, care must be taken to ensure that participants from these settings do not fear that their treatment depends on their participation in research studies.
A key strength of the study is the large sample size spanning four African countries. Genetic studies of psychiatric conditions are relatively recent in African countries. There is an opportunity to add more ethnographic work and to use cognitive interviews with both the administrators of the UBACC and the participants involved to get detailed qualitative data about their experiences, particularly on non-salient items. With these additional investigations, the potential of additional items to capture these nuances in the LMIC setting may indeed increase the reliability of the first-pass UBACC.
There are also a few limitations to consider. Per the study protocols, the participants who failed the UBACC after four attempts were not enrolled in the NeuroGAP-Psychosis study, and so, too, were not included in these analyses. As such, we may be missing a subset of information from these excluded participants. The study also did not administer a comparable measure for assessing capacity to consent, therefore limiting an evaluation of the concurrent validity of the UBACC. Additionally, it was not recorded which assessors administered the UBACC, preventing us from testing inter-rater reliability. Due to the limited research on measures of capacity to consent, future studies incorporating these elements would be incredibly beneficial. The addition of an ethnographic arm, including cognitive interviews, in future research would also be insightful as it would help gain a deeper understanding of the use of the UBACC and its validity from the participant’s view in a structured and systematic manner.
5. Conclusion
In conclusion, the present study provides evidence of the utility of the UBACC in assessing the capacity to consent to research studies. It is supported by the convergence of the three-factor solution proposed by the original developer of the instrument. However, the UBACC has low reliability scores and a few low discriminant items that perform in a varied manner. This suggests that further investigation into the content and translation of the UBACC items is required to ensure it is appropriately utilised in this research setting.
Supplementary Material
Appendix A. Supplementary data
Supplementary data to this article can be found online at https://doi.org/10.1016/j.comppsych.2024.152526.
Box 1.
UBACC items and abbreviations.
| UBACC items | UBACC item abbreviations used in the manuscript | Decisional capacity domains (Jeste et al.) |
|---|---|---|
|
| ||
| 1. What is the purpose of the study that was just described to you? | Study purpose | Understanding |
| 2. What makes you want to consider participating in this study? | Why participate | Reasoning |
| 3. Do you believe this is primarily research or primarily treatment? | Research or treatment | Understanding |
| 4. Do you have to be in this study? | Voluntariness | Appreciation |
| 5. If you withdraw from the study, will you still receive regular treatment? | If you withdraw | Appreciation |
| 6. If you participate in this study, what will you be asked to do? | Study activities | Appreciation |
| 7. Please describe some of the risks or discomforts that you may experience. | Risks | Understanding |
| 8. Please describe some of the possible benefits of this study. | Benefits | Understanding |
| 9. Is it possible that being in this study will not have any benefit to you? | No benefits | Appreciation |
| 10. If you agree to be in this study, is it possible that your DNA and health information will be used for other research studies? | Broad consent | Appreciation |
Acknowledgments
We express our immense gratitude to the NeuroGAP-Psychosis participants in Ethiopia, Kenya, South Africa and Uganda who volunteered to participate in the study. We would also like to thank all the staff in the host institutions involved in the study.
Funding
This work was supported by the Stanley Center for Psychiatric Research at the Broad Institute of MIT and Harvard. DA, LBC, BG, KCK, DJS, and ST are supported in part by the United States’ National Institute of Mental Health (NIMH) [R01MH120642]; AS, BG, and KCK are also supported by NIMH [U01MH125045]; KCK and ST are also supported in part by NIMH [U01MH125047]. KJK is supported by the United States National Institutes of Health Fogarty International Center [K01TW012180].
Footnotes
Declaration of competing interest
None.
CRediT authorship contribution statement
Patricia Kipkemoi: Writing – review & editing, Writing – original draft, Visualization, Validation, Software, Methodology, Investigation, Formal analysis. Mary S. Mufford: Writing – review & editing, Writing – original draft, Visualization, Validation, Software, Methodology, Investigation, Formal analysis. Dickens Akena: Writing – review & editing, Resources, Methodology, Funding acquisition, Conceptualization. Melkam Alemayehu: Writing – review & editing, Resources, Project administration, Methodology, Data curation. Lukoye Atwoli: Writing – review & editing, Resources, Methodology, Funding acquisition, Conceptualization. Lori B. Chibnik: Writing – review & editing, Resources, Methodology, Funding acquisition, Conceptualization. Bizu Gelaye: Writing – review & editing, Resources, Methodology, Funding acquisition, Conceptualization. Stella Gichuru: Writing – review & editing, Resources, Project administration, Methodology, Data curation. Symon M. Kariuki: Writing – review & editing, Resources, Methodology, Funding acquisition, Conceptualization. Karestan C. Koenen: Writing – review & editing, Resources, Methodology, Funding acquisition, Conceptualization. Edith Kwobah: Writing – review & editing, Resources, Project administration, Data curation. Joseph Kyebuzibwa: Writing – review & editing, Project administration, Data curation. Rehema M. Mwema: Writing – review & editing, Project administration, Data curation. Charles R.J.C. Newton: Writing – review & editing, Resources, Methodology, Funding acquisition, Conceptualization. Adele Pretorius: Writing – review & editing, Resources, Project administration, Methodology, Data curation. Dan J. Stein: Writing – review & editing, Resources, Methodology, Funding acquisition, Conceptualization. Anne Stevenson: Writing – review & editing, Resources, Methodology, Funding acquisition, Conceptualization. Rocky E. Stroud: Writing – review & editing, Resources, Project administration, Methodology, Data curation. Solomon Teferra: Writing – review & editing, Resources, Methodology, Funding acquisition, Conceptualization. Zukiswa Zingela: Writing – review & editing, Resources, Methodology, Funding acquisition, Conceptualization. Kristina J. Korte: Writing – review & editing, Resources, Project administration, Methodology, Data curation.
Data availability
Coded individual-level data that do not allow researchers to identify participants are available upon request to researchers who meet the criteria for data sharing of the NeuroGAP Consortium. Any researcher affiliated to a public or private research institution who complies with the NeuroGAP Consortium standards can submit a research application to [email address].
References
- [1].Jeste DV, Palmer BW, Appelbaum PS, Golshan S, Glorioso D, Dunn LB, et al. A new brief instrument for assessing decisional capacity for clinical research. Arch Gen Psychiatry 2007;64(8):966–74. 10.1001/archpsyc.64.8.966. [DOI] [PubMed] [Google Scholar]
- [2].Dunn LB, Nowrangi MA, Palmer BW, Jeste DV, Saks ER. Assessing decisional capacity for clinical research or treatment: a review of instruments. Am J Psychiatry 2006;163(8):1323–34. 10.1176/ajp.2006.163.8.1323. [DOI] [PubMed] [Google Scholar]
- [3].Seaman JB, Terhorst L, Gentry A, Hunsaker A, Parker LS, Lingler JH. Psychometric properties of a decisional capacity screening tool for individuals contemplating participation in Alzheimer’s disease research. J Alzheimers Dis 2015;46(1):1–9. 10.3233/JAD-142559. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [4].Duron E, Boulay M, Vidal JS, El Bchiri J, Fraisse ML, Rigaud AS, et al. Capacity to consent to biomedical research’s evaluation among older cognitively impaired patients. A study to validate the University of California Brief Assessment of capacity to consent questionnaire in French among older cognitively impaired patients. J Nutr Health Aging 2013;17(4):385–9. 10.1007/s12603-013-0036-5. [DOI] [PubMed] [Google Scholar]
- [5].Campbell MM, Susser E, Mall S, Mqulwana SG, Mndini MM, Ntola OA, et al. Using iterative learning to improve understanding during the informed consent process in a south African psychiatric genomics study. PloS One 2017;12(11):e0188466. 10.1371/journal.pone.0188466. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [6].Thomas S DeVellis Robert F. & Thorpe Carolyn T. (2022) scale development: theory and applications. California: SAGE publications, Inc., 320 pages, $60.00 softcover. Pers Psychol 2022;75(1):243–4. 10.1111/peps.12499. [DOI] [Google Scholar]
- [7].Moye J, Gurrera RJ, Karel MJ, Edelstein B, O’Connell C. Empirical advances in the assessment of the capacity to consent to medical treatment: clinical implications and research needs. Clin Psychol Rev 2006;26(8):1054–77. 10.1016/j.cpr.2005.04.013. [DOI] [PubMed] [Google Scholar]
- [8].de Ayala RJ. The theory and practice of item response theory. pp. xv, 448. Guilford Press; 2009. [Google Scholar]
- [9].Stevenson A, Akena D, Stroud RE, Atwoli L, Campbell MM, Chibnik LB, et al. Neuropsychiatric genetics of African populations-psychosis (NeuroGAP-psychosis): a case-control study protocol and GWAS in Ethiopia, Kenya, South Africa and Uganda. BMJ Open 2019;9(2):e025469. 10.1136/bmjopen-2018-025469. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [10].R Core Team. R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing; 2023. Retrieved from, https://www.R-project.org/. [Google Scholar]
- [11].Kaiser HF. The application of electronic computers to factor analysis. Educ Psychol Meas 1960;20(1):141–51. 10.1177/001316446002000116. [DOI] [Google Scholar]
- [12].Cattell RB. The scree test for the number of factors. Multivar Behav Res 1966. 10.1207/s15327906mbr0102_10. [DOI] [PubMed] [Google Scholar]
- [13].Horn JL. A rationale and test for the number of factors in factor analysis. Psychometrika 1965;30(2):179–85. 10.1007/BF02289447. [DOI] [PubMed] [Google Scholar]
- [14].Ruscio J, Roche B. Determining the number of factors to retain in an exploratory factor analysis using comparison data of known factorial structure. Psychol Assess 2012;24(2):282–92. 10.1037/a0025697. [DOI] [PubMed] [Google Scholar]
- [15].Jöreskog KG. A general approach to confirmatory maximum likelihood factor analysis. Psychometrika 1969;34(2):183–202. 10.1007/BF02289343. [DOI] [Google Scholar]
- [16].Brown TA. Confirmatory factor analysis for applied research. 2nd ed. Guilford Publications; 2015. [Google Scholar]
- [17].Browne MW, Cudeck R. Alternative ways of assessing model fit. Sage Focus Editions 1993;154:136. [Google Scholar]
- [18].Yu C-Y. Evaluating cutoff criteria of model fit indices for latent variable models with binary and continuous outcomes. Los Angeles: University of California; 2002. [Google Scholar]
- [19].Nunnally JC. The assessment of reliability. Psychometric Theory; 1994. [Google Scholar]
- [20].Kline RB. Principles and practice of structural equation modeling. 4th ed. Guilford Publications; 2015. [Google Scholar]
- [21].Brink PJ, Wood MJ. Advanced design in nursing research. 2nd ed. Sage Publications; 1998. [Google Scholar]
- [22].Kral MJ, Ramírez García JI, Aber MS, Masood N, Dutta U, Todd NR. Culture and community psychology: toward a renewed and reimagined vision. Am J Community Psychol 2011;47(1–2):46–57. 10.1007/s10464-010-9367-0. [DOI] [PubMed] [Google Scholar]
- [23].Levy D, Splansky GL, Strand NK, Atwood LD, Benjamin EJ, Blease S, et al. Consent for genetic research in the Framingham heart study. Am J Med Genet A 2010;152A (5):1250–6. 10.1002/ajmg.a.33377. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [24].Mandava A, Pace C, Campbell B, Emanuel E, Grady C. The quality of informed consent: mapping the landscape. A review of empirical data from developing and developed countries. J Med Ethics 2012;38(6):356–65. 10.1136/medethics-2011-100178. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Data Availability Statement
Coded individual-level data that do not allow researchers to identify participants are available upon request to researchers who meet the criteria for data sharing of the NeuroGAP Consortium. Any researcher affiliated to a public or private research institution who complies with the NeuroGAP Consortium standards can submit a research application to [email address].