A culturally tailored research ethics training curriculum for American Indian and Alaska Native communities: a randomized comparison trial

Abstract

The primary aim of this study was to develop an American Indian and Alaska Native (AIAN) tailored research with human subjects curriculum that would increase the participation of AIAN members in research affecting their communities. We used a community-engaged research approach to co-design and evaluate a culturally tailored online human subjects curriculum among a national sample of AIAN community members (n = 244) with a standard nationally used online curriculum (n = 246). We evaluated pre-and post-test measures to assess group differences in ethics knowledge, perceived self-efficacy to apply such knowledge to protocol review, and trust in research. Analysis of regional tribal differences assessed curriculum generalizability. Using an 80% correct item cut-off at first attempt as passing criterion, the tailored curriculum achieved a 59.3% passing rate versus 28.1% in the standard curriculum (p < .001). For both arms, participants reported a significant increase in trust in research and in research review efficacy. Participants took less time to complete the training and reported significantly higher acceptability, satisfaction, and understandability of the curriculum for the tailored curriculum. This culturally tailored research ethics curriculum has the potential to increase participation in AIAN communities in research affecting tribal members. The AIAN curriculum achieved significantly higher levels of participants’ research ethics knowledge, self-efficacy in reviewing research protocols, trust in research, and completion of the training requirements. Culturally grounded training curricula may help remedy the impact of historical research ethics abuses involving AIAN communities that have contributed to mistrust of research and lack of community engagement in research.

Introduction

American Indian and Alaska Native (AIAN) health research lags behind other ethnic minority communities, despite higher rates of health disparities among AIAN compared to non-indigenous U.S. populations (Kunitz, 2008; Late, 2005; Moy, Smith, Johansson, & Andrews, 2006). AIAN communities retain cultural and tribal diversity, while reporting disproportionately high rates of death from alcoholism (552% higher), diabetes (182% higher), unintentional injuries (138% higher), homicide (83% higher), and suicide (74% higher) (Indian Health Service, 2013). A major impediment to research and population health across Indian country is mistrust in the scientific establishment based on over 100 years of experience with the conduct of culturally insensitive, unethical, and harmful research studies resulting in tribal stigmatization (Sloat & Epstein, 1996; Hodge, 2012; Lawrence, 2000; Sahota, 2011) and data failing to adequately inform AIAN prevention and treatment services (Mohatt & Thomas, 2006). In addition to research mistrust, the absence of a culturally relevant research ethics curriculum for AIAN community partners serves as a significant barrier to research in AIAN communities (Caldwell et al., 2005; Pearson, Parker, Fisher, & Moreno, 2014). AIAN community partners have reported the Collaborative Institutional Training Initiative (CITI) and National Institutes of Health (NIH) trainings have taken several days and up to 15 hours to complete. After taking the training, community partners have raised concerns about the jargon, lack of cultural and contextual relevance, absence of discussion about community risks and benefits, and questioned their own ability to apply the concepts to the research for which they provide oversight (Hatcher & Schoenberg, 2007; Pearson et al., 2014). Online research ethics curricula for federally funded research are not written for community research partners (Anderson et al., 2012b), nor do they address research ethics issues unique to AIAN research contexts, thus creating barriers to research with AIAN communities.

Increasingly, AIAN communities require researchers to adhere to tribal research regulations (Ball & Janyst, 2008) and to provide meaningful roles for tribal research partners (Thomas, Donovan, & Sigo, 2010). AIAN community institutional review boards (IRBs) meet community-level concerns (Fleischman, 2007), while many non-tribal IRBs lack the knowledge to adequately assess risks and benefits specific to AIAN communities (Deloria, 2002; Sahota, 2011). Human subjects research training that integrates community cultures and contexts can increase engagement and the number of AIAN partners leading research and research review efforts (Andersen, Belcourt, & Langwell, 2005). Engaging AIAN communities throughout the research process reduces research mistrust, increases tribal participation, and improves the population validity of research designs and human subjects protections, leading to the sustainability of evidence-based interventions to reduce health disparities rooted in historical and contemporary systemic inequities.

A history of research malfeasance has created AIAN mistrust of research. Deprecatory findings published without community review, typified by the Barrow Alcohol study (Mohatt et al., 2004), stigmatized AIAN communities, and contributed to mistrust. More recently, the misuse of the Havasupai Tribe’s blood samples by an Arizona State University researcher harmed the tribal community when researchers reported the tribe was ‘inbred’ (Harmon, 2010). One-sided portrayals of AIAN have resulted in the dissemination of stigmatizing findings that have grossly misinformed health practitioners and the public about AIAN life and culture (Poupart, Martinez, Red Horse, & Scharnberg, 2000). Ethical violations can result in pathologizing AIAN (Gilchrist, 1997; Mitchell & Baker, 2005; Mohatt & Thomas, 2006; Trimble, 2008) and AIAN ways of life (Bishop, 1997; Sinclair, 2003). These ethical breaches have left AIAN communities wary of research practices based on exploitation, racism, and majority ethnocentrism. As a result, AIAN communities strive to protect tribal members and communities when engaging in research (Sahota, 2011). However, federal regulations that govern research with human subjects are written to protect individuals from research harm and do not discuss protection for community-level harms. Limiting the application of the ethical principles of beneficence, respect, and justice to the protection of individual participants fails to protect communities from harm.

Outside the United States, indigenous groups worldwide rely on both international codes of ethics and indigenous research principles and ethics frameworks. These indigenous ethics guidelines establish the ethical standards and professional expectations for investigators conducting research among indigenous communities. Several communities have established best practices for research, including human subjects trainings that incorporate indigenous ethical principles (Dene, Masuzumi, & Quirk, 1993; Royal Commission on Aboriginal Peoples, 1996). Aboriginal and Maori communities have also established research ethics frameworks (Hudson, Putaiora Writing Group, & Health Research Council of New Zealand, 2010; Walker et al., 1996), though these focus on rights established with the Australian and New Zealand governments, explicitly incorporating indigenous law and policies in the research framework. Community engaged frameworks for developing research protections for indigenous communities in the United States have begun to emerge (Sahota, 2011), emphasizing tribally specific ethical principles, along with tribal sovereignty established under federal Indian law.

As community members assume equal partnership in the research process, they achieve greater equity with investigators to ensure studies meet the highest standards of human subjects protections (CIOMS & WHO, 2002; DuBois et al., 2011; Hyatt et al., 2009a; NIH, 1979; World Medical Association, 2008). Many research ethics training curricula, including the CITI, Family Health International training (Family Health International 2009), NIH Protecting Human Research Participants guide (NIH, 2008), and other university-based trainings, meet required ethics training standards. However, they were developed for the scientific community at large and provide examples from research settings with little relevance to indigenous community engaged research (CEnR) (Flicker, Travers, Guta, McDonald, & Meagher, 2007).

Indigenous CEnR focuses on the protection of AIAN cultures and knowledge systems, the inclusion of community oversight (Beauvais, 1999; Fisher & Ball, 2003) and acknowledges historical trauma, tribal sovereignty, and other contextual variables (Brave Heart-Jordan & DeBruyn, 1995; Duran & Duran, 1995) throughout the research process including the review and approval of research protocols. Indigenous CEnR also requires academic partners to comply with tribal laws and regulations (Baydala, Placsko, Hampton, & Bourassa, 2006; Thomas, Rosa, Forcehimes, & Donovan, 2011). The few ethics training approaches developed specifically for community training in human subjects protections and research in the U.S. are narrow in scope, focusing on responsibilities of field staff, techniques for primary data collection, and policy advocacy, and miss the opportunity to develop understanding of human subjects protections (Carroll-Scott, Toy, Wyn, Zane, & Wallace, 2012; Goodman, Dias, & Stafford, 2010; Hyatt et al., 2009a).

Although funders may require training, IRBs determine what constitutes sufficient training for those engaged in research (DHHS, 2009). Thus, IRBs can facilitate community-grounded, culturally centered human research ethics training that moves beyond meeting the minimum institutional research requirements defined by IRB policies and federal regulations. These institutions can support the dissemination and utilization of an AIAN-focused curriculum to advance ethical research in diverse communities and improve ethical reviews of research proposals that involve culturally distinct communities. When the training curricula selected by institutions do not match the needs of indigenous populations, AIAN are excluded from becoming contributing members to the research. Currently, there is an absence of research ethics training specifically tailored to the needs of AIAN, thus resulting in a significant barrier to AIAN research participation.

The CITI (CITI, 2000), developer of widely used online modules for ethics training, supports ethical and respectful CBPR/CEnR partnerships. However, CITI falls short in providing training addressing stigmatized (i.e. MSM) (Anderson et al., 2012a; Pearson et al., 2014), and ethnically and racially diverse (E/RD) communities. These gaps include a lack of: (1) ethics topics and examples relevant to stigmatized and E/RD populations; (2) clear, simple language; and (3) community-level oversight, risk, and benefit concerns (Cochran et al., 2008; Pearson et al., 2014). In recognition of these gaps, CITI provided permission to use their social–behavioral research with human subjects curriculum to serve as a comparison for this study.

Our primary aim was to provide a culturally tailored training curriculum that would increase the engagement of AIAN community members as co-researchers in research affecting their communities as well as participation of community members as research participants. We assessed whether the community- developed curriculum increased AIAN community members’ involvement in research through improved knowledge in research ethics, research self-efficacy, and research trust.

Methods

Curriculum development

Using a CeNR approach, three expert panels (n = 29) participated in an iterative curriculum and pre- and post-test item development process. Panel members set forth the curriculum foundation and approved all content. Panel reviews included: (1) ‘cultural relevance’ (research relevant to unique aspects of AI/AN, sovereignty, culture, and laws); (2) ‘clarifying concepts’ (removing jargon, simplifying language, and expanding explanation of the Belmont Report principles at the community level); (3) adding examples relevant to AIAN populations (i.e. misuse of data, focus on events likely in small, rural community settings vs. larger, urban settings); and, (4) adding community level risk and benefit issues pertinent to AIAN. The AIAN community panel (n = 12), composed of AIAN members with experience conducting research in AIAN communities, provided the first level of review. Rather than simply adapting the CITI, this panel stressed the need to begin with the Code of Federal Regulations (The Common Rule, 2009) and the Belmont Report (National Instiute of Health, 1979) to examine how they apply to AIAN research. This emic approach brought AIAN culture and lived experience into the revision process, and most importantly, identified AIAN individual and community-level risk, benefits, privacy, and informed consent concerns. The Scientific panel of AIAN and allied principal investigators (n = 12, 11 AIAN) conducted the second review. This panel verified content and added examples reflective of AIAN culture. The Ethics panel (n = 5, three AIAN), comprised of IRB members and ethicists, provided the third level of review. These panelists ensured that recommended new content met human subjects certification criteria. They also provided examples and suggestions regarding ethical issues specifically confronting IRBs evaluating research with AIAN communities, e.g. tribal approval processes. Panel members provided continued feedback on the curriculum during its development.

Modules

The online training curriculum topic areas covered in the 10 modules of the culturally tailored curriculum parallel those of the standard online CITI modules for social–behavioral research. The format of the AIAN-tailored curriculum includes quizzes after every module and text boxes that highlight important information and give examples of the main content within each section and links to additional information. The modules include: (1) a history of research in Indian country and description of tribal sovereignty; (2) a history of ethical regulations, the Belmont principles of respect for persons, beneficence, and justice, and their applications in AIAN communities; (3) the definition of ‘human subjects research’; (4) the types, roles, and responsibilities of IRBs, and exempt, expedited, and full board review; (5) calculating risks and benefits, assessing probability and magnitude of research risks, and benefits; (6) strategies to ensure privacy and confidentiality; (7) required elements of the consent process and how to ensure participants’ understanding; (8) definition of vulnerable populations and examples of individual and personal vulnerabilities in research settings; (9) special considerations for AIAN research involving children; and, (10) unanticipated problems and reporting requirements. Each module ended with six quiz questions: two original CITI questions, two slightly modified CITI questions (i.e. clarifying or simplifying language), and two new questions applying the Belmont principles in an AIAN setting.

A description of the full AIAN-tailored adaptations is beyond the scope of this article. Here, we provide a few illustrations. Module two of the AIAN curriculum presents the Belmont Principles, including indigenous perspectives emphasizing the importance of establishing trust and equitable partnership with tribal members and their leadership. For example, the principle of beneficence must include consideration of the potential benefits and harms to participating tribes or AIAN communities. The principle of respect must recognize tribal sovereignty and tribes’ inherent right to govern research that take place on their lands. This includes early discussions regarding data ownership, data use, publications, and dissemination activities with tribal representatives. The principle of justice requires that sample selection must consider distributive justice and ensure that AIAN communities are neither excluded from nor exploited in research. In discussing the definition of human subjects research, module three notes that although the current definition relies on the term ‘living’ person, many tribal members view deceased individuals, their stories, and their belongings (otherwise known as their data) as sacred, which means that they retain rights to privacy and confidentiality. In addressing privacy, module six notes that some tribal ceremonies that take place in public are sacred, should be treated as private information, and that tribal anonymity is as important as protecting participant identifiers. It also emphasizes the obligation of researchers to protect cultural data (such as objects or teachings), including excluding cultural information in reports or publications without tribal approval. The full curriculum is available by request to the first author.

Curriculum pilot and debriefing

We recruited 49 AIAN participants through our expert panel to pilot the online training. Participants took the online pre-post surveys and an in-depth phone debriefing interview to assess: (1) feasibility of delivery the training online; (2) appropriateness of the format, language level, clarity of content, questions, and directions; and, (3) the psychometric properties of the knowledge quiz items. Pilot participants were recruited across the U.S., with 63.3% from urban settings and 73.5% female. The pilot resulted in minor editorial changes to the curriculum. Following the psychometric analysis, the final quiz was composed of five items from each module. Identical quiz questions were used in the CITI and the AIAN curricula.

RCT participants and recruitment

We recruited and consented 490 AIAN. Eligible participants were 18 years of age or older with an interest in research, had not taken human subject training in the last five years, had an email address, and could access the internet. This sample size achieved 80% power to detect a 15% or more difference in pass rates of knowledge quizzes between the CITI and the AIAN curricula at the post-training assessment. Participants received $150 for the 4–5-hour online training and assessment. Data were collected from February 2016 to January 2017. Recruitment was stratified to sample tribal members by 10 representative cultural regions, defined in the Native North American Almanac as geographical areas in which AIAN communities shared common ecological environments, food production, or language to account for regional difference (Champagne, 2001) and urban/rural settings. Tribal affiliation was collected, though not retained, to validate cultural region classification. Drawing on Census Bureau data (Norris, Vines, & Hoeffel, 2012), we oversampled rural settings to ensure tribal representativeness, resulting in 38% rural/62% urban participation for a total of 20 strata. The University of Washington Institutional Review Board approved the study under protocol #47991. The expert panel and representatives from our community partners (described below) provided approval of the final curriculum.

Recruitment efforts also relied on the American Indian Higher Education consortium (AIHEC) (serving AIAN students from over 230 federally recognized tribes across 37 AIAN tribal colleges and universities), the National Congress of American Indians (NCAI), and the networks of panel members and four investigators. Recruitment included extensive advertising, word of mouth, postings on stakeholders’ websites (i.e. AIAN research institutes and centers), local health boards, and national NCAI and AIHEC conferences.

Sociodemographic, geographic, and other descriptive information for the overall sample and by study condition are provided in (Supplemental Table 1). Geographic cultural region distribution matched the purposive sampling plan and randomization achieved balance in all pre-specified demographics, geographic cultural regions, and potential confounding variables; there were no statistically significant differences between study arms.

RCT procedures

Participants called a toll-free telephone number, were screened by study staff, and provided oral consent. Tribal affiliation was collected to ensure a representative sample. These data and personal identifiers were destroyed at the end of data collection. Participants received information for logging into either the AIAN or CITI curriculum according to a computer-generated randomization list. To ensure the balance of group characteristics and to reduce opportunities for selection bias, we used a variable permuted randomization block design where the block size itself was randomly selected (i.e. blocks of four and eight) (Hedden, Woolson, & Malcolm, 2006), randomizing by cultural areas and rural/urban setting upon consent. Blinding and random assignment were maintained through staff training and continuous supervision by key members of the research team. All investigators, staff, and participants were blinded to outcome measurements during data collection (Schulz, Altman, Moher, & CONSORT Group, 2010).

Outcomes

Knowledge quiz scores (48 items; five in section 2–9 and three items in 10) assessed understanding of content knowledge using true–false and multiple-choice questions. Percent correct at first attempt was calculated for each module. Interest in research (six items, five-point Likert-type scale from ‘not at all’ to ‘extremely’) was assessed after completion of the modules and assessed involvement as co-researchers, research with AIAN in general, conducting research involving their tribal community, partnering with academic or government organizations, serving on an IRB, and as a research participant in the role of research subject, (α = .82). Research review self-efficacy (eight items, five-point scale from ‘not at all confident’ to ‘extremely confident’) assessed confidence in reviewing research protocols, specifically in making recommendations to ensure informed consent, applying the principles of beneficence, respect, and justice, and knowing ways to protect privacy (α = .94). Hall’s (2006) 12-item trust in research scale (Hall et al., 2006) originally focused on trust in medical research and was adapted so that ‘doctor’ was replaced with ‘researcher’. Sample items include: ‘Health researchers treat people like ‘guinea pigs” and ‘it’s safe to be in a research study’ (α = .89). Time to take the training was self-reported. Two previously validated scales assessed participants’ perception of the relevance of the materials (eight items, α = .86; i.e. subject covered, information reflective of culture, quality of the information conveyed) (Ayala & Elder, 2011) and satisfaction (five items, α = .90; i.e. readability, format, style of the materials, visual appearance, and range of topics) (Kalbach, 2009). A final scale assessed ease and understandability, and included six items, (α = .71). Items included measures of ease of accessing and using the online training, understanding vocabulary, whether quiz questions asked relevant questions, and access to materials and technology. Responses ranged on a five-point scale from ‘not at all’ to ‘extremely relevant’, ‘confident’, and ‘satisfied’, as appropriate. Mean scores were calculated for each measure.

Sociodemographics and potential moderators

We collected data on participants’ gender (male/female), age, highest grade or post-high school degree completed (i.e. vocational, associate, bachelor), employment status (temporary, part- or full-time), top three research topic interest areas, and years of research experience.

Statistical analysis

We conducted an intent-to-treat analysis, in which participants remained in study arms as initially randomized. We assessed participants’ characteristics, pre-training interest, trust in research, and research review efficacy using descriptive statistics. Differences between study arms were examined using chi-squared tests for categorical variables, and t-tests for continuous variables. We assessed reliability of updated constructs for research interest, trust in research, and research review efficacy using Cronbach’s alpha. Within-individual differences in these constructs before and after training were compared using paired t-tests. For primary analyses, we focused on two outcomes: (1) a binary outcome of whether the participant passed the knowledge quiz after training; and, (2) a continuous outcome of actual quiz score. Following CITI’s algorithm, participants received a passing score on the knowledge test if 80% or more questions were correct. The continuous quiz score reflected the percent of questions correctly answered. We compared passing rates between study arms using chi-squared tests, and quiz scores between study arms using t-tests. We applied logistic and linear regression to examine the relationship between training and outcomes while controlling for age, education, urban/rural setting, prior ethics training exposure, and cultural regions. We examined effect modifications by age and education by including interaction terms in the regression analyses.

Results

As seen in (Supplemental Figure 1), of the 711 individuals screened for eligibility, 221 were excluded. A total of 490 participants were randomized to the intervention (AIAN curriculum, n = 244) and control (CITI curriculum, n = 246) groups. Among the 490 participants, 17 participants never logged in and seven participants logged in but never proceeded to surveys. A computer problem resulted in loss of pre-assessment survey data for 15 cases and pre- and post-survey data for two cases in the control arm. However, CITI was able to provide post-training knowledge scores for 15 missing cases, leaving two participants missing all survey outcomes and knowledge scores. Overall, 26 participants (eight in intervention arm [3.3%] vs. 18 from control arm [7.3%], p = .05) were dropped due to missing data. The remaining 464 participants formed our intent-to-treat sample, of which 429 (92.5%) completed the 4-hour training and pre- and post-assessments, and 35 (7.5%) had some missing responses in surveys and quiz scores. Intervention arm participants had a slightly higher retention rate compared to the control arm. There were no relevant significant differences between study arms in sociodemographic, geographic cultural regions, or potential moderators (i.e. gender [male/female], age, highest grade or degree completed, employment status, Supplemental Table 1). The top three rated research interests were mental health, violence, and substance use.

Intervention participants were more engaged in answering survey and quiz questions, with 14.5% of control participants missing one or more outcomes, compared to only 7.6% intervention participants (p < .01). There were no statistically significant differences between participants who completed all assessments and those who did not complete in terms of age, past research experience, tribal recognition status, cultural region, urban/rural setting, employment, research interest, tribal enrollment status, and state of residence. Less educated participants were more likely to drop out. Among those without a college degree, 14.5% missed one or more outcomes, compared to 6.8% with a college degree (Pearson χ²_{df = 1}p = .01).

Main outcomes

Across training module quizzes, intervention arm participants out-performed those in the control by a range of 6.8%–20.2% across all modules (Table 1). Overall, AIAN curriculum participants answered an average of 78.9% quiz items correctly, compared to 65.3% (p < .001) among CITI participants. Using an 80% correct score as passing criterion at first attempt, the AIAN curriculum arm achieved a 59.3% passing rate versus a significantly lower 28.1% passing rate in the CITI arm (p < .001).

Table 1.

Comparison of knowledge quiz scores post-training by study arm among 490 AIAN community members.

	Total				Test statistic
	N	Combined	AIAN curriculum	CITI curriculum
Post-knowledge scores		N(%)	n(%)	n(%)
Total score	450	72.2 (18.9)	78.9 (14.2)	65.3 (20.9)	t = 8.09***
History of ethical principles	450	70.0 (29.3)	73.9 (23.0)	66.0 (34.4)	t = 2.88**
Human subjects research	442	73.3 (28.8)	83.2 (19.38)	63.0 (33.2)	t = 7.86***
IRBs function and roles	434	74.3 (28.8)	82.3 (21.3)	65.9 (33.0)	t = 6.17***
Risk & benefits	431	85.7 (23.1)	90.5 (15.6)	80.6 (28.0)	t = 4.59***
Confidentiality	429	66.5 (25.8)	71.5 (21.6)	61.3 (28.6)	t = 4.15***
Informed consent	429	71.7 (24.5)	79.4 (20.0)	63.8 (26.1)	t = 6.98***
Vulnerability	429	80.9 (25.7)	86.3 (19.0)	75.3 (30.2)	t = 4.56***
Research with children	429	56.7 (27.0)	59.8 (26.4)	53.4 (27.3)	t = 2.45**
Unanticipated problems/reporting Requirements	429	77.9 (28.1)	84.8 (21.2)	70.7 (32.2)	t = 5.37

^*P < .05;

^**P < .01;

^***P < .001;

t = t test.

For each arm, within-participant pre/post-test comparisons of self-efficacy in conducting ethical reviews of research protocols, trust in research, and interest in conducting research (Table 2) indicated that, on average, participants in both arms reported a significant increase in self-efficacy of ethical reviews (pre-test 2.99 to post-test 3.94 p < .001). Trust in research increased for the AIAN curriculum but not the CITI curriculum. Interest in research decreased slightly for both arms comparing scores between pre-test (mean score: 4.35) and post-test (mean score: 4.22), (p < .001). For both curricula there was a slight, significant drop in scores reflecting likelihood of engaging in research (mean score: pre-test 4.35 to post-test 4.22, p < .01). On average, at pre-test 96.0% of the participants reported they were somewhat to extremely likely to engage in research and at post-test this dropped to 92.4%. The magnitude of change for research review self-efficacy and interest in research was similar across both arms, suggesting that both trainings had similar impact on these outcomes.

Table 2.

Comparison of outcomes measures for pre/post-data analysis by study arm and within study arm among 490 AIAN community members.

		Pre-training score	Post-training score	Difference	Test statistic
Research review efficacy
Total score	413	2.99 (1.01)	3.94 (.72)	.95	t = 19.6***
Difference in AIAN curriculum group	218	3.04 (1.04)	4.02 (.63)	.98	t = 14.7***
Difference in CITI curriculum group	195	2.93 (.98)	3.85 (.79)	.91	t = 12.9***
Trust in research
Total score	414	3.15 (.39)	3.23 (.38)	.07	t = 3.62***
Difference in AIAN curriculum group	218	3.11 (.39)	3.22 (.36)	.11	t = 3.9***
Difference in CITI curriculum group	196	3.20 (.38)	3.23 (.39)	.03	t = 1.08
Interest in research
Total score	414	4.35 (.62)	4.22 (.75)	.13	t = 4.08***
Difference in AIAN curriculum group	218	4.40 (.55)	4.31 (.67)	.1	t = 2.45**
Difference in CITI curriculum group	196	4.29 (.68)	4.12 (.82)	.17	t = 3.27***

^*P < .05;

^**P < .01;

^***P < .001;

t = t test.

Moderation analysis

Using multi-variable linear regression we examined the association between quiz scores and study arm, while controlling for gender, age, education, prior ethics training experience, rural/urban setting, and cultural region. The intervention arm was associated with, on average, a 14.2-point higher quiz score (p < .001), and having a college degree was associated with a 9.4-point higher quiz score (p = .03). No other modification effect was detected.

Acceptability, feasibility, and understandability

Those who completed the AIAN curriculum were more satisfied with the materials than those who took the CITI curriculum (see Table 3). AIAN curriculum participants reported significantly higher mean scores for satisfaction, acceptability, and reported greater difficulty in understanding the CITI vocabulary and greater difficulty in accessing the online training curriculum.

Table 3.

Comparison of acceptability and feasibility measures by study arm among N = 467 AIAN community members.

	Combined (N = 425) (M, SD)	AIAN curriculum (n = 237) (M, SD)	CITI curriculum (n = 230)	Test statistic
Acceptability †(Post assessment)
Relevance: overall score (M, SD)	4.0 (.65)	4.2 (.54)	3.8 (.70)	t = 6.21***
Relevant to research goals	4.0 (.92)	4.1 (.94)	3.8 (.91)	t = 2.81***
Research protections relevant	4.2 (.84)	4.4 (.74)	3.9 (.88)	t = 5.85***
Topics coverage important	4.2 (.85)	4.5 (.65)	3.9 (.93)	t = 7.25***
Information present clearly	4.1 (.83)	4.2 (.82)	4.0 (.85)	t = 1.67
Images appropriate for the text	3.9 (.95)	4.0 (.90)	3.8 (1.0)	t = 1.97*
Appropriate examples	4.0 (.92)	4.3 (.80)	3.8 (1.0)	t = 5.53***
Information presented in an interesting way	3.7 (1.0)	3.9 (.95)	3.5 (1.1)	t = 4.23***
Relevant to research with AIAN	4.2 (.86)	4.5 (.70)	3.9 (.93)	t = 6.50***
Satisfaction †(Post assessment)
Satisfaction: overall score (M, SD)	3.9 (.77)	4.1 (.66)	3.7 (.83)	t = 5.26***
Format of the materials	3.8 (.85)	4.0 (.73)	3.7 (.93)	t = 4.10***
Style of presentation	3.8 (.93)	4.0 (.82)	3.6 (1.0)	t = 4.54***
Range of topics covered	4.1 (.80)	4.3 (.70)	4.0 (.87)	t = 2.98**
Readability	3.8 (.98)	4.1 (.88)	3.6 (1.0)	t = 4.74***
Visual appearance	3.8 (1.0)	4.0 (.88)	3.5 (1.1)	t = 5.30***
Time to take the training (hours)
Number of hours	4.3 (2.5)	3.9 (2.11)	4.8 (2.87)	t = 3.62***
Time spent was reasonable (n, %)	397 (93.9)	209 (96.3)	188 (91.3)	x2 = 4.67*
Ease & understandability
Difficulty understanding words use (vocabulary)	2.1 (.95)	1.9 (.91)	2.2 (.98)	t = 3.50***
Ease of access to only materials	4.3 (1.02)	4.5 (.93)	4.0 (1.06)	t = 4.89***
Had technology problems (n, %)	44 (10.4)	10 (4.6)	34 (16.5)	x2 = 16.1***
Chapter quiz represented important lessons	3.8 (.95)	3.8 (.92)	3.8 (.96)	t = .23
Multiple-choice questions were confusing	2.3 (1.14)	2.3 (1.18)	2.3 (1.10)	t = .42
Quiz assessed how well I understood the material	3.9 (.84)	4.0 (.83)	4.0 (.86)	t = .39

Notes: M = mean, SD = standard deviation, N = frequency, % = percent; t = t test;

χ² = Pearson chi-square; t-test and chi-square test were conducted as appropriate;

^*P < .05;

^**P < .01;

^***P < .001.;

^δδ5-point Liket scale responses dichotomized at mostly or extremely compared to not at all, a little, somewhat.

Discussion

This study highlights the value of drawing upon community engagement strategies to develop effective culturally appropriate research ethics curriculum for AIAN community members. This approach expanded ethics training to include both individual and community level protections, including: respect for tribal sovereignty; unique confidentiality and privacy concerns; tribal or AIAN entity approval of research; and, understanding of cultural and contextual differences regarding research risk as grounded in historical events stemming from colonization to more present breaches. The modules recognized that structural inequalities rather than participant characteristics constitute the primary source of vulnerability for AIAN research participation. Increasingly, AIAN communities conduct randomized control trials that align with indigenous research priorities and ways of knowing. As opposed to the traditional RCT treatment versus a placebo RCT, acceptable RCT designs in AIAN settings have included cluster and waitlist control designs most of which use a community-engaged approach. As tribal communities assume leadership in research conducted in their communities, ethics review of research protocols plays a critical role in ensuring that research avoids cultural misunderstandings, including tribal stigmatization, or culturally inappropriate, irrelevant, and disrespectful science that may lead to mistrust and damage the long-term viability of research opportunities (Hyatt et al., 2009b; Mohatt & Thomas, 2006). Although the training was developed for AIAN community members interested in research, future research can assess the extent to which the training may be relevant for anyone reviewing or conducting research with AIAN communities.

The AIAN-tailored ethics training curriculum, as compared to the standard CITI, resulted in significantly higher total knowledge scores and scores on individual modules. Participants who took the AIAN curriculum (compared to the CITI curriculum) reported higher levels of trust in research and research review efficacy post-completion. These higher scores demonstrate that when the Belmont principles are contextualized within a community setting, members increase their retention of information and their skills to implement research protections, thus increasing trust in the research process. The AIAN curriculum was developed by AIAN representatives across the U.S. with a national scope, while acknowledging that there is great diversity across tribal communities and the importance of local AIAN community review and approval of research. This entry-level research with human subject AIAN curriculum is generalizable across the U.S. as there were no regional, urban, or rural differences, thus supporting the national dissemination across AIAN communities.

Some limitations require further investigation. Less educated participants were more likely to drop out of the study. Consideration should be given to making the training even more accessible, such as providing a trainers’ toolkit so the training can be delivered in-person. In-person trainings would help address varying degrees of educational inequality as well as benefit those that have limited or no internet connections or where computer access or knowledge is limited. Although at post-training, over 92% of the participants reported likely to engage in research, we saw a slight decrease in the level of interest in research in both arms, between pre- and post-training assessments. This suggests that the training may have contributed to a sense by participants that achieving the necessary research knowledge and training presents a major challenge. Further investigation is needed to understand the lower level of interest post-training.

Historical unethical research practices have generated AIAN community mistrust of research. In addition, academic researchers and institutional/ethical review boards (IRB/ERB) often lack familiarity with the risks and benefits unique to diverse communities, hampering the review of culturally specific ethics issues. As a result, practitioners of community-based participatory research (CBPR) and CEnR have raised questions about how well standard research ethics training fits with the principles and practices of CBPR and CEnR (Anderson et al. 2012a). The absence of culturally relevant human subjects research training for community partners has been noted as a barrier to CBPR/CEnR. Currently, online ethics trainings for federally funded research are not written for community research partners (Anderson et al. 2012a), nor do they address research ethics issues unique to diverse cultural and environmental contexts. This validated ethics training curriculum addresses the gap and offers a tool to improve understanding of indigenous ethics principles, reduce concerns related to potential for research harm, and enhance research benefits. Moreover, the process by which it was developed offers guidance for other communities that experience both health inequities and a history of research harm and mistrust.