Exploring Institutional Research Ethics Systems: A Case Study From Uganda

Abstract

Background

The increasing globalization of research drives a need for greater research ethics capacity in low resource countries. Several programs have attempted to expand research ethics capacity by training individuals, but few have focused on broader research ethics systems and institutions. This study describes and applies an institutional research ethics model to assess the institutional research ethics capacity of Makerere University College of Heath Sciences (MakCHS) in 2011.

Methods

Internal and external stakeholders conducted the assessment of MakCHS using the multidimensional Octagon framework. Five methods were used to collect data on current ethical processes and institutional relationships.

Results

MakCHS scored in the mid range on all Octagon domains, with some variation between external and internal assessments. The external Octagon scores suggest that MakCHS’s areas of strengths are in identity, structure, relevance, target groups, and working environment; needs are greater in the areas of production, competence, and systems of finance and administration. Discrepancies in external and internal assessment can serve as a useful platform to shape ongoing discussions and strategic efforts.

Conclusions

The assessment identified strengths, opportunities, and challenges for institutional research ethics capacity at MakCHS. We believe this systematic approach was helpful in evaluating research ethics needs and provides a benchmark for institutions to measure progress over time.

Over the past two decades, the health research landscape has become progressively more globalized, with increasing volumes of research trials shifting to low- and middle-income countries (LMICs) (Glickman et al. 2009; Thiers, Sinskey, and Berndt 2008; Vidyasagar 2006). In addition to shifting disease burdens, cost advantages and perceptions of decreased regulatory burdens have, in part, encouraged investigators to locate studies in LMICs and the National Institutes of Health (NIH) to fund more international studies (National Bioethics Advisory Commission 2001). For example, since 2002, the number of research investigators regulated by the Food and Drug Administration (FDA) located outside of the United States has increased by 15% annually (Getz 2007). Further analyses suggest that the greatest proportional increase in clinical trials sites has been in areas of Africa, the Middle East, and parts of Eastern Europe (Glickman et al. 2009). This increasing globalization of research highlights the need for researchers, health professionals, and institutions to be trained in research ethics (Ijsselmuiden et al. 2012; Singer and Benatar 2001).

To address this need, several initiatives have funded or implemented programs to train LMIC professionals in research ethics and on how to apply research ethics principles on a local level, including the African Malaria Network Trust (Nyika et al. 2009), the International Bioethics Education and Career Development Award of the National Institutes of Health’s Fogarty International Center (Millum et al. 2013), the Strategic Initiative for Developing Capacity in Ethical Review (2003), the European and Developing Country Clinical Trials Partnership (2009), and the Society and Ethics funding programs of the Wellcome Trust (2014).

These programs have been crucial in developing research ethics education in LMICs, and all support capacity building. Many have primarily focused on training individuals or strengthening institutional review boards (IRBs) in LMICs, while others focus on partnerships between a Northern institution and an LMIC institution. In addition, online resources for building research ethics capacity have become increasingly more available, for example, through platforms from the World Medical Association (2014), The Ethics Application Repository (2014), and Global Health Reviewers (2014).

However, in order to encourage fully functional research ethics systems in LMICs, we believe training programs should also employ a systems approach that directs attention to the developmental needs of institutions, organizational models, infrastructure, and financing (Hyder et al. 2009). A systems approach is inherently concerned with sustainability, acknowledging that institutional-level investment is critical for supporting networks of individuals and future growth. An institutional commitment to research ethics should involve the following components: (1) the establishment of a justifiable research agenda; (2) the protection of research participants; (3) training of institutional members; (4) the establishment of institutional priorities and structures that promote ethical conduct; and (5) the strengthening of communications with regional, national, and international stakeholders, especially institutional leadership (Hyder et al. 2009). Therefore, capacity development must also occur on institutional levels with a strong focus on supportive models of partnership. We see this focus on institutional-level capacity as critical for enabling a wider “culture of ethical conduct” within the developmental context of LMICs (Edwards, Hifnawy, and Silverman 2014).

Developed with funding from the Fogarty International Center, the Johns Hopkins–Fogarty African Bioethics Training Program (FABTP) began in 2000 with a model that initially focused on bringing individual African researchers to the United States to receive 6 months of training at Johns Hopkins University, followed by a 6-month, mentored, in-country practicum (Ali 2012; Hyder et al. 2007). In 2010, recognizing the changing research ethics landscape in Sub-Saharan Africa and the growing value of an institutional approach to research ethics training, FABTP restructured its program to focus on year-long institutional partnerships with African institutions (Hyder et al. 2013). In 2011, FABTP entered a 1-year research ethics partnership with the Makerere University College of Heath Sciences (MakCHS) in Uganda.

Broadly, the goal of FABTP’s institutional training program was to operationalize a commitment to research ethics by first conducting a needs assessment, which would include learning about local objectives, and then targeting capacity development to institutional needs. In this article, we describe our systematic approach for assessing institutional research ethics needs at MakCHS. We begin by providing background on MakCHS and then present the institutional research ethics needs assessment approach, methodology, and process. We hope this process for assessing research ethics capacity facilitates the critical evaluation of institutional commitments to research ethics in LMICs going forward.

MAKERERE UNIVERSITY COLLEGE OF HEALTH SCIENCES

MakCHS in Uganda was formally established in 2007 as a constituent college of Makerere University, the country’s largest and second oldest educational institution (Namisango and Wamai 2012); however, it was not fully operational until January 2011, when its first principal was appointed. MakCHS comprises four schools, including Biomedical Sciences, Public Health, Medicine, and Health Sciences. When our partnership started in 2010, at least 10 professionals with graduate- or certificate-level training in bioethics and research ethics from foreign institutions worked closely with MakCHS; some were academic and administrative staff, while the rest were based at other institutions affiliated with MakCHS. Together, these professionals constituted what they called a “Bioethics Working Group” and formed the backbone of research ethics training at MakCHS. The substantial research agenda at MakCHS, the large number of international collaborations, and the lack of dedicated time for bioethics resulted in growing expectations placed upon the limited number of faculty and staff members. Subsequently, the institution has been actively seeking more capacity and engagement in bioethics and research ethics.

In 2008, MakCHS released its Research and Innovations Policy, detailing support for institutional review boards (IRBs) and their role in overseeing ethical conduct and research at the institution (Makerere University 2011). Until 2011, MakCHS had two IRBs—one each at the schools of Medicine and Public Health; since this work was conducted, the schools of Biomedical Sciences and Health Sciences have established two new IRBs. At the time, MakCHS was also actively planning to establish a core bioethics center and a master’s program in research ethics. MakCHS is one of the most mature African universities in its management of human resources and commitments to research ethics; its leadership has noted its dedication to increasing training, leadership development, and funding for ethics. To this end, MakCHS was selected to partner with FABTP in 2011 to further improve research ethics capacity.

Each FABTP partnership began with a baseline assessment of needs, capacity, and resources of the African university to best target work and goals for the year and facilitate the development of concrete plans for addressing institutional needs. It is important to note that this assessment occurred during a transitional period for MakCHS, and the college was still establishing mechanisms and administrative structures necessary to support its research agenda. This article describes the approach taken for conducting a baseline assessment at MakCHS, as well as the results of the assessment. We then discuss the broader implications for institutional engagement in research ethics in LMICs.

METHODS

Framework for Assessing Institutional Research Ethics Capacity

FABTP performed a systematic joint baseline needs assessment of MakCHS early in the partnership (January–March 2011). Though evaluation methods have been developed for health systems research, no existing tools adequately assessed institutional research ethics capacity in LMICs (Malek, Geller, and Sugarman 2000; Forum for Ethical Review Committees in the Asian and Western Pacific Region 2008). Therefore, we adapted the Octagon model created by the Swedish International Development Cooperation Agency (2002), a framework designed for assessing the capacity of nongovernmental organizations in African countries. The framework evaluated organizations according to eight domains: (1) basic value and identity; (2) structure and organization; (3) implementation of activities; (4) relevance of activities to the stated goals; (5) capacity of staff and management to carry out activities; (6) organization’s administrative, financing, and accounting systems; (7) relationships with target groups and stakeholders; and (8) relationship of the organization to the larger environment within which it operates.

This adapted Octagon model involved a partner-based process and, as such, is different from other self-assessment tools that have their own important role in evaluating ethics capacity (Silverman et al. 2014; Sleem, El-Kamary, and Silverman 2010). For example, this model differs from other assessment tools, such as those created by the Association for the Accreditation of Human Research Protection Programs (2012) and the Strategic Initiative for Developing Capacity in Ethical Review (World Health Organization 2003), in its focus on a wide range of institutional-level measures, focus on a systems approach, and use of a deliberative process. This model lends itself to a rapid evaluation of the institutional research ethics system within these eight domains as described in the following (Table 1).

Table 1.

Eight-domain Octagon framework for institutional research ethics capacity assessment

Domain		Key elements
1	Basic values and identity	Formulation of organization’s vision and mission for research ethics Formulation of relevant strategies for research ethics
2	Structure and management	Application of a clear division of duties and responsibilities for research ethics Application of democratic rules and transparency in decision making
3	Implementation of activities	Planning for the implementation of activities for research ethics Follow-up and learning from work done on research ethics (feedback)
4	Relevance of activities	Content of activities in research ethics correspond with the vision/goal for research ethics Strategies are consistent with goals in research ethics
5	Capacity	Professional qualifications and experience of the ethics staff are appropriate Ability of management to lead research ethics programs
6	Systems for finance and administration	Administration of financial resources and accountability Research ethics procedures and routines are documented and planned
7	Target groups	Dialogue and relationships with target groups in research ethics Legitimacy for research ethics work and active participation in networks
8	Working environment	Activities are in accordance with government rules/regulatory policies for research ethics Influence of and on national health research system

The first domain, Basic Values and Identity, focuses on whether an organization has clearly articulated its aims and objectives for research ethics and analyzes whether the organization has specific strategies to achieve these goals. Highest points are awarded if the aims and objectives with regard to research ethics are available in written form and if clear strategies are outlined to achieve these goals. The second domain, Structure and Management, assesses how an organization is structured for research ethics and how these responsibilities are allocated. Questions focus on whether research ethics duties are clearly defined for employees and whether the organization abides by principles of transparency, accountability, and democratic rules. Highest points are awarded if members of the organization know their purpose within the research ethics group and feel that responsibilities are clearly allocated. The organization should also have provided an operational plan outlining these research ethics divisions to promote transparency.

The third domain, Implementation of Activities, evaluates the implementation of research ethics activities and evaluates an organization’s ability to systemically carry out an operational plan in which research review and teaching of ethics are actively implemented. We also defined it to assess whether the organization requests and incorporates feedback on its current ethics operations. Highest points are awarded if institutional management has created and implemented an operational plan for research ethics to achieve the organization’s objectives. The fourth domain, Relevance of Activities, analyzes whether the content of an organization’s activities in research ethics corresponds with its vision. This domain examines the consistency among goals, existing strategies, and activities to determine the likelihood of achieving the stated objectives. Highest scores are awarded when research ethics activities and working methods of the organization clearly correlate with its vision.

The fifth domain, Competence, centers on whether the staff possesses appropriate research ethics skills and qualifications to fulfill activities crucial to its objectives. It also assesses whether the management is recognized as legitimate by workers—for example, whether researchers believe that institutional leadership understood and was capable of building research ethics capacity and whether the institution was staffed to do so. Highest points are awarded when research ethics job descriptions are clearly documented and selected staff members fully meet the job description criteria. The strength of management is evaluated based on the intellectual support that staff members received for research ethics activities. The sixth domain, Systems for Finance and Administration, examines the financing and administration of the research ethics component within the organization. Here, highest points are awarded when financing is guaranteed through internal budgetary support and there are detailed sources of funding for research ethics activities. This domain also encompasses whether an organization has an efficient administration that systematically documents its activities, typically in the form of standard operating manuals, ethics job descriptions and contracts, and documentation of IRB registration.

The last two domains, Target Groups and Working Environment, focus on relationships and context, analyzing the organization’s collaborations as well as the surrounding academic environment. The organization should have maintained strategic relationships with internal stakeholders (e.g., clinicians or researchers) as well as other institutions pursuing similar research ethics aims (“target groups,” for example, another university or institute in the same city) to foster local and regional networks and avoid duplicating efforts.

The seventh domain awards highest points if the organization identified and involved clear target groups in its activities. In addition, organizations should have considered stakeholder feedback and included key constituencies in decision-making processes. The eighth domain addresses the regulatory context provided by the national (or state) government and national health research system. This explores how the institutional research ethics activities and policies reflected consistency with higher national authorities. Highest points are awarded when research ethics activities are consistent with local, national, and multinational laws, policies, and research ethics guidelines.

Applying the Institutional Assessment Approach to This Baseline Assessment

MakCHS participated in a needs assessment exercise involving a mixed-methods approach to examine research ethics capacity at the beginning of the partnership. Five methods were used to collect data to inform the Octagon framework, including: (1) review of institutional documents, (2) an institutional survey, (3) a survey of IRB members, (4) in-depth interviews with key stakeholders, and (5) focus-group discussions. Results from each method were used by the MakCHS leadership and the FABTP teams, respectively, to generate scores for each Octagon domain.

The study protocol was reviewed and granted exempt status by the Johns Hopkins Bloomberg School of Public Health institutional review board.

Surveys

The document review explored institutional policies, teaching syllabi, and training materials related to research ethics. Building on previous work by the World Health Organization (2006), the FABTP team developed an institutional survey containing sections pertaining to an institution’s leadership and policies, current research ethics practices, educational opportunities, existing research structure, financial commitment to research ethics, national government policies, and IRB information. The institutional survey consisted of 194 questions encompassing all eight domains of the Octagon (see example questions in Table 2). This survey had been administered previously to our 2010 partner, the University of Botswana (Hyder et al. 2013), and was modified based on that experience for use in Uganda.

Table 2.

Institutional and Research Ethics Committee survey—Illustrative questions

Institutional survey	Institutional leadership, plans, and policies	Does [Institution] have written institution-wide policies that clearly state the kinds of research protocols that must be submitted to the IRB?
	Institutional finances	Does [Institution] set aside financial support for research ethics or for example an Office of Research and Development?
	Formal teaching of research ethics	Does [Institution] offer any type of educational opportunities in research ethics for students? For faculty?
IRB member survey	Human resources	Are there individuals currently on [Institution]’s academic faculty/staff with a significant academic interest in bioethics? Does [Institution] offer any type of educational opportunities in research ethics for students? For faculty?
	Institutional finances	Does the IRB receive sufficient financial support?
	Formal training of research ethics	Do the members of the IRB have adequate training in research ethics required to provide quality reviews?
	Relevance	Are the IRB standard operating procedures sufficiently detailed?
	Implementation	Is the IRB review process clear? Does it ensure that reviews are completed in an appropriate amount of time?

The survey was distributed to MakCHS (Principal) leadership members with explanation that their responses should reflect the institution’s existing research ethics capacity. A cover page disclosing the purpose, procedure, risks, and benefits of completing the survey was enclosed, and the completed survey was collected after 6 weeks.

The IRB member survey was completed by members of the MakCHS schools of Public Health and Medicine IRBs, who were asked to evaluate their committee’s level of functioning, resources, financial support, and training. This survey consisted of 43 questions intended to ascertain the demographics of IRB members, opinions on their institution’s current IRB procedures and policies, and the resources and training opportunities available to them at MakCHS. This five-page survey collected information pertinent to all domains of the Octagon (see example questions in Table 2). The questionnaire was distributed, following oral consent, to members of the MakCHS Schools of Public Health and Medicine IRBs, and all members were requested to respond anonymously.

Interviews

Qualitative data were collected via semistructured interviews with key informants representing institutional leadership in academia and research, including principal leadership and school deans of MakCHS, IRB chairs, and senior researchers. Recruitment for in-depth interviews began in early January 2011, and participants who agreed were audio-recorded. The interviews were used to elaborate on institutional policies, experiences with strategic development and organizational issues, and understanding of research ethics across all of the domains.

Focus-group discussions were held with key stakeholder groups that had targeted experience with research ethics, including IRB members, researchers at MakCHS, and staff members with prior training in bioethics and college leadership. Each group contained stakeholders from a similar category who were asked about their experiences, knowledge, and opinions of procedures related to research ethics at MakCHS, protocol review, ethical conduct, and the institution’s overall organizational structure with regard to research ethics (Table 3). These discussions were semistructured and moderated in English with four to eight participants in each. Transcripts from in-depth interviews and focus-group discussions were analyzed thematically by broad issues as matched to the Octagon framework to identify issues and quotes.

Table 3.

In-depth interviews and focus-group discussions—Illustrative questions

Research ethics training	Graduate students, research staff, researchers, IRB members	Have you ever taken any courses in bioethics or participated in ethics training?
Research ethics resources	Graduate students, research staff, researchers	If you had a bioethics or research ethics question, where would you go to get an answer?
Perception of strengths and challenges	Research staff, researchers, IRB members	What do you see as [Institution]’s greatest research ethics strengths?
Research ethics and [Institution]’s approach to ethics review	IRB members	What do you feel is the role of the IRB at the [Institution]?
Research ethics influences	Research staff, researchers	How do research ethics issues affect your day-to-day research activities?
Research experience	Graduate students	Have you conducted any research at [Institution] as a graduate student?

Data Analysis

Data collected from all five methods were compiled to determine the final Octagon domain score by the FABTP team and MakCHS team, respectively. Three FABTP team members individually scored MakCHS on every domain, employing a 7-point scale in which 7 is the highest score (excellent) and 1 is the lowest score (very weak). The three team members then convened and generated a score for each domain based on group consensus. The compiled results within the Octagon framework allowed the team to calculate an external assessment score for MakCHS. The two partnership directors at MakCHS completed a similar, but separate, institutional self-assessment of their perceived capacity in all eight domains. Results were compared to determine any differences between FABTP team and MakCHS self-assessment scores.

RESULTS

Makerere College of Health Sciences Case Study

The institutional survey, 15 IRB member surveys, five in-depth interviews (IDIs), and four focus-group discussions (FGDs) were included in the data analysis. The mean age of the 15 IRB survey respondents was 42.4 years; nine respondents were male and six female (Table 4). The majority of IRB survey participants held a master’s degree (53.3%), with seven participants holding doctoral degrees. Almost all respondents (93.3%) reported having received training of more than two days in research ethics. The mean age of FGD participants was 42 years, and 16 (of the 23 participants) were male; IDI participants were on average 49 years old and four of five were male. A majority of FGD participants and all of the IDI respondents held a master’s degree. Ninety-five percent of FGD and all IDI participants indicated they had received formal training in research ethics that lasted longer than two days.

Table 4.

Participant information at Makerere University College of Health Sciences assessment

Characteristic	Group	IRB members* (n = 15)	Focus group (n = 23)	In-depth interviews (n = 5)
Mean age in years		42	42.13	49.0
Gender	Male	9	16	4
	Female	6	7	1
Education level (terminal degree)	Bachelor’s	0	—	0
	Master’s	8	—	5
	Doctoral	7	—	0
Received training in research ethics	Yes	14	22	5
	No	1	1	0

^*IRB members include both institutional review board and research ethics committee members.

Domain 1: Basic Values and Identity

The MakCHS team identified the following documents for review: the MakCHS strategic plan, and standard operating procedures for the schools of Public Health and Medicine IRBs. The written 2010 MakCHS strategic plan had two goals pertaining to MakCHS being a research-centered institution: (1) to engender a more supportive research environment and (2) to enhance networking and dissemination of research findings. Additionally, the two IRBs had documented standard operating procedures that included associated forms for reviewing and monitoring study protocols, as well as informed consent procedures.

While the 2010 MakCHS strategic plan did not explicitly discuss a specific vision for research ethics, interviewees openly discussed ongoing efforts to strengthen research ethics capacity by establishing a master’s degree in bioethics and a university-wide course in research ethics. Discussions with interviewees and FGD participants revealed a widespread commitment, voiced by different respondents and stakeholders, to protecting human subjects in research through institutional channels and to strengthening research ethics capacity within MakCHS. There was also support for creating an ethics center within the university that could organize and pool resources for research ethics:

We’ve been thinking that we really needed a place, a home where we can coordinate our activities … with the center we’d have all the resources, including some bit of funds, reference materials. (FGD Participant)

Domain 2: Structure and Management

In its organizational documents, it was clear that MakCHS had defined a leadership position to oversee its research activities; however, no similar position existed for research ethics. The university had an Office of Research, which was responsible for organizing research activities; discussions with stakeholders revealed a possible plan for managing bioethics through the same office with any future bioethics center reporting to research leadership. At the time of the assessment, MakCHS lacked an organizational chart clarifying how the Office of Research related to other departments in the College. Through the assessment, this type of clear organizational plan was identified as a need.

For the IRBs, there were clear definitions of leadership and strict membership policies that outlined specific criteria for gender representation and for diversity in the background of members. IRB members believed that board member duties were clearly defined despite the lack of an organizational chart. Almost all members (93%) either agreed or strongly agreed that the IRB standard operating procedures (SOPs) were sufficiently detailed. This finding was further explained by the role that IRB members played in writing and revising the SOPs, which were based on those issued by the Uganda National Council for Science and Technology (UNCST). One interviewee outlined how specific the SOP is in terms of timing for different protocol reviews and how the procedures were followed in forming the IRB agenda:

[The protocol] is recorded on the agenda; so, what happens, they [the IRB secretariat] receive the proposals, and our SOP says two weeks before the meeting. And so when it gets to this time period, it generates a list of these proposals; the new review, the continuing review, the exempt and so on. (FGD Participant)

However, IRB member responses also indicated some frustration in the role of the IRB. This was captured by an IRB member who remarked:

The challenge is that we are an ethics committee and then that we are [also] a scientific review committee. (IDI Participant)

Additionally, responses suggested that there was an institutional culture of encouraging staff and faculty members to serve on institutional committees or in capacities that utilize their existing expertise:

I would say in our appointment job description [not formally], but to be on our staff, you are expected to be on a number of committees at our institution and if you are an expert in research ethics, we expect you to be part of that research ethics committee. In any case, if you are not able to be part, you supervise students when they do research. (FGD Participant)

Domain 3: Implementation of Activities/Production

Information was reviewed to better understand implementation at MakCHS of training programs, IRB work, teaching, and other research ethics-related activities. The assessment tool suggested that MakCHS demonstrated strength in areas of implementation, particularly in its research ethics training for students and faculty members. According to results from the institutional survey, MakCHS offered educational opportunities, mainly in the form of lectures and courses in research ethics for its students, as well as workshops for its faculty and research members, with some of the opportunities having been available for at least 5 years. For example, an FGD with researchers indicated that the School of Public Health offered a 10-week course in research ethics established 2 years prior.

Regarding specific policies, interviews and FGD analyses of institutional policies did suggest some confusion around the particular types of protocols that require approval from a MakCHS IRB and where an IRB proposal must be submitted. As one researcher indicated, while the procedure for submitting a protocol was clear in theory,

It is mostly affiliation to whichever department. Those who are affiliated with the School of Public Health will submit there and those that are affiliated with the School of Medicine will submit there. (Researcher, FGD)

In reality, the procedure was not always followed: “Then [for] other people, I think it’s a matter of preference” (Researcher, FGD).

When asked whether the MakCHS IRB review process was efficient and completed on time, 8 of 15 members either were neutral or disagreed. This lack of perceived IRB efficiency could be attributed to reports that the IRB often did not have a sufficient number of members in attendance to vote and also lacked incentives and recognition of its members for their service:

We have an attendance problem. There are two-thirds of the members, or 50% of the members, who attend continuously. It is usually the same people. And then, there is this one-third who rarely ever attends … when it comes to transitioning out the members … because we need institutional memory … you transition out the one-third. (FGD Participant)

Another concern included the lack of a standardized review checklist for IRB members. As one focus-group discussion indicated, only the secretariat receiving protocols had a checklist for completion, but not for review:

So they [secretariat] don’t review the ethics of the proposal, they just review the eligibility … They review the preliminary submission forms [using] the checklist and the SOPs to see whether this is a protocol that should be reviewed by our IRB. (FGD Participant)

In addition, the IRBs did not maintain an adequate follow-up review system for local proposals. Members indicated that given the large volume of submissions, only a few received a follow-up every year:

Sometimes you read the proposal and you are bit unclear as to whether the researcher actually implemented research … you make your recommendations … but there’s no formal way that you can be certain that these are the changes, in doing the implementation … I think there is no sort of feedback. You are not so certain that [exactly] what has been approved is exactly what has been implemented. (FGD Participant)

It was reported that the majority of studies that were afforded subsequent follow-up were those with international funding.

Domain 4: Relevance of Activities to Stated Goals

FGD participants and interviewees highlighted many activities that corresponded to MakCHS’s goals in research ethics while also offering recommendations for strengthening connection between activities and goals. For example, while one target population—members of IRBs at MakCHS—appeared eager to engage in research ethics training for quality reviews (93% in favor), only 53% believed that sufficient or advanced training opportunities exist.

One in-depth interviewee echoed this sentiment in relation to providing research ethics training for faculty members:

There is definite lack of formal, either in house or on the job, training for our faculty, as far as research ethics is concerned … we have been able to hold a number of seminars, trainings on research issues … but it has been generally on research methodologies and maybe how to write good scientific papers and things like that. But the formal training for faculty is what is lacking. (IDI Participant)

The perceived lack of opportunity for advanced IRB member training was likely exacerbated by demands external to their committee service. In-depth interviewees made suggestions for future training activities that could be implemented gradually, with refresher courses rather than long-term formal training:

You may once in a while have a five-day training program for the IRB members, because they need more detailed training and then you have refresher courses or update courses after that. So, for the IRB members, you can have a one-week, once in a while, but it may be a bit hard getting all of them to attend for the whole week. (IDI Participant)

External grants for ethics-related activities from the African Malaria Network Trust and the European and Developing Countries Clinical Trials Partnership had been received for IRB capacity development and from the National Institutes of Health for developing a formal training proposal for research ethics and responsible conduct of research (RCR) training. Pursuit of these grants also demonstrated a strong interest at MakCHS in ethics capacity development and success in securing peer-reviewed grant awards in the area of research ethics.

Domain 5: Capacity of Staff and Management to Carry Out Activities

A review of the institutional and IRB questionnaires demonstrated that MakCHS offers training for personnel hired in the Office of Research and the IRBs; interviews and surveys suggested that, generally, faculty and administrative staff members had prior ethics training. It was reported that staff members received brief training on processes and protocol management, usually from existing or outgoing staff members. One interviewee also encouraged a commitment to increasing the amount of refresher training opportunities, especially for newer staff, emphasizing that

Every day, we are counting new staff. And yes, they have some idea [about research ethics], but it doesn’t mean that they have got it all. I think learning is a continuous process. (IDI Participant)

Eleven of the 15 members indicated that MakCHS IRB members were often very busy with other daily tasks, making thorough review of research protocols difficult. To address this, in-depth interviewees made suggestions for future training activities that could be implemented concurrently:

By the time you have trained enough people for you to have the right members in the IRB … it will have taken us about two years. So initially I wanted a request for mandatory training, but I would expect that people on the IRB should be trained during their time on the IRB. That may be better and easier. (IDI Participant)

IRB members were required to undergo a minimum of a full-day in-person training course. This was supplemented by additional workshops, a seminar, and an online course. This induction training occurred for members as well as IRB administration:

The committee has a life cycle, so usually when the membership changes, we have an induction training. And that training includes general training in ethics and then specific training on the roles and functions of the IRB. (FGD Participant)

Sixty-six percent of IRB members agreed that they had adequate initial training in research ethics to provide reviews. An in-depth interview with a MakCHS leader suggested that IRB members become more confident throughout their service on the committee. However, IRB members felt that they did not have enough continued training in advanced research ethics, as one interviewee commented:

I think … when new members joined the committee there is always the induction training for them … but then we need in-person workshops [later]. (FGD Participant)

Many MakCHS staff participated in the Bioethics Working Group, which was a group of Ugandan bioethics professionals housed under the UNCST and facilitated by the Uganda Society for Health and Sciences. This group provided technical expertise and ethics training and MakCHS regularly invited them to participate in activities.

Domain 6: Systems and Finance

The assessment tool revealed that MakCHS allocated a portion of its funds to support the institutional Office of Research, but did not have funds specifically set aside for bioethics-related activities. This was in the context of generally low institutional funding and the fact that bioethics was subsumed within research at MakCHS. External grants for ethics-related activities (as mentioned earlier) had been received for IRB capacity development and for developing a formal training proposal for research ethics, demonstrating success in securing peer-reviewed grant awards in the area of research ethics.

For the IRBs, an operating budget was provided from fees collected for reviewing study protocols, but otherwise there was no operating budget. Seventy-three percent of IRB members felt that the financial support provided to the committee was insufficient. The fees for protocol reviews paid by researchers, which ranged from $100 for resubmissions and continuing reviews up to $400 for expedited reviews, provided some capital for the committee. As one IRB member explained: “So they pay … that money goes into a pool. And that pool is what facilitates the functioning of the secretariat” (FGD Participant).

Another in-depth interviewee clarified that funds generated by these protocols were not directly provided to the committee, but instead were channeled to the university for allocation: “You see, if you collect moneys of the IRB, that money belongs to the university. It is up to the university to decide” (FGD Participant). Still, shortages of funds affected facilitation of research ethics and research protocol follow-up. One IRB member commented in the questionnaire that monitoring of approved research proposals had been difficult, mainly due to the lack of funding.

None of the policy documents or plans specifically mentioned allocation of funds for teaching of research ethics (apart from the external grants mentioned earlier) or for the development of an ethics center. Interviewees generally discussed a lack of funding for bioethics-related activities.

Domain 7: Relationships With Target Groups and Stakeholders

As an institution, MakCHS had multiple formal external collaborations in research ethics with other organizations, including the African Malaria Network Trust, the European and Developing Countries Clinical Trials Partnership, the National Institutes of Health, the Fogarty African Bioethics Training Program, and the Medical Education Partnership Initiative. In addition, it had a relationship with the World Health Organization Regional Office for Africa. Further, according to an FGD with IRB members, the two IRB committees also interacted with each other, attended training sessions together, jointly reviewed abstracts, and organized an annual scientific conference. While the School of Public Health IRB did not explicitly report any formal direct relationships with outside organizations, it was clear from IDIs and FGDs that the committee did have, at least, informal connections with the UNCST.

Domain 8: Relationship of Institution to its Environment

An examination of the environment surrounding MakCHS illustrated how advanced the institution was in its aims to encourage research ethics training. In Uganda, there were national regulations that addressed research with human subjects as well as national laws regarding the ethical conduct of research. Adherence to these regulations was mandatory, and the penalty for noncompliance was a reprimand from the UNCST, which had the ability to discontinue research projects permanently or until noncompliance was addressed. Though there were national regulations that recommended scientific review, ethics review, research coordination, and creation of policy guidelines, institutional IRBs were largely required to enforce these on an institutional basis. An IRB member explained the balance between national regulations and subsequent institutional enforcement:

There are policies that the IRBs have been provided by the National Council of Science and Technology. So we do have national guidelines and standards, that’s the basis. Thereafter, the IRB is required to have its own institutional policies or standard operating procedures. (FGD Participant)

Discussions with FGD participants and interviewees revealed that MakCHS collaborated closely with the Bioethics Working Group (mentioned earlier), which provided technical expertise, on-site monitoring of studies, ethics training, and curriculum development. The working group was regularly contracted by the government to provide oversight for research studies. FGD participants reported that the Bioethics Working Group generally carried out two compliance visits to separate sites per month, with a group of three or four individuals visiting each site. Sometimes these visits were based on suspected noncompliance (e.g., when sites neglected to complete required reporting) and at other times were simply for monitoring purposes. FGD participants noted that UNCST and Uganda Society for Health Scientists had a memorandum of understanding under which both groups were expected to jointly fund such site visits (e.g., transportation and per-diems).

Multiple members of the working group were also MakCHS faculty and staff members and offered training and courses at MakCHS. In-depth interviews highlighted perceptions of the university as an important provider of ethics training even beyond the national level:

So we are very key in the region, so we feel if we start that program [graduate program in bioethics] plus other shorter courses, we think we will help to promote ethics in the region. (IDI Participant)

INTERNAL AND EXTERNAL OCTAGON ASSESSMENT

After analyzing the data from all five sources, the MakCHS team and FABTP team separately generated a numerical score between 1 (lowest) and 7 (highest) assessing MakCHS research ethics capacity in each of the domains (Figure 1). Figure 1a suggests that the MakCHS team perceived that it retained a very strong institutional identity, clearly documented job descriptions, and hired personnel that possessed appropriate skills. Further, it saw implementation of activities and its position within its environment as relatively robust. By its own assessment, MakCHS leadership identified more room for improvement in the areas of funding for research ethics advancement, relationships with target groups, and ensuring that activities clearly correspond to the institutional vision and goals. Similarly, the FABTP team perceived that MakCHS had strength in the areas of identity, structure, relevance, target groups, and working environment, and had greater need in the areas of systems of finance and administration, competence, and implementation of activities.

Figure 1.

(a) Makerere University College of Health Sciences self-assessment. (b) FABTP assessment of Makerere University College of Health Sciences.

The external and internal institutional assessments of MakCHS were similar in identifying which areas were stronger at baseline and which areas would require the greatest focus of additional work. The two assessments differed, however, in the raw scores for each domain. For example, MakCHS gave itself a score of 7, the highest, for two of the domains; the lowest score on its self-assessment was a 3 for the financing and target group domains. In contrast, the external FABTP evaluation awarded scores in the range of 3 to 5, with only financing receiving the score of 3 (Figure 1). These differences indicated that MakCHS perceived greater institutional research ethics capacity for itself at the baseline of this particular partnership than the external counterpart FABTP team in a majority of domains.

What was critical about these assessments, and what one might argue was a virtue of the tool when used in this way, was that such differences provide an opportunity for discussion. Assessments are by definition subjective, based on likely differing views about how much capacity warrants particular scores. The benefit of the tool was not in the absolute scores, but in recognizing where there were relative strengths versus challenges, and how differences in perceptions by partners or stakeholders could be relevant for guiding future capacity development.

DISCUSSION

This article describes an operational approach to identifying areas of strength and need in the context of developing research ethics capacity in LMICs, using the case study of conducting a baseline assessment at MakCHS. We emphasize that these findings exist in the resource-constrained context of Uganda, a low-income country that ranks 120th out of 144 countries in terms of the inequality-adjusted Human Development Index (United Nations Development Programme 2014).

The approach taken focused on the institution and the environment in which it is situated, and is informed by the views of leadership as well as of internal stakeholders with the goal of identifying elements that could help to guide the focus of future capacity development efforts. It is important to recognize key characteristics of the way this process operated: It occurred within the context of a longer term partnership for capacity development; it was intended to be time-sensitive; and it was intended largely to serve the internal needs of the institution and partnership, rather than to serve in any way as an “objective” performance assessment of the institution’s capacity. Given its expressly subjective nature, it was inherently a deliberative tool that encouraged greater dialogue and helped inform plans for capacity development. As such, other institutions may also find this tool, and how we have adapted it, to be of value to them both to benchmark where they are regarding research ethics capacity and to reflect on capacity development over time.

This approach to an institutional assessment of research ethics capacity revealed a shared view that MakCHS in Uganda has a strong institutional identity with aspirations to be a research-centered college by 2019. As previously noted, this case study was carried out soon after the establishment of the college to encompass the four new schools; thus, it was conducted during a time of transition, exploration, and the creation of administrative structures and college-wide policies (Pariyo et al. 2011). It also was deliberately conducted at the beginning of a Fogarty-funded research ethics partnership and served as a baseline anchor against which to target and measure future capacity development efforts.

MakCHS’s published strategic plan for the college outlines its goals of building a supportive research environment, and some faculty and staff members discussed institutional plans around research ethics in interviews and discussions; however, the written strategic plan contained no specific mention of the advancement in research ethics. The lack of institutional policies framing the growth of research ethics was likely a reflection of the early stage of MakCHS’s structured planning in this arena, and indeed, why it wanted to enter into a partnership focusing expressly on research ethics.

In the period since this baseline assessment was conducted, MakCHS not only has established an institutional research leadership position, but also has drafted a strategic plan for bioethics itself. Additional efforts that are responsive to the eight Octagon domains can help guide the various components that ultimately need to be in place in any institution to have a solid, sustainable, and high-quality program of research ethics. These range from generating funding (e.g., MakCHS has successfully obtained a grant for further ethics capacity development) and development of institutional curricula for research ethics, to requiring training for all active researchers and collaborating with national authorities (e.g., the National Council of Science and Technology, which oversees regulatory approval for Uganda).

Limitations

While we believe that the Octagon model was a useful assessment tool, and helped identify the areas on which a baseline assessment should focus, this case study also revealed important limitations. While the assessment tool systematically highlighted broad domains of interest and offered some specific indictors, employing methods that can capture all possible elements affecting institutional capacity for research ethics was difficult. For example, while our methods gathered information about appropriate job descriptions and assessing whether employees had the right skill set, questions remain about institutional efficiency and use of these resources. In a similar vein, the domain scores might not fully capture issues around the role of internal target groups, the local academic environment, internal power dynamics, transparency, and the level of institutional autonomy: all factors that are difficult to measure and that might affect decisions and perceptions considerably.

There was also a tendency for interviews to focus heavily on issues pertaining to the IRBs, which was perhaps unsurprising as IRBs are often one of the most tangible components of institutional research ethics capacity, and one that often has urgency (e.g., in relation to securing grant funds), while training may not.

While we completed interviews with many MakCHS stakeholders, it was possible that important viewpoints of stakeholders who did not complete interviews were not represented. Additionally, it should be noted that colleges are not independent entities within MakCHS, and this assessment did not include the perspective of the overall university-wide Directorate of Research and Graduate Studies. This directorate housed all research activities in the university and was led by a director and a deputy director for research and compliance to ethical standards.

CONCLUSION

The Octagon approach provided a useful tool for, and a structured way to approach, conducting a baseline assessment of research capacity. It served well the needs we had when establishing a partnership between Johns Hopkins and MakCHS. The approach to assessing research ethics capacity received a favorable response from MakCHS, where the institution was eager to collaborate with the FABTP team. The systematic approach to an institutional evaluation of research ethics capacity using mixed methods has informed the FABTP–MakCHS partnership in meaningful ways and provided a road map for strong collaboration. It has led to more dialogue within the institution about building research ethics capacity moving forward and partially served as the impetus for MakCHS to join the African Bioethics Consortium, alongside the universities of Zambia and Botswana; this consortium is devoted to strengthening research ethics capacity regionally and across the continent.

We believe this process and the associated tools can provide insights into research ethics assessments in LMIC institutions, particularly in contexts with limited institutional data. We look forward to additional applications of this approach in Africa and beyond.