Ethics and regulatory complexities posed by a pragmatic clinical trial: a case study from Lilongwe, Malawi

Abstract

Background

Pragmatic clinical trials generally rely on real world data and have the potential to generate real world evidence. This approach arose from concerns that many trial results did not adequately inform real world practice. However, maintaining the real world setting during the conduct of a trial and ensuring adequate protection for research participants can be challenging. Best practices in research oversight for pragmatic clinical trials are nascent and underdeveloped, especially in developing countries.

Methods

We use the PRECIS-2 tool to present a case study from Lilongwe in Malawi to describe ethical and regulatory challenges encountered during the conduct of a pragmatic trial and suggest possible solutions.

Results

In this article, we highlight the following six issues: (1) one public facility hosting several pragmatic trials within the same period; (2) research participants refusing financial incentives; (3) inadequate infrastructure and high workload to conduct research; (4) silos among partner organisations involved in delivery of health care; (5) individuals influencing the implementation of revised national guidelines; (6) difficulties with access to electronic medical records.

Conclusion

Multiple stakeholder engagement is critical to the conduct of pragmatic trials, and even with careful stakeholder engagement, continuous monitoring by gatekeepers is essential. In the Malawian context, active engagement of the district research committees can complement the work of the research ethics committees (RECs).

Introduction

There is an increasing recognition that results from conventional clinical trials done in specialised, highly controlled research settings may not be uniformly generalisable to real world practice¹. Pragmatic clinical trials have the potential to generate real world evidence. While interest in pragmatic trials has increased dramatically², the differences between explanatory and pragmatic approaches in clinical trials were already first highlighted five decades ago³. Schwartz and Lellouch state that the objective of an explanatory trial is understanding: to discover the efficacy of an intervention in ideal circumstances. By contrast, the objective of a pragmatic trial is decision-making: to evaluate the effectiveness of an intervention in usual clinical conditions³. The availability of funding, the infrastructure for streamlined data collection including electronic medical records (EMR) and the development of innovative clinical trial designs contribute to the increasing use of pragmatic trials⁴.

Despite the increase, there is a growing consensus that the current international guidelines for ethical conduct of research were written with explanatory trials in mind rather than pragmatic trials^5–7. Authors claim that there are four aspects where conflicts arise when applying the existing ethical and regulatory frameworks⁴. First, researchers and members of research ethics committees (RECs) fail to distinguish research from practice in order to demarcate the activities that must undergo ethical review for the protection of research participants. Second, it is not clear from the regulations how best to determine the need for obtaining consent for pragmatic trials. Third, if consent is needed, what must be disclosed in the consent process in pragmatic trials is hard to determine. Fourth, it is not clear how to best conduct appropriate research oversight. Although there are growing efforts to provide guidance^7,8, the discussion is currently dominated by those working in Canada and United States^4,9. The dominant discourse, therefore, may focus on ethical considerations more relevant to developed countries than developing ones.

However, the use of pragmatic trials, including innovative clinical trial designs, is increasing in developing countries and is generating debate about what form an appropriate research oversight system should take in this context^10–12.

For example, there is an ongoing debate on whether the RTS, S/AS01 Malaria Vaccine implementation (MVIP) in Malawi constitutes a pragmatic clinical trial or not, and consequently whether and how to demarcate the activities that must undergo ethical review for the protection of research participants. The government of Malawi through the Ministry of Health and Population (MoHP) and its partners are implementing the new malaria vaccine using a cluster randomised design to inform decision making¹³. The implementors have obtained ethical approvals from relevant RECs and have registered it as an observational trial on ClinicalTrials.gov (NCT03806465). Only the programme evaluation component in the implementation is subjected to research oversight, while the intervention component i.e. the RTS, S/AS01 Malaria vaccine, is not. Due to this arrangement, some experts have questioned the MVIP, labelling it “... a serious breach to research standards, and a violation to the Ottawa statement and the Council for International Organizations of Medical Sciences (CIOMS) guidelines”^14–16. In contrast, the sponsors of MVIP have indicated that “this is a pilot introduction of a new vaccine and not a research activity”¹⁷. However, an in-depth analysis by van der Graaf et al. (2020), concluded that the MVIP has a substantial research component, and that it is prudent to apply ethical norms for research involving humans, such as the CIOMS guidelines. In addition, they stated that the ethical requirements of informed consent and independent ethical review have not been met. Meanwhile, the Malawi research guidelines appear to provide guidance for explanatory trials rather than pragmatic trials, including the issues of waiver of consent¹⁸, and the guidelines are silent on programme evaluation.

In light of the ethical and regulatory complexities posed by pragmatic clinical trials, this paper aims to present a case study of the issues encountered during the implementation of a trial titled “Developing and Assesing a Male Engagement Intervention for Option B+ in Malawi: A Randomized Controlled Trial in Lilongwe,” otherwise known as Timasamalirana study. We focus on issues important for consideration in future trials. Futhermore, to systematically discuss the issues, we have used the Pragmatic-Explanatory Continuum Indicator Summary 2 (PRECIS-2) tool which was developed to help trialists work through their study designs¹⁹. Following Nichols et al., the PRECIS-2 tool can also be used to conduct an ethical analysis⁹. We hope that such a case study will not only help REC members and pragmatic trialists, but will also raise awareness among health care system leaders, front-line health workers and partner organisations involved in health care delivery. In this case study analysis, note that some authors were involved in the design and implementation of the Timasamalirana trial.

The Timasamalirana Study

Couple-based approaches, especially couple HIV testing and counselling, have been encouraged by the World Health Organisation, but two important implementation questions remained: 1) assessing the strategies used for recruiting couples, and 2) understanding the impacts of couple-based strategies on HIV treatment and treatment as prevention. For these reasons, Timasamalirana study was developed to addresses the two issues. The study was conducted in Lilongwe, Malawi. It enrolled newly diagnosed HIV-positive women who were eligible for Option B+ and had a male partner in Bwaila Hospital's catchment area. Timasamalirana is a Chichewa word meaning “We care for one another.” The study started in 2018, was completed in 2020, and managed to enrol 500 women, randomized to a standard of care (SOC) arm (N=250) versus a couple-based intervention arm (N=250). All participants were followed for one year, with follow-up visits occurring 6- and 12-months after enrolment. Women in the intervention arm were given an invitation card for their male partners to present to the clinic for important pregnancy information. For those who did not present to the clinic, a trained community worker contacted the partner by phone and/or conduct a community visit to encourage the male partner to present to the clinic. When a couple presented to the clinic, a trained counsellor provided them with important pregnancy information.

They were also given the opportunity to receive couple HIV counselling and testing at this initial visit and additional couple counselling at six and twelve months.

Women in the SOC arm did not receive any couple-based intervention procedures until their exit visit. On their exit visit, viral loads were checked. According to the findings presented at the 2021 AIDS conference, the findings show viral suppression was 81.5% in the SOC arm and 88.0% in the couple-based intervention arm (RD 6.6, CI:-0.8, 14.0) (p=0.08). In addition, greater viral suppression was observed among women who were more or equal to 25 years old, married or cohabiting, in a relationship of more or equal to 1 year, and who did not report recent history of partner violence.

The Pragmatic-Explanatory Continuum Indicator Summary 2 (PRECIS-2)

Loudon, Treweek, Sullivan, Donnan, Thorpe and Zwarenstein(19) describe the PRECIS-2 tool as a framework to guide study teams to prospectively consider the pragmatic or explanatory nature of their trial across nine domains (Fig. 1). Each domain has a scale of 1 to 5 to score a clinical trial as either more pragmatic or more explanatory. The nine domains are: Eligibility; Recruitment; Setting; Organisation; Flexibility-delivery; Flexibility-adherance; Follow up; Primary outcome; Primary analysis. The nine domains were developed after extensive review and consensus reached by stakeholders experienced in clinical trials.

Fig 1.

PRECIS-2 tool(20)

The Timasamalirana study is more pragmatic because of the following; (1) It had broad recruitment criteria, because the aim was to mimic the population likely to receive the intervention in usual clinical care setting, that is, ‘new HIV positive women with partners within Bwaila hospital catchment area.’ (2) Recruitment of research participants came from patients who came to seek routine clinical care at the facility; (3)The study activities were embedded into usual clinical care conditions; (4) Follow-up of participants was aligned to routine ART clinic visits and data collection used the routine ART programme monitoring and evaluation tools.

Using the PRESIC-2 TOOL to describe the ethical and regulatory complexities

1. Eligibility

According to the PRECIS-2 tool, a highly pragmatic trial is expected to include anyone with the condition of interest who is likely to be a candidate for the intervention if it was being provided in usual care²¹. In the case of Timasamalirana study, Option B+ clients who had male partners within the Bwaila catchment area were eligible to join the study. Option B+ is a treatment that recommends lifelong ART for all pregnant and breastfeeding women regardless of their CD4 count level or World Health Organisation (WHO) clinical stage and 6 weeks of daily nevirapine for the infant. The host facility also cares for many patients living with HIV and has extensive experience in conducting HIV trials in Lilongwe. Yet, despite this broad inclusion cliteria for Option B+ client and the available infrastructure, at some point the Timasamalirana study team were unable to invite all eligible patients. This was due to the presence of new pragmatic trials which had started a few months later after the Timasamalirana study and were recruiting the same study population at the facility hosting the research. In other words, there was local competition to enrol eligible participants. Additionally, there was one particular trial whose intervention resembled Timasamalirana study and their sponsor recommended to recruit patients from this same host facility. Later, the research teams collaborated and devised a plan to avoid conflicts arising from the competition. The research teams agreed that Timasamalarana would be given priority to recruit participants. Some participants were also approached to co-enrol in these studies.

2. Recruitment

One of the key features of pragmatic trials is that they try to preserve usual care clinic conditions in their trials, such as recruiting study participants when they are patients presenting for routine care. To help maintain the usual clinic conditions, the PRECIS-2 tool suggests that incentives or compensation given to research participants e.g. cash payment, vouchers, or travel costs may be better considered as part of the established^19,21. In contrast, the financial established in Timasamalirana study was provided to both the SOC and intervention arms as stipulated by the Malawian REC that reviewed the study protocol. On November 1, 2017, the Malawi government, through the National Health Sciences Research Committee (NHSRC), directed that all human subjects research should provide study participants with US$10 per study visit as compensation for costs. The Malawi research guidance advocates reimbursement for direct costs such as transport, modest meals or communication and also considers time spent²². Timasamalirana reseachers understood that in order to promote fairness, participants in both study arms should receive the compensation. Usually patients spend the whole day at this host facility and it is a high volume facility. However, like many pragmatic trials, the study visits were aligned with routine clinical care visits and this led some participants to wonder why they were entitled to receive financial incentives from the hospital. Consequently, some participants refused to receive the stipend and in certain cases, the stipend was returned on the following day after conducting a study visit.

3. Setting

The PRECIS-2 tool encourages trialists to explicitly consider matching the setting of their trial with the setting where their results are likely to be applied^19,21 otherwise, the relevance of the trial can be reduced. The Timasamalirana study was meant to be embedded into the usual care for antiretroviral (ART) clinic in the antenatal care setting, but the facility infrastructure lacked adequate space to conduct study-related activities. In response to this challenge, a temporary structure was erected close to the ART clinic for additional space. Although the study visits were aligned to routine clinical care visits, major study activities such as consenting and providing the intervention were done in the new structure. Additionally, the host facility is located in the urban setting where migration is high and this adversely affects male engagement interventions. Generally, most men in urban settings prefer to conduct income generating activities rather than spending their time at the antenatal clinic. In addition, the Malawian ART data indicates that retention of patients in high volume urban settings is lower compared to rural settings^23,24. Consequently, conducting the trial in an urban setting only reduced the ability to generalise the findings and also reduced the social value of the study.

4. Organisation

According to the PRECIS-2 tool, a highly pragmatic design would aim to slot the intervention into the usual organisation of care for the condition of interest (e.g HIV/ART clinic), making use of no more than the existing healthcare staff and resources in that setting²⁵. In contrast, in the Timasamalirana study, the existing staff at the host facility were overwhelmed due to high workload. The only way to appropriately implement the study was to recruit additional healthcare staff to support the existing workers to deliver usual clinical care services plus research activities. In some special cases, the host facility staff were willing to provide services to Timasamalirana study and other pragmatic trials when financial incentives were provided. Furthermore, considering that the host facility had multiple stakeholders, there was need to engage all partner organisations, but this was challenging because some organisations did not allow their employees to receive financial incentives from any other organisations for providing care services at the host facility. For that reason, there was resistance to cooperate with some of the staff from such organisations. But authors such as Darsaut and Raymond claim that it is a moral duty for health workers to design and participate in trials that are primarily conceived in the patient's best medical interests²⁶. According to these authors, it is the duty of the health care taker to embrace health system research as a responsible means of responding to uncertainties. Such uncertainties should be researched with appropriate study designs while providing the health care worker.

5. Flexibility-delivery

According to the PRECIS-2 tool, the most pragmatic design approach would leave the details of how to implement the intervention up to providers. Thus, the methodology of how to deliver an intervention is not rigidly prescriptive in the protocol¹⁹. Similarly, the Timasamalirana study recruited clients eligible for Option B+. At the beginning of the trial the usual care ART regimen included a combination of Tenofovir, Lamivudine and Efavirenz but in the course of the trial, the usual care regimen was changed to dolutegravir (DTG)²⁷. A blanket transition to DTG-based regimens for patient groups was planned for January 2019. Surprisingly, although there was availability of trained health care workers and stocks of DTG regimen, the implementation at the host facility was delayed up until around May, 2019. In efforts to follow up with host facility on the causes of the delay, it was found that the team leader was surprised that health providers were not implementing the DTG based regimen, while the health providers argued that they did not know that they could start the implementation. Also, there were unconfirmed reports indicating that the delay was influenced by some researchers whose study objectives would become irrelevant if they were to implement the newly revised treatment guidelines. Despite the delay, changes were effected and Timasamalirana research participants were switched to the new treatment regimen at different time points in the study. The Timasamalirana workers provided counselling in the intervention arm which was tailored towards the use of new DTG-based regimen without necessarily revising the intervention manual or the study protocol. The REC was informed of this change.

6. Primary outcome

PRECIS-2 tool recommends that the choice of primary outcome should include an outcome that is of obvious importance from the patient's perspective. Such outcomes are usually measured in a way that is similar to the way they are measured in usual care¹⁹. One of the outcomes of interest in Timasamalirana study was ‘6 months viral load’ whose data is collected using routine electronic data collection systems. This data is of interest to patients, who often want to know if their viral load is decreasing because they feel confident that the ART is working for them.

Permission to access routine data, i.e. the viral load, was obtained from the gatekeeper, the MoHP through the Lilongwe District Health Office (DHO). In contrast, there was no prior arrangement with the partner organisation that supports MoHP in the implementation of EMR. Despite the DHO's permission, the partner organisation requested a signed agreement on data sharing which included issues of authorship in case of publication. Additionally, access to EMR data posed challenges because of the nature of the computer programming involved as it required rare specialised skills to extract important data elements.

Discussion

Of the nine domains of the PRECIS-2 tool, we have described the challenges encountered using six domains only; adherence flexibility, follow up and primary analysis did not raise complexities for the selected case study.The issues being presented were observed publicly at facility level . The case study analysis approach does not include views from stakeholders. It is possible that social science research accompanying pragmatic trials, including qualitative research with participants and providers, could reveal more ethical challenges in these domains.

Thus, the challenges identified can be summarised into six aspects; (1) one public health facility hosting multiple pragmatic trials within the same period; (2) research participants refusing financial compensation; (3) inadequate infrastructure and high workload to conduct research; (4) silos among partner organisations involved in deliverly of health care; (5) individuals influencing the implementation of revised national guidelines; (6) difficulties with access to electronic medical records.

It is important to note that issues such as EMR data access challenges, infrastructure challenges and silos that exist both within healthcare and between healthcare delivery are also highlighted in a report by the British Columbia Academic Health Science Network (BC AHSN)²⁸. Thus, the problems being raised are not unique to developing country settings only but, perhaps, the challenges are a matter of degree. The BC AHSN recommendations include the need for extensive multiple stakeholder engagement²⁸. In the case of Timasamalirana study, some stakeholder engagement was done, though it was probably inadequate. Selected partner oganisations at the host facility were oriented to the study. It should also be pointed out that meaningful stakeholder engagement requires funding but the researchers had limited budgets to devote to this activity.

Furthermore, the infrastructure challenges will continue to affect the conduct of pragmatic trials and the utility of their outcomes. It will be difficult to demonstrate the mechanisms of how the intervention can be implemented if it were to be rolled out in real practice. In future trials, we suggest that the newly stablished district research committees in Malawi²⁹, as stipulated in the 2019 National Health Research Policy, be engaged from research development to implementation so they could actively provide insights and influence decision making in an effort to support these trials.

Similarly, the district research committees may need to keep track of pragmatic trials taking place in public health facilities to appreciate the landscape/ecology in the health facilities. Normally, International Conference on Hammonisation (ICH) good clinical practice guidelines(GCP) discourage co-enrolment, but according to the PRECIS-2 tool, it is permissible in pragmatic trials²¹. The aim is to preserve the usual clinical care environment such that even if the facility is hosting several trials, participants can enrol in all the studies if eligible. In Timasamalirana's case, the conflicts arising from the competitive enrolment was resolved, but note that the solution used may not always work. This, therefore, calls for members of the REC and researchers to be alerted that not all GCP rules apply to pragmatic trials. Additionally, considering the case where several trials are recruiting the same study population or have similarities in their interventions at the same host facility, issues related to compromised scientific validity are bound to rise. On one hand, a suggestion to address this challenge could be that the RECs may need to have knowledge of the research landscape/ecology of the public facility and think about how the conduct of one trial may impact other ongoing studies in ways that relate to ethics. On the other hand, doing so is fairly controversial, since this would mean judging a study on something other than its own merits, to some extent. Nevertheless, ensuring scientific validity is an important ethical principle in ethics review of research protocols³⁰.

As suggested earlier, the district research committees will need to continue monitoring the pragmatic trials periodically. Their active role has the potential to complement RECs and reduce manipulation by other reseachers whose study objectives may be irrelevant if they are to implement the newly revised treatment guidelines. Practically, Malawian RECs may not yet be able to monitor such important details, yet it is unethical to withhold the implementation of newly revised national guidelines to avoid certain pragmatic research becoming irrelevant. Thus, the district research committees may be able to close the gap.

Regarding the issue of financial compensation to be given to Malawian research participants, this is an ongoing debate³¹, and further efforts to provide guidance for trials have been suggested by Goldon et al.³². However, the research regulators are yet to adopt the recommendations. In addition, the recommendations by Goldon et al. appear to favour explanatory trials. The stipend which were returned by particiants in Timasamalirana trial may suggest that there were some misconceptions surrounding the money given. On a different note, misconceptions aside, it can be an important message to researchers and members of RECs that the patients felt the study procedures are aligned to study visits so there was no justification to receive a compensation. Moving forward, there is need for guidance on the issue of compensation in pragmatic research. In a similar case, anecdotal evidence from mental health pragmatic trial in Balaka district revealed that reseachers encountered challenges in recruiting/retaining participants because they were receiving financial incentives, which was refered to as satanic. Regardless of the education during consenting process, it appears that participants think there is a hidden (possibly nefarious) agenda behind the ‘free’ money offer during routine clinic visit. Additionally, a recent published pragmatic trial from Malawi by Choko, Corbett, Stallard, Maheswaran, Lepine and Johnson et al. included financial incentives as an intervention in one of its trial arms involving HIV self-testing kits and, surprisingly, the findings show that the standard of care performed much better than the intervention arm with financial incentive³³. It could be that the research participant financial compensation in highly specialised research settings does not raise questions compared to the routine clinical care setting. The participants who refuse financial reimbursement ought to be handled sensitively to maintain relationships of trust between the community and researchers.

Lastly, the ideal way to enhance the social value or the relevance of Timasamalirana study was to assess the effectiveness of the intervention in both rural and urban public facilities. Over 80% of the Malawian population live in rural settings yet this trial was exclusively conducted in an urban setting. Thus the application of the results cannot become uniformly generalisable. There were also no prior arrangements to cover what would happen if the intervention was proven successful in the urban settings. Such plans can enhance the social value of such trials. Meaningful stakeholder engagement is also critical to promoting the implementation of successful interventions.

Conclusion

We have raised six issues from this case study, and have suggested how research oversight could be conducted to promote acceptable ethical conduct of pragmatic trials. Some issues identified are similar to developed countries. However, unique issues in the Malawian setting include the host facility conducting several pragmatic trials within the same period, and that some participants refuse reimbursement/compensation. Researchers interested in pragmatic trials and regulators in Malawi may need to work together to promote best practices for the appropriate design, review and conduct of pragmatic clinical trials. To complement the work of Malawian RECs, we also recommend the engagement of the newly established district research committees in the conduct of pragmatic trials. Extensive multiple stakeholder engagement is crucial in pragmatic trials. Future work should examine other pragmatic trials, including programme evaluation conducted in Malawi, to identify if the same ethical issues (or new issues) are raised.

Table 1.

A summary of key ethical/regulatory issues and future considerations

Elements of pragmatisim	Ethical/regulatory issues	Considerations
Eligibility	One public health facility hosting multiple pragmatic trials and studying same study population. Some participants are asked to co-enrol in trials, which could overburden the patients and compromise scientific validity.	District research committees may need to be vigilant in assessing the presence of research activities in public health facilities before approving additional trials.
Recruitment	Compensation in form of money given to research participants is ethical, but patients recruitment in pragmatic trials differs with explanatory trials. The Malawi research guideline compensation indicates that 10 US dollars must be given to each research participant. In some cases, payment may not be appropriate in pragmatic trials.	Malawian RECs need to develop appropriate compensation model for the pragmatic trials
Setting	Partner organisations involved in patient care delivery in public facilities need to cooperate with reseach teams. Conducting research may not be their primary function but their cooperation with the research teams is critical. Urban and rural health facilities are different. Conducting the pragmatic trial only in an urban setting has the potential to reduce the social value of the study.	RECs may need to review stakeholder engagement plan before approving these trials. Research teams need to be clear on the reason they are conducting a pragmatic trial and justify the selection of host facility. The stakeholders involved may find research findings useful and roll out effective interventions even when research teams do not have funds to support post trial access of proven interventions.
Flexibility	Delaying the implemention of new treatment guidelines in an effort to maintain the relevance of a research question is inappropriate.	District research committees may need to have delegated powers from RECs to monitor the implementation of these trials.
Primary outcome	The difficulties to access viral load data in EMR is related to the nature of the computer programming, very few individuals understand it. In addition, due to lack of national guidance on research data sharing, it is not easy to reach a consensus especially on issues of authorhip.	Research teams to ensure that the roles of all stakeholders and partnerships are clearly spelt before the start of the study. REC may need to review stakeholder engagement plans and roles for each partner organisation.

Funding

The work was funded by the NIH Forgarty: D43TW01511- 01-Advancing Reasearch Ethics in Southern Africa (ARESA): Leadership Program.