Open-source mobile digital platform for clinical trial data collection in low-resource settings

Abstract

Background

Governments, universities and pan-African research networks are building durable infrastructure and capabilities for biomedical research in Africa. This offers the opportunity to adopt from the outset innovative approaches and technologies that would be challenging to retrofit into fully established research infrastructures such as those regularly found in high-income countries. In this context we piloted the use of a novel mobile digital health platform, designed specifically for low-resource environments, to support high-quality data collection in a clinical research study.

Objective

Our primary aim was to assess the feasibility of a using a mobile digital platform for clinical trial data collection in a low-resource setting. Secondarily, we sought to explore the potential benefits of such an approach.

Methods

The investigative site was a research institute in Nairobi, Kenya. We integrated an open-source platform for mobile data collection commonly used in the developing world with an open-source, standard platform for electronic data capture in clinical trials. The integration was developed using common data standards (Clinical Data Interchange Standards Consortium (CDISC) Operational Data Model), maximising the potential to extend the approach to other platforms. The system was deployed in a pharmacokinetic study involving healthy human volunteers.

Results

The electronic data collection platform successfully supported conduct of the study. Multidisciplinary users reported high levels of satisfaction with the mobile application and highlighted substantial advantages when compared with traditional paper record systems. The new system also demonstrated a potential for expediting data quality review.

Discussion and Conclusions

This pilot study demonstrated the feasibility of using a mobile digital platform for clinical research data collection in low-resource settings. Sustainable scientific capabilities and infrastructure are essential to attract and support clinical research studies. Since many research structures in Africa are being developed anew, stakeholders should consider implementing innovative technologies and approaches.

Introduction

Sub-Saharan Africa is home to roughly a quarter of the global disease burden,^{1 2} but the number of clinical research studies conducted in the region has always been disproportionally low.³ However, the situation is evolving. Compelling ethical, scientific, pragmatic and economic rationales for conducting clinical research in Africa are crystallising, at the same time there is convergence of substantial enabling influences.^4–6 In recent years there has been notable progress in local regulatory guidance^{4 5} for clinical trials in Africa, increased commitments by several countries to improve their research infrastructure,⁷ and strengthened partnerships between large pan-African research networks and well-funded international research-focused consortia.^8–10 These developments have contributed to increasingly sophisticated scientific capabilities at many African universities, hospitals and research institutes.

Establishment of new infrastructure and research capabilities in Africa presents a unique opportunity to institute highly efficient clinical research systems from the outset. Lessons can be learnt from the experiences in high-income countries where research systems are well established but often complex and in some instances inefficient.¹¹ It may be possible for countries in Africa to implement leapfrog approaches¹² that exploit the advantages of digital and mobile technologies, tools which have permeated and revolutionised many aspects of work and life around the world but have not yet been widely adopted for use in clinical trials.^13–15

We explored the use of a novel platform to support the strictly controlled process of data collection in clinical trials. Specifically, we integrated an EDC system that is broadly used with a mobile data collection system that has been extensively used in healthcare programmes in low-resource settings.^16–23 With this approach data needs to be entered only once, at the point of care on a mobile device, from where it automatically flows into the EDC system. We hypothesised that this process would decrease the time-consuming and resource-consuming overhead of multiple manual transcriptions of data, and thereby also mitigate an important source of potential data errors. The integration is based on the use of commonly available data standards, to allow for extension of the approach to other data collection platforms and EDC systems.

The new digital platform was deployed in a phase-I, healthy volunteer, pharmacokinetic study that was conducted at the Centre for Research in Therapeutic Sciences (CREATES) at Strathmore University in Nairobi, Kenya. We assessed the feasibility of the approach and explored the potential benefits as they relate to efficiency and data quality.

Approach

The typical approach to data collection in clinical trials is illustrated in figure 1. In a first step, clinic staff enters patient data into a paper-based medical record. Next, data is transcribed from the clinical record into a paper case report form (CRF), which is developed specifically for the clinical research study. Not all data from the medical record are transcribed to the CRF—rather, only the subset that is required by the clinical study protocol. The CRF is normally reviewed for accuracy and completeness by quality control (QC) staff at the clinic. Once the CRF is reviewed, the final step involves transcription of CRF data into the EDC system for further review and eventual analysis.

Figure 1.

Data collection in clinical trials. The typical approach involves transcription from paper records to electronic forms (top). In the new model, a mobile digital platform is employed to improve efficiency and quality of data collection (bottom). CRF, case report form; EDC, electronic data capture.

The process of transcribing data multiple times imposes considerable burden on clinical trial operations. Moreover, in some clinics the data entry staff have limited clinical backgrounds and may be unfamiliar with the data they are transcribing; this can predispose to additional data error. Also, in an attempt to minimise disruption to clinical activities, medical records and CRFs are often assembled over a period of time before data is transcribed into the EDC system, which generates delays in availability of data in the EDC system and may cause difficulties with eventual reconciliation of potentially discrepant results. Correction of data errors in the EDC system requires consultation, review and approval by the person who was responsible for originally entering the data in the patient's medical record. Therefore, a recognised best practice is to implement measures to detect and correct data errors in real time, at the point of care when data is being entered for the first time; this process is referred to as the use of ‘edit checks’.

Data collection using the mobile platform differs from traditional methods in several important ways (figure 1). In the new approach, clinic staff enters data in on a mobile device at the point of care. QC staff then logs into an online platform and reviews data for accuracy and completeness. An extract of the data is created and uploaded into the EDC system. Only data fields that are part of the study's CRF are included in the data export; other data fields are securely stored in the mobile platform but not exported. As a final step, the study investigator logs in to the EDC system to validate and sign off on the uploaded data.

Mobile platform

The mobile platform used in this study was supported by the CommCare platform,²⁴ an open source, cloud-based platform for health programmes that supports the development of data capture tools using mobile devices. All access to the CommCare platform is achieved through Hypertext Transfer Protocol Secure (HTTPS) and is cryptographically secure. Data stored in the platform's database is confidential and password-protected. The CommCare system was selected for this study in part because of prior experience with the use of the platform in other research settings.¹⁷ The Samsung Galaxy Tab 4 was used as the mobile device in this study since it was available locally, though the application is capable of running on any modern mobile device that runs Android.

Two members from the CREATES team (KOO and BM) completed online training in CommCare, and received additional in-person training from the Dimagi team on development of the user-interface application (‘App’). The CREATES and Dimagi team developed the data collection App on the CommCare platform, based on the definition of the CRF for the study. This App was then tested and adapted in 2-day field testing sessions. During these sessions, the administrators, nurses, doctors, laboratory technicians and QC staff at CREATES used the App in a mock trial setting, and their usage of the tool was observed. From these observations, iterative refinements to the App were incorporated. Additional data fields were added to the App, reflecting data fields that are part of the medical record but not of the CRF. The sequence of a number of data fields in the App was adjusted, and the CRF pages were grouped into different subforms, to closely fit the local operating procedures at the clinic. Further improvements were made to the App by the implementation of ‘edit checks’ in order to check the validity of a data value on entry of the data on the mobile device. Edit checks were defined by the Data Manager and were entered directly into the App by the CommCare programmer.

Data integration

We implemented, on the CommCare platform, the ability to export data from a CommCare App in the format defined by the Clinical Data Interchange Standards Consortium (CDISC) Operational Data Model (ODM).²⁵ CDISC ODM is a vendor neutral, platform-independent format for interchange and archive of clinical study data, which includes clinical data along with its associated metadata, administrative data, reference data and audit information. The CDISC ODM standard is supported by a multitude of EDC systems.²⁶

The new export functionality on CommCare is not specific to the study in which the functionality was tested, which enables future CommCare Apps to also export data in CDISC ODM format. After completion of the study, a data extract in CDISC ODM standard was created from the CommCare platform and was uploaded to the EDC system.

The functionality was tested by entering data for two dummy patients in the CommCare platform, synchronising that data into OpenClinica and by manually comparing data extracts from both systems.

Electronic data capture

The study data were uploaded and managed in OpenClinica, an open-source software platform for EDC and clinical data management (CDM).²⁷ OpenClinica has been in use since 2006, and has been used in thousands of clinical studies across the world. It is the standard platform for EDC and CDM at CREATES.

Once data is uploaded into the OpenClinica platform it is assigned the status of ‘completed’. The study's principal investigator or delegate is responsible for reviewing, confirming and signing off on the data in OpenClinica, at which point the data is assigned the status ‘signed’. This review and sign off is an essential step in the validation of the overall data collection process. Since the mobile data collection platform is not a validated system in compliance with the requirements of 21 CFR part 11,²⁸ this manual validation step is required to ensure that data transferred from the mobile data collection platform to the EDC system was transferred accurately, correctly and while preserving patient confidentiality. This approach of ‘Extraction and Investigator Verification’, and its compliance with regulations governing the use of electronic source data in clinical trials, is well described and documented.^{29 30}

Training

The end users of the App (including the Principal Investigator, Investigators, Clinicians, Nurses, Laboratory staff and QC team) received training on the use of the mobile devices and the data collection process at the start of the field testing sessions described earlier. Refresher training was organised 1 week prior to initiation of the study for the staff that would support the research study (including 7 nurses, 3 clinicians and 5 laboratory staff). The refresher training covered the proper use of Android tablets, the CommCare platform and the data collection App. The data management staff required no further training as they used the standard OpenClinica platform, and its ODM features.³¹

Assessment of feasibility

The feasibility of this approach was assessed by manual comparison of data extracts from the CommCare platform and the OpenClinica platform. Data were extracted from both systems into Excel spreadsheets. These spreadsheets were then compared for missing data, mismatched data or misrepresented data. This comparison was made three times, first by the clinical team from CREATES, then by a technical team from Dimagi, and finally by a joint clinical and technical team. In addition, a randomly selected subset of participants was selected in the OpenClinica system, and their data values were checked against the data in the CommCare system. This redundancy was built in to limit human errors in the assessment process.

The potential benefits of this approach were identified through unstructured interviews conducted with system end users.

Results

The clinical research study, which involved 38 screened volunteers and 16 recruited participants, was conducted on schedule and without technical complications. On completion of the study, an export of the study data was produced by the CommCare system in the CDISC ODM format. This file was uploaded into the OpenClinica EDC system. Manual assessment of data extracted from both systems confirmed that all mandatory data elements described in the CRF (including safety data, adverse event reports and concomitant medication lists) were successfully uploaded from CommCare to OpenClinica.

The study benefitted in several ways from the high quality of data collection that was associated with the digital platform. First, the implementation of edit checks in the CommCare App limited the occurrence of data errors, greatly limiting the number of data queries that were raised after study completion. With a study of this size, the team was expecting to have to manage around 100 data queries, but with the use of the mobile platform only 21 queries were raised. Second, study investigators determined that the resolution of queries was more efficient compared with previous paper-based systems since the entire data chain and audit trail was readily available online. All study queries were successfully completed in one working day.

The clinical staff expressed enthusiasm at using advanced mobile tools in their daily work. The trainings were reported to be highly useful to understand the functionalities of the CommCare platform. During the study users expressed their perspective that the new platform helped them to better stay organised, that the overall process was less ‘cluttered’ than with the use of paper and that use of the App saved them time.

The new QC process to review data quality in the online CommCare system was generally found to run smoothly. The QC staff was able to easily access and locate forms submitted through the App and review them online. This process was enabled by unique user names and passwords that the allowed QC staff to securely validate or correct data on the online platform.

There were some limitations with use of the new system. A few members of the clinical staff were less familiar with the use of mobile devices and as a result found it more time consuming. QC staff members also reported that it was not always straightforward to identify which forms they had already reviewed, and they provided suggestions for future improvements. The upload of the data from the CommCare system to the OpenClinica system required a number of iterations, as the settings between the two platforms needed to be reconciled. Also, the registration of new study participants and study events needed to be completed in both systems separately. Further extensions of the integration between these systems can be made to also support the automatic exchange of this kind of ‘study metadata’.

Discussion

This pilot study demonstrated the feasibility of using an open-source mobile platform specifically designed for use in low-resource settings to conduct data collection in a clinical research study. Overall, users reported significant advantages associated with the mobile application when compared to the use of paper forms, including increased efficiency, improved quality of data collected during the study and expedited data quality control after study completion. An important factor in the implementation success of the new system was repeated training sessions focused on the use of mobile devices deployed in the study, the study-specific App, and the CommCare platform.

Given that the digital platform used in this study was newly developed, it was not surprising that users identified several features that could be improved to further strengthen the system in the future. The current version employed separate applications in the EDC system and the mobile data collection platform. This necessitated manual synchronisation of study settings (such as the names of data fields and edit checks) to allow for exchange of clinical trial data between applications, and therefore imposed some duplication of efforts. Fortunately, the CDISC ODM standard intrinsically supports automatic exchange of study metadata; incorporation of this functionality into future versions of the system could streamline data synchronisation between applications. Another potential improvement to the system would be to digitalise additional paper-intensive procedures such as the informed consent process. In the future, there may be opportunity to transition most, if not all, trial procedures to a paperless system.

The benefits experienced in this study are generally recognised to be inherent to the use of electronic source data, or ‘eSource’. A number of vendors offering commercial eSource solutions for clinical trials have emerged over the past few years, such as ClinicalInk,³² CliniOps³³ and Target Health.³⁴ In the approach to eSource presented here, we chose to use an existing, open source mobile digital platform that was designed for low-resource settings and the usage of which is rapidly expanding in developing world healthcare programmes. We added to that platform the ability to be used as eSource in the context of a clinical research study. This allows organisations and clinics that have adopted such mobile platforms a smooth path towards supporting high quality, efficient data collection for clinical trials in which they may seek to participate.

Not every platform that can capture or store healthcare data can be used as eSource platform in clinical research studies. All data collected must be ‘attributable, legible, contemporaneous, original, accurate—plus complete, consistent, enduring and available’, which is often referred to as ‘ALCOA+CCEA’.³⁵ Prior to starting our project, we ascertained that the CommCare platform used in this approach meets the requirements of ALCOA+CCEA. The inclusion of the manual validation step illustrated in figure 1 obviates the need for validation of the CommCare platform. As usage of CommCare for data collection in clinical trials grows over time, the investment required for setting up a validated CommCare environment may be justified and the manual validation step can be eliminated.

In addition to engineering a new research tool with high utility, the efforts described played an important role in capability building for clinical research in Africa. Members of the local research team were integrally involved with development of the new system. After completion of the study, they hosted a 2-hour live demonstration at a regional clinical trial workshop attended by ∼50 clinical investigators, and have started support and technology transfer to another phase-I centre in Sub-Saharan Africa to adopt the platform. While still in its early stages, this close engagement of the CREATES team in the development and further support and promotion of this approach regionally are key ingredients to an ultimately self-sustainable model for platform adoption.

Conclusion

This pilot study demonstrated the feasibility and utility of a novel mobile digital platform for data collection and management in low-resource settings. User satisfaction was high, driven largely by improved operational efficiencies and data quality processes. Improvements to clinical trial methodologies benefit research teams and, ultimately, patients. Innovative technologies that improve data-related processes therefore are likely to have an important role to play in biomedical research infrastructure that is increasingly being established in Africa and other low-resource settings.