Optimising data completeness and quality in multicentre randomised clinical trials using central data monitoring: a pragmatic approach

Abstract

Background

Randomised clinical trial data monitoring is essential for ensuring quality and adherence to protocols, often achieved through on-site methods. However, challenges like the COVID-19 pandemic have highlighted limitations. Central data monitoring offers an alternative, leveraging digital capabilities and diverse expertise.

Methods

The SafeBoosC-III trial, randomising 1,601 preterm infants across 70 sites, was a pragmatic trial offering detailed education of the investigators before launch. During the trial, we conducted pre-protocolised central data monitoring encompassing monthly data checks, data completeness assessments, and statistical analyses. Discrepancies were primarily identified through visual interpretation of data.

Results

Throughout the trial, meticulous central monitoring helped keeping missing data at only 1.4%. Of 43,965 data points reviewed, 338 were flagged, with 81 (0.2% of all data and 24.0% of flagged data) corrected after investigation.

Conclusions

The methodology of combining manual investigation with statistical tools effectively minimised data errors, enhancing validity. This approach in the SafeBoosC-III trial emphasises the importance of ongoing central data monitoring in clinical trials. Standardised protocols for such monitoring are crucial for ensuring data quality and accelerating the validation process in future trials.

Supplementary Information

The online version contains supplementary material available at https://doi.org/10.1186/s12874-026-02932-y.

Introduction

Monitoring of clinical trial data is vital for obtaining prime data quality and protocol adherence [1–3], and for optimising statistical power by reducing random errors (like clerical mistakes and missing data) [4, 5]. Systematic errors, stemming from misinterpretation or data fabrication, may introduce bias [4–6]. On-site monitoring, while commonly used according to the good clinical (research) recommendations (GCP) [7], can be costly, time-consuming, and faces challenges during events like the COVID-19 pandemic [8–11]. Moreover, it tends to focus on random errors [12, 13], which have little to no influence on the estimated intervention effect [2, 4].

Conventionally, local investigators oversee data quality with on-site monitors [14]. The digital era, however, enables central data monitoring, involving diverse expertise, complementing safety assessments handled by an independent data monitoring and safety committee [2, 13, 15, 16]. Central data monitoring allows for ongoing assessment, identifying discrepancies, ensuring data integrity, and facilitating prompt corrections within the trial period, thereby minimising the need for extensive 'data cleaning' activities after trial completion [2, 4]. Central data monitoring is also a resource efficient alternative compared to frequent on-site visits.

Here, we follow up on our initial report on central data monitoring [17] – aimed to reduce missingness and optimise quality of entered data – carried out in the SafeBoosC-III randomised clinical trial [18]. We present summaries from all the central data monitoring meetings; provide an overview on what we have learned during the process; and offer perspectives on how to optimise central data monitoring in future trials.

Methods

This report is an update from the first feasibility report on central data monitoring [17] carried out in the SafeBoosC-III randomised clinical trial [18]. The SafeBoosC-III trial was a large, pragmatic, multicentre, randomised clinical trial evaluating the benefits and harms of treatment guided by cerebral oximetry monitoring in extremely preterm infants during the first days after birth [18, 19]. To enhance the quality of the trial intervention and patient care, healthcare providers from sites participating in the SafeBoosC-III trial were offered a web-based training and certification programme prior to trial initiation. More than 900 staff members from sites across China, Europe, India, and the United States completed the programme. We provided the training modules in seven different languages: English, Chinese, Italian, Spanish, Turkish, French, and German [20]. A total of 1,601 extremely preterm infants were randomised to either the experimental or the control group. In the experimental group, the infants underwent cerebral oximetry monitoring using near-infrared spectroscopy, in addition to usual routine monitoring, during the first three days after birth. If the cerebral oxygenation dropped below a predefined hypoxic threshold, the clinician was required to evaluate the clinical status of the infant and decide on an appropriate intervention based on a predefined, published physiology-based treatment guideline [19, 21]. Infants randomised to the control group received clinical monitoring and treatment as usual. Cerebral oximetry monitoring did not result in significant differences between the two groups [18]. On-site monitoring during the trial verified existence of the participant and alive-status. On-site monitoring did not verify other data collected during the trial. The methods used for the central data monitoring has previously been described in detail [17, 22].

Central data monitoring outcomes

Missing data are pervasive in research and can impact clinical trial outcomes by reducing statistical power and potentially biasing results, despite attempts at imputation [5]. Addressing missing data is crucial but challenging, as not all missing values are retrievable, and data correction often remains a manual process. Thus, quality indicators are integral in clinical trials, reflecting protocol adherence and guiding result interpretation. Major deficiencies in protocol adherence identified after completion of patient inclusion may result in the exclusion of participants or entire sites, emphasising the need for ongoing monitoring to optimise protocol implementation [10]. Spotting systematic deficiencies in protocol adherence would allow for continuous optimisation of, for example, the electronic case report form (eCRF) or the protocol.

In our central data monitoring of the SafeBoosC-III trial, we identified quality deficiencies, including metrics related to intervention delivery and primary outcome measures, which are crucial for result validity [13]. The quality deficiencies are described in detail previously (Supplemental File 3) [17], and were used as predefined monitoring signals and warning criteria within a risk-based central monitoring framework, in accordance with FDA and EMA guidance on Risk-Based Monitoring and quality management in clinical trials [23, 24]. Data deviations encompass suspected outliers, misunderstandings, and potentially fabricated data, requiring multifaceted approaches for identification and verification [13, 25].

Central data monitoring group

The central data monitoring group is responsible for reviewing data completeness and quality, and for identifying potential deviations. Comprising a trial manager, a neonatology expert, a statistician or data scientist, and two trialists, the group provided diverse expertise to support comprehensive data interpretation and timely intervention. Notably, the members were independent from the Data and Safety Monitoring Committee (DSMC), thereby preserving objectivity [26]. Additionally, the central data monitoring group remained blinded to intervention group allocation and site origins during meetings, using randomly generated acronyms for each site and no indication of intervention.

Central data monitoring reports

Reports on data completeness and quality – created with R version 4.0.3 (R Core Team, Vienna, Austria) and R Markdown – were two separate reports and updated according to code changes detailed in a changelog.

The data completeness report aided the trial manager by displaying completion status for each participant's data entry. An anonymised site-level summary was included in monthly newsletters and used by the Steering Committee of the trial to have a frequent overview of the trial. The data quality report, critical for monitoring meetings, presented variables stratified by sites, excluding those with insufficient data entries. The Mahalanobis distance analysis were implemented to assist in identifying potential differences between sites [27]. Summarised decisions and conclusions are publicly available (example from 19 November 2020, Supplemental file 2) [22].

The Mahalanobis distance method, used for outlier detection, employed a two-standard-deviation criterion after excluding sites with less than 5 participants [25, 28]. Monte Carlo simulation and multivariable analysis ensured robust estimates over 10,000 iterations for site assessment against the defined limit [27]. Mahalanobis distance included both baseline variables closely associated with the outcomes, and the main outcomes of the trial: gestational age, birth weight, days of mechanical ventilation, patent ductus arteriosus, cerebral and non-cerebral outcomes, and death. Outliers from Mahalanobis distance was evaluated by the central data monitoring group, by looking at data patterns while including expert review perspectives.

Central data monitoring activities

The central data monitoring group divided its responsibilities into two main areas: data completeness monitoring and data quality and deviation monitoring.

Data completeness monitoring began after 100 randomisations, and was conducted monthly to ensure timely completion without overwhelming local investigators. Our approach maintained zero tolerance for missing data, emphasising ethical conduct in clinical trials. The zero tolerance would ideally encompass all data, but, here, we maintained zero tolerance for all outcome data.

Data quality and deviation monitoring started after 400 randomisations, occurring every three months. These meetings allowed for changes in the electronic data entry form, database structures, and protocol adjustments. The subjective nature of assessments necessitated discussions within the central data monitoring group during three-hour meetings, which involved all members to identify discrepancies, determine causation, and decide subsequent actions.

Results

Throughout the trial, missing data were monitored monthly, which was accompanied by a data completion report (example from 17 September 2021, Supplemental file 1) [22]. This was followed by contact to local investigators who were asked to fill in their missing data. On a monthly basis, the trial manager spent approximately 10–15 h on writing the data completion report and on correspondences with local investigators to ensure completion of missing data. This process helped facilitate that the SafeBoosC-III trial, conducted across 70 different sites in 17 different countries in Asia, Europe, and North America, achieved an overall missingness of 1.4% for the primary outcome (22 of 1,601 included participants) [18]. All missing data were either due to withdrawal of consent (n = 20) or withdrawal of an entire site (n = 2).

Throughout the trial a total of 43,965 data points from 31 variables were assessed at 8 central data monitoring meetings. Aside from the previously reported structural issues [17] – which among other things resulted in changes to the eCRF in OpenClinica – 338 entries were flagged – corresponding to less than 0.9% of all monitored data points. Of those 338 entries, 81 (24% and 0.2% of all data points) were later corrected (Table 1). Inquiries labelled as awaiting response were not clarified within four weeks after the meeting but were subsequently handled. Following each of the eight meetings, the trial manager contacted relevant investigators regarding the flagged data entries. Throughout the subsequent month, ongoing communication took place between the trial manager and investigators to clarify whether the flagged data entries were correct or needed correction. After this period, a changelog, including an overview of all flagged data entries as well as a summary on the course of action derived from the correspondence between the trial manager and investigators (e.g. ‘correction of flagged data entry in the eCRF by the investigator’), was prepared and uploaded to the SafeBoosC webpage (example from 19 November 2020, Supplemental file 2) [22]. The trial manager kept a detailed monitoring log with details about issue identification, date of contact, date of response and so forth. Similarly, the eCRF system kept a detailed history of all versions of the data and the user who inputted the information for each field. From the date of each central monitoring meeting and throughout the subsequent month, the trial manager sent and received approximately 60 e-mails. The total workload around each of the 8 central monitoring rounds, was estimated to 15–20 working hours for the trial manager.

Table 1.

Central data monitoring meeting summaries

	Entries flagged n	No course of action n (%)	Correct entry n (%)	Incorrect entry n (%)	Awaiting response n (%)
1st meeting 26 August 2020	62	2 (3.2%)	14 (23%)	40 (65%)	6 (9.7%)
2nd meeting 30 November 2020	35	7 (20%)	19 (55%)	6 (17%)	3 (8.6%)
3rd meeting 02 March 2021	59	1 (1.7%)	45 (76%)	7 (12%)	6 (10%)
4th meeting* 01 June 2021	-	-	-	-	-
5th meeting 02 September 2021	52	1 (1.9%)	32 (62%)	11 (21%)	8 (15%)
6th meeting 28 October 2021	20	1 (5.0%)	8 (40%)	1 (5.0%)	10 (50%)
7th meeting 14 February 2022	85	0 (0.0%)	71 (84%)	14 (16%)	0 (0.0%)
8th meeting 18 April 2022	25	2 (8.0%)	18 (72%)	2 (8.0%)	3 (12%)
Total	338	14 (4.1%)	207 (61%)	81 (24%)	36 (11%)

^*Due to the absence of one of the investigators, detailed notes were not recorded for the responses from this meeting in which 74 entries were flagged. Thus, these 74 flagged entries were excluded from both the total count and the percentage calculations

Throughout the meetings, five sites were identified as outliers using the Mahalanobis distance. Two sites were identified at five meetings, one site at two meetings, and two sites at one meeting. For the most extreme outlier, the investigators were contacted, however no data corrections were made. Overall, the flagged sites had fewer participants included, who were either on mechanical ventilator for a longer time, as per local protocol, or were smaller and more severely ill.

The code used to generate the reports was implemented throughout the process as functions in the ‘clintools’ R-package (https://cran.r-project.org/web/packages/clintools/index.html) [29]. Two functions were developed specifically to facilitate central data monitoring in the future. The first function – cdm.miss – provide a graphical overview on how missingness is distributed stratified by each participant. This facilitates the process of reaching out to local investigators to correct the missingness (Fig. 1). The second function – cdm.fig – was developed to provide an overview of data deviations for both continuous and categorical variables (Fig. 2). These functions were continuously updated to further facilitate the process by optimising the visual overview of the data and aiding the monitors to identify differences.

Fig. 1.

The cdm.miss function from the clintools R-package which provide a graphical overview on how missingness is distributed stratified by each participant. The data shown are merely simulated data based on trial data, but without actual data. Missingness presentation

Fig. 2.

The cdm.fig function from the clintools R-package which provide an overview of data deviations for both continuous and categorical variables. The data shown are merely simulated data based on trial data, but without actual data. Data deviation presentation

Randomisation was completed on 16 December 2021 with the last follow-up on 11 March 2022. Only 49 days later (29 April 2022), the first statistical analyses were conducted on the locked, complete dataset, and on 20 May 2022 the final statistical report was shared at an investigator meeting in Copenhagen.

Discussion

Here, we report on central data monitoring in a large, pragmatic, multicentre randomised clinical trial where the resulting missingness was negligible [18]. Furthermore, the process allowed us to flag 338 potential errors, of which we identified 81 as incorrect entries with a hit-rate of one in four. This process reduced both random errors and systematic errors, thereby increasing the validity of our results [2, 4]. Moreover, together with the huge commitment from the trial managers and local investigators, we were able to carry out our first analysis less than two months after the last follow-up. A key operational challenge during implementation was ensuring consistent interpretation of data definitions across multiple sites and maintaining timely follow-up on flagged data without overburdening local investigators. In addition, the effectiveness of the process depended on sustained engagement from trial personnel and the usability of the electronic data capture system.

Our methodology with manual investigation merged with statistical monitoring, enabled the identification of both random errors and systematic errors [13, 25, 30]. This approach signifies a crucial step toward establishing a standardised central data monitoring system [31]. Despite careful trial design, diverse clinical settings among countries, hospitals, and investigators create ample room for data interpretation errors. Frequent central data evaluation may help detect and rectify such misunderstandings. While an independent DSMCs oversee safety concerns during trials, their focus primarily lies in participant interests and safety assessments [32–34]. In contrast, the central data monitoring group safeguards the scientific integrity of the trial but must remain blinded to the allocation to randomisation groups to ensure unbiased trial management [35].

The relatively few flagged potential errors, 338 out of 43,965 data points from 1,601 participants, was likely attributable to several factors: the pragmatic design of the SafeBoosC III trial, the well-designed OpenClinica data capture software for this trial, the trial managers’ education of investigators, and last, but not least, the dedication of all involved clinical investigators. Consequently, the work of the central data monitoring committee was therefore doable and required up to three hours per meeting. In more complex trials with less well-functioning data capture systems and lower levels of engagement from trial managers and clinical investigators, the work of the central data monitoring committee may be more demanding and time consuming.

From a methodological perspective, we identified limitations of the applied statistical monitoring approach, as we found no indications of fraud or data fabrication [36], despite the fact that the Mahalanobis distance did not provide any additional value beyond standard central data monitoring. Instead, the Mahalanobis distance merely showed an expected difference in participant characteristics across countries and sites in this multicentre trial. The data set available for calculation of Mahalanobis distances, consisting mainly of dichotomous variables, was probably too limited, given the relatively small number of participants at each site, to be sufficiently sensitive to detect site-specific data deviations. More broadly, this highlights that the effectiveness of statistical monitoring tools depends on data structure, sample size per site, and outcome variability. The fundamental challenge in this effort is to balance the sensitivity with the efforts needed to react to the flags raised. While a substantial part of this effort lays on the trial manager, it is particularly important in pragmatic trials to use the capacity of the site primary investigators wisely and to avoid trial fatigue. Based solely on the number of corrected data entries, one might get the impression that the efforts and work invested in central data monitoring may not be cost effective. On the contrary, we find central data monitoring to be highly valuable and well worth the investment. As demonstrated, the mix of pragmatism in the trial design, committed investigators, targeted training, and effective interaction with the data capture system of Open Clinica already secured the correctness of the majority of the data. Most trials are not as fortunate – and even when most data are correct, it is reassuring to get them validated.

The methods implemented in this trial might not be superior to other models, but here we report on a feasible methodology which provide an easy overview of missingness and allow for identification of data deviations. Moreover, the central data monitoring group's proactive data assessment allows for immediate data cleaning during the trial, facilitating earlier statistical analysis and faster publication of trial results. For now, data monitoring is primarily conducted by trial companies and GCP groups – companies which help investigators conduct trials – whereas we demonstrate that central data monitoring in a pragmatic, low-cost trial can still identify incorrect data entries. To date, no peer-reviewed, protocolised, detailed and easy-to-follow guidance on central data monitoring has been published, representing a clear next step for the field of randomised clinical trials.

After finishing this project, GCP recommendations has been updated from R2 to R3 [7, 8]. The new revision now provides more flexibility in the recommended modality for trial monitoring. Previously, central data monitoring was only allowed as an addition to conventional on-site monitoring, but in the newest revision central data monitoring could be the sole monitoring modality. This underlines the need for detailed practical reporting of how central data monitoring should be carried out. Similarly, if central data monitoring should be the sole monitoring modality it further warrants detailed pre-protocolisation. Thus, we believe that our clintools-package (https://cran.r-project.org/web/packages/clintools/index.html) [18] will facilitate easier central data monitoring.

Overall, several lessons can be drawn from this work. Central data monitoring is most effective when embedded in a pragmatic trial design with robust electronic data capture and active investigator engagement. Furthermore, combining automated and manual review processes enables efficient detection of both random and systematic errors. Finally, careful protocolisation of monitoring procedures is essential to ensure reproducibility and to minimise unnecessary burden on participating sites. However, external implementation of our methodology might be limited since we did not meticulously record time spent for every step.

In conclusion, central data monitoring is made possible by web-based, real-time data entry with centrally stored trial data [17]. Furthermore, automated central data monitoring algorithms and reporting tools can help efficient identification of errors or mistakes in the eCRF and incorrect data entries. Finally, central data monitoring allows for timely interventions in cases of suspected misunderstandings. As with other aspects of randomised clinical trials, central data monitoring should be protocolised to ensure an optimal and reproducible process.

Authors’ contributions

The study was designed by the authors who were also trial collaborators (MHO, MLH, SS, JCJ, GG, CG). Data was collected by the trial collaborators (see below). The first version of the manuscript was written by MHO and CG; all authors revised the manuscript; and all authors and collaborators (see below) approved the final version.

Funding

The Copenhagen Trial Unit, Centre for Clinical Intervention Research, The Capital Region, Copenhagen University Hospital ─ Rigshospitalet, Copenhagen, Denmark.

Data availability

The datasets will be made available from the coordinating investigator on reasonable request, which will be approved by the SafeBoosC-III steering committee, while the data and scripts for the present report will be made available after request to the corresponding author.

Declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interest

The authors declare no competing interests.