21 Code of Federal Regulations Part 11–Compliant Digital Signature Solution for Cancer Clinical Trials: A Single-Institution Feasibility Study

Abstract

PURPOSE

Inefficiencies in the clinical trial infrastructure result in protracted trial completion timelines, physician-investigator turnover, and a shrinking skilled labor force and present obstacles to research participation. Taken together, these barriers hinder scientific progress. Technological solutions to improve clinical trial efficiency have emerged, yet adoption remains slow because of concerns with cost, regulatory compliance, and implementation.

METHODS

A prospective pilot study that compared regulatory-compliant digital and traditional wet ink paper signatures was conducted over a 6.5-month period in a hospital-based health system. Staff time and effort, error rate, costs, and time to completion were measured. Wilcoxon rank sum tests were used to compare staff time and time to completion. A value analysis was conducted. A survey was administered to measure user satisfaction.

RESULTS

There where 96 participants (47 digital, 49 paper), 132 studies included (31 digital, 101 paper), and 265 documents processed (156 digital, 109 paper). A moderate reduction in staff time required to prepare documents for signature was observed (P < .0001). Error rates were reported in 5.1% of digital and 2.8% of paper documents, but this difference was not significant. Discrepancies requiring revisions included incomplete mandatory fields, inaccurate information submitted, and technical issues. A value analysis demonstrated a 19% labor savings with the use of digital signatures. Survey response rate was 57.4% (n = 27). Most participants (85.2%) preferred digital signatures. The time to complete documents was faster with digital signatures compared with paper (P = .0241).

CONCLUSION

The use of digital signatures resulted in a decrease in document completion time and regulatory burden as represented by staff hours. Additional cost and time savings and information liquidity could be realized by integrating digital signatures and electronic document management systems.

INTRODUCTION

Clinical trials are essential to evaluating new treatments, establishing standards of cancer care, and improving and prolonging the lives of patients¹; however, 30% of eligible patients will not be asked by providers to participate in a trial.² Physicians drive patient enrollment in clinical trials, yet inefficiencies in the clinical trial infrastructure result in significant physician-investigator turnover³ and a shrinking clinician-scientist workforce.⁴ In one study, upwards of 50% of principal investigators do not conduct more than one study.^2,3,5 Barriers to physician participation in research include workload balance among clinical and research obligations, study activation timelines, physician-investigator and staff time, reporting requirements, and unsatisfactory financial outcomes.³ These barriers are consistent with reported major obstacles to conducting clinical trials broadly across the United States.⁶ Taken together, these barriers hinder scientific progress.

CONTEXT

• Key Objective

• To our knowledge, there are limited prospective, solution-based reports for improving clinical trial efficiency. This study prospectively evaluated the feasibility and adoptability of a regulatory-compliant digital signature system for clinical trial documentation.

• Knowledge Generated

• This single-center, nonrandomized feasibility study demonstrated that digital signatures were superior compared with traditional paper-based systems. Statistically significant savings in costs, staff time, and document processing time were realized. There was no significant difference in quality as defined by error rates. When surveyed, a higher proportion of end users preferred digital signatures and reported that the digital system was easy to use, more efficient, and faster.

• Relevance

• The efficiency achieved with this intervention helps to mitigate physician barriers to participating in research, addresses structural barriers, and decreases administrative burden associated with clinical trial management.

To alleviate physicians of some of these burdens, centralized clinical trial offices (CTOs) composed of highly skilled professionals have been established across the country. However, clinical trial management is complex,^7,8 lengthy, and highly variable,⁹ with barriers to opening, conducting, and completing trials.¹⁰ As such, CTOs are plagued with nonvalue-adding activities, sequential processing, and onerous regulatory requirements, which result in high attrition rates and a diminishing skilled labor force.^11-13 Despite these challenges, CTOs are expected to maintain quality, increase capacity, and accelerate operations without budget or workforce expansion or clear performance metrics.^9,14 These challenges are magnified in the community setting, where study personnel of varying skill, training, and allotted time for research are assigned core research activities in addition to their primary job function.⁵

The clinical trial system is laden with cumbersome and unreliable paper-based processes,^6,9 which are further exacerbated by quality control issues that create redundancies and unacceptable error rates. Approximately 40% of the costs of bringing a new drug to market are related to paper-based processes.¹⁵ Integrated paperless systems are needed.

Technological solutions, such as digital signatures, have emerged, yet adoption remains slow because of concerns with cost, resources, regulatory compliance, and logistics. Meanwhile, digital signatures remain the gold standard in highly regulated fields, such as banking, insurance, and real estate, and may serve as a tangible solution to systematize research operations.¹⁶

We conducted a prospective pilot study of digital signatures for regulatory essential documents in cancer clinical trials to assess (1) the feasibility of a 21 Code of Federal Regulations (CFR) part 11–compliant electronic signature system to accelerate document completion time and reduce staff hours and error rates, and (2) the adoptability, convenience, and value of digital signatures among end users.

METHODS

A single-institution, three-campus location, nonrandomized, prospective pilot study comparing digital document distribution and signature to traditional paper routing of essential documents (Table 1) was conducted. This study was not subject to institutional review board review because it did not meet review criteria per 45 CFR part 46.

TABLE 1.

Document Characteristics

A digital signature vendor was selected (DocuSign, San Francisco, CA) based on system security, signature legality, document integrity, implementation ease, costs, flexible digital signature capture methods, system integration ability, and ease of use (Table 2). A cross-functional, interdepartmental task force was assembled (Table 3). The system and securities were validated, 21 CFR part 11 compliance was verified, and an official nonrepudiation letter was filed with the US Food and Drug Administration (FDA). Access roles were configured, system training was performed, and standard operating procedures (SOPs) were developed.

TABLE 2.

Digital Signature System Components

TABLE 3.

Intradepartmental Task Force

Entire clinical trial portfolios of disease management teams (DMTs; n = 13) were prospectively assigned to digital (intervention group) or wet ink signatures of essential documents (control group). To balance the groups, allocations were based on projected disease-specific subject enrollment, number of studies per DMT, and number of end users within each DMT. Neither technological skill nor number of participating sites were known or considered at the time of group assignment. The control group continued with standard practice of hard files routed for wet ink signature through hand delivery, interoffice mail, or e-mail, while the study group exclusively used a digital platform to prepare, route, and obtain signatures. There was no crossover between groups. Data were collected over 6.5 months (Appendix Table A1).

Staff time required to prepare documents was measured in minutes and compared between digital and paper signatures using a Wilcoxon rank sum test. A value analysis was performed using a common strategic value assessment (SVA; IntelliCap, Chicago, IL) model that measures nonlabor savings (printing and document storage) and labor improvement (document creation, distribution, management, and storage). Benchmarking data across the health care and life science industries (73 clinical and nonclinical use cases) were used as comparator data. An anonymous 13-question (Appendix Table A2) survey (REDCap [Research Electronic Data Capture]; Vanderbilt University, Nashville, TN) was distributed to all faculty and staff with active 21 CFR part 11 accounts during the time of the pilot.

The time to completion for each document was calculated using the date sent and the date of document completion. Time was measured in full days and compared between digital and paper signatures using a Wilcoxon rank sum test in SAS 9.4 (SAS Institute, Cary, NC) because time was a non-normally distributed continuous variable. We also stratified the analyses by number of required signatures (one v two or more) and by number of routed sites (single site v multisite).

RESULTS

The number of participants was 96 (47 digital and 49 paper). The number of studies was 132 (31 digital, 101 paper). A total of 265 essential regulatory documents were routed for signature (156 digital and 109 paper). The majority of the documents for both groups required a single signature (88.5% digital v 90.8% paper). The distribution document type was similar across both groups with the exception of Form FDA 1572 (6.4% digital v 15.6% paper) and number of training logs signed (24.4% digital v 5.5% paper). Fewer digital satellite clinic signatures were required (n = 5; 3.2%) compared with wet ink signatures (n = 22; 20.2%).

Staff time to prepare documents for signature was 20% less in the digital group compared with paper (8.0 minutes v 10.9 minutes per document; P < .0001). SVA demonstrated a 19% reduction in total document transaction time, a total of 612 hours of labor savings, and a $25,285 cost savings. Document quality did not improve with digital signatures, with eight (5.1%) digital documents requiring revisions compared with three (2.8%) paper documents; however, error rates were not reported in 7.1% of digital documents and 26.6% of paper documents (Appendix Table A3).

Of 47 surveys distributed, 27 (57.4%) individuals responded (Fig 1). Twenty-one respondents (80.8%) stated that digitally signing documents was very easy to use, and 21 (80.8%) believed that it was very convenient. When asked to compare digital signatures to traditional paper, 88.9% of responders indicated that digital signatures were more efficient and faster. Most responders (85.2%) preferred digital signatures (Fig 1).

FIG 1.

Survey responses.

The median time to complete documents with digital signatures was 3 days (interquartile range [IQR], 1-12 days) and that of traditional wet ink signatures was 7 days (IQR, 2-28 days; Fig 2). The time to complete documents was faster with digital signatures compared with paper signatures (P = .0241). Within the digital signatures group 101 (64.7%) of all signatures necessary for document completion were obtained in < 2 hours. The median time to complete documents routed at a single site for digital signatures was 1 day (IQR, 0-4 days) compared with 0 days (IQR, 0-2 days) for paper signatures. The median time to complete documents routed at multiple sites for digital signatures was 1 day (IQR, 1-16 days) and for paper signatures, 4 days (IQR, 0-16 days; P = .4273). The median time to complete documents that required one signature with digital was 1 day (IQR, 0-5 days) and with paper, 0 days (IQR, 0-4 days; P = .0398). The time to complete documents requiring two or more signatures was similar for both digital and paper (P = .2849; Fig 2).

FIG 2.

Time to completion. (A) Digital v paper. (B) One signature v two or more signatures. (C) Single site v multiple sites. Comparisons by Wilcoxon rank sum test.

DISCUSSION

When comparing the amount of staff time required to prepare and route documents, we found a strong statistical indication that digital signatures were more efficient. Although the analysis did not correct for the learning curve associated with designing digital templates and workflows, the digital process was still faster. Therefore, the time-saving benefit of digital signatures would likely increase the longer the system was used.

The postsurvey assessment showed an overwhelmingly positive response. Users found the process convenient and intuitive and preferred to use digital signatures in the future. No training was required for signers. This demonstrates the ease at which a 21 CFR part 11–compliant digital solution can be implemented.

Traceable, tamper-proof audit trails, system configuration transcripts, and exportable digital certificates with a unique identifying link to the signer were made available to external monitors and auditors. Errors related to form completion or data entry by the regulatory analyst required complete revision because the document security is locked once signed. Errors related to missed signatures/initials, incorrect dating, or signatures in the wrong place on the form were not present in the digital group because the digital signature platform enabled templates for commonly used documents (ie, financial disclosures). For example, specific parameters, conditional logic, and required fields were set. Templates were accessed from the document library and applied to all studies. Signers could not move forward without completing required fields, signing, or dating, thereby eliminating common deficiencies. However, it is important to note that 37.5% of the digital errors in this study were the result of a single template error that was identified and fixed so that the mandatory fields could not be skipped in the future.

Despite long-term efficiencies and cost reductions, transitions to digital signature platforms require a significant initial investment, including software fees, time associated with the development of SOPs, templates, business workflows conducive to electronic signatures, and institutional reviews. A follow-on study conducted across multiple institutions would allow evaluation of the cost effectiveness of transitioning to a digital platform and better characterization of implementation challenges.

Aside from cost, a major barrier to adoption of technology in the clinical research filed has been implementation and concerns with security and privacy. The software provides high availability, fault tolerance, and threat isolation (Table 2). We found system implementation uncomplicated given that the software is a cloud-based, end-to-end, off-the-shelf solution with flexibility for standard or custom interfaces. The backend software ensures minimal disruption by upgrades and new releases that affect operations or costs. The software uses a standard three-layer logical architecture (interface, integration, logic) that controls end user experience and interfaces with standard or custom applications and system processes, such as document routing workflow.

Time to completion was measured from document preparation to document execution. For comparison purposes, document completion time was measured in days rather than in minutes because the paper group did not have the capability of tracking smaller increments of time. The actual completion time for digital signatures was recorded in minutes. While the reduction from traditional paper to digital signatures from a median of 7 days to a median of 3 days is compelling, the superiority and traceability in the intervention group importantly facilitates accountability and enables visibility into document status (ie, opened/read, executed) down to the millisecond.

Sequential signing of paper documents results in document completion delays. For example, a document could sit on a desk while this person is out of the office, which slows the overall process of obtaining multiple signatures, or results in lost original files, which is a serious regulatory concern. Manual workarounds include distribution of documents that require multiple signatures in a group meeting setting, which contributes to faster completion times. However, a remote workforce makes this less possible, necessitating digital signatures. In the digital group, signers could review and sign documents from their mobile device while in the clinic or out of the office. For documents that required multiple signatures but did not require a specific signing order, multiple signatures were obtained on a single document in parallel.

Traditionally, paper documents are difficult to track. Use of automated reminders in the study group ensured that documents were not forgotten or missed because of human error. The digital group benefited from enhanced visibility. The staff mitigated delays by easily identifying where documents were in the signature process. Electronic archiving within the digital solution eradicated compliance issues associated with missing document pages or misfiling.

Clinical trial efficiency and the downstream impact of inefficiencies have been documented. For example, it has been shown that trial development and activation times negatively affect clinical trial accrual rates and significantly affect the likelihood of trial success.¹⁷ To our knowledge, there are limited prospective, solution-based reports for improving clinical trial efficiency. We believe that these data support a follow-on randomized study conducted across multiple independent institutions (academic, community) to assess practical significance of this intervention on clinical trial efficiency. We also believe that these data lend themselves to a subsequent analysis of the relationship between patient participation in research studies and the use of digital signatures in the informed consent and reconsent process.

Implementation of digital signatures in the clinical research continuum have never been more important than in the face of the recent pandemic, where according to a survey of 297 responding US clinical research sites, 31% feared total closure as a result of COVID-19–related stay-at-home orders, and 38% of sites reported difficulties working from home.¹⁸ This, coupled with the ongoing national emphasis on trial acceleration, has amplified the need for paperless integrated systems.

This pilot study was limited to a single institute within a matrixed research-based hospital operating across three campus locations. Groups were not randomly assigned given the exploratory nature of this study and the numerous variables related to number of users, number of studies within each DMT, number and type of documents processed, and variability of workflows. While our method of allocation introduces a potential for bias as a result of assignment methods, we expect this would be alleviated in a follow-up randomized study controlled at the staff and study level. The majority of documents required a single signature; the use of the multisignature functionality in the study group was not fully interrogated. More traditional paper documents were routed for signature at satellite locations than digital documents because of regular onsite meetings where wet ink signatures were obtained. This may have biased the time to completion. However, the low numbers of digital signatures required at satellite locations is a limitation of the study. Data verification and accuracy of documents created by regulatory analysts were not assessed because this pilot study served to simulate real-world experience and because it is uncustomary for a dual check to occur before routing documents. A large proportion of end users did not respond to the survey; however, survey response rates were significantly higher compared with what has been previously reported in electronic surveys geared toward physicians.¹⁹ Information technology integrations were not fully interrogated because this was a pilot study. Additional savings not measured by this pilot study might be realized through integration with e-regulatory binders and clinical trial management systems. Additional benefits in terms of time and cost, had the study continued for a longer duration of time and with a larger sample size, may have been realized.

In conclusion, insights from this single-institution case study demonstrate the implementation ease, accelerated processing, wide user acceptance, and cost savings of digital signatures over traditional paper-based solutions. These unparalleled savings allow scarce resources to be redirected to value-adding activities to enable research sites to offer tomorrow’s most promising treatments today.

Appendix

TABLE A1.

Data Collection Elements

TABLE A2.

End User Survey

TABLE A3.

Error Types

AUTHOR CONTRIBUTIONS

Conception and design: Therica M. Miller, Jenny Lester, Beth Y. Karlan, BJ Rimel

Administrative support: Jenny Lester

Provision of study material or patients: Jenny Lester, BJ Rimel

Collection and assembly of data: Therica M. Miller, Jenny Lester, Beth Y. Karlan, BJ Rimel

Data analysis and interpretation: All authors

Manuscript writing: All authors

Final approval of manuscript: All authors

Accountable for all aspects of the work: All authors

AUTHORS' DISCLOSURES OF POTENTIAL CONFLICTS OF INTEREST

The following represents disclosure information provided by authors of this manuscript. All relationships are considered compensated unless otherwise noted. Relationships are self-held unless noted. I = Immediate Family Member, Inst = My Institution. Relationships may not relate to the subject matter of this manuscript. For more information about ASCO's conflict of interest policy, please refer to www.asco.org/rwc or ascopubs.org/cci/author-center.

Open Payments is a public database containing information reported by companies about payments made to US-licensed physicians (Open Payments).

Therica M. Miller

Other Relationship: University of Southern California, AOC, Alliance Foundation Trials, Children’s Hospital of Los Angeles, Forte Research, George Washington University

Beth Y. Karlan

Honoraria: Merck

Research Funding: VBL Therapeutics (Inst), AstraZeneca (Inst)

Patents, Royalties, Other Intellectual Property: US and European Union patent on gene signature

Other Relationship: Elsevier

BJ Rimel

Honoraria: Genentech

Consulting or Advisory Role: AstraZeneca, Tesaro, Genentech, Roche, Deep 6 AI, Clovis Oncology

No other potential conflicts of interest were reported.