Abstract
Objective
To best meet our point-of-care research (POC-R) needs, we developed ProjectFlow, a configurable, clinical research workflow management application. In this article, we describe ProjectFlow and how it is used to manage study processes for the Diuretic Comparison Project (DCP) and the Research Precision Oncology Program (RePOP).
Materials and methods
The Veterans Health Administration (VHA) is the largest integrated health care system in the United States. ProjectFlow is a flexible web-based workflow management tool specifically created to facilitate conduct of our clinical research initiatives within the VHA. The application was developed using the Grails web framework and allows researchers to create custom workflows using Business Process Model and Notation.
Results
As of January 2021, ProjectFlow has facilitated management of study recruitment, enrollment, randomization, and drug orders for over 10 000 patients for the DCP clinical trial. It has also helped us evaluate over 3800 patients for recruitment and enroll over 370 of them into RePOP for use in data sharing partnerships and predictive analytics aimed at optimizing cancer treatment in the VHA.
Discussion
The POC-R study design embeds research processes within day-to-day clinical care and leverages longitudinal electronic health record (EHR) data for study recruitment, monitoring, and outcome reporting. Software that allows flexibility in study workflow creation and integrates with enterprise EHR systems is critical to the success of POC-R.
Conclusions
We developed a flexible web-based informatics solution called ProjectFlow that supports custom research workflow configuration and has ability to integrate data from existing VHA EHR systems.
Keywords: learning healthcare, veterans, embedded pragmatic, diuretic, precision oncology, data sharing, clinical trial, randomization
Introduction
The Veterans Health Administration (VHA) is the largest integrated health care system in the United States, providing care for over 9 million Veterans at over 1255 facilities across the nation.1 Over the past decade, the VA Office of Research and Development (VA ORD) has supported the implementation of point-of-care research (POC-R)2 with an emphasis on point-of-care clinical trials (POCCT), that is pragmatic clinical trials3 that compare approved treatment options when clinicians are in equipoise.
The POC-R design aims to embed research workflows within day-to-day clinical operations with minimal interference to care, thereby allowing study endpoints, treatment deviations, and patient compliance data to be extracted from a patient’s longitudinal electronic health record (EHR). POC-R also leverages EHR data for study recruitment. This clinically embedded design is advantageous in reducing research costs and promoting realization of a learning healthcare system by facilitating the translation of research evidence into clinical practice.4,5
Workflow management software that can accommodate complex POC-R workflows while also integrating with enterprise EHR infrastructures is crucial to the success of point-of-care initiatives. Although numerous software options exist, many out-of-the-box tools lack flexibility,6 forcing researchers to either sacrifice integration with EHR systems or substantially modify their preferred study workflows. Within the VHA, software selection is further complicated by requirements for compliance with robust data security rules as well as integration with its mature and rigid EHR infrastructures. Indeed, formal VHA approval for the installation and use of commercial or open source tools with protected health data may take years.
We developed a flexible web-based informatics solution called ProjectFlow that supports custom research workflow configuration and has ability to integrate data from existing VHA EHR systems. In this article, we describe how ProjectFlow is used to support a range of research study processes, from a focused patient enrollment process into a national data repository to the multiple complex processes of a multicenter POCCT.
Research Precision Oncology Program
Research Precision Oncology Program (RePOP) is the research component of the national clinical program named the Precision Oncology Program (POP). RePOP recruits and enrolls patients interested in sharing their de-identified electronic health data with the Precision Oncology Data Repository (PODR) for the purpose of innovating VHA cancer treatment.7,8 This includes, but is not limited to, building genomic-based outcome prediction engines for clinical decision support. Cancer is a multifaceted disease, requiring an enormous amount of aggregate data to develop generalizable findings. To this extent PODR acquires patient genomic, clinical health, and medical image data (eg, radiological, pathological, and others). At present, RePOP utilizes ProjectFlow to support the contact and enrollment of eligible patients.
Diuretic Comparison Project
Diuretic Comparison Project (DCP) is a POCCT which evaluates the relative effectiveness of two widely prescribed thiazide-diuretics used in the treatment of hypertension. Specifically, the VHA national outpatient prescription database shows that over a million veterans were prescribed a thiazide-type diuretic each year from 2003 to 2008 with over 95% receiving hydrochlorothiazide (HCTZ) and less than 2.5% receiving chlorthalidone (CTD).9 However, recent evidence suggests CTD may not only be more effective for managing symptoms but also less expensive.9 To better understand the relative effectiveness of these medications, the DCP trial compares cardiovascular outcomes in patients treated with HCTZ versus CTD. At present, DCP is actively enrolling patients and utilizing the ProjectFlow application to support trial recruitment, consent, randomization, and prescription order monitoring.
MATERIALS AND METHODS
Key ProjectFlow functionalities
Customizable workflows and role-based access control
ProjectFlow is a flexible web-based workflow management tool created to facilitate the conduct of national POC-R within the VHA. ProjectFlow allows researchers to create custom workflows using the standards-based Business Process Model and Notation (BPMN, version 2.0). BPMN employs graphical elements as representations of business processes.10,11 Researchers may generate their study-specific workflows using the Eclipse IDE12 and Activiti plugins.13 ProjectFlow was developed using the Grails web framework. Previously known as “Groovy on Rails,” Grails is an open source web application framework which uses Apache Groovy, a Java syntax compatible language.14 ProjectFlow also employs “plug-and-play” web-services to ensure smooth integration with EHR databases through synchronous (immediate response) or asynchronous (delayed response) communication.
When creating POC-R workflows, ProjectFlow users first define “clinical elements.” A clinical element is an object with specified properties that may move through a workflow. For example, a clinical element could be a physician, patient, medication, genomic sample or even a hospital location. Using BPMN 2.0 notation, users may design simple or complex workflows that a clinical element may move through (Figure 1A). The creation of study-specific workflows requires a general understanding of workflow notation as well as a moderate to strong understanding of database structures and relevant EHR data fields. Furthermore, a clear understanding of study-specific clinical processes is essential. Although initially a steep learning curve, we have found that staff members with sufficient training are able to contribute to complex workflow development within a few hours.
Figure 1.
Workflow creation and integration with VHA EHR systems. (A) Example of simple BPMN 2.0 workflow: User defined clinical elements proceed through user designed study workflows. The figure depicts a workflow for updating a patient’s status. More specifically, once the “Update Patient Status” task is completed, synchronous web-service communication transmits the updated information to the database. If an error occurs in transmission, this will be registered via the “IT log” pathway. (B) Data flow utilized by the ProjectFlow web-based application: (B1) Clinicians utilize the computerized patient record system (CPRS) user interface to access and enter patient data into VistA. The DCP study utilizes “View Alerts” embedded within CPRS to facilitate trial recruitment, randomization, and prescription ordering. (B2) VistA data are transferred nightly to the VA Corporate Data Warehouse (CDW) where it resides for secondary operational and research use. (B3) Scheduled, nightly, extract transform load processes extract relevant EHR data from CDW into the study database (Study DB) which is utilized by (B4) ProjectFlow as needed for patient recruitment, randomization, prescription tracking and monitoring. Authorized study staff may access the ProjectFlow system and CPRS via their VHA workstation.
ProjectFlow employs Role-Based Access Control (RBAC) to manage how its users interact with clinical elements and workflows. RBAC allows control over which data a user (role) may access as well as the scope of functions (tasks) each role may perform. To keep patients’ personally identifiable information secure, a study may restrict access to certain data elements depending on the role or expand access as the role changes.
Integration with enterprise EHR systems
As noted previously, POC-R utilizes longitudinal EHR data for patient recruitment and monitoring. ProjectFlow databases query VHA EHR systems regularly to obtain recent relevant data. The Veterans Health Information Systems and Technology Architecture (VistA), developed in 1999, serves as the backbone of the VHA’s EHR15 until the transition to the Cerner EHR (fully operational by 2028) is complete. Clinicians access and enter patient data into VistA via the Computerized Patient Record System (CPRS) user interface. VistA data is transferred nightly to the VA Corporate Data Warehouse (CDW) where it resides in Microsoft SQL (MSSQL) databases for secondary use. Access to CDW data for research is managed by the VA Informatics and Computing Infrastructure (VINCI). To facilitate access to CDW systems the ProjectFlow application is hosted on virtual machines (VMs) within the VINCI environment. A generalized depiction of data flow for the ProjectFlow system is shown in Figure 1B.
Similar to our group’s insulin POCCT pilot16,17 we have further leveraged VHA EHR functionalities through the use of modified CPRS/VistA interfaces. However, this time we have applied modifications on a national level as opposed to just a single VHA site. To provide a specific example, the DCP study utilizes CPRS “View Alerts” to facilitate provider consent to patient enrollment and randomization (Figure 2). As these alerts are embedded within the CPRS interface, the data entered within them becomes part of the patient’s longitudinal health record and the official system of record.
Figure 2.
Integration with clinical interfaces, CPRS view alerts. (A) DCP utilizes CPRS “View Alert” screens to obtain provider consent to contact a patient (top panel) and (B) obtain provider consent to randomize consented patients (bottom panel). Providers may “Discontinue” or “Sign” these requests (red arrows). As the view alerts are embedded in CPRS the provider response also becomes part of the patient record. ProjectFlow queries and tracks these provider responses as they appear within the CDW.
ProjectFlow user dashboards and assigned task prioritization
Users interact with ProjectFlow by navigating within their dashboard (Figure 3). For example, Figure 3A shows what a user in the “Nurse” role would see after selecting the clinical element “Patient.” ProjectFlow also helps users manage activities by prioritizing tasks associated with each of their assigned roles. For example, the “Filter by Tasks” dropdown (Figure 3A2) lists the number of patient-related tasks that must be performed within each stage of a workflow. Prior to using ProjectFlow for study management, new staff undergo a 1.5-h training. Trainees must demonstrate a clear understanding of study workflows as well as how to navigate and manage their tasks using the application dashboards. For more details on dashboard functionalities, see the Figure 3 caption.
Figure 3.
ProjectFlow dashboard showing “Patient” clinical element views for the “Nurse” user/role. (A) Clinical elements appear at the top of the dashboard (red oval). In this example, a “Nurse” has already selected the element “Patient.” (A1) The “All” tab lists tasks (complete, incomplete or on hold) assigned to any user/role. The “Assigned To Me” tab displays only tasks the Nurse role may execute. The Nurse has the ability to “release” the task after which it would appear in the “Patient Tasks” tab which lists all unassigned patient tasks. (A2) The “Filter by Tasks” dropdown shows which tasks require action by the Nurse as well as the number of patients for which that task must be performed. (A3) Data associated with a given patient or task may be queried using the search fields. (A4) In order to complete a task, the Nurse clicks the arrow “>” to open the “Complete Task” view. (B1) The Complete Task view displays relevant patient data. (B2) For this particular task the Nurse must decide whether or not to cancel the patients existing prescription order. (B3) After data entry, the task is completed by hitting the “Next” button. (B4) To view the workflow in its entirety and see which stage of the workflow a patient is currently in; the Nurse may select “View Workflow.” (B5) The “History” panel lists completed workflow steps for the patient as well as which users completed them and when.
Pausing workflows and audit tracking
Clinical studies occasionally encounter unexpected situations that require additional discussion before proceeding. ProjectFlow allows clinical elements to be placed on “HOLD” by flagging them within a workflow to allow time for issue resolution (Figure 3A4, red circle). When an element is placed on hold, the user may enter questions/comments or even assign follow-up to a different role. The role performing the follow-up may choose to provide the necessary information or may instead remove the element from the workflow entirely. Furthermore, to assure research integrity, clinical studies must maintain a record of actions for longitudinal auditing purposes. ProjectFlow facilitates record maintenance by logging critical information for each clinical element. Specifically, each element’s entire path through a workflow is recorded, including but not limited to, the users performing the tasks, when each task is completed as well as all instances whereby an item was placed on hold or re-assigned to another user/role for completion.
Customized study workflows supported by ProjectFlow
Conceptual descriptions of the clinical study workflows supported by ProjectFlow for both RePOP and DCP are shown in Figure 4. As ProjectFlow accesses enterprise EHR data for provider and patient eligibility screening, both studies utilize it to automate and track study recruitment and enrollment.
Figure 4.
RePOP and DCP tasks managed and tracked by ProjectFlow. (A) RePOP workflows primarily support recruitment and enrollment (consenting) of patients. (B) DCP workflows not only support recruitment and enrollment but also study randomization and monitoring of prescription orders. Detailed descriptions of the workflows are provided in the main text.
RePOP workflows
RePOP recruits and enrolls patients interested in sharing their de-identified electronic health data with the Precision Oncology Data Repository (PODR) for the purpose of innovating (VA) cancer treatment7,8 To aid recruitment efforts, the ProjectFlow database has nightly extract transform load (ETL) processes that harvest the most recent available data for the RePOP cohort. The cohort consists of patients who, as part of their standard of care, have undergone or intend to undergo tumor-targeted genomic sequencing to further personalize their oncology treatment. Utilizing the “ProviderRolodex” workflow (Figure 4A, blue), providers are emailed study details and asked if their patients may be contacted to participate. If physician approval is granted, the “PatientConsent” workflow (Figure 4A, purple) is used to generate and manage postal mailings containing detailed study information, consent forms, and pre-paid return envelopes. The returned mailings with signed consents are tracked by study staff in ProjectFlow.
DCP workflows
DCP is a POCCT which evaluates the relative effectiveness of two widely prescribed thiazide-diuretics used in the treatment of hypertension, that is HCTZ and CTD. DCP utilizes existing enterprise EHR data for provider and patient eligibility screening and recruitment (Figure 4B, blue, orange). First, EHR data housed in CDW is queried to identify qualifying providers and their associated living patients currently treated with HCTZ for hypertension. Providers’ contact information is loaded into the trial DB and the ProjectFlow workflow “EmailProviders” is initiated (Figure 4B, blue). Providers are emailed study details and notified that they will receive a CPRS “View Alert,” as exemplified in Figure 2A, to confirm or decline participation in the DCP trial; this process is managed and tracked via the “Physician Consent” workflow (Figure 4, orange). Providers may give permission for DCP staff to contact their qualifying patients about enrolling in the trial. Provider response to the View Alert, that is “sign” to approve, “discontinue” to decline, is subsequently captured by ProjectFlow via daily CDW extracts.
For consenting providers, a “Patient Search” algorithm pulls contact information for their eligible patients into the trial database and ProjectFlow application. The study Call Center then telephones eligible patients to inform them about the study and ask if they would like to participate. This patient contact and consent process is scripted and tracked within the ProjectFlow “Call Center” workflow (Figure 4B, green).
Consented patients are processed through the “RandomizePatient” workflow (Figure 4B, purple). Where relevant, physicians receive a view alert to approve or decline patient randomization to CTHD (Figure 2B). Patients approved for randomization to CTHD are mailed detailed drug safety information. Those declined randomization are removed from the study eligibility list and sent a decision letter.
RESULTS
Current status and application usage
The ProjectFlow application supports multiple study management functions and both RePOP and DCP have used the application to track recruitment and enrollment of providers and/or patients. It should be noted that patient enrollment numbers are most often related to the length of time a VHA site (hospital) has been participating in the study and not site-specific effort or engagement. To protect site confidentiality and assure continued enrollment safety, we do not specify VHA hospital locations (city, state) nor site-specific enrollment/randomization numbers.
Research Precision Oncology Program
Since 2017 over 11 600 tasks have been executed in ProjectFlow by project managers, research nurses, and IT staff to track enrollment for RePOP at 41VHA hospitals in 29 geographically distributed US states. As of January 2021, it has enrolled >370 patients and evaluated >3800 potentially eligible patients at 40 VHA hospitals across the United States. With the intent of enabling translational research to benefit our Veterans, de-identified, longitudinal clinical, targeted tumor sequencing, and medical imaging (CT and pathology slide) data from consented RePOP patients is available to all researchers via the Veterans Precision Oncology Data Commons (VPODC).7,8
Diuretic Comparison Project
Since 2017, DCP has utilized ProjectFlow to conduct and track over >230 000 tasks. To do this the application has been accessed >15 000 times by call center staff, research nurses, project managers, and IT staff. It has also been used by the study call center to track phone calls made for recruitment as well as patient consent. As of January 2021, ProjectFlow has supported tracking of (and prescription ordering) for over 3500 providers and over 10 000 patients at 63 VHA hospitals across 39 geographically distributed US states.
DISCUSSION
In contrast to explanatory trials, where the supporting infrastructure from patient selection to data collection is optimized to evaluate treatment efficacy, a pragmatic trial’s infrastructure aspires to compare treatment effects with minimal interference in clinical practice; this often means relying on workflows that utilize the EHR. Indeed, merging research and clinical processes enables the use of longitudinal EHR data for study recruitment, enrollment, and monitoring. This can help reduce study costs and facilitate realization of a learning healthcare system through the direct translation of research evidence into clinical practice.4,5 To successfully implement a pragmatic trial as a point of care clinical trial using the EHR requires flexibility and ease of integration that are not readily available in most commercial and open-source trial management tools. Here, we have described our efforts with developing a software application that can support complex, customized workflows for POC-R task management while simultaneously integrating with VHA EHR infrastructures.
Initiated in 2016, the DCP study is one of VHA’s largest POCCTs to date. Like other lengthy studies, DCP has evolved since its inception, including ongoing adjustments to the study protocol and overarching design. ProjectFlow’s flexible workflow configuration has allowed us to adapt to these changing requirements and better manage more fluid tasks and successfully maintain continuity of study operations over-time. RePOP similarly has been able to iterate through protocol versions while the workflows in ProjectFlow were adapted accordingly. An example of this flexibility was a change from multi-level consent to one-level consent, which was accommodated by updating the variables in the consent steps to reflect the new consent form.
ProjectFlow’s use of rule-based access control has also proven valuable in unexpected ways. Both RePOP and DCP studies have spanned several years and naturally undergone staff turn-over. Creating “trainee” roles with limited task scope and data access has helped new personnel learn our overarching POC-R study processes and workflows before moving into more formal, “expert” roles with greater scope and access. Similarly, the task prioritization feature of ProjectFlow’s dashboards has not only helped experienced users manage their workload but increased new staff efficiency by serving as a reminder system that enhances learning. The application’s “Note” text-entry feature, within its “Hold” functionality, has allowed staff to pause workflows and communicate on an individual case basis when questions arise. This has been particularly useful for DCP, which enrolls thousands of patients, as it can reduce the need for separate email communication thereby helping maintain all necessary information within one auditing system.
In addition to customizable workflows, one of ProjectFlow’s most useful features is its ability to integrate seamlessly within the VHA EHR ecosystem. This minimizes disruption in clinical care while facilitating data management tasks. More specifically, for the DCP study, combined use of embedded CPRS View Alerts for data capture and CDW connections for data extraction greatly minimized the need for manual data entry by study staff and prescription monitoring via chart review. These features have also made ProjectFlow a useful recruitment tool for the VHA Integrating Pharmacogenetics in Clinical Care (I-PICC)18 trial aimed at assessing if genetic testing can effectively aid personalization of statin medications for cardiovascular disease treatment.
Limitations
ProjectFlow grants users significant flexibility in the types of workflows they may create. However, we have found that formal creation of BPMN workflows may require greater knowledge of database design and web-service protocols than the average trialist or clinical user may have. Thus, the overall maintenance and configuration of the application likely requires oversight by a software development team rather than being managed solely by trialists or clinicians. Additional work is required for reducing application complexity to allow management by the average user.
Furthermore, as ProjectFlow’s primary database is optimized to facilitate maintenance of the application itself, additional database structures designated for tracking and reporting study-specific outcomes on a project by project basis would be useful. More specifically, we have found that formal reporting may be complicated for clinical and reporting staff who are less familiar with the innerworkings of the application and workflows. Presently, to close this gap, database engineers create study-specific views and tables for reporting which may then be used by trialists for reporting.
CONCLUSION
POC-R aims to integrate research processes within day-to-day clinical operations and utilize patient EHR data for study recruitment, monitoring, and outcome reporting. The implementation of ProjectFlow to support both the DCP and RePOP studies afforded us significant flexibility to create customized study workflows for task management while simultaneously integrating with VHA enterprise EHR infrastructures. To date, ProjectFlow has facilitated management of study recruitment, enrollment, randomization, and drug orders for over 10 000 patients for the DCP clinical trial. It has also helped us evaluate over 3800 patients for recruitment and enroll over 370 of them into RePOP for use in data sharing partnerships and predictive analytics aimed at optimizing cancer treatment in the VHA. More recently, the application has also been used to support the VHA I-PICC trial.
FUNDING
Work for ProjectFlow development and implementation was supported by the Department of Veterans Affairs, Veterans Health Administration, Office of Research and Development, Cooperative Studies Program. All work was conducted under approved IRB protocols for CSP2010 and CSP597. The views expressed are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs nor the United States government.
AUTHOR CONTRIBUTIONS
Technical leadership for application development was provided by DE, NM, RA and RD. System development, quality assurance monitoring, development testing and database management was jointly performed by SLD, JF, QL, SS, RS, SD, SG, NM and TAF. All authors have contributed to aspects of this project.
ACKNOWLEDGMENTS
We thank Ryan Cornia, Brian Ivie, Brad Adams, Lalindra DeSilva for their contributions to software creation and Pat Woods, Maura Flynn, Christal Sadatis, Amanda Guski, Cynthia Hau, Karen Pierce-Murray, and Corri Dedomenico for their help in development testing.
Contributor Information
Rupali Dhond, VA Boston Healthcare System, Boston, Massachusetts, USA; Department of Medicine, Boston University School of Medicine, Boston, Massachusetts, USA.
Danne Elbers, VA Boston Healthcare System, Boston, Massachusetts, USA; University of Vermont, Burlington, Vermont, USA.
Nilla Majahalme, VA Boston Healthcare System, Boston, Massachusetts, USA.
Svitlana Dipietro, VA Boston Healthcare System, Boston, Massachusetts, USA.
Sergey Goryachev, VA Boston Healthcare System, Boston, Massachusetts, USA.
Ryan Acher, VA Boston Healthcare System, Boston, Massachusetts, USA.
Sarah Leatherman, VA Boston Healthcare System, Boston, Massachusetts, USA.
Tori Anglin-Foote, VA Salt Lake City Healthcare System, Salt Lake City, Utah, USA.
Qingzhu Liu, VA Salt Lake City Healthcare System, Salt Lake City, Utah, USA.
Shaoyu Su, VA Salt Lake City Healthcare System, Salt Lake City, Utah, USA.
Ramana Seerapu, VA Salt Lake City Healthcare System, Salt Lake City, Utah, USA.
Robert Hall, VA Boston Healthcare System, Boston, Massachusetts, USA.
Ryan Ferguson, VA Boston Healthcare System, Boston, Massachusetts, USA.
Mary T Brophy, VA Boston Healthcare System, Boston, Massachusetts, USA; Department of Medicine, Boston University School of Medicine, Boston, Massachusetts, USA.
Jeff Ferraro, VA Salt Lake City Healthcare System, Salt Lake City, Utah, USA.
Scott L DuVall, VA Salt Lake City Healthcare System, Salt Lake City, Utah, USA.
Nhan V Do, VA Boston Healthcare System, Boston, Massachusetts, USA; Department of Medicine, Boston University School of Medicine, Boston, Massachusetts, USA.
CONFLICT OF INTEREST STATEMENT
The authors have no competing interests to declare.
DATA AVAILABLILTY STATEMENT
Not applicable. The trials mentioned are ongoing and analysis of those data will be reported elsewhere.
REFERENCES
- 1.VHA. About Veterans Health Administration Veterans Health Administration. 2016. https://www.va.gov/health/aboutvha.asp. Accessed January 2020.
- 2.VAORD. Point of Care Research (POC-R). 2020. https://www.research.va.gov/programs/csp/point-of-care.cfm. Accessed January 2020.
- 3.Loudon K, Treweek S, Sullivan F, Donnan P, Thorpe KE, Zwarenstein M.. The PRECIS-2 tool: designing trials that are fit for purpose. BMJ 2015; 350: h2147. [DOI] [PubMed] [Google Scholar]
- 4.Staa TP, Goldacre B, Gulliford M, et al. Pragmatic randomised trials using routine electronic health records: putting them to the test. BMJ 2012; 344: e55. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Vickers AJ, Scardino PT.. The clinically-integrated randomized trial: proposed novel method for conducting large trials at low cost. Trials 2009; 10: 14. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Nourani A, Ayatollahi H, Dodaran MS.. Clinical trial data management software: a review of the technical features. Rev Recent Clin Trials 2019; 14 (3): 160–72. [DOI] [PubMed] [Google Scholar]
- 7.Do N, Grossman R, Feldman T, et al. The Veterans precision oncology data commons: transforming VA data into a national resource for research in precision oncology. Semin Oncol 2019; 46 (4–5): 314–20. [DOI] [PubMed] [Google Scholar]
- 8.Elbers DC, Fillmore NR, Sung FC, et al. The Veterans affairs precision oncology data repository, a clinical, genomic, and imaging research database. Patterns (N Y) 2020; 1 (6): 100083. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Ernst ME, Lund BC.. Renewed interest in chlorthalidone: evidence from the Veterans Health Administration. J Clin Hypertens (Greenwich) 2010; 12 (12): 927–34. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Ko R, Lee S, Lee EW.. Business process management (BPM) standards: a survey. Business Process Mgmt J 2009; 15 (5): 744–91. [Google Scholar]
- 11.White SA.New capabilities for process modelling in BPMN 2.0. In: Fischer L, ed. BPMN 20 Handbook. 2nd ed. Lighthouse Point, FL: Future Strategies Inc; 2012: 17–30. [Google Scholar]
- 12.Burnette E.Eclipse IDE Pocket Guide. Sebastopol, CA: O'Reilly; 2005.
- 13.Activiti. Activiti GitHub Repository. https://github.com/Activiti/Activiti. Accessed January 2020.
- 14.Judd CM, Nusairat JF, Shingler J.. Beginning Groovy and Grails from Novice to Professional. New York, NY USA: Apress; 2008: 441. [Google Scholar]
- 15.Brown SH, Lincoln MJ, Groen PJ, Kolodner RM.. VistA–U.S. Department of Veterans Affairs national-scale HIS. Int J Med Inform 2003; 69 (2–3): 135–56. [DOI] [PubMed] [Google Scholar]
- 16.D'Avolio L, Ferguson R, Goryachev S, et al. Implementation of the Department of Veterans Affairs' first point-of-care clinical trial. J Am Med Inform Assoc 2012; 19 (e1): e170–6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Fiore LD, Brophy M, Ferguson RE, et al. A point-of-care clinical trial comparing insulin administered using a sliding scale versus a weight-based regimen. Clin Trials 2011; 8 (2): 183–95. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Brunette CA, Miller SJ, Majahalme N, et al. Pragmatic trials in genomic medicine: the integrating pharmacogenetics in clinical care (I-PICC) study. Clin Transl Sci 2020; 13 (2): 381–90. [DOI] [PMC free article] [PubMed] [Google Scholar]