Abstract
Issues
A challenge in treatment research is the necessity of adhering to protocol and regulatory strictures while maintaining flexibility to meet patients’ treatment needs and accommodate variations among protocols. Another challenge is the acquisition of large amounts of data in an occasionally hectic environment, along with provision of seamless methods for exporting, mining, and querying the data.
Approach
We have automated several major functions of our outpatient treatment research clinic for studies in drug abuse and dependence. Here we describe three such specialized applications: the Automated Contingency Management (ACM) system for delivery of behavioral interventions, the Transactional Electronic Diary (TED) system for management of behavioral assessments, and the Protocol Workflow System (PWS) for computerized workflow automation and guidance of each participant’s daily clinic activities. These modules are integrated into our larger information system to enable data sharing in real time among authorized staff.
Key Findings
ACM and TED have each permitted us to conduct research that was not previously possible. In addition, the time to data analysis at the end of each study is substantially shorter. With the implementation of the PWS, we have been able to manage a research clinic with an 80-patient capacity having an annual average of 18,000 patient-visits and 7,300 urine collections with a research staff of five. Finally, automated data management has considerably enhanced our ability to monitor and summarize participant-safety data for research oversight.
Implications and conclusion
When developed in consultation with end users, automation in treatment-research clinics can enable more efficient operations, better communication among staff, and expansions in research methods.
Keywords: contingency management, ecological momentary assessment, protocol workflow, clinical trials, computer automation, clinical decision support system, substance abuse
Introduction
In addiction and most other fields of healthcare, treatment research is exceedingly complex, because the inherent complexity of providing care is combined with the strictures of systematic protocol implementation and data collection. For example, the need to follow a pre-established (and usually relatively rigid) research plan can conflict with the flexibility necessary to meet the patients’ treatment needs. Further complications are imposed by legal, regulatory, and organizational standards governing treatment and research. Many of these challenges can be eased by informatics and systems automation [1–3].
In this paper, we describe the application of informatics and automation of processes at the treatment research clinic we operate at the National Institute on Drug Abuse in Baltimore, Maryland, using an enterprise-level infrastructure (i.e. a very-large-scale computing system in which many users at different locations can simultaneously perform functions that can be centrally integrated). We needed a system having a high level of interoperability and communications with other external and internal systems. The architecture had to allow for, among other things, a robust departmental autonomy to preserve the “ownership” of research data by different investigators. The integration and interoperability of research-oriented systems and business-oriented enterprise systems, obviously, require an initial investment in areas such as business-process re-engineering, behavioral changes among the system users, technical or functional changes, and usability considerations. This investment is well justified by the system’s advantages, including increased quality of service, reduction in errors, resource savings, shortening of study participants’ visits, and acceptance by its intended users.
One category of challenges in addiction treatment research is the variability of protocol specifications and workflow management, such as introduction of new clinical procedures and data-collection forms and questionnaires: the system has to be flexible in terms of how and when data are collected and entered. Another category of challenges is the need for investigators, study physicians, counselors, and research technicians to have access to data (either archived data or live-updating data) in different formats at different times: the system has to be flexible in terms of how data are displayed and exported.
In this paper, we present our approach to these challenges and suggest areas for future consideration.
Methods and Results
We developed the Human Research Information System (HuRIS), an informatics infrastructure—i.e., an electronic environment for the collection, organization, and retrieval of information—that enables data and resource sharing in real time among authorized users at our clinic. Users on both the clinical side (e.g., nurses recording participants’ vital signs and doctors writing medication orders) and on the research side (e.g., clinical coordinators completing reporting requirements or downloading data for analysis) have access to the information on demand. At the core of this informatics infrastructure reside the clinical and research records of participants compiled over the entire history of their study participation, and sometimes over multiple studies. The computerized recording of participants’ information starts from the time of their initial consent for screening. Data collected by our intake personnel under a screening protocol become part of the participants’ clinical research records. This recording continues as participants are admitted to a clinical trial and persists throughout their progress within the prescribed activities until they are discharged. The electronic recording of participants’ activities enables the use of this information as a research resource to different groups at different locations, in current and future protocols, as permitted by human-subjects-protection regulations. HuRIS resides in a network that is self-contained (with no connection to the Internet), and an audit trail of changes is automatically maintained. User permissions to access various components of the system are centrally controlled and assigned using a highly flexible interface, and all access is logged.
Figure 1 schematically shows some of the modules implemented for our clinic’s automation. Because the modules are interoperable, new ones can be added when necessary. These modules are seamlessly integrated into HuRIS, thus leveraging the power of the wide variety of components native to HuRIS. Below we describe three of the modules: Automated Contingency Management (ACM) [4], the Transactional Electronic Diary (TED) [5, 6] and the Protocol Workflow System (PWS) [7]. We chose these three because they were specifically developed to address issues in our outpatient treatment research clinic.1
Figure 1.
Integration and interoperability of several applications for the automation in our outpatient research clinic. PWS – Protocol Workflow System; TED – Transactional Electronic Diary; ACM – Automated Contingency Management. Some of the connections among these three applications and various other automated systems inherent in HuRIS are also shown: AE – Adverse Event; MRV – Medical Record Viewer; TQS – Test and Questionnaire System; RX – Pharmacy; DSS – Decision Support System; DQE – Data Query/Export; LDME – Laboratory Data Management and Exchange; CRM – Clinical Recruiting Management system.
Automated Contingency Management (ACM)
Contingency management (CM) is a behavioral intervention that has been successfully used in drug-dependent patients [8]. In a typical implementation of CM, desired behaviors (such as drug abstinence or compliance with clinic rules) are monitored and reinforced via delivery of incentives such as vouchers with monetary values that escalate as the behaviors are uninterruptedly sustained.
Although CM is effective, it can be resource-intensive to administer and subject to human error. We therefore developed an Automated Contingency Management (ACM) system, as shown in Figure 2, which manages abstinence or adherence reinforcement by a combination of vouchers and prizes [9, 10]. ACM systematically assigns participants to stratified experimental groups, tracks participants’ progress and prize-selection history, and calculates earnings, thus reducing the influence of human error and increasing workflow efficiency. At each visit, a case-matching algorithm locates the central clinical record of the participant and provides a link to the relevant data, enabling ACM to access, in real time, information such as participants’ drug-test results in order to calculate reinforcers earned, and then to track the number and delivery of prize draws or the amount of voucher reinforcement. The prize-drawing system can accept input from manual drawings, but also provides a computerized drawing procedure that allows for transparent (i.e. invisible to the participant) variation of the probability of winning based on specific protocol requirements in a way not possible with manual drawing. This capability is important because studies of reinforcement parameters can help identify procedures that lower costs while preserving the efficacy of the treatment intervention [11]. All of our ACM module’s features, taken together, enhance clinical investigators’ confidence that treatment interventions are being implemented as intended and that accurate records are being maintained.
Figure 2.
Schematic diagram of the Automated Contingency Management (ACM) flow sequence. The reinforcements of adherence and abstinence refer to the reinforcement decisions on protocol compliance and drug abstinence, respectively.
ACM has allowed us to manage an annual average of 3,100 vouchers. Additionally, we have been able to conduct an annual average of 3,200 prize draws, of which 51% resulted in small prizes ($1–$5), 9% resulted in large prizes ($20) and 0.6% resulted in jumbo prizes ($100). The automated prize-recording system has allowed us to analyze data not previously available except after labor-intensive data entry. For example, we have found that drug use in the days after a given prize draw was influenced more by whether any material prize was won than by the size of the prize [12].
In keeping with the aims of contingency management, ACM is flexible enough to target nearly any desired behavior. In ongoing work, we are using ACM to target compliance with self-monitoring requirements in an electronic-diary study that uses the module described below.
Transactional Electronic Diary (TED)
Experience sampling is a research technique in which participants record their behaviors and moods in real time in their day-to-day environments [13–15]. In experience sampling, participants may be instructed to initiate a diary entry whenever prompted by a beep (random sampling) or whenever the participant experiences a specified mood or event (event-contingent sampling). Random sampling gathers data on base rates of daily events, while event-contingent sampling collects data on occasions of specified events. Ecological Momentary Assessment (EMA) is a refinement of experience sampling in which randomly timed assessments are combined with event-contingent assessments [16]. To our knowledge, EMA had never been attempted in a large sample of cocaine- and heroin-abusing participants prior to our implementation. EMA data on the precipitants of relapse to drug use, and on the time course of vulnerability to relapse, may lead to the development of more effective treatments.
For our EMA studies, we developed the handheld Transactional Electronic Diary (TED), which provides the means to ask a sequence of questions throughout the day in the research participant’s natural environment and to record the answers in a date- and time-stamped fashion. TED consists of the following three applications: a handheld program, as shown in Figure 3, for running the experiment; a handheld program for configuring user and system specific parameters; and a Windows program for reviewing the data collected by TED and subsequent importing and integration into HuRIS. TED is a highly configurable EMA program with branching capability, in which question presentation is contingent on participants’ answers to previous questions. Our prior clinical experience with standard questionnaires has shown that participants frequently become confused and fail to complete the questionnaire correctly when they are given branching questions on paper (e.g., “If you did not use drugs, skip to question 11”). To implement branching while using the most economical handheld devices, we developed a high-level specification and implemented a finite-state automaton that we have described in detail elsewhere [5].
Figure 3.
Example screens of questions and response options from random prompt and event contingent entries in the Transactional Electronic Diary (TED) program.
Using TED, we have collected an annual average of 15,000 real-time behavioral assessments, of which 10% represent event-contingent sampling and 90% random sampling. Preliminary data analyses have confirmed that the use of TED on PalmPilots with participants in methadone maintenance was highly successful; for example, we were able to recognize differential patterns of craving and use of cocaine and heroin [17], including patterns of craving that were differentially predictive of subsequent relapse [18]. Figure 4 shows the patterns of response to the "who," "what," "where" questions collected when participants made randomly prompted entries and event contingent entries (that is, when they had used or craved heroin or cocaine).
Figure 4.
Summary of results from three questions - Who are you with, Where are you now, What were you doing (just before making this entry) - collected using the Transactional Electronic Diary (TED) in 114 heroin/cocaine users receiving methadone maintenance therapy. Data shown are the percentage of entries endorsed for each response category in response to randomly timed prompts and on occasions of craving or using heroin or cocaine (event contingent entries).
Each of the two modules we have discussed above, TED and ACM, was designed to accommodate ongoing changes within and across research protocols. The third module we discuss below, our Protocol Workflow System (PWS), incorporates even greater flexibility and manages a broader range of activities.
Protocol Workflow System (PWS)
We developed the PWS in order to automate the workflow of our clinic. Each time a participant visits our research clinic, as shown in Figure 5, the PWS seamlessly guides his or her entire sequence of required activities, from admission desk through specimen collection, interviews, counseling appointments, and nurses’ station (typically for dispensing of medication). Upon completion of the first scheduled task, the clinic staff at the next station receive immediate notification that the task was completed, along with real-time access to the data associated with it. (When appropriate, these data can be displayed in the context of the participant’s older, stored data.) At most stations, this is done with desktop or laptop computers; at the station where observed urine specimens are collected, we use a large wall-mounted touch screen display, eliminating the need for a keyboard. The tracking and alerting system provided by our PWS results in very efficient collaboration among clinical staff.
Figure 5.
Activity diagram showing a hypothetical participant’s entire clinical workflow for a single visit; the workflow is seamlessly guided by the PWS. The shaded box shows a series of tasks one after another triggered by completion of the previous scheduled task.
For any given protocol, the initial setup of the PWS occurs a few weeks before the first participants are enrolled. The user interface for this setup, which we call the event-schedule interface, lets protocol administrators define and assemble the elements of treatment and research observations along with their scheduled times and frequencies. This event-schedule interface has been highly flexible, allowing automation of all of our existing protocols; it is also highly flexible in dealing with modifications to protocols as they run. We designed the PWS as a system with its specific rules externalized (that is, specified by the end user rather than built in); this approach keeps the behavior of the system stable from users’ perspectives even when the PWS application itself is periodically upgraded [19]; the same principle is used in commercial software in which user preferences are stored in files that are separate from the software itself.
After an individual participant is admitted to a protocol, his or her event schedule is generated automatically based on the workflow already specified in the PWS. The PWS also provides the flexibility to schedule events manually when warranted—for example, to reschedule missed clinic visits following inclement weather, or to schedule a counseling session in accordance with the counselor’s workload. For such scenarios, the PWS provides a user-friendly interface to modify a participant’s schedule within guidelines set forth by the protocol. Subsequent to the implementation of PWS, we have been successfully capable of managing a research clinic with an 80-patient capacity having an average of 18,000 patient-visits and 7,300 urine sample collections per year with a research staff of five.
Discussion
Based on our experience and on reports from our clinic staff, the use of informatics at our clinic appears to have enabled smoother clinical operations, more efficient communication among staff, and expansions in our research methods. Detailed usability analysis will be reported elsewhere, but we have found that fewer staff are needed to support the research program, and we have been able to relieve staff of tedious double data entry and data checking. In addition, the time to data analysis at the end of each study is substantially shorter because there is no delay in data availability. Finally, automated data collection, recording, storage, and retrieval have enhanced our ability to monitor and summarize participant-safety data and prepare reports for research oversight.
We can attribute our success to both behavioral [20] and technical [21] factors. The behavioral factors occurred on environmental, organizational, group, and staff levels. At the environmental level, we benefited from the affordability of computer hardware in general and PDAs (“Personal Digital Assistants,” handheld devices) in particular. At the organizational level, there existed a mandate for centralization as well as budget constraints rendering increased efficiency a high priority and not simply an option. At the group level, the geographical expansion of the various activities associated with our research protocols demanded a need for real-time and simultaneous access to the clinical and research data: hardcopy charts were no longer satisfactory. At the staff level, a computer-savvy generation of physicians and staff, highly motivated to use automated systems, significantly contributed to the success of this implementation.
The technical factors contributing to the success of this implementation included a consistent, user-friendly interface that shortened the learning curve for new users and for the adoption of newly released modules by current users. This intuitive interface also allowed for more efficient and faster navigation to the desired page for completion of the intended task, resulting in more tasks being completed in a shorter time. The availability of highly qualified computer scientists and programmers was another factor contributing to the success of our approach. Also, built-in controls and range checking allowed for preventing entry of some erroneous data. Various techniques were implemented to provide as much information as possible to the user in a single screen or to provide easy-to-use links and pointers while avoiding cluttered pages full of data.
Possible future applications of informatics at our clinic include: automated drug dispensing using our existing computerized dosage-calculation system to fill standing and special prescriptions; and greater use of decision-support systems, for example, to detect patterns in adverse events with an early-alert mechanism and to predict which participants are likely to drop out before protocol completion, which in turn would help predict enrollment needs.
Readers interested in automating their own research clinics should be aware that automation in healthcare settings (which often have much in common with clinical-research settings) can result in a variety of unintended adverse consequences, such as loss of procedural flexibility, or commission of errors that would not have been possible prior to automation [22][23][24]—and that such problems do not necessarily resolve over time [25]. The severity of such problems may largely depend on whether the system is developed in close consultation with its eventual users [26, 27]. In general, we have observed that nearly any degree of automation can enhance the efficiency of clinic operations and permit new areas of research; thus, even on limited budgets, many research clinics may benefit from automating to the extent that they can.
Acknowledgements
This research was supported by the Intramural Research Program of the NIH, National Institute on Drug Abuse. The authors gratefully acknowledge graphic support by András Frenyo in the preparation of this paper.
Footnotes
Interested readers can contact the first author for more information on obtaining copies of TED. They can also request more technical information that would assist them in implementing similar processes. Plans are underway to make the ACM and PWS stand-alone applications such that an interested reader could adopt the systems into their operations.
References
- 1.Chung TK, Kukafka R, Johnson SB. Reengineering clinical research with informatics. J Investig Med. 2006;54(6):327–333. doi: 10.2310/6650.2006.06014. [DOI] [PubMed] [Google Scholar]
- 2.Sung NS, Crowley WF, Jr, Genel M, Salber P, Sandy L, Sherwood LM, et al. Central challenges facing the national clinical research enterprise. JAMA. 2003;289(10):1278–1287. doi: 10.1001/jama.289.10.1278. [DOI] [PubMed] [Google Scholar]
- 3.Zerhouni EA. Clinical research at a crossroads: the NIH roadmap. J Investig Med. 2006;54(4):171–173. doi: 10.2310/6650.2006.X0016. [DOI] [PubMed] [Google Scholar]
- 4.Vahabzadeh M, Lin J-L, Epstein DH, Mezghanni M, Schmittner J, Preston KL. Computerized contingency management for motivating behavior change: automated tracking and dynamic reward reinforcement management; Proc. 20th IEEE International Symposium on Computer-Based Medical Systems (CBMS 2007); 2007. pp. 85–90. [Google Scholar]
- 5.Lin J-L, Vahabzadeh M, Mezghanni M, Epstein DH, Preston KL. A high-level specification for adaptive ecological momentary assessment: real-time assessment of drug craving, use and abstinence. AMIA Annu Symp Proc. 2005:455–459. [PMC free article] [PubMed] [Google Scholar]
- 6.Vahabzadeh M, Epstein D, Mezghanni M, Lin J-L, Preston K. An electronic diary software for ecological momentary assessment (EMA) in clinical trials; Proc. 17th IEEE Symposium on Computer-Based Medical Systems (CBMS 2004); 2004. pp. 167–172. [Google Scholar]
- 7.Vahabzadeh M, Lin J-L, Gupman A, Schmittner J, Preston K. Automating variations in clinical research protocols workflow; Proc. 17th IEEE Symposium on Computer-Based Medical Systems (CBMS 2004); 2004. pp. 248–253. [Google Scholar]
- 8.Petry NM. A comprehensive guide to the application of contingency management procedures in clinical settings. Drug Alcohol Depend. 2000;58(1–2):9–25. doi: 10.1016/s0376-8716(99)00071-x. [DOI] [PubMed] [Google Scholar]
- 9.Peirce JM, Petry NM, Stitzer ML, Blaine J, Kellogg S, Satterfield F, et al. Effects of lower-cost incentives on stimulant abstinence in methadone maintenance treatment: a National Drug Abuse Treatment Clinical Trials Network study. Arch Gen Psychiatry. 2006;63(2):201–208. doi: 10.1001/archpsyc.63.2.201. [DOI] [PubMed] [Google Scholar]
- 10.Petry NM, Martin B, Simcic F., Jr Prize reinforcement contingency management for cocaine dependence: integration with group therapy in a methadone clinic. J Consult Clin Psychol. 2005;73(2):354–359. doi: 10.1037/0022-006X.73.2.354. [DOI] [PubMed] [Google Scholar]
- 11.Ghitza UE, Epstein DH, Schmittner J, Vahabzadeh M, Lin J-L, Preston KL. Randomized trial of prize-based reinforcement density for simultaneous abstinence from cocaine and heroin. J Consult Clin Psychol. 2007;75(5):765–774. doi: 10.1037/0022-006X.75.5.765. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Ghitza UE, Epstein DH, Schmittner J, Vahabzadeh M, Lin J-L, Preston KL. Effect of reinforcement probability and prize size on cocaine and heroin abstinence in prize-based contingency management. Journal of Applied Behavior Analysis. doi: 10.1901/jaba.2008.41-539. in press. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Csikszentmihalyi M, Larson R. Validity and reliability of the experience-sampling method. Journal of Nervous and Mental Disease. 1987;175(9):526–536. doi: 10.1097/00005053-198709000-00004. [DOI] [PubMed] [Google Scholar]
- 14.Hormuth SE. The sample of experiences in situ. Journal of Personality. 1986;54(1):262–293. [Google Scholar]
- 15.Wheeler L, Reis HT. Self-recording of everyday life events: origins, types, and uses. Journal of Personality. 1991;59(3):343–354. [Google Scholar]
- 16.Stone AA, Shiffman S, Atienza AA, Nebeling L, editors. The science of real-time data capture. New York: Oxford University Press; 2007. [Google Scholar]
- 17.Preston KL, Willner-Reid J, Lin J-L, Vahabzadeh M, Epstein DH. Daily and weekly patterns in heroin/cocaine craving and use. American Psychological Association Annual Convention; August, 2008; Boston, MA: 2008. [Google Scholar]
- 18.Epstein DH, Preston KL, Schmittner J. 114 cocaine and heroin abusers, 226 PalmPilots: initial experiences with Ecological Momentary Assessment at a methadone clinic. 69th Annual Scientific Meeting of the College on Problems of Drug Dependence; Quebec City, Canada. 2007. [Google Scholar]
- 19.Stead WW, Miller RA, Musen MA, Hersh WR. Integration and beyond: linking information from disparate sources and into workflow. J Am Med Inform Assoc. 2000;7(2):135–145. doi: 10.1136/jamia.2000.0070135. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Kukafka R, Johnson SB, Linfante A, Allegrante JP. Grounding a new information technology implementation framework in behavioral science: a systematic analysis of the literature on IT use. J Biomed Inform. 2003;36(3):218–227. doi: 10.1016/j.jbi.2003.09.002. [DOI] [PubMed] [Google Scholar]
- 21.Hornbæk K. Current practice in measuring usability: challenges to usability studies and research. International Journal of Human-Computer Studies. 2006;64(2):79–102. [Google Scholar]
- 22.Campbell EM, Sittig DF, Ash JS, Guappone KP, Dykstra RH. Types of unintended consequences related to computerized provider order entry. J Am Med Inform Assoc. 2006;13(5):547–556. doi: 10.1197/jamia.M2042. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Weiner JP, Kfuri T, Chan K, Fowles JB. "e-Iatrogenesis": the most critical unintended consequence of CPOE and other HIT. J Am Med Inform Assoc. 2007;14(3):387–388. doi: 10.1197/jamia.M2338. discussion 389. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Campbell EM, Sittig DF, Ash JS, Guappone KP, Dykstra RH. In reply to: "e-Iatrogenesis: The most critical consequence of CPOE and other HIT". J Am Med Inform Assoc. 2007 [Google Scholar]
- 25.Ash JS, Sittig DF, Poon EG, Guappone K, Campbell E, Dykstra RH. The extent and importance of unintended consequences related to computerized provider order entry. J Am Med Inform Assoc. 2007;14(4):415–423. doi: 10.1197/jamia.M2373. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Ovretveit J, Scott T, Rundall TG, Shortell SM, Brommels M. Implementation of electronic medical records in hospitals: two case studies. Health Policy. 2007;84(2–3):181–190. doi: 10.1016/j.healthpol.2007.05.013. [DOI] [PubMed] [Google Scholar]
- 27.Scott JT, Rundall TG, Vogt TM, Hsu J. Kaiser Permanente's experience of implementing an electronic medical record: a qualitative study. Bmj. 2005;331(7528):1313–1316. doi: 10.1136/bmj.38638.497477.68. [DOI] [PMC free article] [PubMed] [Google Scholar]