Abstract
Background
In a highly publicized joint report, the National Academy of Engineering and the Institute of Medicine recently recommended the systematic application of systems engineering approaches for reforming our health care delivery system. For this to happen, medical professionals and managers need to understand and appreciate the power that systems engineering concepts and tools can bring to redesigning and improving health care environments and practices.
Objective
To present and discuss fundamental concepts and tools of systems engineering and important parallels between systems engineering, health services, and implementation research as it pertains to the care of complex patients.
Design
An exploratory, qualitative review of systems engineering concepts and overview of ongoing applications of these concepts in the areas of hemodialysis, radiation therapy, and patient flow modeling.
Results
In this paper, we describe systems engineering as the process of identifying the system of interest, choosing appropriate performance measures, selecting the best modeling tool, studying model properties and behavior under a variety of scenarios, and making design and operational decisions for implementation.
Conclusions
We discuss challenges and opportunities for bringing people with systems engineering skills into health care.
KEY WORDS: health care engineering, patient modeling, systems approach, systems engineering, health services research
INTRODUCTION
The U.S. health care delivery system is poorly prepared to meet the growing health care needs of its population.1 Current limitations result in unexplained practice variations, gaps between evidence and practice, inequitable patterns of utilization, unsustainable cost increases, and poor safety.2–4 Health care costs consume a growing proportion of the economy, leaving the public, insurers, industries, and government straining under a financial burden. Unnecessary services are provided far too often because there is little coordination across sites or among providers,5 yet care management, crossdisciplinary care, and preventive care are often uncovered or poorly reimbursed. Notably, 45% of the U.S. population have chronic conditions requiring care management. Of this population, 60 million, or roughly half of those with chronic conditions, have multiple conditions.6 Current care delivery systems are not designed to support the care of these complex patients, which requires multiple providers and services.1
A recently published joint report from the National Academy of Engineering (NAE) and Institute of Medicine (IOM) advocated the widespread application of systems engineering tools to improve health care delivery.7 Systems engineering focuses on coordination, synchronization, and integration of complex systems of personnel, information, materials, and financial resources.8 This is achieved through the application of mathematical modeling and analysis techniques. Over the past 30 years, the continuing development and application of systems engineering methods has enabled the unprecedented growth in the manufacturing, logistics, distribution, and transportation sectors of our economy.7 Although drawing direct parallels between other economic sectors and health care delivery is problematic, many functions common to both have been significantly improved in other sectors through engineering analysis. These include inventory control and logistics, scheduling, operations management, project planning, facilities design, process flow analysis, resource synchronization, engineering economic analysis, and many others.7 We believe that these engineering approaches, properly modified and applied, can provide similar high-level impacts in health care delivery. Thus, our objective here is to provide an overview of key systems engineering concepts and methods and explore some illustrations of how these are being used to improve health care delivery.
SYSTEMS ENGINEERING CONCEPTS
Systems engineering focuses on the design, control, and orchestration of system activities to meet performance objectives. Before discussing systems engineering tools and applications, we provide some basic definitions and key concepts.
Systems Definition and Operational Behavior
A system is a set of possibly diverse entities (patients, nurses, physicians, etc.), each performing some set of functions. The interaction of these entities as they perform their various functions gives rise to a global system behavior.
System State, Resources, and Customers
The state of a system is an instantaneous snapshot of its status. For example, the state of a medical/surgical unit in a hospital might indicate that room 210 is occupied by patient 0901, wheelchair 11 is idle, and nurse 21 is on break. Thus, room 210 is allocated to a patient, wheelchair 11 is available for allocation, and nurse 21 is not currently available. Many levels of resolution are possible in specifying the information content of the system state.
System Operation, Event Occurrence, and State Evolution
As a system operates, it moves from one state to the next through the occurrence of enabled events. An event is enabled when the preconditions for its occurrence are met. An event occurs when its associated actions are performed. For example, if an ICU has 30 beds, with 29 occupied and one available, and there is a patient in the emergency department requesting an ICU bed, then an ICU bed allocation event is enabled and can be performed. This event is performed when the bed is assigned to the patient. If another patient arrives needing an ICU bed, the ICU bed allocation event is not enabled because a bed is not available.
The sequence of states that the system traverses over some time horizon is referred to as the state trace of the system. There are many system state traces that might possibly evolve and because of the presence of uncontrollable events such as walk-in or ambulance arrivals, the future state trace is not always predictable or controllable.
System Performance Measures
From a systems engineering perspective, a performance measure is a statistic computed from information in a given system trace. Examples of performance measures could be number of patients waiting for at least 30 minutes, the number of diabetic patients who received an A1C blood test, or number of ICU beds available during peak hours. System state traces that yield desirable performance measures are preferable to those that do not.
Event-based Systems
Health care delivery systems are examples of event-based systems. The states of these systems evolve with the occurrence of enabled events. The occurrence of events often initiates work actions or physical processes (such as “deliver radiation treatment”) that consume time. Typically, the time interval between consecutive events appears random.
Systems Modeling
Systems modeling is the activity of identifying the most relevant system characteristics and representing them in a mathematical model. The model is then analyzed to learn about and improve the behavior of the original system. This process is significantly different from the hypothesis-based clinical trial mode of research prevalent in medical research.
With this background, we now describe the process a systems engineer undertakes in a health care improvement project. There are six fundamental steps, which are conducted iteratively. Each of the six will be briefly described.
Define system purpose and scope, specify required functions and resource types, and develop relevant performance measures along with desired performance thresholds.
Specify, collect, and develop required data through data collection methods.
Design, validate, and verify appropriate system models. This involves selecting the right modeling tools, building and validating the model.
Use the model to learn about system behavior to find the best design alternative. The engineer often develops appropriate experiments for the studying the model and analyzing the results.
Use the results of step 4 to determine how to configure the system for best performance. This involves specifying equipment requirements, staffing levels and patterns, scheduling procedures, workflows, and so forth. Sensitivity analysis is also important to determine how system performance will be affected by perturbations to nominal conditions.
Develop implementation and evaluation plans and coordinate their performance.
With these six steps in mind, some of the most important engineering methods are listed below with a brief explanation. Many of these originate from the discipline of Industrial Engineering, which is often synonymous with Systems Engineering.
Project management models include project evaluation and review technique (PERT) and critical path method (CPM) techniques. These models capture task dependencies and timing in the execution of a large project.9 For example, Endress et al.10 uses the CPM to analyze patient and work flow in an operating room environment.
Engineering economics and financial engineering models are used to make cost-effective decisions on capital investments and portfolio optimization.11,12 These methods are useful in all steps. As an example, Steenstra et al.13 uses economic analysis techniques to analyze the effectiveness of a multistage “return to work” program for workers with low back pain.
Statistical modeling is used to capture relationships, patterns, correlations, and probabilistic structure in data. In systems engineering, statistical methods are essential for input modeling and analysis in step 2, for any required experimental design and analysis in step 4, and for many quality control applications where a performance or quality characteristic can be monitored and controlled over time. Specific techniques include regression, design of experiments, and statistical quality control.14–16 As an example, Parachoor et al.17 uses statistical process control to benchmark hospital performance indicators against peer organizations.
Stochastic processes model the random nature of complex systems and processes. These types of models can be used to derive expected values and variances of performance measures under a variety of conditions. They can also be used to develop optimal decision policies, that is, decision-making rules that optimize the expected system performance while minimizing risk. Queuing models, Markov chains, Brownian motion, and Markov decision processes (MDP) are common types of stochastic models.18 They are usually applied in steps 3 and 4. Hauskrecht and Fraser19 apply MDP to the treatment of heart disease.
Operations research models are well suited for optimal resource allocation, determining how to cost effectively distribute resources. Operations research methods also provide the analytical foundation for important systems applications such as patient flow, inventory control, and scheduling. Common techniques include linear programming, network flow analysis, and dynamic programming.20 These models are most effective in steps 3–5. As an example, Brennan et al.21 models pricing options, discount rates, and organizational structure in regional health information organizations.
Human factors models can be used to optimize human performance in complex systems. These models can capture both cognitive and ergonomic concerns and include the area of human computer interactions.22 These models support steps 3–5. As an example, Koubek et al.23 provides a framework for process usability assessment.
Process flow models capture how work tasks need to be sequenced, coordinated, and synchronized. Interleaving of the work processes of system components allows the modeling of resource competition and delays.24 These methods support steps 1–5. As an example, Gupta et al.25 uses flow modeling for capacity planning in cardiac catheterization.
Discrete event simulation is perhaps the most commonly applied systems engineering tool. In essence, these models mimic system behavior in accelerated time. Simulation models can easily capture a plethora of operational detail and are easily used in experimentation.26,27 Discrete-event simulation is very useful for steps 3–5. Hung et al.28 use simulation to study patient flow in a pediatric emergency department.
For additional discussion of systems engineering tools in improving health care delivery, we refer the reader to the NAE/IOM report7 and to the recently published Handbook of Operations Research/Management Science Applications in Health Care, edited by Brandeau et al.29 For more general applications and differing perspectives on systems engineering, we refer the reader to the INCOSE website at http://www.incose.org/.
EXAMPLE SYSTEMS ENGINEERING APPLICATIONSTO HEALTH CARE DELIVERY
We now provide two examples of how the tools discussed above are being applied in health care. The first focuses on medical decision making whereas the second focuses on hospital management.
Therapeutic Optimization
Therapeutic optimization models the social and clinical aspects of an individual patient, treatment options, and relevant environmental factors to customize a patient’s treatment. These models might use the patient’s age, physical mobility, comorbidities, social support, and so forth in selecting the optimal treatment. We now briefly discuss several examples.
Hemodialysis is a method for removing waste products from the blood stream of renal failure patients. The procedure requires an arteriovenous (AV) fistula/graft, which joins an artery and vein. Maintaining AV access is a significant problem for many hemodialysis patients because of progressive stenosis. Mean patency (state of being unblocked) of a native fistula is about 3 years and approximately 18 months for a synthetic graft. Studies show that early stenosis detection and treatment intervention (e.g., angioplasty or surgery) can successfully reduce the chances of thrombosis.30,31 Stenosis increases the pressure in the access by reducing its diameter, eventually rendering it unusable. Figure 1 illustrates the growth of the venous access pressure ratio (VAPR) with increasing stenosis before and after intervention. AV patency can be modeled as an optimal stopping time problem (a common problem in stochastic processes) in which, the pressure in the access represents the state of the system. Decisions include when to intervene with treatment, when to prepare a new AV site, and when to abandon the current site and move to the next, all with the objective of optimizing patency and expected patient life.
Figure 1.
Sample data for VAPR
Therapeutic optimization is being used in liver transplantation.32 When a liver becomes available, the transplant surgeon can either accept or reject it based on liver condition, patient compatibility, and waiting list characteristics. This decision can be modeled using Markov decision processes (MDP). These models can be solved to find an optimal decision rule that maximizes the expected transplant success. The MDP approach is also useful for timing liver transplants from living donors to maximize patient life expectancy.33 Schaefer et al.34 review of MDP approaches for treatment modeling.
Therapeutic optimization is being used to optimize radiation therapy in cancer treatment. In this treatment, a specified amount of radiation needs to be applied in a localized cancerous region. To achieve this, the radiation must pass through healthy tissue, which it can damage. Radiation treatment technologies have evolved significantly and are now highly computerized so that very fine beam modulation is possible. Optimization models have been developed for controlling the modulation mechanism so that the cancerous region receives the prescribed dose while collateral damage to surrounding health tissue is minimized.35–37 The methods being applied here include mix-integer and network flow programming from operations research.
Therapeutic optimization has also been applied to model kidney allocation problems,38 HIV treatment,39 seizure warnings,40 and vaccine protocols.41
Hospital Operations Modeling
Large hospitals are highly complex systems that are poorly understood, extremely costly, and rife with inefficiency. Because of this complexity, there are no detailed models that capture the overall operation of these systems from a systems engineering perspective. The authors of this paper are partnering with health providers to develop event-based (Petri net42,43) models of acute care hospitals. The objective is to analyze decision policies for bed allocation, patient transfer and discharge, and staff scheduling to help reduce costs and improve quality of care.
This type of work requires (1) the definition of the hospital state, that is, the instantaneous state of hospital resources, patients, and staff; (2) the events that move the hospital from one state to the next; (3) projections on how patients will continue to evolve through their care plans; (4) how patients following care plans load hospital resources; and (5) surgical schedules and projections capturing how patients arrive at the hospital. Along with this, technology must be in place to automatically monitor and update system state by tracking patients and key resources.
At our partner’s facility, digital displays are coupled with radiofrequency identification and other sensor and data entry technologies to provide real-time patient tracking and bed allocation information at a glance. Displays are mounted in strategic positions within the hospital and in a centralized control room where a bed controller focuses on managing patient flow. Up-to-date information on where patients are, how long they have been there, whether they are waiting, and, if so, how long, is displayed and logged. Departmental state sequences or traces are automatically saved to one or more databases as patients are admitted, discharged, and transported throughout the system.
Much of the trace data are in the form of HL7 messaging (a health care information systems message protocol44) between the hospital information systems and the tracking software. For each patient admitted, the information systems generate a sequence of HL7 messages for postmedication orders, lab requests, transfer and transportation orders, and so forth. In fact, the set of HL7 messages generated for a given patient represents much of what happens to the patient during the patient’s stay. Because patient care plans for the individual patient are rarely formally recorded, as such, they tend to evolve with the patient stay, and exist in a piece-meal fashion in the minds of physicians, nurses, and discharge planners. One objective of this research is to make the care plans explicit so they can be used for modeling hospital processes, workload planning, and quality improvement. Figure 2 provides the reconstructed care paths of two cardiac patients both requiring percutaneous cardiovascular procedures. Their care paths were reconstructed from HL7 messages, although in the interest of space, only the most significant events are illustrated. By comparing treatment paths of these two patients diagnosed under the same diagnosis-related group (DRG), it is straightforward to identify the differences and similarities in their treatments.
Figure 2.
Sample results for patient even traces for DRG 558
If patient care plans can be successfully reconstructed from HL7 messaging, then the following benefits are likely. It will be possible to very rapidly develop representative care plans for classes of patients, which capture and reflect what is actually happening to patients in the facility. The resource needs of representative care plans can then be established, that is, for each patient of a given type, it will be possible to use that patient’s representative care plan to estimate the workload that the patient imposes on the system. One very important element of the hospital state can be modeled: The set of patients currently admitted and their residual care plans (what they have left to do). This will enable the development of operational models that can provide insight into the short-term behavior of the hospital, whether certain resources will be overloaded in the short-term, or whether the hospital is likely to go on a divert status. These models will then support experimentation with new decision policies.
A sample Petri net model of a generic emergency department is shown in Figure 3. The model explicitly represents the event enabling and state transition structure of the system. Such models support significant systems analysis activities. For example, the model can be used to determine where the ED is most resource-constrained and the impact of altering resource levels. It can also be used to explicitly define divert states and then, for any nondivert state, compute the number and sequence of events that must occur to reach the nearest divert state. Finally, the model can be used to identify circular resource dependencies that can cause difficulties in system operation.
Figure 3.
Sample Petri net model of a generic emergency department
Currently, we are developing similar models of all hospital departments. We will then merge these models into one hospital model that will capture resource dependencies that propagate across the system. It will then be possible to explicitly see how discharge policies affect daily operations in the ED and OR, and how ED congestion and OR scheduling policies affect medical surgical wards, ICU, and lab facilities over the short-term.
IMPEDIMENTS TO SYSTEMS ENGINEERING APPROACHES IN HEALTH CARE
Although systems engineering holds great promise for improving health care delivery, there are significant challenges impeding its acceptance. The NAE/IOM report7 discusses the following challenges, which help lay the groundwork for implementation.
From the examples discussed above, it is clear that systems engineering techniques have extensive data requirements. The management of this data requires expensive integrated information systems that are often not available in the health care sector.
Reimbursement practices and regulations provide little incentive for investing in quality and systems improvement. Thus, managerial support for systems improvement can be difficult to obtain. Indeed, there seems to be little understanding of the characteristics that health care organizations need to foster to enable adoption of systems engineering techniques and solutions. One possible approach is to use appreciative inquiry to help uncover enablers and motivators that will help systems engineering methods gain wider acceptance.45
Health care has a culture of rigid division of labor. This functional compartmentalization does not optimally support the application of tools that transcend and span functional areas, especially when they induce significant changes in traditional relationships. Furthermore, because engineering professionals have not traditionally had a presence in health care delivery, the uncertainty associated with the role and status of another functional area can spawn skepticism and fear. This underscores the importance of considering social and communication issues in the evaluation and implementation of any sociotechnical system.46
Very few health care providers are trained to think analytically about how health care delivery should function. Thus, it is often difficult for these professionals to appreciate the contributions that systems engineering approaches can bring. Conversely, engineering professionals often have little, if any, education in health care delivery.
Fortunately, universities, professional organizations, government agencies, and philanthropic foundations are beginning to act. In the future, we believe these organizations will provide leadership, training, and funding opportunities.
SUMMARY AND CONCLUSION
In this paper, we described systems engineering as the process of identifying the system of interest, choosing appropriate performance measures, selecting the best modeling tool, studying model properties and behavior under a variety of scenarios, and making design and operational decisions for implementation. We covered the basic concepts and tools of systems engineering and provided examples of ongoing work in their application to complex patients. Potential impediments were also discussed.
In conclusion, systems engineering approaches have been instrumental in coordinating the growth, operation, and synchronization of many information-rich and technologically complex economic sectors, most notably manufacturing, transportation, and supply chain logistics. Whereas we are excited about the future of health care engineering and anticipate its ultimate success in helping to reengineer health care delivery, we believe widespread success will only come when a critical mass of health care organizations recognize its value through concrete examples. Only then will these organizations promote the organizational changes needed for its adoption.
ACKNOWLEDGMENTS
This research was partially supported by funding from the Department of Veterans Affairs, Veterans Health Administration, and also partially supported by HSRD Center grant no. HFP 04-148. The authors also appreciate the comments of attendees of the VA State of the Art Conference on Complexity, the comments of the SOTA and SGIM reviewers, and comments from Steven Witz, PhD. The opinions expressed here are those of the authors and do not necessarily reflect those of the Veterans Health Administration.
Conflict of Interest None disclosed.
REFERENCES
- 1.Starfield B. Deficiencies in US medical care. JAMA. 2000;284(17):2184–5. [DOI] [PubMed]
- 2.McGlynn E, Asch S, Adams J, et al. The quality of health care delivered to adults in the United States. N Engl J Med. 2003;348(26):2635–45. [DOI] [PubMed]
- 3.American College of Physicians. The Advanced Medical Home: a Patient-centered, Physician-guided Model of Health Care. United States: American College of Physicians Monograph; 2005.
- 4.Kohn L, Corrigan J, Donaldson M. To Err is Human Building a Safer Health System Committee on Quality of Health Care in America. Washington DC: Institute of Medicine National Academy Press; 1999.
- 5.Murphy J, Chang H, Montgomery J, Rogers W, Safran D. The quality of physician–patient relationships: patients experiences, 1996–1999. J Fam Pract. 2001;50:123–9. [PubMed]
- 6.Wu S, Green A. A Projection of Chronic Illness Prevalence and Cost Inflation. Santa Monica, Calif: RAND Health; 2000.
- 7.Proctor P, Compton WD, Grossman J, Fanjiang G. Building a Better Delivery System: a New Engineering/Health Care Partnership, Committee on Engineering and the Health Care System. Washington DC: National Academy of Engineering and Institute of Medicine, National Academy Press; 2005.
- 8.Kossiakoff A, Sweet W. Systems Engineering Principles and Practice. New York: Wiley; 2003.
- 9.Kerzner H. Project Management: A Systems Approach to Planning, Scheduling, and Controlling, 8th edn. New Jersey: Wiley; 2003.
- 10.Endress A, Aydeniz B, Wallwiener D, Kurek R. The critical path method to analyze and modify OR-workflow: integration of an image documentation system. Minim Invasive Ther Allied Technol. 2006;15(3):177–86 [DOI] [PubMed]
- 11.Park C. Fundamentals of Engineering Economics. New Jersey: Prentice-Hall; 2003.
- 12.Neftci S. Principles of Financial Engineering. New York: Academic Press; 2004.
- 13.Steenstra I, Anema J, van Tulder M, Bongers P, de Vet H, van Mechelen W. Economic evaluation of a multi-stage return to work program for workers on sick-leave due to low back pain. J Occup Rehabil. 2006;16(4):557–78. [DOI] [PubMed]
- 14.Ross S. Introductory Statistics, 2nd edn. New York: Academic Press; 2005.
- 15.Montgomery D. Design and Analysis of Experiments, 6th edn. New Jersey: Wiley; 2004.
- 16.Montgomery D, Peck E, Vining G. Introduction to Linear Regression Analysis, 3rd edn. New Jersey: Wiley-Interscience; 2001.
- 17.Parachoor S, Rosow E, Enderle A. Knowledge management system for benchmarking performance indicators using statistical process control (SPC) and virtual instrumentation (VI). Biomed Sci Instrum. 2003;39:175–8. [PubMed]
- 18.Durrett R. Essentials of Stochastic Processes. New York: Springer; 2001.
- 19.Hauskrecht M, Fraser H. Planning treatment of ischemic heart disease with partially observable Markov decision processes. Artif Intell Med. 2000;18(3):221–44. Mar [DOI] [PubMed]
- 20.Rardin R. Optimization in Operations Research. New Jersey: Prentice-Hall; 1997.
- 21.Brennan P, Ferris M, Robinson S, Wright S, Marquard J. Modeling participation in the NHII: operations research approach. AMIA Annu Symp Proc. 2005;76–80. [PMC free article] [PubMed]
- 22.Lehto M, Buck J, Buck M. Introduction to Human Factors and Ergonomics for Engineers. Hillsdale, NJ: Lawrence Erlbaum Associates; 2007.
- 23.Koubek R, Benysh D, Buck M, Harvey C, Reynolds M. The development of a theoretic framework and design tool for process usability assessment. Ergonomics 2003;46(1–3):220–41. [DOI] [PubMed]
- 24.vanderAalst W, vanHee K. Workflow Management: Models, Methods, and Systems. Cambridge, Mass: The MIT Press; 2004.
- 25.Gupta D, Natarajan M, Gafni A, Wang L, Shilton D, Holder D, Yusuf S. Capacity planning for cardiac catheterization: a case study. Health Policy. 2007;82(1):1–11. [DOI] [PubMed]
- 26.Law A, Kelton D. Simulation Modeling and Analysis, 3rd edn. London: McGraw-Hill; 1999.
- 27.Kelton D, Sadowski R, Sturrock D. Simulation with Arena, 4th edn. London: McGraw-Hill; 2007.
- 28.Hung G, Whitehouse S, O’Neill C, Gray A, Kissoon N. Computer modeling of patient flow in a pediatric emergency department using discrete event simulation. Pediatr Emerg Care 2007;23(1):5–10. [DOI] [PubMed]
- 29.Brandeau M, Sainfort F, Pierskalla W, eds. Handbook of Operations Research/Management Science Applications in Health Care. Dordrecht: Kluwer Academic Publishers; 2004.
- 30.Sullivan KL, Besarab A, Bonn J, Shapiro MJ, Gardiner GA, Moritz MJ.Hemodynamics of Failing Dialysis Grafts. J Radiol. 1993;186(3):867–72. [DOI] [PubMed]
- 31.Kaye M, Baird C, McCloskey B, Oscar G, D’Avirro M. Two years sequential hemodynamic data on polytetrafluoroethylene (PTFE) grafts used for hemodialysis. Proc Eur Dial Transplant Assoc. 1979;16:266–71 [PubMed]
- 32.Alagoz O, Maillart L, Schaefer A, Roberts, M. Determining the acceptance of cadaveric livers using an implicit model of the waiting list. Oper Res. 2007;55(1):24–36. [DOI]
- 33.Alagoz O, Maillart L, Schaefer A, Roberts M. The optimal timing of living-donor liver transplantation. Manage Sci. 2004;50(10):1420–30. [DOI]
- 34.Schaefer A, Bailey M, Shechter S, Roberts M. Medical decisions using Markov decision processes. In: Sainfort F, Brandeau M, Pierskalla W, eds. Handbook of Operations Research/Management Science Applications in Health Care. Dordrecht: Kluwer Academic Publishers, 2004;597–616.
- 35.Shepard D, Ferris C, Olivera G, Mackie T. Optimizing the delivery of radiation therapy to cancer patients. SIAM Rev. 1999;41(4):721–44. [DOI]
- 36.Rardin R, Preciado-Walters F, Langer F, Thai V. Column generation for IMRT cancer therapy optimization with implementable segments. Ann Oper Res. 2006;148(1):65–79. [DOI]
- 37.Rardin R, Preciado-Walters F, Langer F, Thai V. A coupled column generation, mixed-integer approach to optimal planning of intensity modulated radiation therapy for cancer. Math Program. 2004;101:319–38.
- 38.Su X, Zenios S. Patient choice in kidney allocation: a sequential stochastic assignment model. Oper Res. 2005;53(3):443–55. [DOI]
- 39.Jeffrey A, Xia X, Craig I. When to initiate HIV therapy: a control theoretic approach. IEEE Trans Biomed Eng. 2003;50(11):1213–20. [DOI] [PubMed]
- 40.Pardalos P, Chaovalitwongse W, Iasemidis L, et al. Seizure warning algorithm based on optimization and nonlinear dynamics. Math Program Ser B. 2004;101:365–85. [DOI]
- 41.Wu J, Wein L, Perelson A. Optimization of influenza vaccine selection. Oper Res. 2005;53(3):456–76. [DOI]
- 42.Peterson J. Petri Net Theory and the Modeling of Systems. New Jersey: Prentice-Hall; 1981.
- 43.Haas P. Stochastic Petri Nets Modeling, Stability, Simulation. New York: Springer; 2002.
- 44.American National Standards Institute. HL7 Version 3 Normative Edition, Health Level Seven. United States: American National Standards Institute; 2006.
- 45.Cooperrider D, Whitney D. Appreciative Inquiry. San Francisco, CA: Berrett-Koehler Communications; 1999.
- 46.Goldstein M, Coleman R, Tu S, et al.Translating research into practice: sociotechnical integration of automated decision support for hypertension in three medical centers. J Am Med Inform Assoc. 2004;11(5):368–76. [DOI] [PMC free article] [PubMed]