Abstract
Objective: Home telemonitoring improves clinical outcomes but can generate large amounts of data. Automating data surveillance with clinical decision support could reduce the impact of translating these systems to clinical settings. We utilized time-motion methodology to measure the time spent on activities monitoring subjects in the two groups of a home spirometry telemonitoring randomized controlled trial: the manual nurse review (control) group and the automated review (intervention) group. These results are examined for potential workflow effects that could occur when the intervention translates to a clinical setting. Materials and Methods: Time motion is an established industrial engineering technique used to evaluate workflow by measuring the time of predefined, discrete tasks. Data were collected via direct observation of two research nurses by a single observer using the repetitive or snap-back timing method. All observed tasks were coded using a list of work activities defined and validated in an earlier study. Reliability data were collected during a 2-h session with a secondary observer. Results: Reliability of the primary observer was established. During 35 h of data collection, a sample of 938 task observations were recorded and coded using 46 previously defined and 5 newly defined work activities. Between-group comparisons of activity time for subjects in the two study groups showed significantly more time spent on data review activities for the automated review group. Reclassification of the 51 observed activities identified 15 activities that would translate to a clinical setting, of which 5 represent potentially new activities. Conclusions: Implementing an intervention into a clinical setting could add work activities to the clinical workflow. Time-motion study of research personnel working with new clinical interventions provides a template for evaluating the workflow impact of these interventions prior to translation from a research to a clinical setting.
Keywords: home health monitoring, telehealth, e-health
Introduction
Home telemonitoring has been advocated as a way of managing chronic conditions through frequent data collection and transmission to health providers.1,2 A recent meta-analysis concluded that although telehealth interventions, including home telemonitoring, positively affected clinical outcomes, additional research on the clinical and cost effectiveness is needed.3 This is especially pertinent in the context of home telemonitoring systems, which generate large amounts of clinical data. Coupling automated decision support with these systems allows increased surveillance without corresponding increases in personnel.4 Successful translation of home telemonitoring clinical decision support (CDS) from the research to clinical setting requires understanding the impact of increased surveillance on the timing and volume of clinical work activities.
Time-motion study is an established methodology used to evaluate the efficiency of work processes. The principles of time-motion study evolved from the scientific management principles of Frederick W. Taylor5 and the industrial engineering techniques of Frank and Lillian Gilbreth.6 Developed in the early 20th century to increase the productivity of repetitive and standardized manufacturing tasks, time-motion studies systematically analyze work processes through direct observation and timing.
The breadth of time-motion studies indicates its growing acceptance as a reliable tool to evaluate the workflow of healthcare delivery. Time-motion studies have been used to make recommendations for improving the efficiency of endoscopic7 and dermatologic procedures,8 as a basis for determining the personnel time and cost of a research intervention,9 to understand the workflow of intensive care remote monitoring10 and to evaluate the impact of computerized physician order entry on clinician time.11 The present time-motion study measured the workflow in a home spirometry telemonitoring randomized controlled trial (RCT) and investigated whether differences exist in the time spent on work activities related to the control and the intervention groups. The information was then used to explore the potential work impact on clinical personnel, of translating the telemonitoring intervention to a clinical setting.
Materials and Methods
Setting
This time-motion study was conducted in the home spirometry telemonitoring research program at the University of Minnesota. An RCT of lung transplant recipients to validate the efficacy of home spirometry CDS against the current standard of care, home spirometry manual nurse review, is underway. Development and validation of the home spirometry telemonitoring protocol were previously reported.4
To understand the purpose and results of this time-motion study, a description of the home spirometry telemonitoring RCT protocol is provided. Lung transplant recipients enrolled in the RCT were randomized into the control (manual nurse review) or the intervention (CDS review) group. The research protocol required blinding of all clinical personnel, which included physicians, transplant coordinators, and respiratory therapists, to subject group assignment. Transplant coordinators manage the ongoing clinical care of all lung transplant recipients and, although not part of the RCT, interact frequently with the research nurses. Specifically, transplant coordinators receive pulmonary function alerts for the lung transplant recipients participating in both the control and the intervention groups. The RCT protocol specified that two research nurses manage the recruitment, adherence, data collection, and monitoring of all subjects in the two-armed trial.
All RCT subjects conducted home spirometry and transmitted pulmonary function and respiratory symptom data weekly from their home using a commercially available device (Home Spirometer; Trans Viva, Minneapolis, MN). These transmissions were automatically formatted into the home spirometry monitoring (HSM) report and sent electronically to the research nurses and the home spirometry CDS system. The nurses manually analyzed the HSM report for control group subjects (n = 26) and notated “Alert” on the report when predetermined pulmonary function and symptom criteria were detected. The CDS system, using a Bayesian predictive model, analyzed the HSM report for intervention group subjects (n = 28) and generated an e-mail “Alert” to the research nurses when pulmonary function and symptom criteria were detected.12 To maintain transplant coordinator blinding, alerts for subjects in both study groups were manually generated and delivered by the research nurses, which included paper copies of home spirometry data and graphical representations of past spirometry data. The transplant coordinators used these data to determine whether additional follow-up was required; research nurses did not determine or provide direct patient care.
Data Collection
This time-motion study was approved by the University of Minnesota Institutional Review Board. Two masters-prepared nurses with 3 or more years of project experience shared responsibilities on the previously described home spirometry telemonitoring RCT. These research nurses were observed by a health informatics doctoral candidate with a nursing background (R.C.). A single observer was used to minimize the variability of time measurements. During the time-motion data collection period, 54 home spirometry RCT subjects transmitted weekly data. All work activities of the research nurses were directly observed and measured, with the exception of subject's clinic visits; during this task, the observer remained in the waiting area. Activities unrelated to the research nurses work, such as breaks and lunch, were recorded but not included in the analysis. With the exception of a bimonthly project team meeting, only one of the two research nurses worked at any given time.
Data were collected for 9 days over a 2-week period using the repetitive or snap-back timing method.5 At the start of each data collection session, the observer coded the research nurse being watched, noted the clock time, and “snapped-back” a digital stop-clock to zero. As the research nurse moved from one task to another, the stop-clock was snapped-back to zero and three pieces of information were coded: the elapsed time from the stop-clock, the activity code (or description) for the task conducted, and the subject identification number for tasks performed for a specific subject. These observations comprised the sample used in this time-motion study.
The activity codes used in this study were previously defined and validated.13 Each activity code delineated a discrete task and similar activities were grouped into categories (e.g., investigator e-mail and clinical staff e-mail grouped into e-mail category) (Table 1). A detailed description was provided for tasks that did not correspond to a predefined activity code. For tasks involving multiple subjects (e.g., printing the HSM report) or no subjects (e.g., working on quarterly newsletter), the subject identification number was left blank.
Table 1.
Total Time of Observation Sample by Activity Code: Reclassification of Observation Sample
| Category/Activity Code | Description | Activity Typea | Total Timeb (min) | Category/Activity Code | Description | Activity Typea | Total Timeb (min) |
|---|---|---|---|---|---|---|---|
| A | Data review | E | Mail/letters | ||||
| A.1 | Spirometry report | Cc | 36.8 (2%) | E.1 | Mail or write/edit letters | R | 55.84 (3%) |
| A.2 | Alert preparation | R | 29.92 (1%) | E.2 | Birthday/special event cards | R | 26.36 (1%) |
| B | Computer | F | Telephone calls | ||||
| B.1 | Weekly summary | R | 76.96 (4%) | F.1 | Call primary physician office for events | R | 45.3 (2%) |
| B.2 | Review spreadsheet | R | 26.1 (1%) | F.2 | Call member of research team | R | 17.06 (1%) |
| B.3 | Update spreadsheet | R | 154.84 (7%) | F.3 | Call transplant coordinators | R | 3.86 (0%) |
| B.4 | Chart progress note | R | 66.7 (3%) | F.4 | Subject adherence call | Cc | 36.12 (2%) |
| B.5 | Print schedule/graphs | R | 34.36 (2%) | F.5 | Review voicemail | C | 5.58 (0%) |
| B.6 | Alert follow-up | R | 0.56 (0%) | F.6 | Call to the office | R | 8.34 (0%) |
| B.7 | Log event into database | R | 7 (0%) | F.7 | Subject check-in call | C | 13.44 (1%) |
| B.8 | Review subject data in the electronic medical record | C | 92.94 (4%) | F.8 | Learning center calls | Cc | 1.54 (0%) |
| B.9 | Check clinic schedule in the electronic medical record | R | 12.14 (1%) | F.9 | Phone call to TIS | R | 4.44 (0%) |
| B.10 | Send alert to transplant coordinator | R | 60.16 (3%) | G | Verbal face-to-face | ||
| B.11 | Review progress notes | R | 17.88 (1%) | G.1 | Investigators | R | 210.26 (10%) |
| B.12 | Update recruitment flow diagram | R | 23.58 (1%) | G.2 | Research nurses | R | 101.8 (5%) |
| B.13 | Document interaction with TIS | R | 2.76 (0%) | G.3 | Discussion with coordinators | R | 17.52 (1%) |
| B.14 | Review SF36 data from TIS | R | 21.66 (1%) | G.4 | Interact with physician | C | 0.5 (0%) |
| C | H | Other | |||||
| C.1 | Request graphs | Cc | 18.84 (1%) | H.1 | Files/forms on new subjects | C | 0.78 (0%) |
| C.2 | Investigator e-mail | R | 93.36 (4%) | H.2 | Research team meeting | R | 93.34 (4%) |
| C.3 | Clinical staff e-mail | R | 6.74 (0%) | H.3 | Newsletter | R | 143.8 (7%) |
| C.4 | TIS e-mail | R | 1.86 (0%) | H.4 | Make copies | C | 17.8 (1%) |
| D | Clinic | H.5 | Review paper information | R | 45.26 (2%) | ||
| D.1 | Review subject records | C | 42.16 (2%) | I | Organizational | ||
| D.2 | Interact directly with subject | C | 82.2 (4%) | I.1 | Gather equipment and supplies | Cc | 0.76 (0%) |
| D.3 | Deliver graphs | R | 9.74 (0%) | I.2 | Planning and processing | R | 69.02 (3%) |
| D.4 | Review pulmonary function tests with respiratory therapist | C | 3.64 (0%) | I.3 | Filing (electronic/paper) | R | 50 (2%) |
| D.5 | Waiting | R | 13.96 (1%) | I.4 | Gathering paperwork | C | 17.46 (1%) |
| D.6 | Walk to or from clinic | R | 145.76 (7%) | J | Nonspecific activity | R |
R, research; C, clinical.
Values include all activities observed, regardless of whether or not conducted for a participant.
Potential new work activities for clinical transplant coordinator.
TIS, transplant information services; SF36, a health survey.
Reliability
Observer reliability was assessed during a 2-h observation session using activity codes and times collected by the primary observer and a secondary nurse observer (R.L.). The secondary observer was a doctoral-prepared nurse researcher who collected time-motion data for an earlier study and was familiar with the research nurse activities. Both observers used the data collection method described earlier.
Analysis
Statistical analyses were conducted using SAS software version 9.1 (SAS Institute, Cary, NC). Interobserver agreement and reliability were determined using the kappa statistic and intraclass correlations. Between-group comparisons of research nurse time for subject-specific activities in each of the 10 categories were analyzed using the Wilcoxon rank sum test. Summary statistics were calculated to describe the task frequency among the categories for each study group.
Results
Observer Reliability
Both observers watched one research nurse perform 46 tasks during a 2-h observation session. The observers recorded the same time for 9 tasks and differed by < 1 min for each of the remaining 37 tasks (1 s for 10 tasks, 2–9 s for 21 tasks, 11–19 s for 5 tasks, 45 s for 1 task). The intraclass correlation was 0.99. Observer agreement in categorizing the research nurse tasks into predefined activities was strong (45/46), with a kappa statistic of 0.97 (95% CI = 0.91–1.00). The timing and activity coding of the primary observer were determined reliable in the recording of study observations.
Observation Sample
A total of 938 observations were recorded during 35 h of data collection conducted over 9 days. The observations were coded using 46 activities defined in a previous study13 and 5 new activities. The new activities resulted from recent changes to the home spirometry telemonitoring protocol and were placed into the computer (3), e-mail (1), and telephone (1) categories. The nonspecific activity category captured activities that consume nurses' time but did not correspond to existing work categories. For example, printer malfunction observations originally coded as print activities were recoded as nonspecific activities.
A total of 51 activities were used to code the sample of 938 observations. The total time spent on each of the 51 coded activities is shown in Table 1 and the total time spent on each category of activities is shown in Table 2. Although the computer category accounted for almost half the activities conducted, they comprise only a quarter of the observed time. The data review category displays a similar trend: it accounted for 12% of the activities but consumed only 3% of observed time. Both categories contain frequently occurring activities that are performed quickly. The clinic and other categories displayed a reverse trend. Activities in the other category (e.g., research team meetings, quarterly newsletter, filing/copying) account for only 3% of total activities but consume 14% of the observed time. A small number of activities in the clinic category (e.g., subject clinic visits) accounted for 14% of the observed time.
Table 2.
Frequency and Total Time of Observation Sample by Category
| Category | Frequency of Observationsa | Total Time (min) of Observationsa | Time (min) Median (min/max) |
|---|---|---|---|
| Computer | 402 (43%) | 597.64 (28%) | 0.83 (0.03, 21.67) |
| Data review | 114 (12%) | 66.72 (3%) | 0.37 (0.05, 2.78) |
| Verbal face-to-face | 97 (10%) | 330.06 (16%) | 2.00 (0.17, 19.45) |
| Organizational | 93 (10%) | 137.24 (7%) | 1.12 (0.23, 8.35) |
| 59 (6%) | 120.78 (6%) | 1.83 (0.42, 8.30) | |
| Phone calls | 55 (6%) | 135.66 (6%) | 2.12 (0.18, 7.83) |
| Clinic | 48 (5%) | 297.46 (14%) | 4.12 (0.17, 33.72) |
| Mail/letters | 25 (3%) | 82.18 (4%) | 2.38 (1.13, 9.22) |
| Other | 28 (3%) | 300.96 (14%) | 6.18 (0.67, 46.67) |
| Nonspecific activity | 17 (2%) | 31.56 (2%) | 1.23 (0.15, 9.27) |
Includes all observations from the 2-week data collection period.
During data collection, subject identification numbers were noted when activities were performed for a specific subject and left blank for activities involving multiple or no subjects. Activities conducted for specific subjects accounted for 34% of the observed time and was nearly equally divided between subjects in the control (16%) and the intervention (18%) groups. Activities conducted for multiple or no subjects accounted for 66% of the observed time and included writing and disseminating the quarterly newsletter, attending research team meetings, communicating with research team members, and engaging in organizational tasks (e.g., filing and making copies).
Between-Group Comparison
Between-group comparisons of the time spent on subject-specific activities were performed at the category level because of inadequate sample size at the activity level. These comparisons yielded only one significant result: data review for the intervention group required significantly more time than data review for the control group (Table 3). When the individual activities in the data review category were analyzed, differences were not significant. This most likely resulted from the low frequency of the two data review activities. Summary statistics for the frequency and the rate of observations per subject (Table 3) reveal little difference in the activities performed for the two study groups.
Table 3.
Relative Frequency and Time of Observation Sample by Category and Study Group
| Category | Study Groupa | Frequency of Observationsb | Rate of Observationsc | Time (min) of Observationsc | Time (min) Of Observations Median (min/max) | p-VALUEd |
|---|---|---|---|---|---|---|
| All categories | I | 251 | 4.83 | 7.06 | 0.75 (0.07, 33.72) | 0.506 |
| C | 261 | 4.66 | 6.07 | 0.70 (0.05, 16.00) | ||
| Data review | I | 59 | 1.13 | 0.73 | 0.45 (0.08, 2.78) | 0.028e |
| C | 51 | 0.91 | 0.42 | 0.30 (0.05, 2.33) | ||
| Computer | I | 144 | 2.77 | 2.99 | 0.68 (0.07, 7.70) | 0.951 |
| C | 157 | 2.80 | 3.14 | 0.67 (0.07, 5.73) | ||
| I | 0 | 0.00 | — | — | — | |
| C | 4 | 0.07 | 0.13 | 1.74 (1.47, 2.13) | ||
| Clinic | I | 12 | 0.23 | 1.61 | 3.42 (1.00, 33.72) | 0.487 |
| C | 8 | 0.14 | 0.99 | 4.08 (2.05, 16.00) | ||
| Mail/letters | I | 8 | 0.15 | 0.59 | 2.27 (1.35, 7.80) | 1.000 |
| C | 10 | 0.18 | 0.48 | 2.78 (1.20, 3.70) | ||
| Phone calls | I | 16 | 0.31 | 0.91 | 2.05 (0.58, 7.83) | 0.925 |
| C | 16 | 0.29 | 0.70 | 2.38 (0.38, 5.05) | ||
| Verbal face-to-face | I | 0 | 0.00 | — | — | — |
| C | 4 | 0.07 | 0.07 | 0.49 (0.30, 2.47) | ||
| Other | I | 2 | 0.04 | 0.04 | 1.09 (0.85, 1.33) | 0.540 |
| C | 1 | 0.02 | 0.01 | 0.80 (0.80, 0.80) | ||
| Organizational | I | 7 | 0.13 | 0.14 | 0.85 (0.52, 1.67) | 0.353 |
| C | 6 | 0.11 | 0.10 | 0.73 (0.23, 2.50) | ||
| Nonspecific activity | I | 3 | 0.06 | 0.04 | 0.57 (0.22, 1.45) | 0.377 |
| C | 4 | 0.07 | 0.02 | 0.38 (0.15, 0.45) |
I, intervention group (clinical decision support review); C, control group (manual nurse review).
Only observations conducted for a specific participant are included in this frequency. Participant-specific activities account for 34% of all activities conducted during the time-motion study.
Columns are adjusted by the total number of participants randomized to each group (I = 26, C = 28) and are reported per participant per week.
Nonparametric Wilcoxon rank sum test was used.
A p-value of <0.05 was considered statistically significant.
Reclassification of Activities
The observed activities were reclassified as either research or clinical, to identify activities required for the translation of home spirometry CDS from a research to a clinical setting. Research activities (e.g., data collection and management) were needed to support the research protocol and would not translate to clinical care. Clinical activities (e.g., promoting home spirometry adherence and managing home spirometry equipment) were needed to support the home spirometry CDS intervention and would translate to clinical care. Of the 51 activities observed during this study, 36 were reclassified as research and accounted for 82% of the observed time. The 15 activities reclassified as clinical accounted for only 18% of observed time. Within the reclassified clinical activities, five represent potentially new work activities (Table 1).
Discussion
The implementation of new technology into clinical workflow can trigger positive and negative unintended consequences,14–16 encourage workarounds,17 and potentially increase worker frustration18 and opportunities for new errors.19 Prior time-motion studies have quantified the impact of technology interventions on clinician workflow,11,20,21 but to the best of our knowledge, the methodology has not been used to analyze whether differences exist in the time spent on activities for subjects in a home spirometry CDS group and subjects in a manual review group. This analysis is important from an informatics perspective because home spirometry can generate large amounts of data that require frequent analysis.4 In our study, subject group assignment impacted the time for data review activities, with significantly more time spent on these activities for the intervention group subjects. This finding was unexpected because CDS automates data review functions for the intervention group.
The research protocol of the home spirometry telemonitoring RCT stipulated that all clinical personnel managing subjects care are blinded to group assignment. Consequently, this blinding produced an unintended consequence. Each week, the research nurses manually analyzed the HSM report for all subjects, noting whether an alert was warranted, and then compared the home spirometry CDS alerts with their own analysis of intervention subjects' data. When disagreement occurred, the subject's electronic medical record was reviewed and discussed with the principal investigator to understand the reasoning behind the CDS alert; the nurse did not interfere with or override the CDS decision. This detailed review was necessary to confidently answer questions from blinded clinical staff about the manually generated pulmonary function alerts for both the control and the intervention subjects and it helps explain why significantly more time was spent reviewing data for the intervention group. Although this difference was statistically significant, the amount of time, per subject, per week, spent on data review tasks for the intervention group subjects was very small (0.73 min) and likely not of clinical significance or importance.
The activities observed during this time-motion study ranged from research protocol management to supporting clinical-type tasks such as subject adherence. Before home spirometry CDS is translated to a clinical setting, it is critical to understand the additional activities of the personnel interacting with the system. In our setting, transplant coordinators are responsible for the ongoing management of lung transplant recipients and currently receive the home spirometry RCT alerts. Clinical-type activities performed by the research nurses, such as managing home spirometry equipment, scheduling home spirometry training sessions, monitoring adherence to home spirometry, and requesting/reviewing home spirometry data, would translate into new work activities for the transplant coordinators. These activities accounted for 5% of the total time-motion observation time.
Translation of the CDS system requires automatic generation of the manually prepared alerts. Although automating alert generation does not affect clinical team activities, organizational informatics resources must be deployed in the translational effort. The automated alerts would presumably provide information similar to the manually generated home spirometry telemonitoring RCT alerts. Enrolling all currently managed lung transplant recipients in home spirometry would yield greater alert generation.
There may be several limitations to this time-motion study. The sequential nature of data collection over a 2-week period may not capture all activities performed by research nurses, especially infrequent and periodic activities. Increasing the length of data collection would increase the sample size of observations and provide a better estimate of activity time. Shadowing the research nurses, which is typical to time-motion study, could cause abnormal activity patterns, but did not occur in this study. Because of the small incidence of some activities, the frequency of tasks between the two groups was evaluated using summary statistics. It is also not clear how the results might have been affected if greater numbers of alerts were generated or if the study had been undertaken in multiple transplant centers. This study does not evaluate the productivity of the observed research nurses. The results of this study are limited to the clinical setting where the RCT was conducted and we do not know if they generalize to other lung transplant settings. However, the use of time-motion methodology to measure the impact of research interventions prior to clinical translation could be generalized to other settings.
Conclusions
This time-motion study compared the frequency and the time of activities conducted for the control and the intervention groups of a lung transplant home spirometry telemonitoring research study. Between-group comparisons of time yielded only one significant result: more time was spent on activities in the data review category for the intervention (CDS) subjects. This difference can be attributed to an unintended consequence of clinical staff blinding to group randomization. Before sending the manual alerts, research nurses must verify their understanding of why a CDS alert was generated so they can knowledgably answer the clinical staff questions on the intervention subjects. There were no significant time differences among the other categories.
We caution against concluding from these results that home spirometry CDS will increase clinical staff workload. Prior lung transplant home spirometry telemonitoring research provides strong evidence that the additional workload could be offset by improved patient outcomes. Home spirometry has yielded earlier detection of pulmonary decline than clinic spirometry and is a reliable and safe alternative.22 Postlung transplant survival has corresponded with increased early adherence to the home telemonitoring protocol.23
Before home spirometry CDS is translated from a research setting to a clinical setting, it is critical to identify the additional work activities of personnel interacting with the system and measure the time-intensity of these activities. Time-motion study of the home spirometry research nurses and reclassification of observed work activities as either clinical or research provide a preliminary understanding of additional clinical workload. Although the majority of these activities are research-type activities, several new clinical-type activities were identified. This time-motion study of research personnel provides a template for evaluating the workflow impact of a research intervention before translation to a clinical setting.
Although this study did not reveal savings attributable to the automated data review, it is anticipated that home spirometry CDS would be “time neutral” or, in contrast to our results, even positive, if the activities needed to preserve blinding were eliminated. Better understanding the time and therefore the personnel costs of utilizing home spirometry CDS in clinical settings requires time-motion studies of actual clinical implementation. This type of information is fundamental to the evaluation of new technology and assists in understanding the impact of the technology on clinical practice, staffing requirements and patterns, and, ultimately, operational revenues.
Acknowledgments
This research was supported by grants from NIH (no. R01 NR009212) and NLM (no. LM07041).
Disclosure Statement
No competing financial interests exist.
References
- 1.Hersh WR. Helfand M. Wallace JA. Kraemer DF. Patterson PK. Shapiro S. Greenlick MR. Clinical outcomes resulting from telemedicine interventions: A systematic review. BMC Med Inf Decis Making. 2001;1:5. doi: 10.1186/1472-6947-1-5. . DOI:10.1186/1472-6947-1-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Hersh WR. Wallace JA. Patterson PK. Kraemer DF. Nichol WP. Greenlick MR. Krages KP. Helfand M. Telemedicine for the Medicare population: Pediatric, obstetric, and clinician-indirect home interventions. Evidence Rep Technol Assess (Summ) 2001;24(Suppl):1–32. [PMC free article] [PubMed] [Google Scholar]
- 3.Dellifraine JL. Dansky KH. Home-based telehealth: A review and meta-analysis. J Telemed Telecare. 2008;14:62–66. doi: 10.1258/jtt.2007.070709. [DOI] [PubMed] [Google Scholar]
- 4.Finkelstein SM. Scudiero A. Lindgren B. Snyder M. Hertz MI. Decision support for the triage of lung transplant recipients on the basis of home-monitoring spirometry and symptom reporting. Heart Lung. 2005;34:201–208. doi: 10.1016/j.hrtlng.2004.09.003. [DOI] [PubMed] [Google Scholar]
- 5.Barnes RM. Motion and Time Study. New York: Wiley; 1958. [Google Scholar]
- 6.Mundel ME. Motion and Time Study: Principles and Practice. New York: Prentice-Hall; 1950. [Google Scholar]
- 7.Harewood GC. Chrysostomou K. Himy N. Leong WL. A “time-and-motion” study of endoscopic practice: Strategies to enhance efficiency. Gastrointest Endosc. 2008;68:1043–1050. doi: 10.1016/j.gie.2008.03.1116. [DOI] [PubMed] [Google Scholar]
- 8.Fisher J. Lotery H. Henderson C. Time in motion—Testing efficiency in the dermatology procedure setting. Dermatol Surg. 2009;35:437–444. doi: 10.1111/j.1524-4725.2009.01076.x. [DOI] [PubMed] [Google Scholar]
- 9.Findorff MJ. Wyman JF. Croghan CF. Nyman JA. Use of time studies for determining intervention costs. Nurs Res. 2005;54:280–284. doi: 10.1097/00006199-200507000-00011. [DOI] [PubMed] [Google Scholar]
- 10.Tang Z. Weavind L. Mazabob J. Thomas EJ. Chu-Weininger MY. Johnson TR. Workflow in intensive care unit remote monitoring: A time-and-motion study. Crit Care Med. 2007;35:2057–2063. doi: 10.1097/01.ccm.0000281516.84767.96. [DOI] [PubMed] [Google Scholar]
- 11.Yen K. Shane EL. Pawar SS. Schwendel ND. Zimmanck RJ. Gorelick MH. Time motion study in a pediatric emergency department before and after computer physician order entry. Ann Emerg Med. 2009;53:462–468e. doi: 10.1016/j.annemergmed.2008.09.018. .1. [DOI] [PubMed] [Google Scholar]
- 12.Troiani JS. Carlin BP. Comparison of Bayesian, classical, and heuristic approaches in identifying acute disease events in lung transplant recipients. Stat Med. 2004;23:803–824. doi: 10.1002/sim.1651. [DOI] [PubMed] [Google Scholar]
- 13.Harrington K. Lindquist R. Van Wormer A. Lindgren B. Finkelstein SM. Analysis of nursing workflow in a home monitoring study. Telemed e-Health. 2009;15:106–106. [Google Scholar]
- 14.Ash JS. Berg M. Coiera E. Some unintended consequences of information technology in health care: The nature of patient care information system-related errors. J Am Med Inform Assoc. 2004;11:104–112. doi: 10.1197/jamia.M1471. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Ash JS. Sittig DF. Poon EG. Guappone K. Campbell E. Dykstra RH. The extent and importance of unintended consequences related to computerized provider order entry. J Am Med Inform Assoc. 2007;14:415–423. doi: 10.1197/jamia.M2373. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Harrison MI. Koppel R. Bar-Lev S. Unintended consequences of information technologies in health care—An interactive sociotechnical analysis. J Am Med Inform Assoc. 2007;14:542–549. doi: 10.1197/jamia.M2384. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Koppel R. Wetterneck T. Telles JL. Karsh BT. Workarounds to barcode medication administration systems: Their occurrences, causes, and threats to patient safety. J Am Med Inform Assoc. 2008;15:408–423. doi: 10.1197/jamia.M2616. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Sittig DF. Ash JS. Guappone KP. Campbell EM. Dykstra RH. Assessing the anticipated consequences of computer-based provider order entry at three community hospitals using an open-ended, semi-structured survey instrument. Int J Med Inf. 2008;77:440–447. doi: 10.1016/j.ijmedinf.2007.08.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Han YY. Carcillo JA. Venkataraman ST. Clark RS. Watson RS. Nguyen TC. Bayir H. Orr RA. Unexpected increased mortality after implementation of a commercially sold computerized physician order entry system. Pediatrics. 2005;116:1506–1512. doi: 10.1542/peds.2005-1287. [DOI] [PubMed] [Google Scholar]
- 20.Asaro PV. Boxerman SB. Effects of computerized provider order entry and nursing documentation on workflow. Acad Emerg Med. 2008;15:908–915. doi: 10.1111/j.1553-2712.2008.00235.x. [DOI] [PubMed] [Google Scholar]
- 21.Amusan AA. Tongen S. Speedie SM. Mellin A. A time-motion study to evaluate the impact of EMR and CPOE implementation on physician efficiency. J Healthc Inf Manage. 2008;22:31–37. [PubMed] [Google Scholar]
- 22.Finkelstein SM. Snyder M. Stibbe CE. Lindgren B. Sabati N. Killoren T. Hertz MI. Staging of bronchiolitis obliterans syndrome using home spirometry. Chest. 1999;116:120–126. doi: 10.1378/chest.116.1.120. [DOI] [PubMed] [Google Scholar]
- 23.Yoon HJ. Guo GF. Hertz M. Finkelstein S. Adherence to home monitoring and its impact on survival in post-lung transplantation patients. Am Med Inf Assoc Annu Symp Proc. 2008;2008:835–838. [PMC free article] [PubMed] [Google Scholar]