Abstract
Objective: To investigate time differences in recording observations and an early warning score using traditional paper charts and a novel e-Obs system in clinical practice.
Methods: Researchers observed the process of recording observations and early warning scores across 3 wards in 2 university teaching hospitals immediately before and after introduction of the e-Obs system. The process of recording observations included both measurement and documentation of vital signs. Interruptions were timed and subtracted from the measured process duration. Multilevel modeling was used to compensate for potential confounding factors.
Results: In all, 577 nurse events were observed (281 paper, 296 e-Obs). The geometric mean time to take a complete set of vital signs was 215 s (95% confidence interval [CI], 177 s–262 s) on paper, and 150 s (95% CI, 130 s–172 s) electronically. The treatment effect ratio was 0.70 (95% CI, 0.57–0.85, P < .001). The treatment effect ratio in ward 1 was 0.37 (95% CI, 0.26–0.53), in ward 2 was 0.98 (95% CI, 0.70–1.38), and in ward 3 was 0.93 (95% CI, 0.66–1.33).
Discussion: Introduction of an e-Obs system was associated with a statistically significant reduction in overall time to measure and document vital signs electronically compared to paper documentation. The reductions in time varied among wards and were of clinical significance on only 1 of 3 wards studied.
Conclusion: Our results suggest that introduction of an e-Obs system could lower nursing workload as well as increase documentation quality.
Keywords: vital signs, electronic charting, early warning score, time and motion studies
BACKGROUND AND SIGNIFICANCE
Safe care of inpatients requires clinicians to regularly measure and document their vital signs. In many hospitals, vital signs are documented on paper charts and interpreted with the aid of early warning score (EWS) systems. Calculation of an EWS involves assigning an integer score to each vital sign and then aggregating the scores. The total score reflects the degree of physiological abnormality. It is used to determine whether care needs to be escalated and the frequency of subsequent observations.
Paper charts have multiple shortcomings. Errors in EWS calculation, omission of key data, and illegible handwriting contribute to misinterpretation of paper notes.1,2
Computerized systems for recording vital sign observations and calculating an EWS, called e-Obs systems, have previously been identified as a more effective way of identifying patients at risk of clinical deterioration.3 The introduction of health care information technology (IT) systems has historically been met with mixed success.4,5 A key factor in determining end-user acceptance is the effect on workload.6
Evidence regarding whether e-Obs systems decrease nursing workload is mixed. In a classroom environment, Prytherch et al.7 demonstrated a 1.6-times reduction in the time to document vital signs and compute an EWS compared to pen and paper. By contrast, Yeung et al.8 observed the practices of 24 nurses within a clinical setting, and found an increase in time to document observations electronically rather than with pen and paper.
The effect of implementing an electronic observation and EWS system on the time it takes to complete the task in clinical practice has not been studied. Oxford University Hospital’s (OUH) National Health Service (NHS) Trust planned to replace a paper chart–based EWS system with the SEND e-Obs system in a phased rollout.9 This created the opportunity to establish the effect of introducing an e-Obs system on the time it takes to record vital sign observations across 3 wards in 2 university teaching hospitals.
METHODS
We conducted a before-and-after observational study between November 2014 and December 2015 on 3 medical inpatient wards in 2 university teaching hospitals that form part of the OUH NHS Foundation Trust. We used time-motion methods to measure how much time was spent taking and documenting patients’ vital signs.
The study was approved as a service evaluation for OUH Foundation Trust (Datix: 3196).
Aim
The primary objective of this study was to determine whether introduction of an e-Obs system alters the time required to record a complete set of vital sign observations.
Pre-intervention
Prior to the intervention, patients’ vital signs data were recorded on existing paper observation charts.10 These were: heart rate, respiratory rate, blood pressure, temperature, oxygen saturation (SpO2), oxygen therapy, and consciousness via the Glasgow Coma Score or alert, voice, pain, unresponsive (AVPU) score. Charts were routinely kept in patients’ nursing folders, alongside other care plans and charts. Nursing folders of all patients were located at the nursing stations, rather than at the bedside, on all observed wards.
Intervention
The paper chart was replaced with an e-Obs system, SEND, a description of which has previously been published.9 In brief, the SEND application is accessed using a tablet mounted on a roll-stand alongside the vital signs monitor. Each patient is identified by scanning a barcode on his or her ID wristband. Vital signs data are manually entered using the tablet’s touchscreen. The vital signs are graphically charted as they are entered, allowing easy comparison with previously entered data. Upon completion, all data are transmitted immediately to a central server, and the system provides clinical advice based on the automatically calculated EWS.
Data collection procedures
Two clinically trained observers watched nurses on the study wards before and after the intervention. We collected ward-level data, including staff levels, staff seniority, and ward specialty, at the start of the study and monitored for any changes throughout the study period. Staff seniority was categorized as: care support workers and student nurses, nurses (NHS band 5), and senior nurses (NHS band 6 and above).
Nurses undertaking observation sets were observed Monday to Friday between 9 a.m. and 5 p.m. Nurses were aware that they were being observed and had the opportunity to refuse to consent prior to each observation.
On each ward, observations were conducted over two 5-week periods, one before and one after the implementation of SEND. The preimplementation period occurred 16–9 weeks prior to implementation of the system. The post-implementation period took place 4–8 weeks thereafter. The pre- and post-implementation periods were separated by 8–12 weeks. We chose this separation to allow for the training and bedding-in effects of the intervention while minimizing the risks of confounder variables such as changes in staff population.
Our decision to measure the time of nursing tasks followed the precedent of previous studies.11–13 We divided the observation recording process into 2 subtasks: View Chart and Take Vital Signs. View Chart was defined as the task of locating the chart and “opening” it, ready to record vital signs. Take Vital Signs was defined as the task of measuring and documenting vital signs. We defined actions that marked the start and end of each task, as shown in Table 1.
Table 1.
Definitions of task time points in the preintervention and intervention groups
| Task | Times | Control (paper chart) | Intervention (SEND e-Obs) |
|---|---|---|---|
| View Chart | Start (i) | Nurse arrives at notes source (eg, the nursing station) | No equivalent (notes stored in database) |
| Finish (i) | Nurse finished collecting all sets of notes required for the observation and has all sets in hand | No equivalent (notes stored in database) | |
| Start (ii) | Nurse is at patient’s bedside and touches notes to open them | Nurse scans patient’s wristband identifier to access SEND record | |
| Finish (ii) | Vital signs chart is open at the bedside | Vital signs chart is visible to nurse | |
| Take Vital Signs | Start | First piece of vital sign monitoring equipment is attached to patient | |
| Finish | Nurse completes final piece of documentation on paper vital signs chart | Nurse presses “Save Obs” to submit a set of vital signs in SEND | |
The View Chart task consists of 2 mutually exclusive time periods. These relate to (i) locating the nursing notes within a ward and (ii) locating the observation chart within the nursing notes.
Tasks can be interrupted by competing events that require the nurse’s attention. We defined an interruption as anything that caused an ongoing task to be halted. All interruptions were timed and classified (see supplementary material for further details).
All data were recorded electronically in real time on tablet devices using software developed for the study. The software contained timers for each task and for interruptions, which allowed concurrent tasks and interruptions to be accurately recorded. The software also had rules to ensure logical consistency, such as preventing the start of an interruption when no other task was in progress.
Observer training
We used high-fidelity simulation to train the study observers prior to data collection. Testing scenarios included a mix of paper-based and SEND-based vital sign recording as well as a variety of interruptions. In each scenario, the observers were asked to record study data using the data-collection software. The 2 additional independent observers, who took no further part in the study, concurrently recording study data, also using the data-collection software.
Interobserver variability was assessed by calculating the range of times for each task for each scenario. A high value for the range, with respect to the mean task time, would indicate uncertainty as to when tasks should be started or stopped, or problems with the data-capture software. Unconscious bias was assessed by ranking the observers (fastest to slowest) for each task within each scenario. Consistently high or low rankings indicated an unconscious propensity to be faster or slower than the true time.
In the event of high interobserver variability or evidence of unconscious bias, we planned to retrain observers and repeat the scenarios. The results and analysis of the scenarios are available as online supplementary material.
ANALYSIS OF OUTCOME MEASURES
Outcomes
The primary outcome was the difference in task completion time, the time needed to take a set of vital signs and compute an EWS. This was calculated as the sum of times required to complete View Chart and Take Vital Signs, excluding the duration of any concurrent tasks and interruptions.
The secondary outcome measures were the differences in times to complete the View Chart and Take Vital Signs subtasks pre- and post-intervention. Ward-level analysis was undertaken post hoc.
Outcome analysis
We limited analysis a priori to observations where all of the vital signs were documented and an EWS score was calculated.
We assessed time differences using a linear mixed-effects model. The first level of the model was a fixed-slope random intercept linear regression to take into account the clustering of multiple observations by the same nurse. The number of interruptions and nurse seniority were identified as potential confounders and included as covariates in the model.
The second level of the model used a random slope and random intercept to account for differences between wards. Non-normal distribution data were log-transformed prior to analysis. We assessed the validity of the transformation by checking the normality of the model residual distributions (available as supplementary material). The back-transformation of logarithmic values means that all times and confidence intervals are presented as geometric means. The effect size was then calculated as the ratio of geometric means pre- and post-intervention.
Statistical analysis was performed using SAS software, version 9.4.14
RESULTS
A total of 606 sets of vital sign recordings were observed during the study period. We excluded 29 incomplete observation sets from analysis. Of the 29, 6 were missing 1 vital sign, 3 were missing multiple vital signs, and 20 were missing EWS scores for at least 1 vital sign. We analyzed 281/297 (94.6%) paper observations and 296/309 (95.8%) e-Obs. In all, 153–280 observations were taken per ward across both periods (Table 2). The majority of staff observed were band 5 nurses. Full details are shown in Table 2.
Table 2.
Ward-level data for the 3 study wards
| Ward | Ward 1 | Ward 2 | Ward 3 | |||
|---|---|---|---|---|---|---|
| Specialty | Medicine: Infectious Diseases | Medicine: Hematology | Medicine: Acute General | |||
| Number of nursing staff trained to record vital signs | 32 | 33 | 29 | |||
| Study phase | Before | After | Before | After | Before | After |
| Senior nurses observed | 1 | 2 | 3 | 1 | 0 | 1 |
| Nurses observed | 9 | 12 | 21 | 22 | 9 | 13 |
| Care support workers and student nurses observed | 4 | 2 | 1 | 4 | 2 | 3 |
| Total, n (%) | 14 (44) | 16 (50) | 25 (76) | 27 (82) | 11 (38) | 17 (59) |
| Total observations | 86 | 67 | 139 | 141 | 86 | 86 |
| Complete observations | 79 | 66 | 133 | 132 | 84 | 83 |
The geometric mean task completion time was lower using e-Obs (150 s; 95% CI, 130 s–172 s) than when charting on paper (215 s; 95% CI, 177 s–262 s). The overall treatment effect ratio was 0.70 (95% CI, 0.57–0.85, P < .001) (Table 3), equivalent to a 30% reduction in time for the e-Obs system compared to the paper system.
Table 3.
Model outputs for a random offset multilevel linear regression model in which level 1 = nurse, level 2 = ward
| Comparison | Paper: Geo mean (95% CI) | e-Obs: Geo mean (95% CI) | Geometric mean ratio (95% CI) | P value |
|---|---|---|---|---|
| Ward 1 | 319 s (225 s–451 s) | 117 s (92 s–150 s) | 0.37 (0.26–0.53) | <.001 |
| Ward 2 | 204 s (146 s–285 s) | 200 s (159 s–253 s) | 0.98 (0.70–1.38) | .91 |
| Ward 3 | 153 s (109 s–216 s) | 143 s (112 s–183 s) | 0.93 (0.66–1.33) | .70 |
| Overall | 215 s (177 s–262 s) | 150 s (130 s–172 s) | 0.70 (0.57–0.85) | <.001 |
A geometric mean ratio <1 implies that the time for observations using e-Obs is less than with paper. The model accounts for correlation between multiple observations of the same nurse.
At the individual ward level, the treatment effect ratio was 0.37 (95% CI, 0.26–0.53, P < .001) in ward 1, equivalent to a 63% reduction in time. In ward 2, the treatment effect ratio was 0.98 (95% CI, 0.70–1.38, P = .91), equivalent to a 2% reduction in time. This corresponded to a task completion time of 204 s (95% CI, 146 s–285 s) pre-intervention and 200 s (95% CI, 159 s–253 s) post-intervention. In ward 3, the treatment effect ratio was 0.93 (95% CI, 0.66–1.33, P = .70), equivalent to a 7% reduction in time. This corresponded to a task completion time of 153 s (95% CI, 109 s–216 s) pre-intervention and 143 s (95% CI, 112 s–183 s) post-intervention. The treatment effect ratios on wards 2 and 3 were not significant (ward 2: 0.98, 95% CI, 0.70 s–1.38 s, P = .91; ward 3: 0.93, 95% CI, 0.66 s–1.33 s, P = .70).
Of the 2 subtasks, View Chart and Take Vital Signs, we observed the greatest time savings in the latter. The geometric mean (95% CI) time to complete the View Chart task was 18 s (13 s–27 s) before the intervention and 13 s (10 s–17 s) after the introduction of SEND (treatment effect ratio 0.36, P = .052). The geometric mean (95% CI) time to complete the Take Vital Signs task was 194 s (156 s–241 s) on paper and 140 s (120 s–164 s) using the e-Obs system (treatment effect ratio 0.72, P = .005).
DISCUSSION
Our study, conducted in a real-world environment, demonstrates that documentation of vital signs using a well-designed e-Obs system can be faster than paper charting. We observed a statistically significant reduction in task completion time in the studied sample. The reduction remained significant, even after accounting for variation in ward, individual nursing behavior, nursing seniority, and number of interruptions.
Subgroup analysis by ward highlighted that the amount of time saved can vary considerably between individual wards. We observed a clinically significant reduction in geometric mean task completion time on ward 1 from 345 s to 114 s, whereas time savings on wards 2 and 3 were smaller and less clinically relevant (Table 3).
Introduction of an e-Obs system was also associated with reduced variability in the time taken to record vital signs. It seems likely that the system was driving a standardization in the process of recording and documenting vital signs. Process standardization is recognized to be associated with improved quality of care.15
The main time saving occurred in the Take Vital Signs subtask. This occurred despite the SEND system’s including a timer to encourage clinical staff to count respiratory rate over a full 60 s. Respiratory rate is known to be a particularly important indicator of adverse clinical events,16 and longer measurement periods have been associated with increased data accuracy.17
The success of time-motion methods depends on how the observed tasks are defined.18 In this case, we attempted only to measure the direct effect of e-Obs observation chart recall and vital sign data entry and EWS calculation. In doing so, we may have underestimated the true overall time saving of e-Obs. For instance, the SEND e-Obs system might reduce the amount of travel required to take observations by ensuring that the equipment and documentation devices are always in the same location. We chose not to include this measure, as the outcome would be highly dependent on local ward organization, rather than the introduction of e-Obs.
Increased efficiency is not the only benefit of an e-Obs system. SEND incorporates a number of features designed to reduce errors. In common with other e-Obs systems,7,19 SEND automatically calculates EWS scores, thereby eliminating EWS calculation errors. Such errors can delay timely identification of patients at risk of deterioration.20 Furthermore, the system identified patients using barcodes on their identification wristbands. Patient identification via barcodes has been associated with error reduction in other clinical settings, including drug prescribing and blood transfusion.21,22
LIMITATIONS
The sampling of vital sign recording sessions across the 3 wards was uneven. We chose to observe for a fixed period before and after intervention to minimize confounding from time-dependent covariates. However, variation in practice between wards led to oversampling in ward 2 and undersampling in wards 1 and 3. As the largest time saving occurred where the fewest samples were taken (ward 1), the likely effect of our sampling differences is to underestimate the effects on task completion time.
During observation sessions, we aimed to observe all observations taken. In order to be present at the bedside, we could only study vital sign recording when the ward nurse agreed to being observed. We did not observe recording practice outside weekday working hours. The choice of 9 a.m. to 5 p.m. on weekdays to undertake the study was pragmatic, given researcher availability and the need to minimize the impact of the study on the wards. It is theoretically possible that observation recording is systematically biased according to time of day, although this does not seem likely.
Measurement of the primary outcome measure could have been affected by the fact that participants were aware that they were being observed. Being under scrutiny can stimulate an improvement in performance (the Hawthorne effect).23 Another potential source of bias in time-motion studies comes from the demand effect, where participants aim to please the study investigators. However, participants were not aware of the study objectives at the time of consent.
Before-and-after studies are limited in their ability to account for temporal variations in confounding variables. The lack of a control cohort who never receive the intervention hampers the modelling of confounding effects.24 Due to the practicalities of rolling out e-Obs to the hospitals, alternative study methodologies were not possible. We limited the effect of temporal variations in confounding variables by observing nurses over a relatively short period close to the time of the intervention. We did not observe any external changes that could plausibly have affected the efficiency of vital sign recording.
Relevance to other work
Three studies have compared electronic and paper vital sign entry. Vital sign recording took longer in a hospital that recorded vital signs in an electronic patient record compared to 2 hospitals that recorded vital signs on paper.8 In contrast, 2 studies suggest that vital sign entry using electronic devices at the bedside is more time efficient than using paper.25,26 None of these studies reported calculation of an EWS.
Two previous studies used selected observation sets to assess the effect of an e-Obs system (VitalPAC™; System C) on the time to record vital signs and calculate an EWS. Time savings were seen compared to paper in a classroom-based study.7 All the fictitious vital signs sets used in this study scored >0 on the EWS system. This contrasts with the clinical environment, where the majority of vital signs score zero.27 Consequently, the time to calculate an EWS manually may have been higher than in our study. Mohammed et al. found marginal improvements when nurses inputted 10 vignettes with the e-Obs system after initial training. However, inputting the same 10 vignettes was, on average, over 10 seconds quicker than paper after the nurses had used the e-Obs system for 4 weeks in clinical practice.28 This improvement is similar to the smallest median ward change found in our study.
Our study adds to previous findings by observing the use of an e-Obs system in clinical practice with real patients. The study sample size was much larger than previous comparable work in this area.
CONCLUSIONS
In our 3-ward study, e-Obs was associated with a statistically significant reduction in the overall time needed to record vital sign observations and calculate an EWS when compared with paper. In subgroup analysis, the time saved varied by ward. These variations may be due to differences in ward practice and require further investigation.
The results of this study, taken in conjunction with previous work, supports the assertion that a well-designed system can save significant time in clinical practice. These time savings, in addition to the data quality benefits of electronic systems, present a convincing case for the adoption of e-Obs systems as part of routine inpatient care.
Supplementary Material
ACKNOWLEDGMENTS
We thank Soubera Yousefi and David Vallance for their help in collecting observation data during the study.
Funding
This work was supported by the National Institute for Health Research Biomedical Research Centre (Oxford). PW is employed by the OUH NHS Trust. DW (grant: DFRWAO00), JK, and TB were funded by the NIHR Biomedical Research Centre.
Competing interests
DW, JK, TB, and PW were part of the team that developed SEND.
Contributors
PW, JK, and DW conceived and designed the study. DW, TB, and JK acquired the study data. DW, JT, SG, and TB analyzed and interpreted the data. All authors were involved in drafting and critically revising the article and have approved the final version for submission.
SUPPLEMENTARY MATERIAL
Supplementary material is available at Journal of the American Medical Informatics Association online.
References
- 1. Wilson SJ, Wong D, Clifton DA et al. . Track and trigger in an emergency department: an observational evaluation study. Emerg Med J. 2012;303:186. [DOI] [PubMed] [Google Scholar]
- 2. Subbe CP, Gao H, Harrison DA. Reproducibility of physiological track-and-trigger warning systems for identifying at-risk patients on the ward. Intensive Care Med. 2007;33:619–24. [DOI] [PubMed] [Google Scholar]
- 3. Smith GB, Prytherch DR, Schmidt P et al. . Hospital-wide physiological surveillance: a new approach to the early identification and management of the sick patient. Resuscitation. 2006;711:19–28. [DOI] [PubMed] [Google Scholar]
- 4. Nguyen L, Bellucci E, Nguyen LT. Electronic health records implementation: an evaluation of information system impact and contingency factors. Int J Med Inform. 2014;8311:779–96. [DOI] [PubMed] [Google Scholar]
- 5. Bossen C, Jensen LG, Udsen FW. Evaluation of a comprehensive EHR based on the DeLone and McLean model for IS success: approach, results, and success factors. Int J Med Inform. 2013;8210:940–53. [DOI] [PubMed] [Google Scholar]
- 6. Pynoo B, Devolder P, Voet T et al. . Assessing hospital physicians’ acceptance of clinical information systems: a review of the relevant literature. Psychologica Belgica. 2013;532:15–31. [Google Scholar]
- 7. Prytherch DR, Smith GB, Schmidt P et al. . Calculating early warning scores: a classroom comparison of pen and paper and hand-held computer methods. Resuscitation. 2006;702:173–78. [DOI] [PubMed] [Google Scholar]
- 8. Yeung MS, Lapinsky SE, Granton JT, Doran DM, Cafazzo JA. Examining nursing vital signs documentation workflow: barriers and opportunities in general internal medicine units. J Clin Nurs. 2012;21(7–8):975–82. [DOI] [PubMed] [Google Scholar]
- 9. Wong D, Bonnici T, Knight J, Morgan L, Coombes P, Watkinson P. SEND: A system for electronic notification and documentation. BMC Med Inform Decis Mak. 2015;15:68. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Tarassenko L, Clifton DA, Pinsky MR, Hravnak MT, Woods JR, Watkinson PJ. Centile-based early warning scores derived from statistical distributions of vital signs. Resuscitation. 2011;828:1013–18. [DOI] [PubMed] [Google Scholar]
- 11. Potter P, Wolf L, Boxerman S et al. . Understanding the cognitive work of nursing in the acute care environment. J Nurs Adm. 2005;35(7–8):327–35. [PubMed] [Google Scholar]
- 12. Wolf LD, Potter P, Sledge JA, Boxerman SB, Grayson D, Evanoff B. Describing nurses’ work: combining quantitative and qualitative analysis. Human Factors. 2006;481:5–14. [DOI] [PubMed] [Google Scholar]
- 13. Hendrick A, Chow MP, Skierczynski BA, Lu Z. A 36-hospital time and motion study: how do medical-surgical nurses spend their time? Permanente J. 2008;123:25. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. Base SAS 9.4. Cary, NC: SAS Institute Inc.;2013. [Google Scholar]
- 15. Batalden PB, Davidoff F. What is “quality improvement” and how can it transform healthcare? Qual Saf Health Care. 2007;16:2–3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16. Cretikos MA, Bellomo R, Hillman K, Chen J, Finfer S, Flabouris A. Respiratory rate: the neglected vital sign. Med J Aust. 2008;18811:657. [DOI] [PubMed] [Google Scholar]
- 17. Simoes EA, Roark R, Berman S, Esler LL, Murphy J. Respiratory rate: measurement of variability over time and accuracy at different counting periods. Arch Dis Child. 1991;6610:1199–203. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18. Zheng K, Guo MH, Hanauer DA. Using the time and motion method to study clinical work processes and workflow: methodological inconsistencies and a call for standardized research. J Am Med Inform Assoc. 2011;185:704–10. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19. Pullinger R, Wilson S, Way R et al. . Implementing an electronic observation and early warning score chart in the emergency department: a feasibility study. Emerg Med J. 2016. http://journals.lww.com/euro-emergencymed/Abstract/publishahead/Implementing_an_electronic_observation_and_early.99344.aspx. Accessed August 31, 2016. [DOI] [PubMed] [Google Scholar]
- 20. Smith AF, Oakey RJ. Incidence and significance of errors in a patient ‘track and trigger’ system during an epidemic of Legionnaires’ disease: retrospective casenote analysis. Anaesthesia. 2006;613:222–28. [DOI] [PubMed] [Google Scholar]
- 21. Poon EG, Keohane CA, Yoon CS et al. . Effect of bar-code technology on the safety of medication administration. N Engl J Med. 2010;36218:1698–707. [DOI] [PubMed] [Google Scholar]
- 22. Turner CL, Casbard AC, Murphy MF. Barcode technology: its role in increasing the safety of blood transfusion. Transfusion. 2003;43:1200–09. [DOI] [PubMed] [Google Scholar]
- 23. McCambridge J, Witton J, Elbourne DR. Systematic review of the Hawthorne effect: new concepts are needed to study research participation effects. J Clin Epidemiol. 2014;673:267–77. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24. Goodacre S. Uncontrolled before-after studies: discouraged by Cochrane and the EMJ. Emerg Med J. 2015;327:507–08. [DOI] [PubMed] [Google Scholar]
- 25. Meccariello M, Perkins P, Quigley LG, Rock A, Qiu J. Vital time savings: evaluating the use of an automated vital signs documentation system on a medical/surgical unit. J Healthc Inf Manag. 2010;244:46–51. [Google Scholar]
- 26. Wager KA, Schaffner MJ, Foulois B, Swanson Kazley A, Parker C, Walo H. Comparison of the quality and timeliness of vital signs data using three different data-entry devices. Comput Inform Nurs. 2010:284:205–12. [DOI] [PubMed] [Google Scholar]
- 27. Clifton DA, Clifton L, Sandu D et al. . ‘Errors’ and omissions in paper-based early warning scores: the association with changes in vital signs—a database analysis. BMJ Open. 2015:57:e007376. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28. Mohammed M, Hayton R, Clement G, Smith G, Prytherch D. Improving accuracy and efficiency of early warning scores in acute care. Brit J Nurs. 2009:181:18–24. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.