Bibliographic Information - Author (s) - Yr Published/Submitted - Publication - Author Affiliations - Funding	Study - Design - Facility/Setting - Time Period - Population/Sample - Comparator - Study bias	Practice - Description - Duration - Training - Staff/Other Resources - Cost	Outcome Measures - Description (s) - Recording method	Results/Findings - Type of Findings - Findings/Effect Size - Stat. Significance/Test(s) - Results/Conclusion Bias
Etchells E [1,3], Adhikari NKJ [1,2,3], Cheung C [1,3], Fowler R [1,2,3], Kiss A [3], Quan S [4], Sibbald W [1,2,3], Wong B [1,3] - Year: 2010 - Publication: Quality & Safety in Health Care - Affiliation: [1] Department of Medicine, University of Toronto, Toronto, Ontario, Canada. [2] Interdepartmental Division of Critical Care, University of Toronto, Toronto, Ontario, Canada. [3] Sunnybrook Health Sciences Centre, Toronto, Ontario, Canada. [4] University Health Network, Toronto, Ontario, Canada - Funding: New Age provided the system and services at a reduced cost; all other study costs funded by the University of Toronto Department of Medicine Quality Partners Program	- Design: Randomized Controlled Study - Facility/Setting: Four general medicine clinical teaching units at Sunnybrook Health Sciences Centre, Toronto, Ontario – Canada. - Time period: 02/2006 – 05/2006 -Population/Sample: 165 critical values evaluated on 108 patients with full response time data. Note: Critical values for patients under care of ED or critical care physician excluded. Excluded critical values for troponin. - Comparator: Telephone call to the patient’s ward by laboratory technician - Study bias: If time of order was not documented, the time of administration of treatment was used to calculate response time. For pilot study, physicians did not consistently document times of their orders.	-Description: Automated real-time paging system that sends critical laboratory values directly from laboratory information system to an alphanumeric pager carried by the responsible housestaff physician. - Duration: 4 months (02/2006 – 05/2006) - Training: Not reported except that they encouraged residents to document response times - Staff: Pager carried by senior resident on weekdays and call resident on nights and weekends, Study nurse reviewed written medical record. Laboratory technician verifies accuracy of test result and inputs into lab informational system (lab tech also telephones patient’s ward with critical lab value). 3 study physicians reviewed data for each critical value. - Other resources: Not reported - Cost: Not reported	- Description: Time to respond (interval between the acceptance of the critical value into the laboratory information system to the writing of an order on the patient’s chart in response to the critical value). - Recording Method: Electronic patient record for laboratory results. Chart review by nurse for written medical record (medication and treatment handwritten orders) around the time of critical value. Study physicians reviewed data for each critical value.	- Type of Findings: Comparison - Findings/Effect Size: Median time to respond Practice: 16 minutes (IQR 2–141 min) Comparator: 39.5 minutes (IQR 7–104.5min) (p=0.33) ES = Not calculable from data provided - Statistical Significance/Test(s): Wilcoxon rank sum test, Hodges-Lehman estimate of shift to calculate the median difference in response time, χ2, t tests - Results/conclusion biases: None noted.
Quality Rating: 8(Good) (10 point maximum) Effect Size Magnitude Rating: Substantial Relevance: Direct	Study (3 pts maximum): 2 Potential study bias: sample selection methods may introduce a study bias that would affect results (when time of order not documented, time of treatment was used to calculate response time (−1)	Practice (2 pts maximum): 2	Outcome measures (2 pts. maximum): 2	Results/findings (3 pts maximum): 2 Data do not permit calculation of an effect size (−1)
Kuperman GJ [1,2]; Teich JM [1,2]; Tanasijevic MJ [2]; Ma’Luf N [2]; Rittenberg E [2]; Ashish Jha MA [2]; Fiskio J [1]; Winkelman J [2]; Bate, DW [1,2] - Year: 1999 - Publication: Journal of the American Medical Informatics Association - Affiliations: [1] Partners HealthCare Systems, [2] Harvard Medical School - Funding: Partly from research grant (R01 -Agency of Health Care Policy and Research)	- Design: Randomized Controlled Study - Facility/Setting: Brigham & Woman’s Hospital; 720 bed tertiary- care academic medical center in Boston, MA - Time period: Medical: 12/1994 – 01/31/1995 Surgical: 09/01/1995 – 10/30/1995 - Population/Sample: 178 subjects (medical and surgical inpatients); 192 alerts (94 intervention, 98 controls); 4 laboratory tests with critical values and/or alert situations - Comparator: Critical values telephoned by lab technologists to patient floor (nursing staff). - Study bias: None noted	- Description: Computer system to detect critical conditions and automatically notify the responsible physician via the hospital’s paging system. Physician identified from automated “coverage list” database that identifies primary physician for each patient at any given time. If alert not acknowledged after 15min, border of computer on patient’s floor turns red, nurse responds to alert; if after 30 min no acknowledgement, workstation in telecommunication beeps, phone operator reviews alert and calls floor. - Duration: 2 months at each service unit at different time periods (total 4 months). - Training: Not reported; outcomes assessed by trained reviewers. - Staff: Computer technicians, physician, nurses, unit secretary, telephone operator, reviewers and lab supervisor/manager; clinical alerting system, digital pager, computer workstation - Other resources: Not reported - Cost: Not reported	- Description: Time to Treat (TTT) - Time interval from the filing of the alerting result to the ordering of appropriate treatment (other outcome of study: Time to Resolution - time interval from the filing of alerting result to the arrival time in the laboratory of a bedside test demonstrating the alerting condition was no longer present) - Recording Method: Occurrence log, Chart review	- Type of Findings: Comparison - Findings/Effect Size: Time to treat: Practice median time: 1.0 hour (60min) Comparator median time: 1.6 hours (96min) (p=0.003); Practice mean, 4.1 vs. Comparator mean 4.6 hours, (p = 0.003) Physicians reviewed 65/94 intervention alerts (69%) –median time to treatment of 65 alerts: 0.5 hours. Nurses reviewed 7 alerts (7%) and 22 alerts (23%) reviewed by telecommunication staff who communicated them to floor. d = 0.434 (CI =0.148–0.720) - Statistical Significance/Test(s): Wilcox rank sum statistic. - Results/conclusion biases: Favorable conclusions not supported; but instead are contradicted by reported findings. Authors note that differences are not significant but focus on direction of effect.
Quality Rating: 7 (Fair) (10 point maximum) Effect Size Magnitude Rating: Substantial Relevance: Direct	Study (3 pts maximum): 2 Study design: sample size too small and may not be representative of the results of the practice and may not be generalizable. (−1)	Practice (2 pts maximum): 2	Outcome measures (2 pts. maximum): 2	Results/findings (3 pts maximum): 1 Uncontrolled deviations: results reported not clearly attributable to practice being evaluated; conclusions not supported by work (−2)
Park H [1]*, Min WK [1], Lee W [1], Park H [2], Park CJ [1], Chi HS [1], Chun S [1] - Year: 2008 - Publication: Annals of Clinical & Laboratory Science. - Affiliations: [1] Asan Medical Center, University of Ulsan College of Medicine, Seoul, Korea; [2] Kangbuk Samsung Hospital, Sungkyunkwan University School of Medicine, Seoul, Korea; * Current affiliation: Catholic University of Korea, College of Medicine, Seoul, Korea - Funding: Korea Health Industry Dev Institute.	- Design: Before-After - Facility/Setting: 2,200 bed tertiary care urban, academic medical center, Seoul Korea; ICU and general wards. - Time period: Two 12-month study periods: Pre: 01/01/2001 –12/31/2001 Post: 07/01/2005 – 06/30/2006 - Population/Sample: alerts for critical hyperkalemia: Pre: 121 alert calls (ICU: 56; general wards 65) Post: 96 alert calls (ICU: 31; general wards 65) - Comparator: Lab tech telephones nurses on inpatient floor to notify of patient CV. Nurse then informs physician of patient’s CV result. Call documented in lab log. - Study bias: None noted.	- Description: Short text message service (SMS) system for notifying physician of critical values by sending message to their personal data assistant (PDA) phones. ) -Text messages w/ patient information and test result is transmitted to appropriate physician via PDA phones - Duration: 12 months (07/01/2005 – 06/30/2006) - Training: Not discussed. - Staff: Lab technicians, nurses, physicians - Other resources: Computer software and PDA phones for all physicians - Cost: Not reported.	- Description: Time to receipt - Time interval in minutes from dispatching critical value result alert to acknowledgement by responsible caregiver (other outcome from study: Clinical response rate -the frequency of clinical responses divided by total # critical value alerts) - Recording Method: Occurrence log	- Type of Findings: Prestest-Posttest Pretest-posttest - Findings/Effect Size: Time to receipt Pre: Total: Median = 213 minutes; Mean 343.3 (sd 369.6) n = 121 ICU: Median = 193 minutes; Mean 306.9 (sd 336.2) n = 56 General wards: Median = 249 minutes; Mean = 374.7 (sd = 396.1) n = 65 Post: Total: Median = 74.5 minutes ; Mean = 203.2 (sd = 294.1) n = 96 ICU: Median = 93 minutes; Mean =270.6 (sd = 366.7) n = 31 General wards: Median = 63 minutes; Mean = 171.1 (sd =249.3) n = 65 d = 0.414 (CI = 0.143–0.685) - Statistical Significance/Test(s): Mann-Whitney U-test showed significant difference between clinical response times in pre and post (p<0.001). - Results/conclusion biases: Practices compared based on data collected during notably different time periods (2001 v. 2005)
Quality Rating: 7 (Fair) (10 point maximum) Effect Size Magnitude Rating: Substantial Relevance: Direct	Study (3 pts maximum): 1 Sample size too small: may not be representative of the results of the practice and may not be generalizable. (−1). Setting sufficiently distinctive and results may not be generalizable to other settings. (−1)	Practice (2 pts maximum): 2	Outcome measures (2 pts. maximum): 2	Results/findings (3 pts maximum): 2 Appropriateness of statistical analysis: Compares two practices and their estimates based on data collected during notably different time periods (2001 v. 2005) (−1)
Piva, E [1], Sciacovelli, L [1], Zaninotto, M [1], Laposata, M [2], Plebani, M [1] - Year: 2009 - Publication: American Journal of Clinical Pathology - Affiliations: [1] Department of Laboratory Medicine, Padua University School of Medicine, Padua, Italy; [2] Vanderbilt University Hospital, Nashville TN - Funding: Self-funded	- Design: Before-After - Facility/Setting: Padua Hospital, teaching hospital and research center, >300 bed inpatient hospital; Annual test volume: >1 million. Padua, Italy - Time period: Pre: 01/2007–02/2007 (first 2 months of study) Study period: 01/2007–12/2007 (1 year) Post: 01/01/2008 – 02/28/08 (2 months) - Population/Sample: Study period: 7,320 CVs (4,392 routine testing; 2,928 emergency testing) 82% found in inpatients. Post and post breakdown: Not reported - Comparator: Telephone CV notification system - Study bias: Did not exclude emergency critical values data	- Description: Automated alerting system which involves the use of a computerized database (i.e., HIS; LIS) of test results. Once critical value identified and validated by clinical pathologist in charge, transmission of database creates an e-mail message for automated notification which generates an SMS (text) to cell phone of referring physician (clinician on duty) and at the department level (an alert message flashes on monitors until physician or nurse in charge of notification confirms message is received (flashing alert stops after 60 minutes). - Duration: 2 months (01/01/2008 – 02/28/08) - Training: Not discussed - Staff: physicians, clinical pathologist, nurses, laboratory - Other Resources: Not reported - Cost: Not reported	- Description: (1) Time to receipt: - Time from detection of CV in minutes to acknowledgement by responsible clinician (2) Timeliness of reporting - % CV results reported within 1 hr:; # unsuccessful notifications w/in 1 hr/total # of CVs - Recording method: Medical reports and HCIS (health care information system)	- Type of Findings: Pretest-Posttest - Findings/Effect Size: (1) Time to receipt - Pre: Average 30 min; Post: Average 11 min (2) % Reported within 1 hour Pre: >50% –unsuccessful Post: 10.9% - unsuccessful ES = Not calculable from data provided - Stat. Significance/ Test(s): None reported (used MedCalc software) - Results/Conclusion Bias: No post sample data (numerator or denominator); sample size only for pre-practice period.
Quality Rating: 7 (Fair) (10 point maximum) Effect Size Magnitude Rating: Substantial Relevance: Direct	Study (3 pts maximum): 2 Potential study bias: sample selection (includes emergency patients CV) may introduce a study bias that would affect results (−1)	Practice (2 pts maximum): 2	Outcome measures (2 pts. maximum): 2	Results/findings (3 pts maximum): 1 Appropriateness of statistical analysis: Does not provide data sufficient to allow/ verify calculation of an effect size (−1) Sample sufficiency: Number of pre/post sample not reported (−2)
Saw S [1], Loh TP [1], Ang SBL [2], Yip JWL [3], Sethi SK [1] - Year: 2011 - Publication: Clinical Chemistry - Affiliation: [1] Department of Laboratory Medicine, National University Hospital, Singapore. [2] Department of Anaesthesia, National University Hospital, Singapore. [3] Department of Cardiology, National University Hospital, Singapore - Funding: Self-funded	- Design: Before-After - Facility/Setting: National University Hospital, Singapore; 1,000-bed tertiary teaching hospital with a full suite of clinical services; laboratory receives in excess of 4,000 clinical samples daily - Time period: Pre: 03/2008 – 05/2008 Pilot: 06/2008 – 08/2008 Implementation: 04/2009 – 03/2010 Post: 03/2010 – 05/2010 -Population/Sample: approximately 4,692 critical laboratory results in each test period - Comparator: Manual call center system - Study bias: None	-Description: Fully automated short message system (critical reportable results – CRR) using information technology engine that automatically alerts physicians to critical values. The CRR engine software was loaded onto the health care messaging system (HMS) –an existing platform used by call center to maintain and retrieve departmental physician rosters. Physician is required to reply within 10 minutes of critical value receipt, otherwise alert is sent to more senior physician from roster (and/or trigger manual intervention from call center). - Duration: 12 months (04/2009 – 03/2010) - Training: Not reported - Staff/Other resources: Steering committee with representatives from nursing professionals, various clinical divisional leaders, laboratory professionals, hospital administrators, information technology department, the call center, and a local private software development partner - Cost: Not reported	- Description: Time to receipt (for pre and post: includes time to validate critical result); time to acknowledge Pre: acknowledgment by a person from ordering location and the reporting was considered complete. Post: acknowledgement by physician - Recording Method: All CRR-HMS transactions are electronically captured, and an audit trail traceable to the sender and receiver is recorded.	- Type of Findings: Pretest-Posttest - Findings/Effect Size: Time to receipt/respond Pre: Median = 7.3 minutes (96.8% of critical results communicated within 1 hour); Mean = 14.6 minutes Post: 11 minutes (92.9% of critical results acknowledged within 1 hour); mean = 18.3 minutes Excluding time taken by laboratory to validate critical values at post: Median = 2.0 minutes; Mean = 4.7 minutes (validation time not available for pre) d = −0.462 (CI = −0.571 to −0.353) OR = 0.433 (CI = 0.355 – 0.527) - Statistical Significance/Test(s): Not discussed - Results/conclusion biases: CRR response includes delay in escalation logic; manual response time stopped after any staff acknowledged critical result. Both pre and post estimates include time to verify result
Quality Rating: 6 (Fair - exclude) (10 point maximum) Effect Size Magnitude Rating: N/A Relevance: N/A Excluded due to multiple confounds and noticeable measurement bias	Study (3 pts maximum): 3	Practice (2 pts maximum): 2	Outcome measures (2 pts. maximum): 0 Non-comparable measures	Results/findings (3 pts maximum): 1 Results not attributable to practice