Flash Mob Research

Abstract

Background:

Vital signs are critical data in the care of hospitalized patients, but the accuracy with which respiratory rates are recorded in this population remains uncertain. We used a novel flash mob research approach to evaluate the accuracy of recorded respiratory rates in inpatients.

Methods:

This was a single-day, resident-led, prospective observational study of recorded vs directly observed vital signs in nonventilated patients not in the ICU on internal medicine teaching services at six large tertiary-care centers across the United States.

Results:

Among the 368 inpatients included, the median respiratory rate was 16 breaths/min for the directly observed values and 18 breaths/min for the recorded values, with a median difference of 2 breaths/min (P < .001). Respiratory rates of 18 or 20 breaths/min accounted for 71.8% (95% CI, 67.1%-76.4%) of the recorded values compared with 13.0% (95% CI, 9.5%-16.5%) of the directly observed measurements. For individual patients, there was less agreement between the recorded and the directly observed respiratory rate compared with pulse rate.

Conclusions:

Among hospitalized patients across the United States, recorded respiratory rates are higher than directly observed measurements and are significantly more likely to be 18 or 20 breaths/min.

Vital signs, including temperature, pulse rate, BP, respiratory rate, and oxygen saturation, are critical pieces of data in the care of hospitalized patients. Vital sign trends serve as early warning signs for impending sepsis, respiratory failure, cardiopulmonary arrest, and ICU transfer and independently predict mortality.^1‐6 Despite the accepted importance of these data, several large studies have questioned the accuracy of their measurements in hospitalized patients.^7‐11

Temperature, pulse rate, BP, and oxygen saturation are all routinely measured in an automated, noninvasive manner. Respiratory rate, however, remains manually measured in patients who are not mechanically ventilated. Several single-institution studies suggested that respiratory rate often is omitted, is rarely accurate, and clusters around rates of 18 and 20 breaths/min.^11,12 To our knowledge, there are no large multiinstitution studies evaluating the accuracy of manual respiratory rate measurement in hospitalized patients.

A flash mob is defined as “a group of people summoned (as by e-mail or text message) to a designated location at a specified time to perform an indicated action before dispersing.”¹³ The objective of this study was to use the numerical strength of residents in academic centers and a flash mob methodology in a large-scale, single-day investigation evaluating the accuracy of recorded respiratory rates in hospitalized patients not in the ICU, with particular focus on the proportion of respiratory rates with specific values of 18 or 20 breaths/min.

Materials and Methods

Study Design

This was a single-day, multiinstitution, observational study. Internal medicine chief residents from six academic centers established a collaborative network that obtained institutional review board approval at each center, developed a research protocol, and engaged and trained residents in the conduct of this large-scale clinical study. Data were collected on any internal medicine (general or subspecialty service) patient who was primarily managed by resident physicians (teaching service). Patients were excluded if they were receiving mechanical ventilation or positive pressure ventilation with a set respiratory rate or were in an ICU.

Data collection took place concurrently at six sites on June 5, 2012. Each internal medicine resident physician directly observed the patients for whom he or she was responsible, measuring respiratory and pulse rates as per routine clinical care. Residents had been previously educated in a standardized technique for pulse and respiratory rate measurement, with each measurement occurring over a 60-s time interval.¹⁴ Residents documented the directly observed respiratory rate, pulse rate, and time of observation on a standard data collection sheet and then transcribed recorded vital signs from the medical record in a deidentified manner, including the respiratory rate, pulse rate, BP, and temperature. The recorded measurements immediately prior and subsequent to the directly observed vital signs were included along with the time those vital signs were obtained. Because pulse rate measurement typically is automated, comparison between directly observed and recorded pulse rates was used as a control. Study data were managed using a collectively accessible electronic database, REDCap, hosted at Vanderbilt University.¹⁵

Statistical Methods

Descriptive statistics are presented as median (interquartile range [IQR]) or percentage (95% CI) when appropriate. Agreement between directly observed and recorded values was assessed with intraclass correlation coefficient (ICC). Wilcoxon signed rank test was used to assess the difference in respiratory rate measured by two methods. To compare the proportions of patients with values of 18 or 20 breaths/min measured by the two methods, McNemar χ² test was performed. All analyses were done with the statistical programming language R version 2.13.1 (R Project).

Results

Vital sign measurements were collected from 368 patients from the six participating centers. The median respiratory rate was 16 breaths/min (IQR, 14-20 breaths/min) for directly observed measurements and 18 breaths/min (IQR, 18-20 breaths/min) for recorded measurements, with a median difference of 2 breaths/min (P < .001). Of the 361 sets of vital signs with both recorded and directly observed respiratory rate data, 71.8% (95% CI, 67.1%-76.4%) of the recorded values were 18 or 20 breaths/min compared with 13.0% (95% CI, 9.5%-16.5%) of the directly observed values (P < .001) (Fig 1).

Figure 1.

Directly observed vs recorded respiratory and pulse rates. Density plots of the frequency of specific respiratory rate and pulse rate values are shown among directly observed and recorded measurements.

The directly observed and recorded measurements showed only fair agreement for respiratory rate (ICC, 0.26; 95% CI, 0.16-0.35) but substantial agreement for pulse rate (ICC, 0.65; 95% CI, 0.58-0.71) (Table 1). Pulse rate demonstrated consistently higher agreement than respiratory rate at each participating site. This was examined graphically by plotting the difference between each directly observed and recorded measurement vs the average of the two values (Fig 2).¹⁶ Although the graphical representation demonstrates an even distribution of disagreement across the range of pulse measurements, the respiratory rate distribution follows a pattern in which the recorded values are higher than the directly observed values for patients with respiratory rates < 18 breaths/min and lower than directly observed values for patients with respiratory rates > 18 breaths/min (Fig 2). Disagreement between directly observed and recorded values was greatest for patients with the highest and lowest respiratory rates. The effect of the time elapsed between recorded and observed measurements on the difference in their values was quantitatively examined (Table 2). A difference of > 5 breaths/min between the recorded and observed respiratory rate occurred equally in cases in which the measurements were taken within 2 h vs > 2 h apart.

Table 1.

—Agreement Between Directly Observed and Recorded Respiratory Rate and Pulse Rate Values

Site	Respiratory Rate	Pulse Rate
1	0.05 (−0.14 to 0.25)	0.61 (0.47-0.72)
2	0.52 (0.27 to 0.71)	0.59 (0.36-0.75)
3	0.16 (−0.18 to 0.47)	0.83 (0.69-0.91)
4	0.20 (−0.11 to 0.50)	0.48 (0.16-0.71)
5	0.36 (0.12 to 0.54)	0.63 (0.50-0.74)
6	0.22 (−0.03 to 0.44)	0.67 (0.46-0.80)
Total	0.26 (0.16 to 0.35)	0.65 (0.58-0.71)

Data are presented as intraclass correlation coefficient (95% CI). Respiratory and pulse rates were assessed both collectively and by participating site.

Figure 2.

Bland-Altman plot of respiratory and pulse rates. Differences between directly observed and recorded values vs the mean of those measurements are shown. Each dot indicates a specific patient. Black dots indicate ≤ 2 h between measurement of directly observed and recorded values. Red dots indicate > 2 h between measurements.

Table 2.

—Impact of Time Between Respiratory Rate Measurements

Time Between Measurements	Observed vs Recorded Respiratory Rate
	≤ −5	−5 to +5	≥ +5
≤ 2 h	10.8 (17)	67.5 (106)	21.7 (34)
> 2 h	13.4 (26)	63.9 (124)	22.7 (44)

Data are presented as % (No.). The difference between the observed and recorded respiratory rate values was stratified by time elapsed between their respective measurements.

Discussion

Although skepticism regarding the accuracy of respiratory rate measurement is not new, this study represents, to our knowledge, the first prospective, multicenter examination of recorded respiratory rates in hospitalized patients, and it was accomplished using a novel flash mob research methodology.^9,11,12 Anticipating that the manual measurement required to assess respiratory rate introduces a bias not shared by the automatically measured vital signs, we investigated the hypothesis that the recorded respiratory rate in hospitalized patients is higher than the directly observed respiratory rate, with specific overrepresentation of 18 and 20 breaths/min. The results from > 350 patients at six academic institutions confirm a widespread discrepancy between directly observed respiratory rates and those recorded in the medical record. As expected, there was substantial correlation between directly observed and recorded pulse rates, with both distributions concentrated in the physiologic range. In contrast, the respiratory rate showed limited correlation between directly observed and recorded values. The distribution of the directly observed respiratory rate followed the expected pattern, with a smooth curve concentrated in the physiologic range. However, the distribution of recorded respiratory rates comprised peaks at 18 and 20 breaths/min, with < 30% of all recorded respiratory rates having any other value.

One potential explanation for these findings is that in a context in which all other vital signs can be effortlessly measured with automated devices, the respiratory rate still requires manual measurement. This results in estimates of respiratory rates, largely favoring 18 and 20 breaths/min, which are falsely considered normal. In addition, respiratory rates may be counted over only 15 or 30 s and multiplied to get rates per minute, which results in an overrepresentation of even values. This study involved > 75 residents at six independent academic institutions across the country, with recorded vital sign measurements from hundreds of nurses or care partners. The wide distribution of these centers confirms that the phenomenon of overrepresented respiratory rates of 18 and 20 breaths/min is not isolated to a single medical center or region but is systemic throughout the broader medical complex.

These results have implications for the manner in which recorded respiratory rates are interpreted—or perhaps largely ignored—in clinical practice. Despite the established body of research linking tachypnea to early detection of hospital-acquired illness like sepsis and respiratory failure, these data suggest that recorded respiratory rates are far more likely than chance to be 18 or 20 breaths/min, even in patients who are directly observed to be markedly tachypneic.^1‐6 Similarly, the underrepresentation of lower respiratory rates in the recorded values suggests that episodes of hypopnea may be underreported, precluding an opportunity for early clinical rescue. Physicians who become accustomed to seeing a respiratory rate of 18 or 20 breaths/min in every patient may be less likely to respond to these values as abnormal when they have been accurately measured. Furthermore, many hospitals are using electronic medical records in an attempt to automate the earlier detection of disease, especially sepsis.^17,18 Inaccurately recorded respiratory rates, especially the overrepresentation of tachypnea, might hinder the ability of these automated systems to accurately detect the systemic inflammatory response syndrome and, thus, sepsis early. Ongoing studies of noninvasive methods to measure respiratory rate, including end-tidal carbon dioxide measurement (capnography), ECG waveform variation, noninvasive acoustic methods, and thoracoabdominal impedance monitoring, may provide a more accurate approach than the currently used count-and-multiply method.^11,19‐21

This study also serves as proof-of-concept for large-scale, resident-driven clinical investigation over an abbreviated time course. The daily care of inpatients is a primary focus for resident physicians, and residents represent a large, clinically-trained group with direct access to patients and motivation to address clinically-oriented research questions. Historically, resident involvement on the front lines of clinical research has been limited by the prolonged time scale of such research. This flash mob research paradigm facilitates significant resident engagement in study design, data collection, and data analysis—the fundamentals of clinical research. The completion of this project, with total time from inception to submission of 90 days, lays the groundwork for future rapid, resident-led collaborative research efforts.

In conclusion, recorded respiratory rates in hospitalized patients are higher than directly observed measurements. Recorded respiratory rates are significantly more likely to be 18 or 20 breaths/min.

Abbreviations

ICC

intraclass correlation coefficient

IQR

interquartile range