Evaluation of a community-based automated blood pressure measuring device

Abstract

Background

Automated devices are widely available in the community for people to measure their blood pressure. We assessed the accuracy and reproducibility of a brand of community-based automated device against the standard mercury sphygmomanometer.

Methods

Same-arm pairs of blood pressure readings were obtained with the Vita-Stat 90550 automated device, a sphygmomanometer and the Omron HEM-705CP automated device in random order on volunteers in 3 community pharmacies using a modified protocol for evaluating blood pressure devices. Comparison of readings between the Omron device and the sphygmomanometer served as a positive control of how well a laboratory-validated automated device could perform in the community. Both the Association for the Advancement of Medical Instrumentation (AAMI) and British Hypertension Society (BHS) criteria were used to assess the accuracy and reproducibility of readings.

Results

The mean blood pressure reading and standard error (SE) of the mean for the 108 volunteers (66 women and 42 men) was 133/77 (SE 2/1) mm Hg with the Vita-Stat device, 131/77 (SE 2/1) mm Hg with the Omron device and 129/76 (SE 2/1) mm Hg with the sphygmomanometer. The mean difference in readings was 4.4/1.0 (standard deviation [SD] 9.4/6.2) mm Hg between the Vita-Stat device and the sphygmomanometer and 1.6/0.6 (SD 9.3/6.4) mm Hg between the Omron device and the sphygmomanometer. Neither automated device met the AAMI accuracy criteria for the systolic readings. The BHS grades were C/A (systolic unacceptable/diastolic acceptable) for each automated device. According to the BHS analytical criterion, all devices achieved acceptable reproducibility grades.

Interpretation

Neither automated device met the AAMI or BHS criteria for accuracy while in use in the community, and neither performed as well in the community as in the laboratory.

Measurement of blood pressure outside the office setting, using ambulatory monitors, home recorders or community-based devices has become popular among both physicians and patients. These devices may help to improve patients' involvement in their care¹ and they may allay physicians' concerns about a possible “white-coat syndrome.” However, incorrect readings could lead to a false sense of security or incorrect clinical decisions.

The British Hypertension Society (BHS)² and the Association for the Advancement of Medical Instrumentation (AAMI)³ have developed laboratory protocols to evaluate automated blood pressure measuring devices. Many devices have failed to meet minimum standards for accuracy and reproducibility.⁴

One community-based device, the Vita-Stat, has been available in various models since 1976, although none has performed uniformly well in community evaluations.^5,6,7,8,9 The newest model, the Vita-Stat 90550, available in about 3000 Canadian communities since 1990, provides 40 million readings yearly (Fred Sarkis, Spacelabs Medical: personal communication, 2000). Hence, we decided to evaluate the Vita-Stat 90550 against the mercury sphygmomanometer for accuracy and reproducibility in the community. To assess how well a laboratory-validated device could perform in the community, we compared the Omron HEM-705CP, which has met both the BHS and the AAMI criteria,¹⁰ against the mercury sphygmomanometer.

Methods

Customers more than 18 years old at 3 North Toronto pharmacies were approached to participate in the study unless they had an irregular pulse, their arm was too large to fit into the Vita-Stat or they failed to provide informed consent. Subjects answered a questionnaire about their age, sex, hypertension history and present antihypertensive medication. The study was approved by the Sunnybrook Health Science Centre's ethics review board and permitted by the participating pharmacies.

The attributes of each device are listed in Table 1. All measures were performed on the left arm to reflect the Vita-Stat device's configuration. Sequential blood pressure readings in duplicate were obtained with the Vita-Stat device, the Omron device and a mercury sphygmomanometer in random blocked order, with each subject resting 2 minutes before the first measurement. Readings were separated by the time required to move between devices to a chair beside the Vita-Stat device. One minute elapsed between paired recordings. Operation of the sphygmomanometer followed the Canadian Hypertension Society guideline.¹¹ Accuracy of sphygmomanometer measurement by the operator (L.M.) was confirmed by comparing readings, on 10 subjects, with those obtained by a hypertension specialist (M.G.M.) using a double-headed Y stethoscope before commencing the study.

Table 1

To conduct the community-based assessment of the devices, we modified aspects of the BHS laboratory protocol² by assuming that the devices should have met laboratory criteria before community placement. We used a convenience sample without criteria based on age, pressure levels or arm circumference because there were no limitations stated on the Vita-Stat machines. Finally, to minimize disruption to customer flow and volunteer time, all sequential paired readings with each device were taken by 1 observer.

Analysis

The mean blood pressure reading and standard error (SE) of the mean for each device by site was obtained. Two-way analysis of variance was used to assess any differences due to device or site effects. Descriptive statistics were used for nominal data.

For analysis of agreement, we used the method of Bland and Altman,^12,13 which plots the mean of 2 measures for comparison on the horizontal axis against the difference between the measures on the vertical axis. The cumulative percentage of readings with absolute differences of 5, 10 and 15 mm Hg were then calculated for comparison against the BHS criteria.

To assess accuracy, we initially calculated the mean of each paired reading and then calculated and plotted the Vita-Stat and Omron devices each against the sphygmomanometer. In addition, we computed the mean difference in readings, the standard deviation (SD) of the mean difference and the 95% limits of agreement¹² for comparison against the AAMI criteria.

To assess reproducibility of each device, we plotted the mean of the duplicate readings by device against the difference between the duplicate readings. We did not reassess reproducibility with AAMI criteria.

Results

Thirty-six subjects agreed to participate at each pharmacy, for a total of 108 subjects (42 men and 66 women); the median age was 59.5 years (range 20–87 years). Forty-six subjects (42.6%) had a previous diagnosis of hypertension, of whom 31 (67.4%) were receiving treatment. The mean arm circumference was 29.3 (SD 3.5) cm (range 20.0–39.0 cm).

Because the analysis of variance showed no device, site or interaction effects, the results were pooled by device. The mean blood pressure reading was 133/77 (SE 2/1) mm Hg for the Vita-Stat device, 131/76 (SE 2/1) mm Hg for the Omron device and 129/76 (SE 2/1) mm Hg for the sphygmomanometer. The Omron device failed to measure the blood pressure in one subject for unknown reasons.

Fig. 1 displays the Bland–Altman plots of the systolic and diastolic readings from the Vita-Stat device versus those from the sphygmomanometer for each of the 108 subjects. The Omron readings versus the sphygmomanometer readings and the reproducibility analyses were plotted similarly (data not shown). The cumulative percentages of readings with absolute differences of 5, 10 and 15 mm Hg are shown in Table 2 for the accuracy and reproducibility assessments of the devices. In terms of accuracy, the BHS grade was C/A (systolic unacceptable/diastolic acceptable) for each of the automated devices. In terms of reproducibility, the Vita-Stat and Omron automated devices and the sphygmomanometer each achieved an acceptable BHS grade (B/A, A/A and A/A respectively).

Fig. 1: Agreement in blood pressure readings from 108 volunteers between a sphygmomanometer and the Vita-Stat 90550 automated device in a community setting, as determined using the Bland–Altman method.¹²,¹³ For each subject, the difference between the readings of the 2 devices is plotted against the mean of the readings from both devices. Horizontal dashed lines represent readings that differ from –15 mm Hg to 15 mm Hg, in increments of 5 mm Hg. White circles = diastolic pressure, black circles = systolic pressure.

Table 2

Table 3 shows the mean differences (and SDs) in blood pressure readings and the 95% limits of agreement between the sphygmomanometer and the automated devices. All of the mean differences in readings between the automated devices and the sphygmomanometer met the AAMI criterion (≤ |5| mm Hg) for accuracy; however, the SDs of the mean differences met the criterion (≤ |8| mm Hg) only for the diastolic readings.

Table 3

Interpretation

There are neither published protocols nor minimum standards for community-based evaluations of automated blood pressure measuring devices designed for community use. The BHS protocol states that a rating of at least grade B/B, under standardized laboratory conditions, is required before a device can be approved as “acceptable” for use in clinical practice.¹⁴ Laboratory studies of the Vita-Stat 90550 resulted in a grade B/C compared with the mercury sphygmomanometer,¹⁵ as opposed to a grade B/A for the Omron HEM-705CP device.¹⁰ On the basis of our community-based readings, both the Vita-Stat and the Omron devices received a grade C/A with the use of a modified BHS protocol.

The AAMI criteria require that all mean differences in readings between the tested device and a sphygmomanometer be ≤ |5| mm Hg and that all SDs of the mean differences be ≤ |8| mm Hg. Neither automated device in our study met these criteria for the systolic readings for accuracy assessment. The 95% limits of agreement suggest, for example, that a systolic blood pressure reading with the Vita-Stat device could be as much as 14.1 mm Hg below or 22.8 mm Hg above the systolic reading with a sphygmomanometer.

Since neither device performed as well in the community as in the laboratory, several possible limitations should be considered. The 3 Vita-Stat machines we tested were not assessed for intra- or interdevice variability before the study because the object of the study was to evaluate machines in everyday use. The observer showed an end-digit preference for “0” over “2”; however, a sensitivity analysis did not demonstrate an effect. Also, there was no bias due to an order effect of the devices. We attempted to offset any regression to the mean by device by averaging the paired readings for the accuracy analysis.

Because the community-based devices state no restrictions to their use, our study protocol had few exclusions. Thirty-two (29.6%) of the subjects were over 70 years old. Clark and colleagues¹⁶ suggested that oscillometric technology may be less accurate in elderly people. Also, 9 subjects (8.3%) had an arm circumference outside the Vita-Stat device limits, and 15 (14.0%) had an arm circumference outside the Omron device limits.

The present study will not reflect device performance for very high or low blood pressure, because all but 2 of the subjects had blood pressures between 100/60 and 180/100 mm Hg. For a clinical cutoff blood pressure of 140/90 mm Hg, as defined by the sphygmomanometer, the Vita-Stat and the sphygmomanometer were in agreement in 91 cases, 63 of which indicated normotensive readings. In 13 cases the Vita-Stat device gave high readings while the sphygmomanometer indicated normotensive pressure, and in 4 cases the Vita-Stat device gave normal readings while the sphygmomanometer indicated high blood pressure.

On the basis of our findings, reliance on blood pressure readings taken by the Vita-Stat 90550 device could result in an inaccurate or missed diagnosis of hypertension or it could lead to an inaccurate assessment of blood pressure control in patients already receiving antihypertensive therapy.