Comparison of Home and Office Blood Pressure Devices in the Clinical Setting

Abstract

BACKGROUND

Self-measured blood pressure (SMBP) monitoring is increasingly used for remote hypertension management, but the real-world performance of home blood pressure (BP) devices is unknown. We examined BP measurements from patients’ home devices using the American Medical Association’s (AMA) SMBP Device Accuracy Test tool.

METHODS

Patients at a single internal medicine clinic underwent up to five seated, same-arm BP readings using a home device and an automated BP device (Omron HEM-907XL). Following the AMA’s three-step protocol, we used the patient’s home device for the first, second, and fourth measurements and the office device for the third and fifth (if needed) measurements. Device agreement failure was defined as an absolute difference in systolic BP >10 mm Hg between the home and office devices in either of two confirmatory steps. Performance was examined by brand (Omron vs. non-Omron). Moreover, we examined patient factors associated with agreement failure via logistic regression models adjusted for demographic characteristics.

RESULTS

We evaluated 152 patients (mean age 60 ± 15 years, 58% women, 31% Black) seen between October 2020 and November 2021. Device agreement failure occurred in 22.4% (95% CI: 16.4%, 29.7%) of devices tested, including 19.1% among Omron devices and 27.6% among non-Omron devices (P = 0.23). No patient characteristics were associated with agreement failure.

CONCLUSIONS

Over one-fifth of home devices did not agree based on the AMA SMBP device accuracy protocol. These findings confirm the importance of office-based device comparisons to ensure the accuracy of home BP monitoring.

Graphical Abstract

Graphical Abstract.

Poorly controlled hypertension is a known risk factor for cardiovascular disease and stroke, affecting over 108 million adults in the United States and resulting in more than $131 billion in healthcare expenditures each year.¹ Despite strides in hypertension (HTN) management in the last decade, recent data from the National Health and Nutrition Examination Survey (NHANES) suggest that only 43.7% of US adults with HTN had blood pressures (BP) at goal in 2018.²

Access to office-based BP monitoring for timely diagnosis and treatment of HTN has been cited as a major contributor to stagnant HTN control rates.³ Meanwhile, self-measured blood pressure (SMBP) has been demonstrated to significantly improve systolic blood pressure (SBP) control at 6 months compared to routine office-based care.⁴ With widespread adoption of SMBP catalyzed by the growth of telemedicine and remote monitoring solutions during the COVID-19 pandemic, many view SMBP as a scalable approach to HTN control at the population level.⁵ However, effective SMBP is predicated on the accuracy of devices used to measure BP at home and their agreement with office BP measurements. While the importance of using devices listed on validation registries is well-documented,⁶ a myriad of patient-level factors can influence individual device accuracy.^7–9 Recognizing this challenge, the American Medical Association (AMA) published a Device Accuracy Test protocol to facilitate comparison between home and office devices prior to beginning SMBP.¹⁰ However, data describing the performance of this accuracy testing protocol in the clinical setting is lacking.

In this study, we examined the real-world performance of home BP devices among internal medicine patients using the AMA’s SMBP Device Accuracy Test protocol. We hypothesized that measurement disagreement between home and office devices would be common due to device characteristics (different algorithms, cuff fit) and potentially related to patient characteristics (e.g., age or body mass index [BMI]). Note that we intentionally avoided the word validation throughout this manuscript, as the SMBP tool is meant to determine if the device is accurate for the patient intending to use the device, and not meant to inform the device’s overall accuracy (the purpose of validation testing).

METHODS

Study population

This study was conducted among adult patients (age 18 years or older) of the Beth Israel Deaconess Medical Center (BIDMC) Healthcare Associates internal medicine clinic between October 2020 and November 2021. Healthcare Associates is an academic outpatient practice consisting of physicians, pharmacists, nurse practitioners, and nurses. The clinical pharmacist saw patients referred by their primary care team for HTN follow-up and comparison of their own BP devices (if they had one) or of a device loaned to them from the clinic.

This project was determined by the BIDMC Institutional Review Board to be human subjects exempt research as a quality improvement (QI) initiative. We examined the data from the BP device comparison visits of 152 patients.

Blood pressure measurement

A clinical pharmacist compared the BP devices according to the AMA’s SMBP Device Accuracy Test protocol.¹⁰ The protocol entailed up to five seated, sequential, same-arm BP readings using a combination of the patient’s home device or a device provided by the clinic and an Omron HEM-907XL (Omron Healthcare Inc., Lake Forest, IL, USA) taken during a clinic visit. All home devices were upper arm devices, though brands and models varied (e.g., Omron could be series 3, 5, 7, and 10). Cuff size was based on range indicators printed on the cuff or via paper tape measurement of arm circumference. Patients requiring cuff sizes outside of the range permissible by the Omron HEM-907XL or outside the range permitted by their home device’s cuff did not participate in this study. We only used cuffs that were indicated for each device. We did not use different cuffs with different brands. We also did not use the Omron HEM-907XL cuff in place of any of the home device cuffs.

The first (measurement A), second (measurement B), and fourth (measurement D) BP readings were measured using the patient’s home device and the third (measurement C) and fifth (measurement E) readings were measured using the office device according to the SMBP protocol (Figure 1). The protocol calls for the first measurement to be discarded and then to average measurements B and D (patient device; Step 1). This average is compared with measurement C (office device) (Step 2). Measurement E was only obtained if the difference between the average home device SBP and average office device SBP fell within 6–10 mm Hg as specified by the Accuracy Test.¹⁰ In these situations, measurement D (patient device) is compared to the average of measurements C and E (office device) (Step 3). A device would pass agreement testing if the difference in the initial comparison between home and office was ≤5 mm Hg (Step 2) or if the difference in the second comparison between home and office was ≤10 mm Hg (Step 3). Diastolic BP (DBP) is not used in the AMA’s SMBP Device Accuracy Test protocol.

Figure 1.

Schematic of the American Medical Association’s Device Accuracy Test Protocol.

Prior to measurement, all patients were positioned with their feet flat on the ground, their backs supported, and their arm elevated at heart level and resting on an adjacent table. Arm selection varied based on practical considerations (room configuration), patient preference, or clinical indications to avoid a specific arm (e.g., arteriovenous fistula). The same arm was used for all BP measurements taken during the testing. All patients underwent a 5-min rest prior to testing, but this rest was not required to immediately precede agreement testing. Moreover, there was no fixed interval pause between measurements in concordance with the AMA protocol.

Other covariates

Baseline demographic data collection was obtained from retrospective chart review and included age, sex, Black race (yes or no), height, weight, and smoking status. Body mass index (BMI) was calculated by dividing weight by the square of patient height and was based on the most recent measurement preceding the visit, no more than 3 years prior the clinic visit. Relevant medical history, including history of diabetes, cardiovascular disease, statin use at the time of the visit, and creatinine at the time of the visit were also documented. The brand of the patient’s device was directly observed and the approximate device age was collected through patient report.

Statistical analysis

We determined the mean SBP and DBP and their corresponding standard deviations for each of the three steps of the AMA protocol. There were three patients with Step 3 BP measurements when it was not indicated. Analyses were restricted to only those measurements that were supposed to undergo Step 3 based on the protocol. The absolute value of the differences in SBP readings between home devices and office devices was also calculated, and used to classify device agreement testing as passing, failing, or borderline. Device agreement failure was defined as a >10 mm Hg absolute difference between the patient’s average SBP measured by the patient’s device compared to the office device. The devices passed if there was a ≤5 mm Hg absolute difference between the home device and office device. Device agreement was considered borderline if the SBP difference was 6–10 mm Hg. For all tested devices, the agreement failure rates and corresponding 95% confidence intervals were determined for Step 2 and Step 3 of the SMBP protocol.

For devices that passed using the SBP criterion described above, the AMA SMBP protocol was extended to DBP readings. Namely, the absolute value of the difference between average DBP measured using the home and office devices was calculated and the failure rates were similarly obtained. A subgroup analysis was conducted to determine failure rates separately for Omron and non-Omron home devices. This subgroup was chosen because: Omron devices were common, all home Omron devices were validated, and Omron was the brand of the office device, making it unique from other brands.

We examined the associations of the following patient and device factors with device failure: patient age, sex, BMI, comorbid diabetes, comorbid cardiovascular disease, and current statin use. Associations were determined via independent logistic regression models adjusted for demographic characteristics (age, sex, and Black race). We examined current statin use as a surrogate for elevated cardiovascular risk.

Two-tailed, paired t-tests were used to calculate P-values for comparing agreement failure rates across devices and P-values < 0.05 were considered statistically significant. All analyses were performed with Stata 15.1 (StataCorp LP, College Station, TX).

RESULTS

Population characteristics

We examined data from 152 BP devices compared in clinic between October 2020 and November 2021. Patients were a mean age of 60.4 years (SD 15.3; range of 26–95 years) with 59% women, 32% Black, and an overall average BMI of 30.3 (SD 5.8) of which 49% were obese (BMI ≥ 30) (Table 1). Devices were 62% Omron, 7% A&D, 1% Microlife, 30% other brands, and 1% unknown.

Table 1.

Baseline characteristics

	Overall population
	N	Mean (SD) or %
Age, year	152	60.4 (15.3)
Women, %	152	59
Black, %	152	32
Hispanic or Latino, %	136	7
Body mass index, kg/m2	149	30.3 (5.8)
Last creatinine, mg/dL	152	0.9 (0.3)
Body mass index ≥30 kg/m2, %	149	49
Diabetes, %	152	17
Cardiovascular disease, %	152	13
Current statin use, %	152	34
Smoking status, %
Never	152	71
Former	152	25
Current	152	4
Approximate device age, years*	108	1.8 (2.3)
Brand
A&D	152	7
Microlife	152	1
Omron	152	62
Other	152	30
Unknown	152	1

*Range: 0–18 years.

Comparison of device measurements

Mean BP measures resulting from the Accuracy Tool may be found in Table 2. The mean SBP from home measurements B and D from Step 1 was 141.8 mm Hg (SD, 18.9) which resulted in a mean difference from the office measure C (Step 2) of 6.2 mm Hg (SD, 5.7). Thirty-seven of the 152 devices were indicated for Step 3 testing and demonstrated a mean difference of 6.3 mm Hg (SD, 4.3).

Table 2.

Mean BP by validation step

Systolic blood pressure	N	Mean (SD)
Home device measurement 1 (A)	152	144.7 (20.1)
Home device measurement 2 (B)	152	142.0 (19.7)
Office measurement 1 (C)	152	139.8 (17.7)
Home device measurement 3 (D)	152	141.5 (19.0)
Office measurement 2 (E), optional*	37	136.9 (17.7)
Step 1 mean: ([B + D]/2)	152	141.8 (18.9)
Step 2 mean: abs(C − [B + D]/2)	152	6.2 (5.7)
Step 3 mean: abs(D − [C + E]/2)*	37	6.3 (4.3)
Diastolic blood pressure
Home device measurement 1 (A)	152	87.9 (11.0)
Home device measurement 2 (B)	152	86.2 (11.1)
Office measurement 1 (C)	152	81.7 (11.4)
Home device measurement 3 (D)	152	85.8 (11.0)
Office measurement 2 (E), optional*	37	78.5 (12.4)
Step 1 mean: ([B + D]/2)	152	86.0 (10.8)
Step 2 mean: abs(C − [B + D]/2)	152	5.9 (4.6)
Step 3 mean: abs(D − [C + E]/2)*	37	5.7 (3.3)

*Step 3 was only performed if systolic blood pressure performance was borderline in Step 2. Of the 42 patients that were eligible for Step 3, there were five patients missing these data, who had Step 2 differences of 5.5, 8.0, 9.0, 9.5, and 10.0 mm Hg.

Of devices tested during Step 2, 55.9% (95% CI: 47.9%, 63.7%) passed agreement testing, 27.0% (95% CI: 20.5, 34.6) were borderline, and 17.1% (95% CI: 11.9, 24.0) failed agreement testing (Table 3). Among the 37 Step 2 borderline cases, 18.9% (95% CI: 9.1%, 35.3%) also failed to agree during Step 3. In aggregate, 22.4% (95% CI: 16.4%, 29.7%) of devices failed agreement using the Accuracy Tool. Performance was comparable between Omron and non-Omron home devices with the proportion of agreement failures being 19.1% (95% CI: 12.3%, 28.5%) for Omron versus 27.6% (95% CI: 17.5%, 40.7%) for non-Omron home devices (P = 0.23).

Table 3.

Failure rates by Omron vs. non-Omron devices

	N	% (95% CI)
All devices
Step 2: Passed devices	152	55.9 (47.9, 63.7)
Step 2: Borderline	152	27.0 (20.5, 34.6)
Step 2: Failures	152	17.1 (11.9, 24.0)
Step 3: Failures (of those tested)	37	18.9 (9.1, 35.3)
Total Failures	152	22.4 (16.4, 29.7)
Omron devices
Step 2: Passed devices	94	60.6 (50.3, 70.1)
Step 2: Borderline	94	24.5 (16.7, 34.3)
Step 2: Failures	94	14.9 (9.0, 23.7)
Step 3: Failures (of those tested)	24	16.7 (6.1, 38.3)
Total failures	94	19.1 (12.3, 28.5)
Non-Omron devices
Step 2: Passed devices	58	48.3 (35.5, 61.2)
Step 2: Borderline	58	31.0 (20.3, 44.3)
Step 2: Failures	58	20.7 (12.0, 33.3)
Step 3: Failures (of those tested)	16	25.0 (8.9, 53.3)
Total Failures	58	27.6 (17.5, 40.7)

P comparing failures across Omron and non-Omron devices was 0.23.

Our examination of alternate protocols demonstrated greater agreement failure rates with the incorporation of the first home measurement (Supplementary Table 1). Notably, restricting to the second home measurement only (rather than a mean of two), resulted in an identical agreement failure rate.

Diastolic blood pressure performance

We examined the accuracy of diastolic blood pressure (DBP) among devices that passed agreement testing (Figure 2). There were 50.0% (95% CI: 41.0%, 59.0%) with a DBP difference ≤5 mm Hg, 35.6% (95% CI: 27.4%, 44.7%) with a difference in DBP between 6-10 mm Hg, and 14.4% (95% CI: 9.1%, 22.1%) with a difference in DBP >10 mm Hg.

Figure 2.

Performance of devices based on diastolic blood pressure among devices that passed the American Medical Association’s device accuracy testing (N = 118). Vertical bars represent 95% confidence intervals.

Participant characteristics and device performance

None of the participant characteristics examined were associated with device performance (Table 4). There was a borderline association between BMI and device failure, with a higher BMI being associated with a higher odds of device failure (OR 1.07; 95% CI: 0.99, 1.15). However, this association was not statistically significant.

Table 4.

Factors associated with device failing

	N	OR (95% CI)	P
Age per 1 year	152	1.00 (0.98, 1.03)	0.71
Female	152	1.10 (0.48, 2.55)	0.82
Body mass index per 1 kg/m2	149	1.07 (0.99, 1.15)	0.073
Diabetes	152	1.51 (0.56, 4.07)	0.42
Cardiovascular disease	152	1.97 (0.63, 6.14)	0.25

Logistic regression models adjusted for age, sex, and Black race. Models were independent and structured as follows: Y_{[Device failure]} = X_{[Exposure of interest]} + X_[age] + X_[sex] + X_{[Black race]}. In cases of age or female sex, these covariates were used only once.

DISCUSSION

In this analysis of 152 patients, we found that over 20% of home devices did not agree with the office BP device within 10 mm Hg for SBP. Moreover, 19% of Omron devices, which are listed in validation registries and were the same brand as the office device, were discordant with the office device. Among passing devices, 14% did not measure DBP within 10 mm Hg of the office device. While higher BMI was suggestive for an association with device failure, there were no statistically significant predictors of device failure. These findings provide much-needed data on discordant measurements between home and office BP devices, which may be informative for clinical decision-making based on measurements obtained outside of the medical office.

Multiple guidelines recommend SBPM as a tool for diagnosing HTN, managing HTN, and as a behavioral intervention.^11,12 However, the success of these clinical applications rests upon device measurement accuracy overall and with the individual patient using the device. A recent study by Picone and colleagues¹³ determined that only 18% of 278 upper arm BP devices sold online had successfully undergone validation testing. The same research group conducted a global study of 2,486 BP devices and concluded that only 10% were listed on the Medaval list and thus considered valid.¹⁴ While professional registries list devices achieving validation criteria, how these products perform with individual patients may be different than suggested by the validation testing (which needs to show the device is accurate within 5 ± 8 mm Hg across the study population).¹⁵ To address this, the AMA published their Device Accuracy Test protocol to evaluate the performance of devices among individual patients compared to an office standard. This accuracy tool was developed based on a study of 92 adults with hypertension that used BP measured by auscultation with a mercury sphygmomanometer as the reference. Absolute differences in SBP were compared between the home BP device and sphygmomanometer readings to create a simplified accuracy tool.¹⁰ However, data on the implementation of this tool is limited, particularly among clinics that use automated oscillometric devices as a reference in place of auscultation.

Our study identified a high proportion of device failures even when restricted to validated devices that were the same brand as the clinic device. However, given different algorithms employed by different BP devices for measurement estimation, some degree of discordance would be expected and should not necessarily question whether a brand of home device will yield an accurate result on average. Moreover, error can also result from the office device itself. Given the substantial variability in BP measurement between office visits,¹⁶ there may be value in rechecking discordant devices on another day or even with a second office device, something not performed by our study. Nevertheless, our findings illustrate a higher-than-expected proportion of disagreement between validated home and office devices, which may be informative for understanding discordant BP values detected during routine clinical care.

The AMA Device Accuracy Test protocol does not incorporate DBP. While DBP has been less of a focus of HTN treatment goals in recent hypertension trials like SPRINT,¹⁷ it continues to be used in guidelines for the diagnosis of HTN. Thus, inaccurate measurements of DBP have the potential to impact patient care, particularly among adults with isolated diastolic hypertension and those with widened pulse pressures.¹⁸ Fourteen percent of devices with accurate SBP had inaccurate DBP measurements using the same standard applied to SBP. Failure rates would be substantially higher if half the SBP standard (5 mm Hg) was used. This finding suggests a potential role for incorporating DBP into current accuracy tools.

Performing in-office device comparisons during clinic visits takes time. While the accuracy tool does not require a 5-min rest prior to measurement, BP measurement should be preceded by at least a 3-min rest.¹⁹ Devices themselves may take 20–45 s from inflation to deflation prior to yielding a result, totaling about 4–5 min for the measurement alone, not including instructions and set-up. As a result, it may be advantageous to compare devices with fewer measurements. Our study indicates that a single home measurement (the second) identified 84% of the failing devices vs. 73% from using the first measurement alone. Thus, if time is limited, a single measurement may be used with the recognition that some discordant devices may be missed with this approach.

Our study has limitations. First, BMI was non-significantly associated with device performance; however, we did not have recent BMI measures for all patients. Moreover, the presence of conical arms was not documented. This represents an important area for future research. In addition, while both office cuff size and home cuff size were always assessed for appropriateness, including wide range cuffs, these sizes were not consistently documented. Wide range cuffs may be less accurate and should be evaluated in subsequent research. Second, it is possible that device age could influence performance. However, many patients did not know when their device was purchased. Greater characterization of device performance over time would be useful in subsequent studies. Third, the Omron brand was disproportionately represented in our study. While other validated devices were also used by patients, we did not have large enough numbers of other brands for more detailed comparisons. Moreover, whether these devices were validated or not was not consistently documented among non-Omron devices. As a result, we were unable to perform a direct comparison across validated and non-validated devices. Fourth, oscillometric BP devices use proprietary algorithms for estimating SBP and DBP, which may have led to more favorable results for Omron devices since the Omron HEM907-XL was our reference device. Fifth, we had no patients with small or extra-large arms thus limiting the generalizability of our findings to these groups. Sixth, five patients with borderline performance at Step 2, did not undergo a Step 3 measurement for unclear reasons. In our analyses, these were not considered to have failed agreement testing based on Step 2; however, it is possible some may have failed Step 3. As a result, our approach may underestimate device failure rates. Seventh, it should be noted that the performance characterized in our study may not generalize to other patient populations such as adults with chronic kidney disease, where devices have been described to have worse performance.²⁰ Eighth, our sample size was based on convenience. Thus, it is possible we were under-powered for analyses looking at factors associated with device failure. Finally, wrist devices were not tested in our study.

Our study had several strengths. First, we included devices that were not listed on the U.S. Blood Pressure Validated Device Listing (validatebp.org) allowing patients to use both previously purchased devices and other devices that were covered by third-party payers. Second, we used an automated, oscillometric device as our reference, reducing possible human error such as deflation speed and zero-digit preference. Moreover, this was the device used in SPRINT,¹⁷ which has strong clinical validity. Third, we performed the validation portion of the visit during actual provider visits demonstrating the feasibility of using this tool clinically. Fourth, we used a larger population than the original study that informed the validation tool,¹⁰ which broadens the implications of our results and provides new data on an aspect of home monitoring, which previously lacked real-world evidence.

Our study has clinical implications. Out-of-office BP monitoring has a U.S. Preventive Task Force Grade A recommendation for diagnostic confirmation of HTN before initiating treatment.²¹ Moreover, home BP monitoring has been on the rise in the U. S. in the context of both the COVID-19 pandemic⁵ as well as the advent of new reimbursement codes.³ While a number of professional societies have published vetted lists of accurate BP devices,²² there are many more of unclear accuracy available for consumer purchase. The AMA’s accuracy tool is intended to help clinicians evaluate the accuracy of devices at the patient-level compared to a clinic-based standard. Our real-world data demonstrates that about 1 in 5 devices, even when restricted to validated brands identical to the clinic reference brand, will yield a SBP that is different from the clinic device by more than 10 mm Hg. Moreover, an additional 14% of devices with an accurate SBP had an inaccurate DBP. These findings highlight a need for caution with SBPM and a potential role for recurrent comparisons with clinic BPs to ensure safety with BP treatment titration based on SBPM measurements.

In conclusion, this QI analysis of 152 clinic patients found that 1 of 5 home BP devices did not agree with office devices based on the AMA SMBP Device Accuracy Test protocol. An additional 14% of devices with an accurate SBP had a discordant DBP. These findings are informative for interpreting SBPM and suggest a role for in-office comparisons between devices to ensure the safety of BP treatment in response to SBPM measurements.

Funding

SPJ is supported by NIH/NHLBI grant 7K23HL135273-02 and by Healthcare Associates for QI related to hypertension.

Conflict of Interest

The authors have no conflicts of interests to declare.