Global proliferation and clinical consequences of non-validated automated BP devices

Abstract

Professional societies, guideline writing committees, and other interested parties emphasize the importance of accurate measurement of blood pressure for clinical and public health decisions related to prevention, treatment, and follow-up of high blood pressure. Use of a clinically validated instrument to measure blood pressure is a central component of measurement accuracy and precision. Despite this, most regulatory authorities do not specify validation requirements that manufacturers must meet to sell their blood pressure measurement devices. Likewise, device validity is not a major area of focus for most consumers and healthcare providers, perhaps because they assume it is a pre-requisite for market approval. This has led to a global proliferation of non-validated blood pressure measurement devices, with only a small minority of blood pressure measurement devices having passed internationally accepted validation protocols. The clinical consequences are likely to be significant because non-validated devices are more likely to provide inaccurate estimates of blood pressure compared with validated devices. Even small inaccuracies in blood pressure measurement can result in substantial misdiagnosis and mistreatment of hypertension. There is an urgent need for clinical validation of blood pressure measurement devices prior to marketing them to consumers. There is also need for simplification of the process for consumers and healthcare providers to determine whether a blood pressure measurement device has successfully met an internationally accepted test of validity.

Introduction

High blood pressure (BP) is related to increased cardiovascular disease (CVD) morbidity and mortality in a directly proportional, continuous, and progressive manner from a systolic BP (SBP) as low as 90 mm Hg to as high as 180 mm Hg(1–3). High BP is not only the single most important risk factor for CVD, but BP lowering provides an effective and practical means to prevent CVD(4). Early proponents of BP measurement, including Janeway and Fisher, drew attention to BP variability and the need to standardize the methods for BP readings(5). In 1966, Armitage and colleagues published laboratory(6) and clinical(7) observations in which they quantified the effect of several factors that predictably influenced level of BP and the effects of within- and between-visit random variability on level of BP. These and subsequent studies have provided the basis for recommendations aimed at accurate estimation of BP.

Accuracy of BP measurements

Although professional societies, guideline writing committees, and others have repeatedly emphasized the importance of accurate BP measurement, a consistent body of evidence suggests this advice is frequently ignored in training settings(8) and in clinical practice(9–11). A meta-analysis of ten clinical studies (N=1484) identified a mean difference, 95% confidence interval (CI), of 6.99, 4.92 – 9.06 mm Hg between SBP measurements in routine clinical practice compared with corresponding values obtained using “research quality” measurements, with the average for routine measurements being consistently higher(10). Many of the individual comparisons in this meta-analysis were ordered, with high values at a routine clinic/office visit triggering the subsequent “research quality” measurement. In the largest comparison of routine and research quality BP measurements, a median of 16 trial BP readings obtained with an automated oscillometric device over a median follow up of 3.2 years in 3074 participants in the Systolic Blood Pressure Intervention Trial (SPRINT) were compared with a median of 17 corresponding routine BP measurements that were entered in the participant’s electronic health record (EHR)(11). It seems unlikely that measurement order would have played a role in this comparison. In the group randomized to the SPRINT Intensive treatment goal, the mean SBP, 95% CI, was 7.3, 7.0–7.6, mm Hg lower for trial measurements compared with the routine outpatient electronic health record readings (Figure 1). The corresponding difference for those randomized to Standard treatment was 4.6, 4.4–4.9, mm Hg. In both treatment arms, the differences were larger for women compared with men. Bland-Altman plots demonstrated low agreement between the two types of BP measurement, with wide SBP agreement intervals that ranged from approximately −35 to 45 mm Hg in both the Intensive and Standard treatment arms. The mean difference in SBP was between 5 and 15 mm Hg in only 43.6% of the Intensive treatment and 33.9% of the Standard treatment group. In addition, the magnitude of the SBP difference varied over time and by trial clinic site. There was no way to correct for the errors in routine BP estimation. These findings underscore the need for accurate BP measurement, especially for those who wish to practice evidence based medicine.

Figure 1.

Comparison of research quality (trial) and routine (clinic) blood pressure measurements in 4,796 Systolic Blood Pressure Intervention Trial (SPRINT) participants. Differences for trial and corresponding electronic heath record mean systolic blood pressures over time in those randomized to Intensive and Standard treatment are displayed in the left hand panel. Bland-Altman plot and clinic comparisons of clinic and trial systolic blood pressure differences by level of systolic blood pressure. Adapted from Drawz P et al.(11).

Use of clinically validated BP measurement devices

Use of a validated BP measurement device is a core component of recommendations to obtain accurate BP readings(12–14). Validation, henceforth referred to as clinical validation, should be performed, independent of the manufacturer, by conducting a detailed study comparing the accuracy (mean difference) and precision (standard deviation of the difference or SDD) of a device relative to blinded, simultaneously performed, two-observer auscultation using the 2018 International Organization for Standardization (abbreviated as ISO) standard that is now recommended as a universal protocol(15–17). During the validation process, care is taken to perform a standardized sequence of measurements under ideal conditions, to analyze the data in a pre-specified manner; and to interpret the findings according to specific criteria. One commonly used criterion for a device to be considered clinically valid is that the mean BP difference between the device-under-test compared to auscultation fall with 5 mmHg and the SDD within 8 mmHg. Clinical validation of distinct BP devices is needed because multiple device components can affect the accuracy of BP measurement, including the pressure transducer, amplifier, signal processing methods, cuff system, and manufacturer-specific proprietary software algorithms used to determine BP.

Several studies have demonstrated that validated BP measuring devices (BPMDs) provide more accurate estimates of BP compared with non-validated instruments(18, 19). Akpolat et al identified 34 upper arm automated BPMDs being used for home sphygomanometry that were characterized as having been (n=21) or not been (n=13) validated(18) in the dabl Educational Trust website. BP values obtained using each BPMD were compared with reference standard measurements obtained by trained observers using a mercury sphygmomanometer. A BPMD was considered to be accurate when the average differences in systolic BP (SBP) and diastolic BP (DBP) readings with the two devices were ≤4 mm Hg. Fifteen (71%) of the 21 validated upper arm BPMDs were deemed to be accurate compared with only four (31%) of the non-validated devices. In a larger study, Jung et al classified only 82 (39%) of 212 upper arm BPMDs used for home BP measurement as valid based on dabl Educational Trust and British Hypertension Society website reports(19). Using a slightly different protocol for comparison of the BPMD and mercury sphygmomanometer BP readings, SBP and DBP differences of ≥5 mm Hg were considered to identify BPMDs that were inaccurate. Inaccurate BP readings were more common for non-validated BPMDs (25 of 130 devices; 19.2%) compared with validated devices (6 of 82; 7.3%).

Validation status of commercially marketed BP measurement devices

In most countries, regulatory authorities do not require, as a condition for market approval, that manufacturers of BPMDs conduct clinical validation testing according to an internationally recognized validation standard(20). Terms like ‘Food and Drug Administration or FDA Cleared’ that are often used by manufacturers do not guarantee clinical accuracy(21). In contrast to ‘high-risk’ and ‘moderate-risk’ devices, such as those that are invasively implanted, BP devices are classified as ‘low-risk’ by regulatory authorities, which reduces the level of regulatory scrutiny that these devices undergo and limits enforcement of false claims. While companies are often asked to submit BP validation data, these can be generated internally, performed with no independent oversight, and need not be peer reviewed or reported in a comprehensive, transparent manner. Further, device manufacturers can file a ‘substantial equivalence to a predicate device’ claim if, in their estimation, a new device is like one that has been previously validated in terms of the device components that can affect measurement accuracy. In such cases, notification can consist of a brief ‘letter to file’ that can circumvent adequate scrutiny and oversight.

Studies that have formally investigated the frequency of prior validation using international standards suggest this occurs rarely(20). Picone et al reported on the validity of BPMDs marketed online in Australia(22). A BPMD was deemed to be valid if it had met the standards required in an internationally accepted protocol for testing in the general population and the results had been published in a peer-reviewed journal or database. Validity was also accepted if the manufacturer claimed the BPMD core technology was identical to that employed in a device that had previously met the required standard. The authors identified 59 businesses that were marketing 972 unique BPMDs online. Only 278 (28.4%) of the devices were based on the recommended upper-arm cuff sphygmomanometry method(12, 13, 23, 24). The percent of BPMDs that had been validated varied from 18.3% for the 278 upper-arm cuff-based devices, to 8% for the 162 wrist-cuff devices, and 0% for the 532 wrist-band wearable devices. The validated devices were more expensive compared with non-validated devices; mean price of 101.1 versus 67.4 Australian dollars, respectively. Upper-arm cuff devices represented 90.5%, 87.8%, and 71.4% of the unique BPMDs marketed by pharmacies, medical, and Australian general retailers. In contrast, the majority (56.5%) of BPMDs marketed by e-commerce businesses were the non-validated wrist-band wearable devices which are not recommended for use(12–14, 23, 24). The e-commerce websites, including Amazon and eBay, stocked a high percentage (92.5%) of the available devices but only 5.5% of them had been validated. The dismal situation identified in Australia has been corroborated globally in an unpublished Medaval analysis of more than 3000 upper arm and wrist cuff BPMDs in which <13% of BPMDs were reported to have been validated(22).

Clinical consequences of inaccurate BP estimation

The extent to which inaccurate estimation of BP results in over or underestimation of hypertension was explored in a regional survey of 1540 adults living in a rural area in Jiangxi, China(25). The analysis was based on an average of BPs at three study visits during which three high quality oscillometric measurements of BP were recorded. Hypertension was defined as either 1) average SBP ≥140 mm Hg, DBP ≥90 mm Hg, or participant report of antihypertensive medication use during the preceding two weeks, or 2) average SBP ≥130 mm Hg, DBP ≥80 mm Hg, or participant report of antihypertensive medication during the preceding two weeks. Figure 2 displays the impact of underestimation or overestimation of SBP/DBP on prevalence of hypertension when classified by the 140/90 SBP/DBP cut-points or taking antihypertensive medication definition. Under and over estimation of SBP/DBP by 2/1 mm Hg resulted in a 5% difference in prevalence. The corresponding difference for a 10/5 underestimate or overestimate of SBP/DBP was 31.3%. For the 130/80 cut-points classification, the difference in prevalence of hypertension for under or over estimation of SBP/DBP by 2/1 mm Hg was 8.8% and by 10/5 was 40.4% (Figure 3). In the 2017–2018 US National Health and Nutrition Examination Survey, a 5 mm Hg SBP and 3.5 mm Hg DBP higher or lower than the standardized NHANES readings increased or decreased the prevalence of hypertension (using the 130/80 mm Hg cut-points) from 32% to 44.4 and 21.9, respectively, in US adults not taking antihypertensive medication. In addition, BP measured without bias but with random error (SD for SBP or DBP of 15 or 7 mm Hg, respectively) resulted in the reclassification of hypertension status in 21.4% of the participants. A similar pattern of misclassification was identified in those taking antihypertensive medication(26). The findings from the Jiangxi and NHANES surveys and other reports indicate that even what might be characterized as a relatively small level of inaccuracy during BP measurement is likely to result in misdiagnosis, mistreatment, and less effective provision of care with adverse health and economic consequences. Studies to determine the health and financial consequences of using clinically invalid BPMDs are needed and should be identified as area of interest by research funding agencies.

Figure 2.

Impact of overestimation or underestimation of systolic/diastolic blood pressure measurements on estimated prevalence of hypertension (average systolic blood pressure ≥140 mm Hg, diastolic blood pressure ≥90 mm Hg, or taking antihypertensive medication) in a Chinese regional survey. Adapted from Fan WG et al.(21).

Figure 3.

Impact of overestimation or underestimation of systolic/diastolic blood pressure measurements on estimated prevalence of hypertension (average systolic blood pressure ≥130 mm Hg, diastolic blood pressure ≥80 mm Hg, or taking antihypertensive medication) in a Chinese regional survey. Adapted from Fan WG et al.(21).

Validation challenges for consumers, practitioners, and manufacturers

Most consumers probably assume that regulatory authorities prevent companies from marketing non-validated BPMDs. During the previously mentioned validation study conducted by Akpolat et al(18), 75% of the study participants reported the primary reason for their selection of a BPMD was simplicity and ease of use. The next most common reason was influence by an advertisement. Less than one percent of the participants cited measurement accuracy as a principal factor in their choice.

Many companies that market BPMDs place little or no emphasis on validation of the devices they are selling. Consequently, consumers and practitioners have little direct access to validation information and face substantial challenges in determining whether a particular device has or has not been validated(27). In a survey of 124 websites in which BPMDs were offered for sale, only six (9%) of the 66 sites that offered a device that had been validated by the British Hypertension Society mentioned this fact on their website(28). As is covered elsewhere in this Special Issue, several validation listings exist, with most providing information specific to a country or region but some provide a broader and more universal listing of validated devices. Manufacturers frequently use different names and model numbers for the same device in different countries. As a result, a BPMD that has been validated may be listed under a name and model number that is different to the corresponding information available to a consumer or their healthcare provider.

The current situation also creates difficulties for device manufacturers who embrace proper clinical validation. Their efforts are often not properly rewarded in the marketplace due to a lack of appreciation for the importance of clinical validation. Providing an exclusive mark or seal of approval that manufacturers could display on their packaging is one solution to encourage and reward companies that invest in proper validation.

Actions that could be taken to simplify and improve the process of identifying clinically validated devices are summarized in Table 1. The optimal approach is implementation of policies that require clinical validation of BPMDs by government agencies. Global and regional organizations such as the WHO and PAHO play an important role in assisting and encouraging governmental agencies in implementing these policies and the related regulatory frameworks. As described elsewhere in this Special Issue, PAHO is working with countries in the Americas to implement such policies(29). In the absence of centralized policies, health systems can develop their own policies and individuals as well as organizations can advocate for effective oversight. Implementation of regulatory policies requires extensive education to explain the need for and facilitate access to intuitive listings of clinical validation for BPMDs. Manufacturers constitute a primary stakeholder required to achieve several of the recommended actions in Table 1. To our knowledge, this special issue of the Journal represents the first attempt to engage both academic leaders and representatives of BPMD manufacturers in a journal collaboration, which includes determining industry perspectives, with the goal of identifying opportunities for increased availability of validated BPMDs, globally. Implementation of BPMD clinical validation policies requires a multifaceted approach which has complexities but is achievable and the tipping point has already been reached in some countries in the Americas. Given the importance of valid BP measurements in clinical practice and public health, the need to develop and implement BPMD clinical validity policies more broadly is urgent. Studies to determine the health and financial consequences of using clinically invalid BPMDs are needed and should be identified as area of interest by research funding agencies.

Table 1.

Recommended actions to facilitate identification of clinically validated BP devices.

Action	Primary Stakeholder(s)
Mandate that all BP devices approved for sale undergo an independent clinical validation study that documents meeting the requirements of an internationally accepted validation standard (ideal state)	Governments, regulatory authorities
Stipulate clearly the changes made to a predicate (previously validated) device that should trigger re-validation versus when ‘substantial equivalence’ and a ‘letter to file’ claim can be made	Regulatory authorities
Provide an easily accessible list of marketed devices available in a country or region according to clinical validation status, with links to full-text validation study reports	Governments, regulatory authorities, manufacturers
Simplify device model nomenclature and avoid unnecessary proliferation of derivative devices or alternative naming of similar models across different regions	Manufacturers
Amalgamate validated device listings and promote their existence, importance, and use	Professional societies, device listing creators
Educate clinicians and consumers regarding the importance of clinical validation and provide simple visual tools such as universally identifiable seals or marks of approval that identify a clinically validated device at the point of sale	Governments, regulatory authorities, manufacturers, professional societies, retailers

Conclusions

Accurate measurement of BP is critically important in determining whether and to what extent a reduction in BP should be prescribed for prevention or management of hypertension. A core component of BP measurement accuracy is use of a valid recording device. Despite emphasis by professional societies, guideline writing committees, and other interested parties, the available evidence suggests that consumers, and healthcare practitioners are unaware of the importance of BPMD clinical validation. BPMDs that have been validated using an internationally accepted protocol are more accurate than devices that have not been validated. Even relatively small inaccuracies result is substantial misdiagnosis and mistreatment of hypertension. There is urgent need for greater oversight of BPMD marketing, including requirements for validation of devices. Hopefully, collaboration between academic leaders, representatives of BPMD manufacturers, and regulators will ensure that only validated BPMDs are able to be marketed for use in clinical practice and research studies.