Validation protocols for blood pressure measuring devices in the 21st century

Abstract

Blood pressure (BP) is a vital sign and the essential measurement for the diagnosis of hypertension. Therefore, its accurate measurement is a key element for the evaluation of many medical conditions and for the reliable diagnosis and efficient treatment of hypertension. In the last 3 decades prestigious organizations, such as the US Association for the Advancement of Medical Instrumentation (AAMI), the British Hypertension Society, the European Society of Hypertension (ESH) Working Group on BP Monitoring, and the International Organization for Standardization (ISO), have developed protocols for clinical validation of BP measuring devices. All these initiatives aim to standardize validation procedures and establish minimum accuracy standards for BP monitors. Unfortunately, only a few of the BP measuring devices available on the market have been subjected to independent validation using one of these protocols. Recently, the AAMI, ESH, and ISO experts agreed to develop a single universally acceptable standard (AAMI/ESH/ISO), which will replace all previous protocols. This major international initiative has been undertaken to best serve the needs of patients with hypertension, a public interested in cardiovascular health, practicing physicians, scientific researchers, regulatory bodies, and manufacturers. There is an urgent need to influence regulatory authorities throughout the world to make it mandatory for all BP measuring devices to have undergone independent validation before approval for marketing. Efforts need to be intensified to improve the accuracy of BP measuring devices, further optimize the validation procedure, and ensure that objective and unbiased validation data become available.

1. HISTORY OF VALIDATION PROTOCOLS

Blood pressure (BP) is a hemodynamically variable phenomenon and a vital sign, and it is also the essential estimation for the identification of patients with hypertension; its accurate measurement is, therefore, a key element for the evaluation of many medical conditions and for the reliable diagnosis and efficient treatment of hypertension.1 Noninvasive auscultatory and oscillometric devices provide an “estimation” of BP rather than a true “measurement,” which can be obtained only using a direct intra‐arterial method. However, in this article intended for practicing clinicians we use the term “measurement,” in order to be in line with the wording used in current guidelines for the management of hypertension in clinical practice. Unfortunately, at the present time formal clinical validation is not mandatory in all countries for BP measuring devices to be put on the market, and there is evidence that fewer than 20% of the devices available have been subjected to validation using an established protocol.1, 2

Validation of BP monitors was started with ad hoc protocols in the 1980s,1 and since then several procedures have been developed by prestigious organizations (Table 1). In 1987 the US Association for the Advancement of Medical Instrumentation (AAMI) introduced a standard including a procedure for clinical validation of an automated BP monitor.3 The British Hypertension Society presented a similar clinical validation protocol in 1990.4 Revised versions of both protocols have been published.5, 6 The German Hypertension League developed another protocol in 1999.7 In 2002, the European Society of Hypertension Working Group on BP Monitoring developed the ESH‐International Protocol (ESH‐IP),8 which was revised with more stringent criteria in 2010.9 The European Committee for Standardization (CEN) published a standard in 2004.10 In 2009 the International Organization for Standardization (ISO)11 developed another standard, which was largely based on the AAMI and CEN standard and has been adopted by the AAMI Sphygmomanometer Committee12 and other national committees worldwide. A revised version of the American National Standards Institute (ANSI)/AAMI/ISO standard was developed in 2013.13 With the expanding use of out‐of‐office BP monitoring (ambulatory and home) in clinical practice, which is almost exclusively performed using electronic devices, and the progressive replacement of office or clinic auscultatory devices with electronic ones, the need for formal clinical validation has now become more crucial than ever.

Table 1.

History of validation protocols

Publication	Organization
1987, 1992, 2002	US Association for the Advancement of Medical Instrumentation (AAMI)3, 5
1990, 1993	British Hypertension Society (BHS)4, 6
1999	German Hypertension League (Deutsche Hochdruckliga) (DHL)7
2002, 2010	European Society of Hypertension International Protocol (ESH‐IP)8, 9
2004	European Committee for Standardization (CEN)10
2009	International Organization for Standardization (ISO)11
2009, 2013	American National Standards Institute/Association for the Advancement of Medical Instrumentation/International Organization for Standardization (ANSI/AAMI/ISO)12, 13
2018	Association for the Advancement of Medical Instrumentation/European Society of Hypertension/International Organization for Standardization (AAMI/ESH/ISO)14

2. COMPARISON OF VALIDATION PROCEDURES

The aforementioned validation procedures have many similarities but also several methodological and technical differences. The main differences between the ESH‐IP9 and the ANSI/AAMI/ISO standard,13 which currently are the most widely accepted and used, may be summarized as follows:

Efficacy measure

The two protocols require a similar level of device accuracy in BP measurement with estimated probability of a tolerable error (≤ 10 mm Hg) at 85%. This is not an ideal level of accuracy but a realistic one, taking into account the performance of currently available technology and the patient's physiologic BP variability within the validation procedure. Further technological development aiming to a higher level of accuracy is desirable.

Sample size

The ESH‐IP requires 33 participants compared to 85 for the ANSI/AAMI/ISO. The larger sample size of the latter protocol results in greater study power and sensitivity and allows the evaluation of subgroups, eg, per cuff size, but requires considerably greater investment in terms of resources and costs.14

General and special populations

The ESH‐IP has been developed for a general population of adults aged ≥ 25 years and requires separate studies for special populations (eg, in pregnancy). The ANSI/AAMI/ISO for general population studies requires participants aged > 12 years and additional 35‐participant subgroups for special populations (apart from pregnancy, which requires 45 women). Thus, for special populations the sample size is similar in the ESH‐IP and the ANSI/AAMI/ISO (33 vs 35 participants respectively). Both protocols consider participants aged ≤ 12 years as a special population.

Reference BP measurement

The ESH‐IP recommends the use of a mercury sphygmomanometer, whereas the ANSI/AAMI/ISO accepts any manometer with maximum error ±1 mm Hg. The ANSI/AAMI/ISO also provides requirements for reference invasive BP measurements.

Methods for comparison

For the comparison of the test device BP measurements with the reference method measurements, a same‐arm sequential BP measurement procedure is recommended in the two protocols. The ANSI/AAMI/ISO also permits a simultaneous (same or opposite arm) BP measurement procedure.

Limb size and cuffs

The ANSI/AAMI/ISO has requirements for the minimum number of participants per cuff size, whereas the ESH‐IP, because of its smaller sample size, does not allow subgroup analyses (eg, per cuff size) but only as special groups (eg, obese).

Ambulatory monitors and stress testing

Only the ANSI/AAMI/ISO presents additional requirements for ambulatory BP monitors and devices for exercise stress testing.

Pass criteria

The ESH‐IP has pass/fail criteria for individual BP readings (Part 1) and individual participants (Part 2), which are based on the number of readings with absolute test‐reference BP difference ≤ 5, 10, 15 mm Hg. The ANSI/AAMI/ISO also has criteria for individual BP readings (Criterion 1) and individual participants (Criterion 2) but based on mean BP differences and their standard deviations. Interestingly, the two approaches are statistically interchangeable with the two protocols allowing similar maximum levels of probability for tolerable error of ≤ 10 mm Hg in BP measurement.14

3. AAMI/ESH/ISO UNIVERSAL STANDARD

Despite their differences, all the established validation protocols (Table 1) have a common objective, which is to standardize the validation procedures and establish uniform minimum standards of accuracy for BP measuring devices.1 The small sample size of the ESH‐IP⁹ reduced the time, resources, and cost for validation studies and thereby allowed widespread application and considerable increase in the number of devices subjected to independent validation.15 Thus, awareness of the importance of using appropriately validated BP monitors was increased. However, this came at the cost of lower study power and accuracy and inability to evaluate subgroups (eg, for cuff sizes, or special populations).15, 16, 17, 18

In 2017 the AAMI, ESH, and ISO committees have reexamined all the major aspects of the validation procedures aiming to develop a single universally acceptable standard.14 The AAMI/ESH/ISO Universal standard is based on the collective experience from all previous standards and protocols and will replace previous protocols.14

This major international initiative has been undertaken to best serve the needs of patients with hypertension, a public interested in cardiovascular health, practicing physicians, scientific researchers, regulatory bodies, and manufacturers and to strengthen the case for independent clinical validation to become mandatory in all countries for BP measuring devices before they are put on the market. The key aspects of the agreed AAMI/ESH/ISO universal validation procedure are presented in Table 2.14

Table 2.

Key aspects of the AAMI/ESH/ISO universal validation standard14

Aspect	Requirement
Efficacy measure	Threshold for BP measurement accuracy acceptance at estimated probability of tolerable error (≤ 10 mm Hg) ≥85%.
Sample size	≥ 85 participants
General/special populations	A general population study includes participants > 12 years. Special populations: age < 3 years; pregnancy; arm > 42 cm; atrial fibrillation; others may be added. Special population studies include ≥ 35 participants (after successful general population study). Pregnancy: N = 45 (15 normotensive, 15 gestational hypertension, 15 preeclampsia). Korotkoff K5 for reference diastolic BP: Children: N = 35 aged 3‐12 years can be included and analyzed together with 50 older participants. Results also reported separately for children (not a pass/fail criterion). Korotkoff K5 for reference diastolic BP.
Cuff sizes	There is a minimum number of participants per cuff depending on number of test device cuffs. Requirements for arm circumference distribution according to range of use of the test device.
Reference BP	Mercury sphygmomanometers or accurate non‐mercury devices.
Data collection	Same‐arm sequential BP measurement is preferred.
Pass criteria	Average BP difference and SD criteria 1 and 2 of ANSI/AAMI/ISO. Absolute BP differences ≤ 5, 10, 15 mm Hg and scatterplots to be presented.

4. THE FUTURE

At the present time, about 1 in 5 BP measuring devices available on the market has been subjected to independent validation using an established protocol.1, 2 Thus, there is an urgent need to expand validation procedures worldwide and eventually succeed in making it mandatory for all devices to be put on the market. The AAMI/ESH/ISO universal standard14 is a landmark initiative towards achieving these goals. International efforts need to be intensified to improve the clinical validation process by ensuring that the proposed protocols are feasible for wide use and that protective measures are implemented to prevent protocol violations and conflicts of interest.18, 19, 20

CONFLICT OF INTEREST

GS, EOB, and BA have conducted validation studies for various manufacturers and advised manufacturers on device development. JW conducted validation studies for various manufacturers. SM has nothing to declare.

Stergiou GS, Alpert BS, Mieke S, Wang J, O'Brien E. Validation protocols for blood pressure measuring devices in the 21st century. J Clin Hypertens. 2018;20:1096–1099. 10.1111/jch.13294