STRIDE BP international initiative for accurate blood pressure measurement: Systematic review of published validation studies of blood pressure measuring devices

Abstract

Blood pressure (BP) is a vital sign, and its measurement is essential for diagnosing and treating hypertension. The accuracy of BP monitors is therefore essential, but unfortunately very few devices available on the market have been validated using an established protocol. STRIDE BP (www.stridebp.org) is an international nonprofit organization with the mission to improve the accuracy of BP measurement and the diagnosis of hypertension. It has a prestigious Scientific Advisory Board and operates in affiliation with the European Society of Hypertension, the International Society of Hypertension, and the World Hypertension League. STRIDE BP provides lists of accurate office, ambulatory, and home BP monitors. STRIDE BP performed a systematic review of 419 published validations (270 articles and 260 devices). In these publications, 50 (12%) of the validations were rejected compared with 129 (31%) rejected by STRIDE BP (P < .001). Of 79 validations approved in publications but rejected by STRIDE BP, 7 (9%) were rejected due to device inaccuracy and 72 (91%) due to inadequate study quality (execution, analysis, and reporting). Errors in conducting and reporting published validations must be avoided. Peer review must ascertain that a comprehensive checklist of all aspects of a validation study have been adhered to. The implementation of a Universal Standard for device validation and the global dissemination of information on accurate devices by STRIDE BP are expected to improve the quality and accuracy of BP measurement, which should have a positive impact on the diagnosis and management of hypertension globally.

1. THE STRIDE BP INITIATIVE

STRIDE BP is an international nonprofit scientific organization founded by hypertension experts from around the world with the mission of improving the accuracy of BP measurement and the global diagnosis and management of hypertension.1 STRIDE BP aims to provide international scientific guidance for the methodology and technology of BP measurement according to the latest scientific evidence to (a) health care providers, including specialists, general practitioners, pharmacists, and nurses; (b) the public, including people with hypertension and other cardiovascular diseases and healthy people with an interest in maintaining cardiovascular health; (c) regulatory bodies; and (d) medical device manufacturers. The STRIDE BP website (www.stridebp.org) will present (a) lists of accurate BP measuring devices; (b) educational sessions on BP monitoring methods; and (c) tools for BP measurement for use in the clinical setting.

STRIDE BP operates in affiliation with the European Society of Hypertension, the International Society of Hypertension, and the World Hypertension League. It is governed by an Executive Management Board and a Scientific Advisory Board composed of 24 experts from 16 countries in all five continents, all of whom are acknowledged authorities in cardiovascular medicine, hypertension, and BP measurement.1

STRIDE BP collected all the published validation studies of BP measuring devices that have been performed using one of the established protocols.2 A vigorous review of all these studies was performed by the STRIDE BP Scientific Advisory Board members aiming to provide guidance on which devices have been properly validated and have proven accuracy.1 This article presents the STRIDE BP database of validated studies, as well as the methodology and the results of their evaluation by the STRIDE BP Scientific Advisory Board members.

2. STRIDE BP VALIDATIONS REVIEW SYSTEM

Validation studies published as full papers in peer‐review PubMed journals and using the following protocols were considered2: (a) Association for the Advancement of Medical Instrumentation (AAMI 1987, 1992, 2002)3, 4; (b) British Hypertension Society (BHS 1990, 1993)5, 6; (c) European Society of Hypertension International Protocol (ESH‐IP 2002, 2010)7, 8; (d) International Organization for Standardization (ISO 2009)9; (e) American National Standards Institute/AAMI/ISO (ANSI/AAMI/ISO 2009, 2013)10, 11; and (f) AAMI/ESH/ISO Universal Standard (ISO 81060‐2:2018).12, 13, 14 The rationale for continuing to use these protocols at the present time until the results of a reasonable number of devices validated using the AAMI/ESH/ISO Universal Standard will become available has been previously published.15

The STRIDE BP Validations Review System uses a unique software program developed to manage the studies' database and to communicate with both internal and external reviewers and also with the STRIDE BP website. For each validation study a “STRIDE BP Validation Study Checklist Report” is developed, which is specific to the validation protocol (and version) and the study population (general, children, pregnancy, etc), and verifies key aspects of the methodology and results (device type, population, sample size, selection criteria, pass/fail criteria, conclusion, etc). The STRIDE BP criteria for approval/rejection of validation studies are presented in Table 1. Two members of the STRIDE BP team independently prepare a Validation Study Checklist Report for each study, which is uploaded to the STRIDE BP Validations Review System. Any disagreement is resolved by the STRIDE BP Executive Management Board, and a final report is generated. Then, each validation study article and the accompanying “STRIDE BP Validation Study Checklist Report” are reviewed independently by two members of the STRIDE BP Scientific Advisory Board. Any disagreement between them is resolved by the STRIDE BP Executive Management Board.1

Table 1.

STRIDE BP criteria for validation study approval/rejection

Criteria for validation study rejection	Protocol violations which are accepted
Validation pass criteria not fulfilled or not reported. Insufficient sample size. Violation in the procedure or sequence of test and reference device BP measurements or the number of observers. Validation in pregnancy without data in women with hypertension and preeclampsia.	Minor violation in age and/or gender criteria. Minor BP distribution criteria violation. Blood pressure level criteria not reported but grossly visible in scatterplot. Minor cuff distribution criteria (if applicable).

Devices that successfully pass the STRIDE BP review process are listed on the STRIDE BP website as “Validated” devices (www.stridebp.org). STRIDE BP uses additional requirements to select “Preferred” devices, suggesting that these are the most appropriate ones to use.1 These additional requirements are (a) upper‐arm cuff device; (b) at least one successful validation study published within 10 years before the current year; and (c) automated storage of multiple readings, or mobile phone, PC or internet link connectivity enabling data transfer (the latter for home BP monitors only).

3. STRIDE BP REVIEW RESULTS

3.1. Validation studies selection

For the development of the STRIDE BP database of published validation studies, a systematic literature search was performed at PubMed from 1990 to June 15, 2019, using the following search algorithms: (a) (validation) AND (blood pressure monitor) AND device; (b) (European Society of Hypertension International Protocol) AND blood pressure monitor; (c) (British Hypertension Society protocol) AND blood pressure monitor; (d) (International Organization for Standardization protocol) AND blood pressure monitor; and (e) (Association for the Advancement of Medical Instrumentation protocol) AND blood pressure monitor. This standard literature search will be performed at monthly intervals to identify newly published validation studies. A total of 284 published articles were identified of which 14 were discarded (5 did not use an established protocol, 4 evaluated aneroid devices, 2 were central BP monitors, 1 was a BP measurement algorithm, and 2 were invasive studies). Thus, 270 articles were subjected to the STRIDE BP peer‐review process.

3.2. Publications

These 270 articles were published in a total of 32 PubMed journals, with 190 of them (70.4%) published by the Blood Pressure Monitoring journal, 16 (5.9%) by the Journal of Human Hypertension, 11 (4.1%) by the Vascular Health Risk Management journal, and 7 (2.6%) by the Journal of Hypertension (5 or fewer publications in 28 other journals). Most of these publications originated from Europe (52%) or Asia (33%) (Table 2). Several articles reported validation data of more than one BP measuring devices and several used more than one protocol to evaluate the accuracy of the same device (Figure 1).

Table 2.

Origin of PubMed publications reporting validation studies of blood pressure measuring devices

Continent	N of published studies	(%)
Europe	141	52	(Italy 39, UK 38)
Asia	89	33	(China 53, Japan 13)
N America	27	10	(USA 21, Canada 6)
C and S America	8	3
Australia	3	1.1
S Africa	2	0.7
Total	270

Figure 1.

STRIDE BP systematic review of published “articles,” validation “studies,” (validations) and validated “devices”

3.3. Test devices

The 270 published papers reported results of 419 validations of which 338 (81%) evaluated upper‐arm cuff devices, 72 (17%) wrist, and 9 (2%) finger devices. Several of these devices were validated using more than one protocol in the same study (eg, AAMI and BHS, or AAMI, BHS, and ESH‐IP), or in more than one independent studies.

3.4. Validation protocols

Of 419 validations, 206 (49%) used the ESH‐IP, 107 (26%) the AAMI/ISO, 104 used (25%) the BHS protocol, and two used (0.5%) the AAMI/ESH/ISO Universal Standard. The use of these protocols in published validations in the last 30 years is shown in Figure 2.

Figure 2.

Cumulative graph of published validation studies of blood pressure measuring devices performed using the European Society of Hypertension International Protocol (ESH‐IP), the British Hypertension Society (BHS) protocol, and the American Association for the Advancement of Medical Instrumentation (AAMI) or the International Organization for Standardization (ISO) standard

3.5. STRIDE BP approved devices

The above studies evaluated a total of 260 devices, and 215 (83%) were approved by STRIDE BP (Office 20; Ambulatory 23; Home 165; for public space/kiosk 4; only for hospital 3; for children 12; for pregnancy 9). Only 110 devices (42%) were classified as “Preferred” according to the above‐mentioned criteria (Figure 1).

4. STRIDE BP VERSUS REVIEW OF VALIDATION STUDIES IN PUBMED JOURNALS

The conclusions of the STRIDE BP review of 419 validations were compared with those of the PubMed publications. In 369 (88%) of 419 published validations, the authors concluded that the test device passed the requirements of the protocol used and in 50 (12%) that it failed. However, the STRIDE BP review process approved only 290 of these validations (69%) and rejected 129 (31%) (P < .001 compared with published results; Table 3). Thus, 79 studies rejected by STRIDE BP have been published in a PubMed journal as device “Pass.” The validations rejected by STRIDE BP and in PubMed journals according to the protocol used are presented in Table 3. The main reasons for STRIDE BP rejecting validation studies and also for rejecting studies that were published in PubMed journals as “Pass” are shown in Table 4. The most common causes of rejection and the validation protocol used are shown in Table 5.

Table 3.

Validations rejected by STRIDE BP and in PubMed publications

	Validation studies reviewed by STRIDE BP	Validation studies rejected by STRIDE BP	Studies rejected by STRIDE BP and in PubMed paper	Studies rejected by STRIDE BP but published as “Pass”
AAMI	71	44 (62%)	16/44 (36%)	28/44 (64%)
ANSI/AAMI/ISO	36	8 (22%)	1/8 (13%)	7/8 (87%)
BHS	104	49 (47%)	17/49 (35%)	32/49 (65%)
ESH‐IP	206	28 (14%)	16/28 (57%)	12/28 (43%)
AAMI/ESH/ISO	2	0	0	0
Total	419	129 (31%)	50/129 (39%)	79/129 (61%)

Table 4.

Main reasons for rejecting validation studies by STRIDE BP (129 studies) and for rejecting studies that were published in PubMed journals as “Pass” (79 studies)

Main reasons of validation study rejection by STRIDE BP*		STRIDE BP rejected studies (%)
		All (N = 129)	Published as “Pass” (N = 79)
Device inaccurate	Key validation criteria not fulfilled	57 (44%)	7 (9%)
Incorrect study execution	Inappropriate sample size	16 (12%)	11 (14%)
Incorrect study execution	Single observer	22 (17%)	18 (23%)
Incorrect study execution	BP measurements protocol violation (sequence, number, repeats)	37 (29%)	29 (37%)
Incorrect study execution	Inadequate reference device, or cuffs for reference and test device	13 (10%)	8 (10%)
Incorrect data analysis	Inappropriate data analysis	27 (21%)	27 (34%)
Incorrect data reporting	Key validation criteria not reported	35 (27%)	30 (38%)

^*Some studies with more than one major reason for rejection.

Table 5.

Most common reasons for rejecting validation studies by STRIDE BP and for rejecting studies that were published in PubMed journals as “Pass”

Reason for study rejection	N of studies	Protocol involved
Studies rejected by STRIDE BP (N = 129) *
Key validation criterion not reported (Criterion 2)	17 (13%)	AAMI 2002 and after
Key validation criteria not fulfilled	16 (12%)	BHS 1993
BP measurement sequence	16 (12%)	BHS 1993
Single observer	13 (10%)	BHS 1993
Preeclamptic women not included	10 (8%)	All
Studies rejected by STRIDE BP but published as “Pass” (N = 79) *
AAMI BP pairs analyzed with BHS method	27 (34%)	AAMI
Key validation criterion not reported (Criterion 2)	16 (20%)	AAMI 2002 and after
BP measurement sequence	14 (18%)	BHS 1993
Single observer	10 (13%)	BHS 1993

^*Issues identified in ≥10 studies.

5. IMPLICATIONS OF THE STRIDE BP APPROACH FOR REVIEWS OF VALIDATION STUDIES

In the last 30 years, several BP monitors have been validated using different established protocols and the results have been published in peer‐review PubMed journals.2, 14, 15 STRIDE BP performed a systematic review of all these studies, which at the present time constitute the basis for selecting accurate devices for use for health care professionals and patients.2, 15, 16

A major finding of this analysis is that in the last 30 years, there has been a consistent and steep increase in the number of devices subjected to validation using an established protocol. Almost half of these studies were published in the last decade, which demonstrates the increasing awareness, interest, and investment in having devices validated (Figure 1).

The STRIDE BP review process rejected 31% of the published validations. The true proportion of failed validations is probably higher as some negative studies might have not been published (publication bias). This is unethical for drug trials, but apparently not for BP measuring devices, because they might be prototypes that can be subjected to further technological development and subsequently pass the validation process. Nonetheless, all negative validations of devices that are currently available on the market should be published in order to inform health care professionals and the public that they should not be used.

The STRIDE BP review of published validations has revealed another two challenging findings. First, 44% of the validations were rejected due to device inaccuracy, whereas the rest were rejected due to poor study quality (incorrect study execution or incorrect data analysis). Second, 61% of the validations rejected by STRIDE BP were published in PubMed journals as device “pass” (Table 3). Of the latter validations, 7 (9%) were rejected due to device inaccuracy and 72 (91%) due to poor study quality.

The issue of quality and reliability of validation study reports published in PubMed journals has been discussed in previous articles.17, 18, 19 Protocol violations and incomplete reporting (missing and unclear data) are common and often missed by the peer‐review process of scientific journals.17, 18, 19 The STRIDE BP review showed that seven studies published as “Pass” did not fulfill the key “pass” criteria of the validation protocol used (Table 3). In addition, 72 studies published as device “pass” were incorrectly conducted, analyzed, or reported, which did not allow firm conclusion on the devices' accuracy. Many of these issues occurred when two protocols were used in the same validation study (eg, BHS BP pairs favoring the test device used for AAMI analysis; AAMI analysis without reporting criterion 2), or when requirements of an older version of the protocol were used long after a new version has been published (eg, AAMI studies not reporting criterion after 2002; BHS with simultaneous measurements after 1993; Table 3).

Such issues in published studies are not unique to validations and there is a considerable body of research showing that even in the most reputable peer‐reviewed journals, some publications can be erroneous and misleading.20, 21 A misleading PubMed publication accepting a BP measuring device as accurate has serious implications. The manufacturer can promote and market the device with the stamp of scientific approval and health care professionals and patients will purchase and use the device in good faith. This is analogous to a drug being given approval based on a flawed clinical trial.

The above‐mentioned errors in conducting and reporting of validation studies published in PubMed journals must be avoided. The methodology for validating BP measuring devices is standard according to the protocol used and must be adhered to in detail. Peer review must therefore ascertain that a structured and comprehensive checklist of all the study's aspects has been adhered to, rather than providing the editor with a subjective opinion as would be common practice for clinical trials. On‐line software that enables day‐to‐day entry of validation data as the study progresses, with rejection of incomplete or inaccurate data according to the protocol requirements and with provision of an automated full study report on successful completion would prevent protocol violations (other than fabrication of entered data) and misreporting.

After three decades on validation studies, the confusion and issues due to using several different validation protocols have been successfully addressed by developing a single Universal Standard (AAMI/ESH/ISO) aimed at replacing all the existing protocols.12, 13, 14 To ensure that the stipulations of the AAMI/ESH/ISO Universal Standard are meticulously implemented and data are correctly analyzed and fully reported, the ESH Working Group on BP Monitoring and Cardiovascular Variability has published a practical guide for investigators performing validation studies.14 The wide implementation of the AAMI/ESH/ISO Universal Standard and the global dissemination of information on accurate BP monitors by STRIDE BP are expected to improve the quality of validation studies and thereby improve the accuracy of BP measurement and the global diagnosis and management of hypertension.

CONFLICT OF INTEREST

GS, EOB, PP, and GP conducted validation studies for various manufacturers and advised manufacturers on device and software development.

APPENDIX 1.

Collaborators

GS Stergiou, Greece; E O'Brien, Ireland; M Myers, Canada; P Palatini, Italy; G Parati, Italy; R Asmar, France; A de la Sierra, Spain; P de Leeuw, Netherlands; E Dolan, Ireland; G Head, Australia; Y Imai, Japan; N Karpettas, Cyprus; K Kario, Japan; A Kollias, Greece; E Manios, Greece; A Mihailidou, Australia; A Murray, UK; T Ohkubo, Japan; A Shennan, UK; J Staessen, Belgium; A Schutte, South Africa; J Wang, China; M Weber, USA; the STRIDE BP Scientific Advisory Board.

Stergiou GS, O'Brien E, Myers M, et al; STRIDE BP Scientific Advisory Board . STRIDE BP international initiative for accurate blood pressure measurement: Systematic review of published validation studies of blood pressure measuring devices. J Clin Hypertens. 2019;21:1616–1622. 10.1111/jch.13710