The quest for accuracy of blood pressure measuring devices

Abstract

The accuracy of blood pressure (BP) measuring devices is fundamental to good practice and scientific research. International guidelines on BP measurement are provided for clinicians who diagnose and treat patients with hypertension, clinical researchers who conduct trials on the efficacy of BP lowering drugs and interventional strategies, epidemiologists who conduct population surveys to determine the demographic consequences of hypertension on society, and researchers who perform meta‐analyses on published research to further influence the practice of medicine and the provision of resources. Although the outcomes of the endeavors of all these groups are dependent on the accuracy of BP measurements, the equipment is often of doubtful accuracy and the methodology of measurement is often poorly described and frequently not standardized. Thus, the fundamental element of hypertension evaluation has been largely ignored by both clinical practitioners and scientific researchers. Here, the authors briefly review the development of efforts to improve and validate the accuracy of BP measuring devices and highlight the deficiencies that persist. We conclude that, to protect the public from the serious consequences of inaccurate BP measurements, the following steps are required: (1) regulatory requirement for mandatory independent validation of all BP measuring devices using a universal protocol; (2) accreditation of laboratories for the performance of BP device validations; (3) online evaluation of validation studies with detection of protocol violations prior to publication of results; and (4) establishment of an independent scientific forum for the listing of accurate BP measuring devices.

If you can't measure something, you can't understand it; if you can't understand it, you can't control it; if you can't control it, you can't improve it. H. James Harrington 1

1. THE PHENOTYPE – BLOOD PRESSURE

Appropriately, cardiovascular disease to which hypertension is the major contributor has been dubbed “the largest epidemic known to mankind.”2 The number of adults living with hypertension worldwide has nearly doubled since 1975, from 594 million to 1.1 billion in 2015.3 A recent systematic analysis that assessed 67 risk factors of disease in 21 regions around the world from 1990 to 2010 showed increased BP as the leading risk factor for death and disability globally.4 However, it is often forgotten that these daunting statistics are based on a variable physiological phenomenon that denotes either normotension when the BP is below a certain and rather arbitrary level or a pathological entity (ie, a disease) when the BP exceeds this level, and that the transformation from normality to abnormality depends exclusively on measurements. Although the severity of hypertension may be influenced by associated risk factors, such as family history, lifestyle, and the presence of associated illnesses such as diabetes mellitus, the diagnosis and all subsequent therapeutic decisions depend on the ability to measure BP accurately.

To diagnose and treat hypertension, a device capable of measuring at least 2 blood pressures, namely systolic and diastolic, is required. Devices may be developed further to measure BP in subjects with varying characteristics, such as children, obese persons, and pregnant persons; and in special circumstances, such as during exercise; and over time, for example, ambulatory blood pressure monitoring (ABPM). However, whatever the circumstances or conditions of measurement, a fundamental requirement is that the device should be accurate.

It is worth emphasizing that although evolving technology can offer the means to measure complex functions of cardiovascular physiology, the basic phenotype remains BP, and all other measurements are meaningless unless BP is measured accurately. This has been stated with similar emphasis in the Lancet Commission on Hypertension: “In hypertension, blood pressure is an almost ideal biomarker. Blood pressure is causally related to the development of the condition, defines the condition, predicts the outcome, is the target of therapeutic interventions, and serves as a surrogate marker to assess the benefit of therapies. Therefore, the role that other biomarkers could have in hypertension requires careful thought.”5

2. DEVELOPMENT OF VALIDATION PROTOCOLS

The “pursuit of accuracy” for BP measuring devices has a long legacy. In 1918 Dr. Faught made a despairing statement: “At the present time the market is flooded with instruments of all descriptions for estimating blood‐pressure, so that it is important that the prospective purchaser should be able to separate the good from the bad.”6 The gold standard method for measuring BP noninvasively is the auscultatory method performed by an expert using a mercury manometer to measure cuff pressure. Most current cardiovascular risk calculators are based on population studies in which this method was used; hence “accurate” means producing results close to auscultatory values. Analysis of population BPs suggests that to avoid over‐ or underdetecting hypertension by more than 20%, systematic errors in diastolic BP and systolic BP should be less than 1 and 3 mm Hg respectively.7 Serious efforts to distinguish the “good from the bad” to avoid disadvantaging patients by inaccurate measurement began only in the 1980s when efforts to standardize the validation of BP monitors began.

The first BP measuring devices to come under scrutiny were those used in research studies. In clinical research it is mandatory that devices are accurate to avoid erroneous recommendations on the demographics of hypertension and treatment efficacy. The London School of Hygiene sphygmomanometer was developed in 1964 to remove observer bias in BP measurement in scientific studies,8 but when it was subjected to validation, it was shown to be inaccurate.9 Similar studies later demonstrated that the Hawksley random zero sphygmomanometer, another BP measuring device also developed as a gold standard for research in 1963, was also inaccurate.10, 11 The consequences of the misleading results from the use of these inaccurate devices in therapeutic and demographic studies on which we base practice today have never been examined.

The first validation studies of BP measuring devices used in clinical practice were conducted with ad hoc protocols. However, with the introduction of electronic measurement and the advent of 24‐hour ABPM, the need for a standardized protocol became compelling, and a series of protocols have been developed over the past 30 years.12 Despite their differences, all these protocols have major similarities and a common objective, namely standardization of the validation procedures to establish minimum standards of performance and to facilitate comparison of different devices. The protocols of the European Society of Hypertension, in particular, have had a considerable influence in motivating more manufacturers to submit devices for independent validation.13 There is now international agreement that a universal protocol for device validation is necessary and the US Association for the Advancement of Medical Instrumentation, the European Society of Hypertension, and the International Organization for Standardization are currently developing a universal protocol (AAMI/ESH/ISO) that is expected to come into effect in 2019 and supersede all previous protocols.14

3. PERFORMING AND REPORTING VALIDATION STUDIES

The development of the universal protocol will undoubtedly be a major development in the history of validation. However, a protocol is of no avail unless it is adhered to in every detail. Unfortunately, experience has shown that the performance of validation studies is often inadequate because of 3 major concerns—protocol violation, inadequate peer review of published results, and manufacturer influence.13, 14 The impact of validation studies has also been weakened by the failure of regulatory bodies to enforce a requirement for accuracy in the approval for devices on the market.

Protocol violations

Protocols for device validation are helpful only if they are adhered to strictly. If protocols are ambivalent, or if directions are not stated clearly, those performing validation studies may unintentionally violate the protocol. It is also possible for investigators to violate the protocol requirements willfully to provide a favorable outcome. Experience has shown that many validation studies published in peer reviewed journals deviated from the protocols and the results are at best questionable and at worst so erroneous that the recommendation approving the device for clinical use is incorrect.13, 14, 15 This serious scientific deficiency can be overcome by establishing an independent off‐site system to provide online validation of data as the study proceeds, thus enabling the validation laboratory to be alerted immediately to a violation that can be corrected and deviations from the protocol prevented.

Inadequate peer review

Unfortunately, protocol violations are often not detected by reviewers of papers submitted for publication. As a result, the devices validated in such studies pass onto the marketplace as “recommended” and, therefore, suitable for clinical practice. This unsatisfactory situation is compounded by the fact that device manufacturers are provided with the ultimate stamp of approval for devices that are inaccurate, and the erroneous conclusion of a published validation study is used to promote and sell an inaccurate device.

Manufacturer influence

The potential for manufacturers, who usually sponsor validation studies, to influence the outcome of a study is potentially a serious concern, the scale of which is not known. There are 4 potential solutions to this issue. First, reputable validation laboratories will have procedures in place that exclude manufacturer personnel from the laboratory and from having any involvement in both data analysis of validation data and the publication of results; they will also provide a declaration of any conflict of interest. Second, a process of independent allocation of devices to laboratories, so that the manufacturer is unaware of the laboratory conducting the validation study, could remove this potential influence. The British Hypertension Society has addressed this issue by setting up a process whereby researchers performing a validation study are blinded from the manufacturer so that the study cannot be influenced.16 A third option is that an independent source could collect the data online throughout the study and analyze them as the study progresses. This would ensure that the data are collected completely and correctly, so that adjustments cannot be made on completion of the study. Finally, validation studies could be performed in externally accredited laboratories as is done for many other measurements that society considers important.17, 18 The financial implications of establishing such laboratories may appear to be prohibitive, but when the long‐term costs and loss of quality of life caused by incorrect BP measurements are considered, external accreditation of laboratories may prove to be cost effective.

Regulatory approval

Authoritative government bodies, such as the National Institute for Health and Clinical Excellence in the United Kingdom19 and the Preventive Services Task Force in the United States,20 ultimately dictate the practice of clinical medicine and it is essential that these bodies are persuaded to differentiate accurate from inaccurate devices in their recommendations.

4. ACKNOWLEDGING DEVICE EQUIVALENCE

The concept of equivalence arose from recognition that manufacturers often provide new models of their BP monitors with technology identical to a device previously validated as accurate, but with variations unrelated to BP measurement.12 However, strict criteria need to be decided for these devices to be considered equivalent to a previously validated one. First, the manufacturer should give signed assurances that the measuring algorithms are identical and that any alterations to the device do not affect its measuring capacity. Second, every feature of these devices must be compared carefully to prove the claim. Third, and most important, an independent board of experts should approve the evidence submitted by the manufacturer for equivalence status. Without credible and transparent scientific endorsement, the granting of equivalence becomes meaningless. There is then the issue as to how long a validation study should remain valid; in the era of rapidly developing technology, inevitably, many components of a device could be changed during a 5‐year period rendering that device “different” and in need of revalidation. It is essential, therefore, that a scientific body completely free from the influence of the manufacturing industry oversees the international listing of BP measuring devices, including their validation status and any equivalence claims for accuracy.

5. THE FUTURE

We are now at a major turning point in the history of device validation. We are aware that as scientists, we have failed to protect the public from the serious consequence of inaccurate BP measuring devices, which by reaching the marketplace are assumed to be accurate. The following 4 steps are now required:

• Regulatory requirement for mandatory independent validation of all BP measuring devices using a universal protocol

• Accreditation of laboratories for the performance of BP device validation

• Online evaluation of validation studies using a standard procedure and aiming to detect protocol violations prior to publication of results

• Establishment of an independent scientific forum for the listing of accurate BP measuring devices

CONFLICT OF INTEREST

EOB and GS have conducted validation studies for various manufacturers and advised manufacturers on device development. MT is a technical assessor for the National Association of Testing Authorities of Australia.

O'Brien E, Stergiou GS, Turner MJ. The quest for accuracy of blood pressure measuring devices. J Clin Hypertens. 2018;20:1092–1095. 10.1111/jch.13279