The pursuit of accurate blood pressure measurement: A 35‐year travail

Appropriately, cardiovascular disease has recently been dubbed “the largest epidemic known to mankind, and hypertension is the major contributory risk.”1 The number of adults around the globe living with hypertension has nearly doubled since 1975, from 594 million to 1.1 billion in 2015.2 A recent systematic analysis that assessed 67 risk factors of disease in 21 regions around the world from 1990 to 2010 showed increased blood pressure (BP) to remain the leading risk factor for death and disability globally.3 When we talk of hypertension, or high BP, it is often forgotten that we are referring to a physiological phenomenon, namely normotension, that can become a pathological entity (ie, a disease) when BP exceeds a certain level, and that the transformation from normality to abnormality is exclusively dependent on a measurement. Although the severity of hypertension may be influenced by associated risk factors, such as family history, lifestyle, and the presence of associated illnesses, such as diabetes mellitus, the diagnosis and all subsequent therapeutic decisions are dependent on being able to measure BP accurately. Put another way, the empirical aphorism of H. James Harrington: “If you can't measure something, you can't understand it; if you can't understand it, you can't control it; if you can't control it, you can't improve it,”4 has been ignored, not only by clinical practitioners but also by scientific researchers.

To diagnose and treat hypertension, it is necessary to have a device capable of measuring at least two levels of BP accurately, namely systolic and diastolic pressures. Devices may be developed further to measure BP in patients with varying characteristics, such as children, obese and pregnant persons, and in certain circumstances, such as during exercise, and over time, such as with ambulatory BP monitoring. However, whatever the circumstances or conditions of measurement, the sine qua non for all forms of measurement is that the device must be accurate.

It is worth emphasizing that although evolving technology can offer the means to measure complex functions of cardiovascular physiology, the basic phenotype remains the BP, and all other methodologies of measurement become meaningless unless BP is measured accurately. This has been stated with similar emphasis in the recently published Lancet Commission on hypertension: “In hypertension, blood pressure is an almost ideal biomarker. Blood pressure is causally related to the development of the condition, defines the condition, predicts the outcome, is the target of therapeutic interventions, and serves as a surrogate marker to assess the benefit of therapies. Therefore, the role that other biomarkers could have in hypertension requires careful thought.”5

The “pursuit of accuracy” for BP‐measuring devices has a long legacy. In 1918, Dr Faught made a despairing statement: “At the present time the market is flooded with instruments of all descriptions for estimating blood‐pressure, so that it is important that the prospective purchaser should be able to separate the good from the bad.”6 However, serious efforts to distinguish the “good from the bad,” so that patients would not be disadvantaged by inaccurate measurement only began in the 1980s when efforts to standardize the validation of BP monitors were made.

1. Creating Validation Protocols

Desirable though it is to have accurate devices for BP measurement in clinical practice, in clinical research it is essential that devices are accurate so as to avoid erroneous recommendations on the demographic characteristics of hypertension and the efficacy of drugs. The London School of Hygiene sphygmomanometer was developed in 1964 to remove measurement bias in scientific studies,7 but when it was subjected to validation, it was shown to be inaccurate.8 Similar studies later demonstrated that another device, also developed as a gold standard for research in 1963—the Hawksley random‐zero sphygmomanometer—was also inaccurate.9, 10 The potential consequences of using these inaccurate devices in many clinical trials on which the therapeutic and demographic recommendations of contemporary practice are based, have never been examined. At that time of performing these early assessments of devices, there was no standardized protocol for the evaluation of BP‐measuring devices and validation studies were conducted using ad hoc protocols.11, 12 With the dawn of electronic measurement and the advent of 24‐hour ambulatory BP monitoring, the need for a standardized protocol became compelling.13, 14

In 1987, the joint American National Standards Institute (ANSI) and the US Association for the Advancement of Medical Instrumentation (AAMI) standard for electronic BP monitors included a clinical validation procedure.15 In 1990, the British Hypertension Society (BHS) published a protocol dedicated to the validation of BP monitors in the clinical setting, which incorporated many of the features of the ANSI/AAMI validation procedure, but also had many important differences.16 The BHS protocol was revised in 1993 and the ANSI/AAMI standard was also revised in 1993 and amended in 2003 and 2006.17, 18, 19, 20, 21, 22 In 1999, the German Hypertension League introduced its own validation protocol.23

In 2002, the European Society of Hypertension (ESH) Working Group on BP Monitoring developed the ESH International Protocol (ESH‐IP), with the major difference that a smaller sample size was required (N=33 compared with N=85 in the ANSI/AAMI and BHS protocols).24 This protocol facilitated validation procedures considerably, mainly due to the smaller sample size required, and has been the most widely used validation protocol worldwide (Figure).25 A revised version of the ESH‐IP with more stringent validation criteria was published in 2010.26 During 2008 and 2009, the International Organization for Standardization (ISO), in conjunction with ANSI and AAMI, published revised standards for nonautomated and automated devices,27, 28 which were revised in 2013.29, 30 In 2009, the ISO developed another standard, which incorporated aspects of EN 1060‐4 and the AAMI SP‐10 (eg, sample size and validation criteria),31 and this has been adopted by the AAMI Sphygmomanometer Committee.32 The German Hypertension League subsequently introduced a validation protocol incorporating many aspects of the ANSI/AAMI/ISO standard.33

Figure 1.

Cumulative graph of validation studies performed according to the European Society of Hypertension International Protocol (ESH‐IP) compared with the British Hypertension Society (BHS) and the Association for the Advancement of Medical Instrumentation (AAMI) protocols from 2002 (publication of ESH‐IP) until June 2009

Despite their differences, all these protocols have major similarities and a common objective, namely the standardization of the validation procedures to establish minimum standards of accuracy and performance, and to facilitate the comparison of different devices. Currently, there is an international initiative to arrive at a consensus on a universal validation protocol through an AAMI‐ESH‐ISO collaboration. The “universal protocol,” which is expected to be published at the end of 2017, will probably come into effect in 2018 or shortly afterwards, to replace all previous protocols (personal communication). The development of a universal protocol will undoubtedly be a major development in the validation process. But, having a protocol, however good, is of no avail unless the validation process that it puts forward is adhered to in every detail. Experience has shown that the performance of validation studies is a major obstacle in the assessment of device accuracy.25, 34, 35, 36

2. Performing and Reporting Validation Studies

The following issues can be identified in the performance of validation studies.

2.1. Protocol violations

Protocols for device validation are only helpful if they are adhered to strictly. If protocols are ambivalent, or if directions are not stated clearly and emphatically, those performing validation studies may, without wishing to violate the protocol, do so because of lack of clarity. There is also the possibility that an investigator may violate the protocol requirements willfully, so as to provide a favorable outcome. Experience with the protocols listed above has shown that several validation studies published in peer‐reviewed journals have not adhered to the protocols and that, as a consequence, the results are at best questionable and at worst so erroneous that the recommendation approving the device for clinical use is incorrect.25, 34, 35, 36 We have examined the results of published validation studies performed on BP‐measuring devices and have shown that violations of the ESH‐IP Protocol occurred in 23 of 78 studies, and we identified eight major violations and many more less important ones.25 The most serious consequence of misleading conclusions is that the devices validated in such studies pass into the marketplace as being recommended and, therefore, suitable for clinical practice. This flawed scientific process is largely due to the lack of a standardized procedure, and refereeing of papers by journal reviewers who do not have the requisite expertise to review validation studies submitted for publication. This unsatisfactory situation is compounded further by the fact that device manufacturers use the result of a published validation study to promote and sell their device. One way of overcoming this unacceptable scientific deficiency is to establish an independent online off‐site system of validation data upload and analysis that will alert laboratories to violations of the protocol and provide ongoing analyses to minimize deviations from the protocol.

2.2. Manufacturer influence

Another problem with validation studies is the potential for manufacturers, who usually sponsor these studies, to influence the performance of the procedure and the analysis of the data. The scale of this problem is not known.

There are four potential solutions to this issue. First, reputable validation laboratories will have scientific procedures in place that exclude manufacturer personnel from the laboratory and from having any involvement in both the analysis of validation data and the publication of results; they will also provide a declaration of any conflict of interest. Second, a process of independent allocation of devices to laboratories, so that the manufacturer is unaware of the laboratory conducting the validation study, could remove this potential influence. The BHS has addressed this issue by setting up a process whereby the researcher performing a validation study is blinded from the manufacturer so that the latter cannot influence the performance of the study.37 A third option is that an independent source could collect the data online throughout the study and analyze them as the study progresses. This would ensure that the data are collected completely and correctly, so that adjustments cannot be made on completion of the study. Finally, validation studies could be performed in designated measurement laboratories, as is done for other measurement systems,38, 39 but the financial implications of establishing such laboratories are at present prohibitive.

3. Acknowledging Device Equivalence

The concept of equivalence arose from recognition that manufacturers often provide new models of their BP monitors with identical BP measurement technology to a device that had been validated as accurate, but with variations unrelated to BP measurement, for instance, a memory facility, data transfer features, and statistical analysis, or cosmetic changes or, in the case of original equipment manufacturer devices, the marketing of identical devices under different housing and brands. As these differences should not affect the measuring accuracy of the device, it would be unreasonable to expect a manufacturer of such a device to subject it to another full, time‐consuming and expensive validation study.40 However, strict criteria need to be decided for these devices to be considered equivalent to a previously validated one. First, the manufacturer should give signed assurances that the measuring algorithms are identical and that any alterations to the device do not affect its measuring capacity. Second, every feature of these devices must be compared carefully to prove the claim. Third, and most importantly, an independent board of experts should approve the evidence submitted by the manufacturer for equivalence status.

4. Communicating the Data of Accuracy

The preceding summary of the problems associated with the establishment of validation protocols and the performance of validation procedures, although scientifically necessary, is of little practical use unless the results of the validation procedures leading to accuracy recommendations are communicated effectively to the desired recipient community. In the past, this recipient audience has not been categorized, and, consequently, the most effective means of communication to the different groups within this audience has not been explored.

4.1. Recipient audience

At least five groups constitute the recipient audience that would benefit from BP device accuracy information, some of which have common characteristics.

• Researchers and scientists: We have each participated on the BP measurement working parties of a number of societies, such as the ESH, the World Hypertension League, the BHS, the AAMI, and the International Standardisation Organisation, and we have been able to observe that while a major aspiration of such groups is to uphold the principles of accuracy and the importance of device validation, there is often inadequate information on specific accuracy of devices. This deficiency within bodies interested in hypertension is even greater in other specialities, such as cardiology and diabetology, where the potential consequences of inaccurate BP measurement may result in erroneous demographic assessments and therapeutic recommendations.

• The public: This most important group includes healthy people who wish to protect their cardiovascular health, but also, and of greatest concern, patients with hypertension. It is forecasted that the global BP monitoring market will exceed US $2.6 billion by 2020,41 an estimate that is based on the current practice of being able to market devices of unknown accuracy to a largely unquestioning public. This eventuality is not of much concern for devices that measure cardiovascular function in healthy people, but the sale of inaccurate devices to patients with high BP or diabetes, who may make therapeutic decisions based on these measurements, is an issue of major concern. Furthermore, measurement errors showing incorrect normal results in seemingly healthy people may delay diagnosis and timely treatment, whereas incorrect abnormal results may lead to unnecessary anxiety and investigation.

• Healthcare providers: This category is composed of the decision purchasers in hospitals, healthcare trusts, pharmacies, and primary care practices, who have an important role in deciding which devices to purchase for theaters, wards, and clinics throughout the healthcare system. Unfortunately, there is evidence that without adequate guidance, the cost of BP‐measuring devices, rather than their accuracy, may be of more importance, or unfounded and unchecked accuracy claims by manufacturers for particular devices may prove persuasive.

• Consumer bodies: Consumer organizations have not given due attention to the scientific requirements to assess BP device accuracy. A popular consumer magazine, WHICH?, with a following of 1.5 million people in the United Kingdom, in its latest review of BP monitors designated devices into “Best Buy”42 and “Don't Buy”43 categories. The magazine states that criteria for accuracy were based on “the British Hypertension Society, US Association for the Advancement of Medical Instrumentation and European Hypertension Society,” according to a process that involved taking “more than 1500 blood pressure readings to bring you our definitive verdict on 22 widely available models.”44 However, there is no evidence that the strict criteria of the ESH‐IP 2010 protocol or the ISO‐81060‐2:2013 standard are adhered to in the WHICH? evaluation for accuracy. In particular, the devices were not tested in the categories of hypertensive patients stipulated in the protocols.44 This is crucial to determining the accuracy of a device, because a device that is accurate in normotensive persons may well be inaccurate in the patients who will use it most, namely patients with elevated BP. Of the 14 devices listed as evaluated by WHICH?,42, 43 only one had fulfilled the criteria of the ESH‐IP 2010 protocol,45 but WHICH? designated this device to the “Don't Buy” category with the following damning assessment: “This futuristic, super‐light wrist blood pressure monitor has lots of pros in terms of its usability, but unfortunately it doesn't cut the mustard—in our tests it was a low scorer on accuracy.”43 So, in summary, the difference between consumer and scientific processes of evaluation highlights the need for closer collaboration to achieve a common goal, which is to provide the user of BP‐measuring devices with the best device.46

• Regulatory bodies: There are many national and international regulatory bodies for devices and it is outside the scope of this review to examine these in detail. Major regulatory influences are the Food and Drug Administration in the United States47 and the European Medicines Agency in Europe.48 Within the latter, each member country of the European Union has its own regulatory authority, which is often associated with allied bodies in regulatory affairs. For example, in Ireland, the Health Products Regulatory Authority has overall responsibility for the regulation of medical devices but is associated with the National Standards Authority of Ireland and the Advisory Committee for Medical Devices.49 However, the major concern of most regulatory bodies is safety rather than accuracy and, although the importance of device accuracy is often stressed, the expertise to make definitive recommendations in this regard is often lacking, and validation studies performed according to accepted international protocols are cited without critical appraisal. Hence, devices appearing on the market must fulfill stringent safety requirements but are not required to have undergone accuracy testing according to one of the above‐mentioned protocols. The collaboration already noted between AAMI‐ESH‐ISO bodies is therefore a welcome initiative that will result in an internationally accepted universal protocol for the assessment of BP device accuracy.

4.2. Methods of communication

Traditionally, the results of validation studies have been published in scientific journals or presented at scientific meetings, which serve as a durable reference source.14 However, such publications reach a relatively small group of scientists and have little impact on most of the groups listed above. Website communication of results has proved an effective means of reaching a broader audience. The dabl Educational Trust website was established in 2004 to provide a listing of BP devices according to accuracy and to provide a methodology that allowed manufacturers to claim equivalence of a new device against one that had been previously validated.50 In 2014, the scientific advisory board of this website resigned and established the Medaval website,51 which has the broader mission of improving accuracy not only for BP‐measuring devices but also for other medical devices, such as central BP monitors, pulse wave velocity monitors, arrhythmia detectors, and blood glucose monitors. As a first step, Medaval has drawn up the most comprehensive presentation of BP‐measuring devices on the Internet, with detailed device information, device manuals, and PubMed links to published validation studies. Medaval provides lists of certified accurate devices for professional and home use approved by its international advisory board of experts, a unique facility for comparative equivalence for devices using the same measurement methodology and a Medaval brand of approval. Lists of accurate devices are being circulated to all national societies involved in hypertension management and to regulatory bodies.

5. Legal Considerations

Living as we do in a litigious age, it is perhaps surprising how little consideration has been given to the potential legal consequences of inaccurate BP measurement. The failure to provide ambulatory BP monitoring to determine BP over 24 hours in hypertensive patients has been acknowledged.52 However, until recently, the legal consequences of a device giving erroneous information to patients with the consequent potential for incorrect diagnostic or therapeutic action has gone unchallenged. The most celebrated instance was the collapse of the biotech start‐up company Theranos, which had attracted investments running to more than $700 million. The story of this debacle has been compellingly recounted by Nick Belton.53 Of greater relevance to BP measurement is the recent ruling by the Federal Trade Commission that the “marketers of a mobile app designed to measure blood pressure have agreed to settle Federal Trade Commission charges that they deceived consumers with claims that their Instant Blood Pressure app was as accurate as a traditional blood pressure cuff.” The Federal Trade Commission ruled that: “The stipulated federal court order prohibits the defendants from making the deceptive claims alleged in the complaint. It also prohibits them from making any claims about the health benefits of any product or device without the scientific evidence to support the claims.”54 The company, with sales of more than $600 000 in 1 year, had done so with a deceptively simple technique—the user had only to place the right index finger over the rear camera lens of a mobile phone and hold the base of the phone over the heart to obtain a BP measurement.55 Not surprisingly, when the device was tested, it was shown to be grossly inaccurate.56 The important outcome of this case is that patients who use devices to manage their own illnesses or medical personnel who use devices to manage patients can seek redress if they are sold inaccurate devices or if manufacturers make spurious claims for devices without sound scientific evidence.

6. Conclusions

The global patient monitoring device market reached $15.9 billion and $16.9 billion in 2014 and 2015, respectively. The market is expected to reach $23.8 billion in 2020, increasing at a compound annual growth rate of 7.1% from 2015 to 2020.57 This massive fiscal potential for measurement devices is most lucrative in the field of cardiovascular health and illness.

It is imperative for the health and protection of people using devices for the measurements of a medical function that they can be assured about measurement accuracy. As scientists, we must ensure that sufficiently stringent validation protocols are made available, that validation studies are properly monitored, that stringent criteria are applied to devices claiming equivalence, that accurate devices are readily identifiable by the public, and that a source of reliable information for healthcare professionals, healthcare administrators, and the public is provided. It is clearly also desirable that manufacturers are influenced by scientific considerations to produce more accurate BP‐measuring devices and this is one of the major objectives of all validation protocols. It is necessary, therefore, that the scientific requirements for validation are clearly communicated to manufacturers.

Indeed, this aspiration is endorsed by the recent Lancet Commission on hypertension, which has called in effect for regulation of BP‐measuring devices:

“Ideally, devices should comply with the validity guidelines of scientific societies, rather than just internal testing by the manufacturer, and this information should be clearly available for the customer. Professional societies could also consider providing a seal of approval or certification of blood pressure devices meeting appropriate accuracy standards, which is particularly important given the rapid developments in wearable technologies marketed without validation testing according to current international expectations. (A Position Statement from the World Hypertension League, International Society of Hypertension and Supporting Hypertension Organizations58) Active warnings on sub‐standard devices can be provided…. The seal of approval could, in turn, be used by manufacturers for marketing… a close collaboration between a wide range of stakeholders such as governments, the mobile communications industry, health‐care professionals, the pharmaceutical industry, and professional societies to not only develop and distribute inexpensive, validated, and certified blood pressure monitors, but also to ensure correct use through simple mobile apps and online education endorsed by the professional societies.”5

The essential steps for improving the accuracy of BP‐measuring devices may be summarized as follows:

• Establish a “universal protocol” to replace all other validation protocols.

• Make the “universal protocol” mandatory before a BP device can be put on the market.

• Improve validation procedures by providing safeguards for performance, analysis, and reporting of validation results.

• Provide comparative equivalence for devices based on updated validation studies.

• Provide lists of accurate devices in different categories.

• Develop a network of affiliated international medical societies to be updated regularly on device accuracy.

• Improve communication with practicing doctors and researchers, pharmacists, healthcare purchasers, and the public.

• Collaborate with and regularly update regulating authorities and consumer bodies.

Conflicts of Interest

EO and GS have conducted validation studies of blood pressure monitors for various manufacturers, advised manufacturers on device development, and chair the advisory board of Medaval.