HISTORY AND JUSTIFICATION OF A NATIONAL BLOOD PRESSURE MEASUREMENT VALIDATED DEVICE LISTING

INTRODUCTION

For over a century, blood pressure (BP) measurement was defined by the indelible standard of the mercury sphygmomanometer.¹ Mercury was responsible for assuring uniformity of BP measurement in the first studies to identify the risks of elevated BP with regard to cardiovascular outcomes. It continued to be the standard of accuracy in BP measurement in clinical practice until recent decades. In 2013–2014, international hypertension societies ultimately concluded that the risk of toxicity superseded any potential benefit of using mercury-based BP devices.^2,3 Accordingly, over the past several years, there has been a revolution in BP measurement. We have transitioned from widespread use of in-office manual aneroid sphygmomanometers (which easily lose calibration) to a broad range of options, including semi- or fully-automated (e.g. oscillometric) devices capable of measuring BP both in and out of the office.

With greater selection in BP devices came additional shortcomings. While oscillometric devices eliminate some degree of human error, many of these newer devices lack the precision afforded by mercury sphygmomanometers. Oscillometric devices, including automated office, home, and ambulatory BP measurement devices, have the potential to yield inaccurate readings, particularly if they do not undergo rigorous validation.^4,5 There is no validation requirement for marketing a BP device in the Unites States.⁶ Moreover, publicly available information regarding validation status of widely used BP devices is very limited.⁷ The American Medical Association (AMA) is leading a group of well-respected individuals and organizations in the field of hypertension and BP measurement to create a transparent and easily accessible resource for identifying validated BP devices, the AMA Validated Device Listing (VDL); the proposed criteria were recently made available for public comment. Our goal in this review is to underscore the importance of this endeavor, and to call on practitioners to engage in discussion and dissemination of this important tool. Here, we provide an overview of the process of device validation, the current practice of in-office and out-of-office BP measurement, and current and future directions with regard to device listings. Of note, the AMA VDL will likely not include finger, wrist, and smartphone BP measurement devices due to generally poor validation of the models currently available;^8–10 the use of these devices is reviewed elsewhere.^11,12

THE VALIDATION AND CLEARANCE PROCESS

How Does an Oscillometric Device Work?

As an oscillometric device cycles from inflation to deflation, cuff-sensed arterial pulses are transduced, filtered, amplified and processed to create a mathematically-derived curve called an oscillometric waveform envelope (Figure 1). Algorithms, proprietary to each device manufacturer, are applied to the envelope to estimate systolic and diastolic BP.¹³ Oscillometry provides an algorithm-dependent estimate of systolic and diastolic BP, which can vary widely between different devices, algorithms and patients.^14,15 Factors such as arrhythmias and medical comorbidities can result in greater error rates and inaccurate readings.^16,17

Figure 1.

Depiction of manual blood pressure measurement (with auscultation of Korotkoff sounds) in contrast to automated oscillometric measurement (which derives a mathematic curve called an oscillometric waveform envelope that uses proprietary algorithms to estimate the systolic and diastolic blood pressures).

Abbreviations: BP = blood pressure; MAP = mean arterial pressure

Device Validation

BP device validation should be performed prior to marketing to assess device accuracy and precision against a reference standard, which is typically auscultation using a mercury or calibrated aneroid sphygmomanometer.^4,5,18 The device should be tested in combination with the cuff(s) to be used with the device, as cuffs can be sources of variability.^19,20 Auscultation is performed simultaneously by two trained observers, with careful attention paid to proper cuffing and technique. Each observer is blinded to the results of the other, and between-observer agreement of 4 mmHg or less is required to obtain an acceptable reference standard measurement. Four reference standard measurements bracketing three device measurements taken in alternating fashion are used for analysis. Criteria for subject selection and methods of analysis are protocol-specific and detailed elsewhere.^4,5,18 The main features of the three main validation protocols, including expert consensus-based criteria required for a passing grade,^4,5,18 are summarized in Table 1.

Table 1:

Major Blood Pressure Device Validation Protocols

Protocol	Year of Last Update	Sample Size Required (Number of Paired Readings)	Criteria Indicating a Valid Device
BHS	1993	85 (255)	Device graded from A to D. Grade A is the highest level of accuracy and requires that the percentage of readings with a difference between the device-under-test and the reference sphygmomanometer of ≤ 5, 10, and 15 mmHg be 65, 85 and 95%, respectively.
ESH IP	2010	33 (99)	Pass requirements are split into two phases and are based on the number of measurements with differences between the device-under-test and reference sphygmomanometer of ≤ 5, 10, and 15 mmHg. See protocol for details. This protocol is being phased out, to be replaced by a joint universal AAMI/ESH/ISO validation protocol requiring 85 subjects.
AAMI/ANSI/ISO	2013	85 (255)	Criterion 1: When analyzed as 255 paired determinations, the mean difference between the device-under-test and reference sphygmomanometer is less than 5.0 mmHg and the standard deviation of the difference is less than 8.0 mmHg. Criterion 2: When analyzed as 85 paired determinations, the standard deviation of the difference between the device-under-test and reference sphygmomanometer is less than 4.79 to 6.95 mmHg (the actual threshold varies according to the mean difference observed – see protocol for details).

Abbreviations: AAMI = Association for the Advancement of Medical Instrumentation; ANSI = American National Standards Institute; BHS = British Hypertension Society; ESH IP = European Society of Hypertension International Protocol; ISO = International Standards Organization;

Device Calibration

BP device calibration can be divided into two categories – static and patient-specific. Static calibration involves checking the pressure registered by the device against a reference standard (usually a mercury column or, if not available, a highly sensitive electronic manometer). A device is considered acceptable if measurements agree within 3 mmHg across the BP range.⁵ It may also involve assessing the device measurements against a simulator that generates sample oscillometric waveforms with known systolic and diastolic BP levels. Simulators are a convenient tool, but are not a substitute for human testing.²¹ The extent to which the static calibration of oscillometric devices drifts over time is unclear; aging of transducers, connectors, and cuffs may contribute to drift.

In patient-specific calibration, a comparison to auscultation is performed in a specific patient for the purposes of identifying if that device can accurately assess BP in that individual. This procedure should be done after identifying the arm with higher BP.²² The optimal, most feasible method to perform patient-specific calibration will vary from practice to practice, understanding that practical issues may exist related to patient-specific calibration, such as inadequate reference devices or time constraints. Simplified protocols have been proposed for use in the clinic.²³ The same arm simultaneous (SAS) method, using a three-way connector which attaches the device to a manometer and stethoscope, is one convenient method.⁴ A stethoscope connected to a smartphone that records the Korotkoff sounds for playback may also be used for this method.^24,25 The accuracy of any method for patient-specific calibration is dependent on the accuracy of the provider’s BP measurement. The accuracy of the SAS method can be compromised if the deflation speed of the device is faster than the recommended 2–3 mmHg per second. However, estimates from mathematical models indicate that this error is only a few mmHg, at most.²⁶ At very rapid deflation rates, such as 10 mmHg/sec, a same arm sequential method may be more appropriate.

Patient-specific calibration is important because a device can be inaccurate in an individual patient despite achieving a passing grade in a validation protocol.^14,15 Assuming the cuff is properly sized, this inaccuracy could be the result of an algorithm that performs poorly in that patient, or of altered arterial characteristics in an individual. There is no easy solution to this problem. Using a patient-specific correction factor generated using the SAS method to adjust for the degree and consistency of differences with auscultatory BP is one option, though has not been formally tested. Multiple comparisons should be performed when generating correction factors. In the authors’ experience, performing five comparisons is optimal. Switching to a device from a different manufacturer that performs better in that patient is another solution when a device performs poorly in a specific individual.

Food and Drug Administration (FDA) Clearance

A common misconception is that the FDA “approves” BP devices for patient use. More accurately, the FDA ‘clears’ a device to be sold on the market. Manufacturers of non-invasive BP devices, which are categorized as Class II or moderate-risk devices, obtain clearance to market their device through a Premarket Notification application detailed in the 501(k) section of the Federal Food, Drug and Cosmetic Act.²⁷ Importantly, manufacturers are required only to demonstrate that a new BP device is approximately as safe and effective as similar devices on the market, termed ‘substantial equivalence’.

It is important to understand the limitations of the 510(k) process.²⁸ It does not contain explicit requirements to demonstrate accuracy and does not mandate which, if any, validation protocol should be used. A device manufacturer is also not required to perform an independent, peer-reviewed validation study. Rather, internally generated data can be submitted. These data are not readily available to the public. In addition, post-approval device modification (including new cuffs and changes to the algorithm) is common. Although the FDA provides guidance to the manufacturer when a device modification should trigger a new 510(k) application, it states that “The burden is on the 510(k) holder to decide whether or not a modification could significantly affect safety or effectiveness of the device.”²⁹ In other words, manufacturers are allowed to decide if a new 501(k) submission is needed. The FDA also recommends that the justification for submitting or not submitting a new 510(k) be recorded in a “letter to file change”; however, submission of this documentation to the FDA is not required unless requested. Additional concerns include inadequate enforcement of false claims including ‘off-label’ use of devices beyond their intended market.⁶ Because of these issues, many devices are sold on the market without rigorous evaluation of the accuracy of the device.^30,31 Manufacturers cannot make statements about the accuracy of devices that have not undergone appropriate validation.³² However, it is not readily apparent to patients and providers which devices on the market are appropriately validated.

THE CURRENT PRACTICE IN BP MEASUREMENT

In-Office BP Measurement

Standard Clinical vs. Research BP Measurement

For most practices, there is a major discrepancy between the careful measurement of BP in clinical research and the routine method used in clinical practice.^33–35 In the typical clinical setting, often only a single measurement is obtained by a medical assistant and not by a registered nurse, mid-level provider, or physician. Most medical assistant measurement errors are related to insufficient rest time, incorrect body position, under-cuffing, and talking during the measurement,^36–38 which combined result in a substantially elevated BP measurement compared to measurement by proper technique.³⁹ The choice of equipment at large centers is often based on vendor contracts, while most smaller offices simply use what has been previously provided with often little to no attention paid to the validation status of the device. Portable aneroid BP sphygmomanometers, which are frequently used in smaller practices, are the most vulnerable to loss of calibration over time.^40,41 Further, calibration of BP measuring devices is likely only performed at larger, mostly academic centers with the support of a clinical engineering department.

Automated office BP (AOBP)

Recent international guidelines encourage the use of multiple preprogrammed, observed or unobserved, fully-automated AOBP readings in routine practice.⁴² AOBP has the potential to improve the accuracy of in-office BP measurements by reducing the white coat effect.⁴³ Recent findings suggest similar results in unobserved (i.e. the mean of three consecutive oscillometric readings with no clinician present) compared to observed (i.e. the mean of three consecutive oscillometric readings with a clinician present) AOBP measurements.^44–46

Out-of-Office BP Measurement

Home BP Monitoring

Home BP monitoring is increasingly common and reflects the growing popularity of mobile health technologies in recent years. While it is a cornerstone of recent hypertension guideline recommendations,⁴⁷ there are many unresolved issues with regard to home BP monitoring. A substantial number of home BP devices on the market are not validated, and the government has no enforcement division to prohibit selling these devices (see section on FDA Clearance).⁶ Although an increasing number of people in North America have arm sizes requiring a ‘large’ or ‘extra-large’ cuff, many home BP monitoring devices only come with a ‘regular’-sized cuff.^38,48 Patients often are not aware of the importance of resting for five minutes, correct posture, and abstaining from talking, using a computer, or watching television during measurements.^36,37 Useful tools in addressing these potential sources of error include a patient instruction handout and, more importantly, in-office individual validation of each patient’s device and periodic review of their measurement technique.^14,15

Ambulatory BP Monitoring (ABPM)

ABPM performs automated BP measurements at regular intervals over a 24–48 hour period, including at least one full cycle of wakefulness and sleep. The United States Preventive Service Taskforce⁴⁹ and American College of Cardiology/American Heart Association⁴⁷ guidelines recommend ABPM measurement for the initial diagnosis of hypertension in many patients due to its prognostic superiority over in-office BP measurement.^50–53 However, inadequate reimbursement balanced with high start-up costs for the devices and software, and the relatively time-consuming interpretation of the results make it difficult for small practices to undertake routine use of ABPM.⁵⁴ Additionally, patient-level barriers include potential disruption of work or sleep, and the need to come back to the office for a second visit to return the monitor within one to two days of the initial visit.

ONLINE DEVICE LISTINGS

Many automated BP devices of different varieties (home, ambulatory, clinic) are currently on the market. These devices vary widely in terms of their supporting data, from no available validation data, to full validation assessments conducted according to the most rigorous protocols.^4,5 To ensure that consumers, health care providers, academic researchers, and industry have access to information summarizing which devices have been validated and which criteria were used, device validation listings have been established. These registries are typically focused on devices that have had some type of validation assessment and are funded, at least in part, by device manufacturer application fees. Validation data are typically reviewed by one or more experts in the field of BP measurement.

Dabl Educational Trust,⁵⁵ established in 1993 by a group of international investigators, was the first widely used device registry. It includes manual and automated devices, with the latter category comprising home, ambulatory, and clinic devices. The Dabl registry divides devices into ‘recommended’ and ‘not recommended’ categories according to validation study results. A limitation of the Dabl registry is the classification of devices validated using the Association for the Advancement of Medical Instrumentation (AAMI)/American National Standards Institute (ANSI)/ International Organization for Standardization (ISO) protocol as ‘not recommended,’ citing ‘questionable evidence.’ In particular, the Dabl registry, although willing to accept as valid the 33-subject (with 99 paired readings) European Society of Hypertension International Protocol (ESH IP) results,¹⁸ excludes studies performed using the more rigorous 85-subject (with 255 paired readings) AAMI/ANSI/ISO protocol.⁴ Notably, the existing protocols are being phased out in favor of a conjoint AAMI/ESH/ISO 85-patient protocol,^56,57 which requires similar high precision and large sample size as the AAMI/ANSI/ISO protocol.⁴ There is no longer any clear scientific oversight or regular updating of the Dabl registry.

Medaval⁵⁸ is a newer device registry that, in addition to listing BP devices, also includes blood glucose meters and pulse oximeters; there is no independent scientific oversight of the registry. In addition, professional hypertension societies have sections of their websites dedicated to VDLs of devices available for purchase in their countries. Examples include Hypertension Canada,⁵⁹ the British and Irish Hypertension Society (BIHS),⁶⁰ and the Japanese Society of Hypertension.⁶¹ The Hypertension Canada device listing requires a manufacturer application and published validation data. It divides devices into those validated using the British Hypertension Society (BHS)⁵ or AAMI/ANSI/ISO protocol⁴ (Gold tier) and ESH IP¹⁸ (Silver tier).

The New AMA VDL

The AMA’s VDL will be established to address some of the limitations of previous and current device listings, requiring more rigorous inclusion criteria. The primary impetus for the AMA VDL was lack of any legal requirement in the United States for devices to undergo rigorous validation testing for clinical accuracy. As a result, many devices have not been tested properly and could yield inaccurate readings. There is no way to know the accuracy of a given BP measurement device without going through clinical validation protocols. Payers, health systems, physicians, and individuals can all purchase these devices in the United States. Yet there is no listing of device available in the United States that have gone through validation testing for clinical accuracy. As a result, it is very difficult for these groups to make an informed decision when choosing a device to use or recommend for self-monitoring of BP. The AMA and a group of experts who came together are seeking to create a regularly updated repository of devices that have been validated for clinical accuracy that will be publicly available online.

For a device to be listed in the AMA VDL, manufacturers will have to include in their application the details provided in Table 2. Importantly, documentation of substantial equivalence to a device already on the market is insufficient for entry into the AMA VDL if a critical component of the new device differs from the previously validated one. Critical components that may affect BP measurement include the cuff, transducer, inflation/deflation process, procedure for waveform acquisition and processing, and algorithm. Validation data must be provided that were generated externally and preferably published in peer-review format (Table 2). As with other registries, the only assurance that validation protocols were performed correctly is by peer-review.

Table 2:

Proposed Entry Criteria for the American Medical Association Validated Device Listing

Criteria
A validation study performed according to one of the following 85-subject protocols*:
1. ANSI/AAMI/ISO 81060–2:2013
2. AAMI/ISO 81060–2:2009
3. ANSI/AAMI SP10:2002
4. BHS Revised Protocol 1993
One of the following methods of summarizing validation data (listed in order of preference):
1. Peer-reviewed publication.
2. Independent third party validation testing by a qualified entity. These may include academic institutions or credible research entities with expertise in BP measurement and knowledge of validation protocols and validation study requirements.

^*The European Society of Hypertension International Protocol¹⁸ alone is not considered acceptable

Abbreviations: AAMI = Association for the Advancement of Medical Instrumentation; ANSI = American National Standards Institute; ISO = International Standards Organization; BHS = British Hypertension Society

A major difference between the AMA VDL and the Canadian and BIHS device listings is the distinction of the validation protocols deemed acceptable. The AMA VDL will not recommend devices tested in validation studies performed using only the ESH IP. This protocol has been judged inadequate for AMA VDL entry due to the low sample size and low number of paired readings; therefore, only the ANSI/AAMI/ISO or BHS protocols are considered acceptable. See Table 3 for a detailed comparison between the AMA, Canadian, and BIHS VDLs.

Table 3.

Comparison of National Society Validated Device Listings

Validated Device Listing Criteria	American Medical Association	Hypertension Canada	British and Irish Hypertension Society
Accepted validation protocols	85-subject protocols only: • AAMI/ANSI/ISO • BHS	85-subject protocols (Gold Tier): • AAMI/ANSI/ISO • BHS 33-subject protocols (Silver Tier): • ESH IP	85-subject and 33-subject protocols: • BHS • ESH IP
Accepted methods for summarizing validation data	• Peer-reviewed publication • Unpublished independent third party validation study	• Peer-reviewed publication	• Peer-reviewed publication • Society-performed validation study
Listed devices	• Only recommended devices	• Only recommended devices	• Recommended and non-recommended devices

Abbreviations: AAMI = Association for the Advancement of Medical Instrumentation; ANSI = American National Standards Institute; ISO = International Standards Organization; BHS = British Hypertension Society; ESH IP = European Society of Hypertension International Protocol

The proposed AMA VDL criteria were open for public comment during the Summer of 2018. In the Fall of 2018, the AMA convened with a panel of experts to address the public comments. The AMA VDL is currently undergoing administrative approvals and starting production; the AMA plans to make the VDL available for public use in early to mid-2019.

CONCLUSION

The removal of mercury-based BP measurement devices resulted in a shift to aneroid and oscillometric methods to determine BP in the office and home settings. Oscillometric device validation is a critical component to ensure accurate assessment of BP and thus promote appropriate BP management. Recent hypertension guidelines emphasize lower BP treatment goals based on carefully obtained in-office readings, to be complemented by home BP monitoring.⁴⁷ Accordingly, greater attention to the accuracy of the devices and methods being used to measure BP both in and out of the office is paramount. Many devices available to both healthcare providers and patients have not undergone validation prior to being placed on the market. Since BP devices are distributed worldwide, uniformity of BP accuracy standards across the world will simplify development of newer devices for manufacturers, while assuring safety for patients. Access to an updated listing of available, validated devices is an important step in improving current methods of BP measurement across healthcare systems and practices nationally. The VDL will also likely facilitate development of national standards for home BP measurement, which may in turn promote inclusion of home BP measurements in practice quality assessments and merit-based incentive programs.

We strongly encourage clinicians to actively take part in discussions around the implementation of the VDL, and to disseminate this valuable tool to the medical community and to patients involved or interested in home BP monitoring.