Skip to main content
American Journal of Respiratory and Critical Care Medicine logoLink to American Journal of Respiratory and Critical Care Medicine
letter
. 2024 May 31;210(3):355–358. doi: 10.1164/rccm.202401-0036LE

Skin Pigmentation and Pulse Oximeter Accuracy in the Intensive Care Unit: A Pilot Prospective Study

Ashraf Fawzy 1,✉, Harith Ali 1, Peter H Dziedzic 2, Niteesh Potu 2, Eusebia Calvillo 2, Sherita H Golden 4,5, Theodore J Iwashyna 1,6, Jose I Suarez 2,3, David N Hager 1, Brian T Garibaldi 1
PMCID: PMC11348963  PMID: 38820169

To the Editor:

As of the beginning of 2024, the U.S. Food and Drug Administration (FDA) requires testing medical-grade pulse oximeters on 10 or more unique individuals (to date, always healthy volunteers) contributing 200 or more paired pulse oximeter (SpO2) and arterial blood (SaO2) measurements, with ⩾15% darkly pigmented (1). Average root mean square error (ARMS) ⩽ 3.0% is the current threshold for FDA clearance of finger-wrapped sensors (2). The real-world accuracy of pulse oximeters has been questioned, with concern that unrecognized or “occult” hypoxemia is associated with delayed therapy, especially among Black patients (3–7). However, previous studies have major limitations: race and/or ethnicity were used as surrogates for skin pigmentation, and paired arterial blood gas (ABG) and SpO2 readings could be 10 minutes or more apart (8). We address these limitations by objectively measuring skin pigmentation and linking SpO2 to the exact time of ABG collection in a sample of critically ill adults that exceeds current FDA requirements for premarket notification submission (1). This study’s design serves as a model for future pre- and postmarket testing (2). Some of the results were reported in a preprint and presented at an FDA Advisory Committee Meeting (9, 10).

Patients in The Johns Hopkins Hospital medical ICU with an arterial catheter were prospectively recruited. The Johns Hopkins University Institutional Review Board approved this study, and informed consent was obtained before data collection (IRB#00318076). Concurrent SaO2 and SpO2 were gathered from patients’ entire hospitalization. ABGs were analyzed by co-oximetry on the ABL825 or ABL827 (Radiometer America). The Masimo SET LNCS Neo-3 wrap oximeter connected to a GE Carescape B850 monitor was used. Vital sign data, including SpO2, was recorded at 2-second intervals. Arterial line waveforms were visually inspected in the Sickbay Platform to determine ABG collection time. For each ABG, the corresponding SpO2 value was the median SpO2 during ABG sampling. Occult hypoxemia was defined as SaO2 < 88% with concurrent SpO2 ⩾ 92%. Comorbidities were categorized and coded based on established International Classification of Disease, 10th Revision algorithms (11).

Skin pigmentation was measured using the Konica Minolta CR-400 chromameter at the forehead, forearm, and finger pad (12). Objective quantification of skin pigmentation using the individual typology angle (ITA) was calculated from the Commission Internationale de l’Eclairage coordinates as [arctan(L*−50)/b*)]×(180/π) for each participant and anatomic site (12). Participants with forehead ITA < −30° were categorized as darkly pigmented (12). Error was calculated as SpO2 − SaO2. The bias (√(SpO2 − SaO2)/N) and ARMS (√([SpO2 − SaO2]2)/N) were calculated according to the International Organization for Standardization document 80601-2-61:2011 (1). Statistical comparison of ARMS was performed using bootstrapping (10,000 sampling replications). Separate mixed effects models compared SpO2 − SaO2 with skin pigmentation ITA at each anatomical site, accounting for repeated measures and controlling for SaO2, pH, heart rate, mean arterial pressure, and vasopressor use.

Twelve patients were enrolled. Five were darkly pigmented (forehead ITA: median, −46.2°; range, −52.5° to −42.0°), and seven were lighter pigmented (forehead ITA: median, 12.6°; range, −6.3° to 48.3°). Forearm pigmentation was similar, but finger pad pigmentation was less distinguishing. Baseline characteristics, comorbid conditions, and hospital diagnoses are presented in Table 1.

Table 1.

Baseline Characteristics, Comorbidities, Hospital Diagnoses, and Characteristics of Paired Arterial Blood Gas Samples and Pulse Oximetry by Skin Pigmentation

  All Patients (N = 12) Darkly Pigmented Patients (n = 5) Lighter-Pigmented Patients (n = 7)
Age, yr, median (range) 55.9 (34.7–79.4) 54 (42.9–70.2) 64.8 (34.7–79.4)
Female 6 (50) 2 (40) 4 (57.1)
Race      
 Asian 1 (8.3) 0 (0) 1 (14.2)
 Black 5 (41.7) 5 (100) 0 (0)
 White 6 (50) 0 (0) 6 (85.7)
Principal inpatient diagnosis      
 Sepsis or infection 9 (75) 4 (80) 5 (71.4)
 Myocardial dysfunction 1 (8.3) 1 (20) 0 (0)
 Hepatic dysfunction 2 (16.7) 0 (0) 2 (28.6)
 Acute respiratory failure during hospitalization 8 (66.7) 4 (80) 4 (57.1)
Comorbid conditions      
 Cerebrovascular disease 4 (33.3) 1 (20) 3 (42.9)
 Congestive heart failure 3 (25) 3 (60) 0 (0)
 Myocardial infarction 2 (16.7) 2 (40) 0 (0)
 Peripheral vascular disease 4 (33.3) 3 (60) 1 (14.3)
 Diabetes (complicated or uncomplicated) 7 (58.3) 3 (60) 4 (57.1)
 Pulmonary disease 6 (50) 3 (60) 3 (42.9)
ABG samples with blood sampling period identified, n 350 96 254
 SaO2 97.1 (95.1–98.4) 97.0 (94.9–98.7) 97.2 (95.3–98.3)
  SaO2 < 90% 9 (2.6) 5 (5.2) 4 (1.6)
  SaO2, 90–94% 44 (12.6) 11 (11.5) 33 (13)
  SaO2 > 94% 297 (84.9) 80 (83.3) 217 (85.4)
 Occult hypoxemia (SaO2 < 88%, SpO2 ⩾ 92%) 3 (0.9) 2 (2.1) 1 (0.4)
 Occult hypoxemia, alternative definition (SaO2 < 90%, SpO2 ⩾ 90%) 3 (0.9) 2 (2.1) 1 (0.4)
 pH 7.43 (7.39–7.47) 7.43 (7.31–7.45) 7.43 (7.40–7.48)
 Heart rate, beats/min 90 (74–105) 98 (79–111.5) 89 (72–101.5)
 Mean arterial pressure, mm Hg 78 (70–91) 73.8 (70–81) 80.3 (70–96)
 Vasopressor use (at time of ABG sampling) 201 (57.4) 80 (83.3) 121 (47.6)
 Blood sampling duration, s 46.1 (28.1–69) 43.6 (28.7–65.8) 46.7 (28–70.3)

Definition of abbreviations: ABG = arterial blood gas; SpO2 = oxygen saturation as measured by pulse oximetry.

Data are given as n (%) or median (interquartile range) unless otherwise noted. Reprinted by permission from Reference 9.

Four hundred ABGs were collected during routine clinical care (darkly pigmented: 119 [median, 24; range, 2–49]; lighter pigmented: 281 [median, 12; range, 1–129]). A corresponding SpO2 was not available, or the arterial tracing was absent or nonpulsatile, for 50 observations, leaving 96 ABGs in darkly pigmented participants and 254 in lighter pigmented participants. SaO2 < 90% was more common among darkly pigmented patients than lighter pigmented patients (5.2% vs. 1.6%). Other metrics, including vasopressor use, are presented in Table 1.

Overall, pulse oximeter bias was 0.62%, and the ARMS was 2.75%. However, among darkly pigmented patients, pulse oximeter bias was 1.05% and the ARMS was 4.15% (95% confidence interval [CI], 2.35–5.72%), which was significantly higher than for lighter-pigmented patients (bias, 0.34% and ARMS, 1.97%; 95% CI, 1.76–2.17%). Occult hypoxemia was present in 2.1% of sample pairs of darkly pigmented participants compared with 0.4% of sample pairs in lighter-pigmented participants. Pulse oximeter error for darkly pigmented participants increased as SaO2 decreased (Figure 1A).

Figure 1.


Figure 1.

(A) Distribution (with median and range) of pulse oximeter error (SpO2 − SaO2) stratified by skin pigmentation and SaO2. (B–D) Estimated mean pulse oximeter error (SpO2 − SaO2) by skin pigmentation represented by the individual typology angle (in degrees) at the (B) forehead, (C) forearm, and (D) finger pad. Graphs are at SaO2, 94%; pH, 7.4; heart rate, 90 beats/min; mean arterial pressure, 75 mm Hg. SpO2 = oxygen saturation as measured by pulse oximetry. Reprinted by permission from Reference 9.

In regression analysis, SpO2 − SaO2 was on average 0.95% (95% CI, 0.18–1.73%) higher for darkly pigmented patients than lighter-pigmented patients. A 10° reduction in forearm ITA (darker pigmentation) was associated with a 0.19% increase in SpO2 − SaO2 (95% CI, 0.10–0.28%). Associations of pulse oximeter accuracy with forehead and finger pad pigmentation were nonlinear but demonstrated significant overestimation of SaO2 by SpO2 for darker pigmentations (Figures 1B–1D). The marginal difference compared with intermediate or light pigmentation (ITA, 41°) was 2.41% (95% CI, 1.67–3.14%) for dark (ITA, −45°), 2.93% (95% CI, 2.10–3.75%) for brown (ITA, −10°), and 1.80% (95% CI, 1.24–2.36%) for tan (ITA, 19°) forehead pigmentation and 2.75% (95% CI, 1.97–3.52%) for brown and 2.51% (95% CI, 1.91–3.11%) for tan finger pad pigmentation.

This is the first real-world clinical study using objectively measured skin pigmentation and precise identification of SpO2 − SaO2 pairs to evaluate the accuracy of pulse oximetry in critically ill adults. The FDA-cleared pulse oximeter in use performed worse in darkly pigmented patients and would not meet established criteria for FDA clearance in this population, whereas performance was adequate for lighter-pigmented patients. The results are independent of saturation, acidosis, heart rate, blood pressure, and vasopressor use, which may compound pulse oximeter error.

Early clinical studies using subjective skin pigmentation scales noted that pulse oximeters were less accurate among darkly pigmented patients (5). By objectively measuring skin pigmentation, we avoid introduction of subjective biases when visually assessing skin pigmentation. Although pigmentation at the forehead or forearm best distinguishes dark and lighter pigmentation, finger pad pigmentation was also strongly associated with pulse oximeter error. The findings of this study are at odds with proprietary laboratory data of a Masimo SET oximeter (13). Other laboratory studies have reported poor accuracy among darkly pigmented volunteers at more severe desaturations (SaO2 < 80%) (14–16).

Although this study addresses prior methodologic shortcomings, limitations remain. This study only evaluated a single pulse oximeter model, limiting generalizability, because bias and accuracy vary by oximeter brand (17). However, the devices tested represent the top two suppliers of pulse oximeter systems in the United States (18). The majority (85%) of SaO2 values were >94%, which is above the target oxygen saturation for some hospitalized patients, potentially limiting generalizability. Furthermore, anatomic placement of the oximeter was unknown. Variability in the number of ABGs per patient may introduce confounding by indication. The relatively small sample size did not allow subgrouping of lighter-pigmented patients, and patient-level factors could not be included in the analysis.

This study creates a framework for future investigations to use objectively measured skin pigmentation and high-resolution vital sign data paired with clinically obtained ABGs that does not rely on study-specific biospecimens. This approach reduces participant and staff burden, while reflecting naturally occurring variations in oxygenation under real care and enhancing the probability of capturing occult inaccuracies. Although in February 2024 the FDA recommended updated testing requirements of pulse oximeters (10), devices may continue to be tested in laboratories on healthy volunteers (19). Regulatory standards should be updated to ensure that devices are tested in real-world settings and include subjects with objectively dark pigmentation with sufficient power to characterize device performance in this subgroup.

Footnotes

Supported by funding from John Hopkins inHealth (the Johns Hopkins Precision Medicine initiative), the John Templeton Foundation, and NIH NHLBI grant K23HL151758 (A.F.).

Author Contributions: A.F. was involved in conception and design of the study; acquisition, analysis, and interpretation of the data; and drafting of the manuscript. H.A., P.H.D., N.P., E.C., and J.I.S. were involved in acquisition and interpretation of the data and reviewed the manuscript for important intellectual content. S.H.G., T.J.I., and D.N.H. were involved in interpretation of the data and reviewed the manuscript for important intellectual content. B.T.G. was involved in conception of the study and interpretation of the data and reviewed the manuscript for important intellectual content. All authors approved the final version of the manuscript.

Originally Published in Press as DOI: 10.1164/rccm.202401-0036LE on May 31, 2024

Author disclosures are available with the text of this letter at www.atsjournals.org.

References


Articles from American Journal of Respiratory and Critical Care Medicine are provided here courtesy of American Thoracic Society

RESOURCES