Performance of the MP570T pulse oximeter in volunteers participating in the controlled desaturation study: a comparison of seven probes

Abstract

Background

The performance of the pulse oximeter was evaluated based on the ISO 80601-2-61:2011 (E) guidelines. This study aimed to determine whether the various finger probes of the MP570T pulse oximeter (MEK-ICS Co., Ltd., Korea) would provide clinically reliable peripheral oxygen saturation (SpO₂) readings over a range of 70-100% arterial oxygen saturation (SaO₂) during non-motion conditions.

Methods

Each volunteer (n = 12) was connected to a breathing circuit for the administration of a hypoxic gas mixture. For frequent blood sampling, an arterial cannula was placed in a radial artery. The following seven pulse oximeter probes were simultaneously attached to each volunteer’s fingers: (1) WA-100 reusable finger probe (MEDNIS Co., Ltd., Korea), (2) MDNA disposable finger probe (MEDNIS Co., Ltd.), (3) IS-1011 disposable finger probe (Insung Medical Co., Ltd., Korea), (4) CJ340NA disposable finger probe (CHUN JI IN Medical Co., Ltd., Korea), (5) Nellcor^TM OxiMax DS-100A reusable finger probe (Medtronic, USA), (6) Nellcor^TM OxiMax MAX-N disposable finger probe (Medtronic), and (7) OXI-PRO DA disposable finger probe (Bio-Protech Inc., Korea).

Results

A total of 275 SpO₂-SaO₂ pairs were included in the analysis. The accuracy of the root mean square (A_rms) of each probe was 2.83%, 3.98%, 3.75%, 6.84%, 3.43%, 5.17%, and 3.84%, respectively.

Conclusions

The MP570T pulse oximeter with WA-100 reusable, MDNA disposable, IS-1011 disposable, Nellcor^TM OxiMax DS-100A reusable, and OXI-PRO DA disposable finger probes meets an acceptable standard of SpO₂ accuracy under non-motion conditions.

INTRODUCTION

A pulse oximeter is a non-invasive medical device used to monitor patients’ oxygen saturation. The peripheral oxygen saturation (SpO₂) measured by this device is an estimate of the oxygen saturation in the artery (SaO₂). In general, a pulse oximeter uses dual-wavelength absorption spectrophotometry to measure SpO₂ [1]. The wavelengths are selected to provide the best separation of absorbance values of oxyhemoglobin and deoxyhemoglobin [1]. The ratio of the two absorbance values is used to calculate SpO₂ values.

Since a pulse oximeter is used for continuous monitoring of oxygenation and plays an important role in the early detection of hypoxia, the SpO₂ accuracy of a pulse oximeter should be validated for patient safety. The performance of a pulse oximeter is evaluated based on clause 201.12.1 of the ISO 80601-2-61:2011 (E) guidelines [2], and a controlled desaturation study is conducted within the range of 70-100% of SaO₂ by administering air-nitrogen-CO₂ mixtures to a patient or volunteer. Pulse oximetry equipment consists of a monitor (electronics, display, and operator-equipment interface), probe, and cable extender, and its performance needs to be evaluated according to international standards. The use of different sensors with the same pulse oximeter monitor is considered to be a difference warranting re-evaluation of its reliability. In particular, the United States of America Food and Drug Administration (USFDA) requires a controlled desaturation study to evaluate the performance of pulse oximeter equipment before it can be sold in the United States [3].

The aim of this study was to determine whether the MP570T pulse oximeter (MEK-ICS Co., Ltd., Korea) with various finger probes would provide clinically reliable SpO₂ readings over a range of 70-100% SaO₂ during non-motion conditions.

MATERIALS AND METHODS

Volunteer population

The study had a single-center, non-randomized design. This study was conducted in a normal environment at the Intensive Clinical Research Room of the author’s hospital according to ISO 14155: 2011 guidelines and the clinical trial guidelines of the pulse oximeter (Ministry of Food and Drug Safety, Korea), which complied with ISO 80601-2-61:2011 (E) guidelines [2]. This study was approved by our Institutional Review Board (no. 2016-0069) and was registered in an international clinical trials registry platform (http://cris.nih.go.kr, no. KCT0004854). Written informed consent was obtained from all volunteers. The inclusion criteria for the participants were: aged 20-50 years, carboxyhemoglobin < 3%, methemoglobin < 2%, and total hemoglobin concentration > 10 g/dl. The exclusion criteria were: a known history of respiratory or cardiovascular disease, smoking habits, evidence of pregnancy, history of syncope, diabetes mellitus, skin diseases that can affect SpO₂ values, such as onychomycosis [4], and body mass index ≥ 35 kg/m². The volunteers were fully informed of the study protocols and completed health assessment questionnaires before enrollment.

Procedure for controlled desaturation study

Volunteers were monitored using end-tidal carbon dioxide partial pressure and fraction of inspired oxygen by using Carescape^® B850 (GE Healthcare, USA). Each volunteer was placed in a semi-Fowler’s position and connected to a breathing circuit to administer the nitrogen-air-carbon dioxide mixtures. A nose clip was applied to prevent breathing of room air. For frequent blood sampling, an arterial cannula was placed in the radial artery of each volunteer. Seven pulse oximeter probes were simultaneously attached to each volunteer’s fingers. The following models were used in this study: (1) WA-100 reusable finger probe (MEDNIS Co., Ltd., Korea), (2) MDNA disposable finger probe (MEDNIS Co., Ltd.), (3) IS-1011 disposable finger probe (Insung Medical Co., Ltd., Korea), (4) CJ340NA disposable finger probe (CHUN JI IN Medical Co., Ltd., Korea), (5) Nellcor^TM OxiMax DS-100A reusable finger probe (Medtronic, USA), (6) Nellcor^TM OxiMax MAX-N disposable finger probe (Medtronic), and (7) OXI-PRO DA disposable finger probe (Bio-Protech Inc., Korea). The same reusable probes were used for all volunteers, and a new disposable probe was used for each volunteer. To minimize light interference from outside, both hands were covered with a blanket. An air warmer (Bair Hugger™, 3M™, USA) was applied to the hands to prevent hypoperfusion. To obtain the SpO₂ value to be referenced when determining the target plateau, a reusable pulse oximeter finger probe (OxiMax^® N-600x, Medtronic) was also used. The probes were placed on the fingers of the hand with the arterial catheter as follows: CJ340NA disposable finger probe (thumb), MDNA disposable finger probe (index finger), OXI-PRO DA disposable finger probe (middle finger), Nellcor^TM OxiMax MAX-N disposable finger probe (ring finger). The probes were placed on the fingers of the other hand as follows: IS-1011 disposable finger probe (thumb), OxiMax^® N-600x (index finger), WA-100 reusable finger probe (middle finger), Nellcor^TM OxiMax DS-100A reusable finger probe (ring finger). Each volunteer was exposed to various levels of induced hypoxia from 70-100% of SaO₂. Each plateau of oxygen saturation was maintained for at least 30 s until stabilization, after which 1 ml of arterial blood was drawn into a heparinized syringe (Fig. 1). The study period consisted of two rounds of hypoxia, and the volunteers were maintained on room air between each round. SaO₂ measurements using a CO-oximeter (ABL90 FLEX, Radiometer Medical A/S, Denmark) were used as a reference for the SpO₂ accuracy.

Fig. 1.

Target plateaus of oxygen saturation and sampling points. The red dots represent the SaO₂ value at each sampling point. According to the SaO₂ stability standard of the pulse oximeter probe placement site, the first sample at each plateau was acquired 30 s after reaching a target plateau, and the successive sample was acquired at least 20 s after the first sample.

Methods of performance evaluation

The measured SaO₂ values were matched with the corresponding SpO₂ values. Reference SaO₂ data outside the range of 73-97% were excluded from the analysis [2]. The SpO₂-SaO₂ pair data were analyzed according to the following statistics.

A_rms (accuracy root mean square)

SpO₂ accuracy was evaluated by the A_rms for the overall range.

$$A_{rms}=\sqrt{\frac{\sum _{i=1}^n{(Sp{O}_{2i}-S_{Ri})}^2}{n}}$$ (1)

where SpO_2ij is the i^th measured SpO₂ value and S_Ri is the i^th measured standard reference value.

Bias

The average difference was calculated to show the bias of the device being tested compared to the reference. The mean bias was calculated for the overall range.

$$Mean bias=\frac{\sum _{i=1}^n(Sp{O}_{2i}-S_{Ri})}{n}$$ (2)

Precision

Precision is a measure of the scatter to be expected in multiple measurements taken with the same pulse oximeter equipment at a given oxygen saturation, considering both the variation among patients and the repeatability of the device. Precision was calculated for the overall range.

$$Precision=\sqrt{\frac{\sum _{i=1}^n{(Sp{O}_{2i}-Sp{O}_{2fit,i})}^2}{n-2}}$$ (3)

where SpO_2fit,i is the value of the curve fitted to the test data at the ith reference oxygen saturation value (S_Ri).

Of these parameters, the pass/fail criteria of pulse oximeter equipment are determined by A_rms. Over a range of 70-100% of SaO₂, A_rms was less than or equal 4% in non-motion conditions.

Statistics

Data are expressed as mean ± standard deviation (SD) for normally distributed continuous variables, median (25-75%) for non-normally distributed continuous variables, or as counts for categorical variables.

RESULTS

A total of 12 volunteers were enrolled in the study, and all volunteers were included in the analysis. The characteristics of these volunteers are presented in Table 1. The mean ± SD time for volunteers to participate in the current study from the first blood sampling to the last blood sampling was 25.8 ± 3.1 min. Of the 293 SpO₂-SaO₂ pairs obtained from these volunteers, 18 pairs were excluded from the analysis because they were outside the defined SaO₂ range. Therefore, 275 pairs were included in the analysis (73 ≤ SaO₂ [%] < 80: n = 91; 80 ≤ SaO₂ < 90: n = 99, 90 ≤ SaO₂ ≤ 97: n = 85). Performance comparison of each probe within the range of 73-97% of SaO₂ is summarized in Table 2. Two of the seven probes (CJ340NA disposable finger probe and Nellcor^TM OxiMax MAX-N disposable finger probe) did not meet the performance evaluation criteria. Observed SpO₂ versus measured SaO₂ in two pieces of pulse oximeter equipment with the lowest (WA-100 reusable finger probe) and the highest (CJ340NA disposable finger probe) A_rms are shown in Fig. 2. In the case of the CJ340NA probe, it was confirmed that overall SpO_2i values corresponding to S_Ri values were high, which means that it is overestimated. For the two cables connected by the MP570T pulse oximeter, Bland-Altman plots and the relationship between S_Ri and bias are presented in Figs. 3 and 4, respectively. There were no adverse events observed throughout the study period.

Table 1.

Volunteer Characteristics

ID	Sex	Height (cm)	Weight (kg)	Age (yr)	Complexion	Ethnicity
1	M	172.1	56.5	19	Medium	Asian
2	M	173	64.6	19	Medium	Asian
3	M	172	79.6	22	Medium	Asian
4	F	162.6	54	27	Medium	Asian
5	F	161.6	47.8	27	Medium	Asian
6	M	182.3	76.35	23	Medium	Asian
7	F	157	49.4	29	Light	Asian
8	F	166.6	50.4	22	Light	Asian
9	F	162.4	57.7	29	Dark	African
10	M	174.1	74.35	41	Dark	African
11	M	171.7	61.8	37	Dark	African
12	F	172.8	62.7	28	Dark	African

Mean ± SD height, weight, and age were 169.0 ± 7.1 cm, 61.3 ± 10.8 kg, and 26.9 ± 6.7 yr, respectively.

Table 2.

Performance Comparison of Each Probe within a Range of 73-97% of Arterial Oxygen Saturation (SaO₂)

Probes	1	2	3	4	5	6	7
Arms (%)	2.83	3.98	3.75	6.84	3.43	5.17	3.84
Mean bias (%)	-1.10	3.29	3.04	6.01	2.63	4.69	2.70
Precision (%)	2.22	2.25	2.19	2.56	2.16	2.07	2.74

1: WA-100 reusable finger probe, 2: MDNA disposable finger probe, 3: IS-1011 disposable finger probe, 4: CJ340NA disposable finger probe, 5: Nellcor^TM OxiMax DS-100A reusable finger probe, 6: Nellcor^TM OxiMax MAX-N disposable finger probe, 7: OXI-PRO DA disposable finger probe. A_rms, bias, and precision were calculated by the following equations: $A_{rms}=\sqrt{\frac{\sum _{i=1}^n{(Sp{O}_{2i}-S_{Ri})}^2}{n}}, Bias=Sp{O}_{2i}-S_{Ri, }Precision=\sqrt{\frac{\sum _{i=1}^n{(Sp{O}_{2i}-Sp{O}_{2fit,i})}^2}{n-2}}$ , where SpO_2ij is the i^th measured peripheral oxygen saturation (SpO₂) value, S_Ri is the i^th measured standard reference value, and SpO_2fit,i is the value of the curve fitted to the test data at S_Ri.

Fig. 2.

Measured SpO₂ (SpO_2i) versus reference SaO₂ (S_Ri) plots. (A) WA-100 reusable finger probe, (B) CJ340NA disposable finger probe. The linear regression line (red solid line) and the equation with R² are shown on the plot. The line of identity (blue dotted line) represents when SpO_2i = S_Ri. SpO₂: pulse oxygen saturation, SaO₂: oxygen saturation of arterial blood, SpO_2i: SpO₂ measured by the device being tested, S_Ri: SaO₂ measured by the reference CO-oximeter.

Fig. 3.

Bland-Altman plots, with mean bias and 95% limits of agreement. (A) WA-100 reusable finger probe, (B) CJ340NA disposable finger probe. Mean error and y = 0 are shown as red solid and black dashed lines, respectively. The upper limit (mean bias + 1.96 × SD) and lower limit (mean bias 1.96 × SD) of the agreement are shown as a gold dotted line. SpO_2i: peripheral oxygen saturation measured by the device being tested, S_Ri: oxygen saturation of arterial blood measured by the reference CO-oximeter.

Fig. 4.

Bias (SpO_2i - S_Ri) versus reference SaO₂ (S_Ri) plots, with mean bias and 95% limits of agreement. (A) WA-100 reusable finger probe, (B) CJ340NA disposable finger probe. Mean error and y = 0 are shown as red solid and black dashed lines, respectively. The upper limit (mean bias + 1.96 × SD) and lower limit (mean bias 1.96 × SD) of the agreement are shown as a gold dotted line. SpO_2i: peripheral oxygen saturation measured by the device being tested, S_Ri: oxygen saturation of arterial blood measured by the reference CO-oximeter.

DISCUSSION

The pulse oximeter is a piece of medical equipment that is commonly used in clinical settings [1], but few medical practitioners understand the accuracy evaluation criteria. Accuracy data obtained from controlled desaturation studies are not required for pulse oximeter equipment in Korea. However, the USFDA requires these data from companies desiring to sell pulse oximeter equipment in the United States [3]. The common manufacturing literature claim for A_rms for pulse oximeters is ± 2–3% over the range of 70–100% SpO₂ [5]. The total number of acceptable SpO₂-SaO₂ pair data obtained from controlled desaturation studies should be sufficient to statistically validate the specified SpO₂ accuracy. Typically, at least 10 volunteers are recruited, and at least 20 arterial blood samples per volunteer are obtained and analyzed with at least 200 data pairs. Moreover, the distribution of the S_Ri values should have a similar density over the entire required range; for example, the ranges of 70-79%, 80-89%, and 90-100% SaO₂ should each have approximately 1/3 of the total data. In general, the lower the SaO₂, the higher the value of A_rms. In the case of WA-100 reusable finger probe, the values of A_rms were 4.06 for 70-79% SaO₂, 2.32 for 80-89%, and 1.40 for 90-100%, respectively.

Particularly, the skin complexion of the study participants should be specified because this affects the accuracy of SpO₂, with dark skin pigmentation resulting in an overestimation of arterial oxygen saturation especially at a low saturation in some pulse oximeters [6]. For some pulse oximeters, at 60–70% SaO₂, SpO₂ overestimated SaO₂ (bias ± SD) by 3.56 ± 2.45% in darkly pigmented subjects, compared with 0.37 ± 3.20% in lightly pigmented subjects (P < 0.0001) [6]. Additionally, another previous study revealed the effect of skin pigmentation on bias [7]. For this reason, the USFDA recommended that controlled desaturation studies should include subjects with a range of skin pigmentation, including at least 2 darkly pigmented subjects or 15% of the subject pool, whichever is larger [3]. For this reason, some Africans participated in the current clinical trial.

On pulse oximeter accuracy evaluation, sex can be a factor affecting bias. In an earlier study, sex was found to be a significant univariate predictor for bias [7]. This may be because female has a smaller pulsatile signal detected by the probe due to the relatively smaller finger size. However, in general, female volunteers have lower hemoglobin levels than male volunteers. Female’s hemoglobin levels were also significantly lower in this study ((female [n = 6]: 13.1 g/dl; male [n = 6]: 14.9 g/dl; Student’s t-test, P = 0.001). Because low hemoglobin levels were prevalent in female, it was not possible to statistically separate the contributions of sex and low hemoglobin to oximeter bias [7]. In the case of the WA-100 reusable finger probe, the A_rms did not show much difference between sexes within the range of 73-97% SaO₂ (male: 2.55%; female: 3.07%).

Through this study, it was confirmed that not only the pulse oximeter but also the connected probes are important for evaluating accuracy. Sensor type was a predictor of bias in SpO₂ estimates at low SaO₂ levels [7]. A prior study also revealed that some of the sensors on the market may be inaccurate [5]. Therefore, it is necessary to check the performance of the pulse oximeter equipment (pulse oximeter and probe) in the clinical field.

In conclusion, these results provide supporting evidence that the SpO₂ accuracy of the MP570T pulse oximeter with WA-100 reusable, MDNA disposable, IS-1011 disposable, Nellcor^TM OxiMax DS-100A reusable, and OXI-PRO DA disposable finger probes pass an A_rms specification of ≤ 4% over the range of 70-100% SaO₂, under non-motion conditions.