Evaluation of Remote Monitoring Device for Monitoring Vital Parameters against Reference Standard: A Diagnostic Validation Study for COVID-19 Preparedness

Abstract

Context:

Vital parameters including blood oxygen level, respiratory rate, pulse rate, and body temperature are crucial for triaging patients to appropriate medical care. Advances in remote health monitoring system and wearable health devices have created a new horizon for delivery of efficient health care from a distance.

Materials and Methods:

This diagnostic validation study included patients attending the outpatient department of the institute. The accuracy of device under study was compared against the gold standard patient monitoring systems used in intensive care units.

Statistical Analysis:

The statistical analysis involved computation of intraclass correlation coefficient. Bland–Altman graphs with limits of agreement were plotted to assess agreement between methods. P <0.05 was considered statistically significant.

Results:

A total of 200 patients, including 152 males and 48 females in the age range of 2–80 years, formed the study group. A strong correlation (intraclass correlation coefficient; r > 0.9) was noted between the two devices for all the investigated parameters with significant P value (<0.01). Bland–Altman plot drawn for each vital parameter revealed observations in agreement from both the devices.

Conclusion:

The wearable device can be reliably used for remote health monitoring. Its regulated use can help mitigate the scarcity of hospital beds and reduce exposure to health-care workers and demand of personal protection equipment.

INTRODUCTION

Pandemics caused by highly contagious diseases have always effected the population adversely and globally. First coronavirus-associated severe acute respiratory syndrome (SARS-Cov-1) infection dates back to 2002–2003. Middle East respiratory syndrome (MERS) grappled the world a decade after in 2013–2014. Yet again, the world is witnessing emergence of SARS associated with novel coronavirus (SARS-Cov-2).[1]

Identification of patients with potential exposure or symptoms can facilitate prompt management. Vital signs – body temperature, respiratory rate, heart rate, and blood oxygen saturation levels – are recognized as critical powerful indicators in respiratory illnesses.[2] Their monitoring forms an integral part in detecting early physiological changes in deteriorating patients. They have an imminent role in triaging the patient to appropriate care and predicting recovery or deterioration.[3,4]

About 82% of SARS-Cov-2 patients exhibit mild symptoms, recover immediately, and do not require hospitalization. Among the patients who require hospitalization, 10%–20% require care in intensive care units (ICUs), 3%–10% require intubation, and case fatality rate varies between 2% and 5%.[5] In patients having mild disease, treatment is mostly symptomatic along with home isolation.[6] The incubation period (time from infection to the onset of symptoms) for COVID-19 varies from 2 to 14 days with the median 5–6 days.[7] There is a need to monitor vital signs in patients stratified for home quarantine.

Wearable remote health monitoring solutions have introduced the possibility of regular monitoring of vital parameters in the outpatient setting. Advances in sensor technology and artificial intelligence coupled with medical conversance have led to a revolutionary development.[4] Recent times have witnessed commercial launches of such devices without clinical validation studies rendering them unsuitable for medical use.[8]

The present study was intended to compare the accuracy of a wearable remote health monitoring device for estimation of SpO₂, respiratory rate, body temperature, and pulse rate.

MATERIALS AND METHODS

The Institutional Research and Ethical Committee approved this validation study that included patients who attended the outpatient department of the institute in April 2020 irrespective of their diagnosis. Individuals with significant deformity, degenerative changes or edema of hand and wrist, localized infection, ulceration or skin breaks involving the wrist, and vascular diseases along with those who refused consent were excluded from the study. Written informed consent before the enrollment was obtained from all of the participants. Participant information sheet in the local language (Hindi) was given and read out to each study participant before their recruitment.

The wearable remote health monitoring system included in the study is manufactured by Electronics Corporation of India Limited (ECIL), a public sector undertaking company, under the Department of Atomic Energy, Government of India, with its unit based at Hyderabad, India. It is designed for measuring SpO₂, respiration rate, pulse rate, and body temperature by noninvasive methods. A semiconductor thermocouple type sensor was used for body temperature measurement. The optical-based approach consisting of a pair of light-emitting diodes – one for infrared (IR) and one for red visible light – and a single photo sensor was used for measurement of SpO₂ and pulse rate. The optical signals were converted to a photoplethysmography (PPG) current, digitized, and then processed by complex and sophisticated algorithms to provide SpO₂ readings in real time. The device had an integrated digital signal processor for faster computation and accurate measurement.

All the recruited patients attending the outpatient department were made comfortable. Wired sensor devices connected to the vital monitoring system were placed on the patient. SpO₂ sensor was applied to the index finger of the left hand and temperature sensor in the left axilla, and the respiratory rate was obtained by piezoresistive sensors placed on the bare chest of the patient. The wrist device made by ECIL, Hyderabad, was placed on the ventral surface of the left hand, 10 cm above the wrist with SpO₂ finger sensor connected to the tip of the middle finger of the left hand.

The obtained values by device under validation was compared with Philips Intellivue Mx550 (Philips Healthcare Canada, Saint-Laurent, QC, Canada) with embedded Masimo sensor technology (Masimo Canada ULC, St Laurent, QC, Canada) for pulse oximetry, endoluminal temperature monitoring, and chest-based piezoresistive respiratory rate monitoring.

All the sensors were placed at predefined anatomical sites by trained medical staff. Data regarding age, gender, and vital parameters were simultaneously entered into datasheet by another researcher. Values of body temperature, pulse rate, respiratory rate, and SpO₂ were documented using both the devices after 20 min of sensor placement as recommended by the manufacturer. Instruments were calibrated before, during, and at the end of the study. Appropriate controls were used for calibration purpose.

Figure 1 shows the system architecture demonstrating flow of information from acquisition to transmission at the command and control center for the discussed solution.

Figure 1.

Evaluated system architecture: Design as submitted by Electronics Corporation of India Limited, Hyderabad

Statistical analysis

The data were entered in MS Excel and analyzed in IBM SPSS Statistics for Windows, version 21 (IBM Corp., Armonk, N.Y., USA) statistical software. Continuous variables were expressed as mean ± standard deviation (SD) and categorical data in frequency and proportion. Intraclass correlation coefficient was calculated. The agreement for each vital parameter versus observation of reference equipment was analyzed by Bland–Altman graphs. On respective graphical representations, three reference lines were drawn for upper limit, lower limit, and mean values. P < 0.05 was considered as statistically significant.

RESULTS

In the present diagnostic validation study, a total of 218 participants formed the part of the study group. Eighteen participants who fall in exclusion criteria were subsequently excluded. The demographical data are represented in Table 1. There were 76% (n = 152) of males and 24% (n = 48) females in an age group of 2–80 years, with a mean ± SD of 34.74 ± 14.96. The mean SpO₂, temperature, respiratory rate, and pulse rate were 96.96 ± 2.09, 97.67 ± 0.79, 97.67 ± 0.79, and 80.87 ± 10.7, respectively.

Table 1.

Intraclass correlation coefficient between the observations of two devices for all the four vital parameters (n=200)

Vital parameter	Intraclass correlation	P
SpO2 (%)	0.92	<0.01
Temperature (°F)	0.95	<0.01
Respiration rate per minute	0.97	<0.01
Pulse rate per minute	0.99	<0.01

Intraclass correlation analysis

Table 1 represents the intraclass correlation between the observed four parameters from the two devices. A strong intraclass correlation in a range of 0.92–0.99 was noted for all vital parameters among both the devices. P value was significant for all the observations.

Bland–Altman comparison between the two devices for agreement

Table 2 shows the mean values for SpO₂, temperature, respiratory rate, and pulse rate from both the devices. Compared with standard equipment, the mean difference was found to be − 0.12, 0.25, 0.49, and 0.27, respectively. The values thus noted from the device under study compared to standard patient monitoring system were found in agreement within the prediction interval, as shown in graphical representation in Figure 2.

Table 2.

Bald-Altman comparison among the participants for all the four parameters

Parameter	Device	Mean±SD	Bald-Altman comparison
			MD	95% CI
SpO2 (%)	Philips Intellivue MX 500	96.96±2.09	Reference
	ECIL device	97.09±2.27	−0.12	−2.36-2.11
Temperature (°F)	Philips Intellivue MX 500	97.67±0.79	Reference
	ECIL device	97.42±0.80	0.25	−0.25-0.75
Respiratory rate per minute	Philips Intellivue MX 500	18.44±4.05	Reference
	ECIL device	17.95±3.93	0.49	−2.05-3.04
Pulse rate per minute	Philips Intellivue MX 500	80.87±10.7	Reference
	ECIL device	80.60±10.61	0.27	−2.69-3.32

ECIL: Electronics Corporation of India Limited, SD: Standard deviation, CI: Confidence interval, MD: Mean difference

Figure 2.

Bland–Altman comparison between two devices for SpO₂(a), temperature (b), respiratory rate (c), and pulse rate (d). X-axis is the mean value for each parameter. Y-axis is the difference in the means from both reference and device under study. Blue line is the mean bias and red dashed lines show 95% limits of agreement

DISCUSSION

Remote health monitoring solutions have seen revolutionary development, owing to improved sensor technology and miniaturized processors.[8] A few studies have discussed features of ideal wearable device systems. The device should be installed once for continuous use and should have very low battery consumption, and there should be no constraints during data transmission and no effect of ambient atmospheric conditions such as temperature, humidity, and motion.[9,10] The present device has a wrist wearable unit, which is comfortable to wear; it is powered by a lithium-ion battery that supports charging by a regular universal serial bus port and can be used for up to 3 days continuously.

Pulse oximetry is considered as an established measurement for assessment of respiratory system. It involves illumination of a small part of human skin (earlobe, fingertip, or toes) at two wavelengths: 660 nm (red) and 940 nm (IR). Absorbed light is then measured depending on the levels of oxy- and deoxyhemoglobin. Both transmission and reflective oximetry approaches described in literature have produced comparable results. Motion artifacts and low perfusion values have always challenged the technique of pulse oximetry; however, owing to improvement in signal analysis and reflectance technology, results have been accurate and reliable.[11] In agreement to the available literature, our results were consistent and strong intraclass correlation (r = 0.92) was noted with values in agreement on Bland–Altman analysis.

In conjunction with blood oxygen saturation, measurement of respiratory rate is noted to predict patient deterioration more effectively. In the opinion of few authors, changes in respiratory rate precede changes reflected by pulse oximetry.[12] Currently, manual assessment of respiratory rate at bedside in the inpatient department is the standard of care. However, this method suffers from limitations due to its intermittent nature dependent on the availability and proficiency of medical staff.[13] Various techniques have been described for remotely measuring respiratory rate. The use of accelerometers and magnetometers to measure linear displacement of the chest has been described previously with variable results.[14,15] Recent development in this technique is derivation of respiratory rate through PPG signals combined with newer algorithms, frequency, and amplitude modulation during consecutive heartbeats. PPG-based technique has alleviated the need to wear chest-based hardware and facilitated the measurement of heart rate, blood oxygen saturation, and respiratory rate using a single wearable device.[16] Our experience with the technology also revealed strong intraclass correlation and strong agreement on Bland–Altman analysis.

Body temperature monitoring through remote wearable health monitoring device is by far the most challenging and still evolving technology. Limited success has been achieved using newer sensors, infrared technology, and advanced algorithms. Despite advances, the outcome is limited by temperature variation in different body parts and effect of ambient atmosphere on the sensor technology.[17] A recent study has validated the accuracy of skin temperature obtained from wrist level comparing it with the tympanic membrane which owing to proximity to temporal artery is equivalent to core body temperature.[18] Our results did reveal strong intraclass correlation and observations in agreement with the available literature.

Apart from the technical limitations, two major issues that have drawn attention of many authors include data security and application of various devices in the absence of clinical validation studies. In the absence of stringent regulations, data ownership becomes unclear. An extremely low number of wearable devices having CE, FDA, or ISO certification for medical use are available in the market rendering a useful technology unreliable. To mitigate this problem, many authors have emphasized the role of validation study before its commercial exploitation.[8]

Many direct and indirect costs are involved when a patient is hospitalized including but not limited to bed occupancy, logistics including personal protection equipment (PPE), and time constraints among already overburdened doctors and health-care workers. The cost of stay per day in a non-ICU setting for a patient of MERS coronavirus and PPE per piece according to a study was USD 173.3 and USD 5.66, respectively. This figure does not include health-care worker fees, cost of investigations, medicines, and ICU stay.[19] For a period of 14 days, the cost comes to the tune of approximately USD 2520 which is equivalent to approximately INR 176,400. This device at a price of less than one-tenth will be a cost-effective model in view of its nonconsumable, reusable character that monitors the vital parameters for entire quarantine period of 14 days, and it comes after a one-time investment to the hospital. The per-day cost of application of this solution will be approximately INR 1000 only. The application of this model will further enable early discharge of the patient to monitored home care. This will free up the hospital beds for more needy patients in queue.

CONCLUSION

Deterioration or recovery of physiological parameters in a patient can be reliably made with the use of studied wearable remote health monitoring system. Its regulated implementation can reduce burden on existing health-care infrastructure and appropriately mitigate the scarcity of hospital beds. In monitoring of patients having contagious infectious diseases, these solutions can reduce avoidable exposure to health-care workers and decrease the demand of PPE and other logistics. In developing nations with skewed doctor-to-patient ratio, such solutions from technology are expected to play a pivotal role in health delivery for all.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.