Abstract
Background
Breath analyser tests are used worldwide to obtain proof of alcohol intoxication and often used in the conviction of traffic violators. These tests are conducted to quickly and painlessly determine the existing concentration of alcohol in arterial blood by measuring the amount of ethanol in exhaled breath, which can be identified with an electrochemical sensor.
At present, the calibration and maintenance of analysers used for these tests are typically performed regularly but lack quality control. Consequently, test results may not be accurate because of calibration deterioration.
The aim of this study was to develop and evaluate the uncertainty of control materials used in breath-alcohol testing at the Bangkok Metropolitan Police Station.
Material and methods
Ethyl alcohol (99.99%; Certified Reference Material grade) diluted at three different concentrations was kept under design conditions. The concentrations were 28, 67, and 134 mg/dL, determined by performing headspace gas chromatography, and the uncertainty was set as ±1.3925, ±2.8736, and ±1.8231 mg/dL (±4.97%, ±4.29%, and ±2.72% for the concentrations, respectively), as per ISO Guide 35:2017.
Results
The total error percentages of the developed control materials were 4.97%, 4.29%, and 2.72% for concentrations of 28, 67, and 134 mg/dL, respectively. Each concentration of the materials was tested by using measurements from 70 breath-alcohol analysers belonging to the Bangkok Metropolitan Police Station.
Conclusion
These control materials are applicable to quality assurance and standards tests and may help to ensure the accuracy of breath-alcohol testing in the future.
Key words: breath alcohol testing, control material, uncertainty
1. INTRODUCTION
According to the World Health Organization’s ‘Global Status Report on Road Safety’, road accidents cause approximately 1.35 million fatalities per year, making it the 8th leading cause of mortality for all ages [1]. Driving under the influence of alcohol (DUI) is an important contributing factor in these accidents [2]. The amount of alcohol consumed is directly proportional to the risk and severity of accidents. In Thailand, despite legislation against DUI, the Ministry of Transport reported that out of 316 DUI accidents per year, 61 were fatal and 293 required medical attention [3],[4].
Additionally, Thai Law stipulates that the blood-alcohol level of drivers above the age of 20 years must not exceed 50 mg/dL. Drivers who are below this age, drivers with a temporary licence, licenced drivers who have received any other type of licence, and drivers whose licence has been revoked or whose application is on hold must have no more than 20 mg/dL alcohol concentration in their blood [5], as measured by police officers using an breath-alcohol analyser at the scene of the accident or when the driver is suspected of DUI.
A breath-alcohol analyser is a device that measures the alcohol in exhaled breath using colorimetric [6], semiconductor [7], or infrared absorption [8]. Detection methods requiring electrochemical-sensor-based devices are the most popular because of their portability, short analytical time, accuracy, good sensitivity, and adequate specificity [9],[10],[11]. However, these methods involve the risk of errors common to all medical laboratory equipment, such as insufficient biological sampling and traceability issues. These errors can be detected through quality control materials [12],[13], [14].
Because electrochemical-sensor-based breath analysers are used routinely by traffic police officers, these instruments are calibrated every six months by an external organisation. However, internal quality control is not regularly performed for this kind of routine task, creating doubt about the reliability of the results in the event of a lawsuit. Furthermore, calibration is expensive. The aim of this study was to develop control materials for breath-alcohol analysers. We evaluated the measurement uncertainty of our control materials according to ISO Guide 35:2017 [15] and compared it to the allowable total error (TEa) specified by CLIA2019 [16]. Finally, we applied the materials to the breath-alcohol analysers used in the Bangkok Metropolitan Police Station. These materials have the potential to make quality control more accessible to all police stations, thereby improving standardised and reliable results.
2. MATERIALS AND METHODS
2.1 Development of control material
Three concentrations of the control material—low, medium, and high—were prepared using 3.267, 8.171, and 16.340 mL of 99.99% ethyl alcohol (HPLC grade, DAEJUNG, Republic of Korea) with 10 L of distilled water each. The samples were mixed by applying inversion and divided into 20 plastic bottles containing 500 mL each. The bottles were sealed with parafilm, an aluminium sheet, and finally a plastic screw cap. All materials were stored at a temperature of 25 ±2 °C and a humidity of 50% ±5% for three months.
2.2 Uncertainty of measurement (MU)
The MU of our control materials was determined as per ISO Guide 35:2017 [15],[17] by using headspace gas chromatography (HSGC; SHIMAZUGC-2010, Japan). These experiments were performed in the toxicology laboratory of the Institute of Forensic Medicine, Police General Hospital, Bangkok, Thailand. The allowable total error; TEa outlined in the CLIA2019 criteria (20%) was used to determine performance characteristics and uncertainty values. Furthermore, the HSGC method was using as the appropriate choice for the measurement procedure.
2.2.1 Homogeneity studies
Standard uncertainties were assessed as bottle-to-bottle heterogeneity (SUbb) on day zero for each control substance concentration. The minimum number of units was then calculated. The bottles were sampled using a simple randomized strategy. Outlier and trend analyses were also conducted. The uncertainties between units were analysed using one-way ANOVA software for Excel.
2.2.2 Characterization study
The standard uncertainty owing to the characterisation study (SUchar) was assessed for each control material concentration. The average result was used as the assigned value for each concentration.
2.2.3 Stability study
Standard uncertainty resulting from long-term instability (SUlts) was assessed over a 3-month period with storage conditions of 25 ± 2°C and 50% ± 5% humidity, and no transportation conditions.
A classic stability study was also conducted. Two bottles of the control material were sampled at six time points: 0, 7, and 14 d, and 1, 2, and 3 months for each concentration. The resulting trends were analysed, and the SUlts at each concentration was evaluated using a t-test.
2.2.4 Expanded uncertainty
The expanded uncertainty (Ux) was calculated from SUbb, SUchar’ and SUIts with a 95% confidence interval (coverage factor k = 2). The equation is:
 |
2.3 Application in breath-alcohol analysers from Bangkok Metropolitan Police Station
Consent and questionnaire surveys were sent to Bangkok Metropolitan Police Stations.
Our control materials were tested with 70 electrochemical breath-alcohol analysers (SD-400 Touch, Lion, UK) by using a wet-bath simulator. Quality was evaluated by using |%BIAS| from the HSGC-assigned value to assess accuracy and %CV for precision, and the total error (TE) was calculated.
3. RESULTS
3.1 MU of developed control materials
The HSGC procedure was evaluated by comparing the repeatability standard deviation (Sr), the number of observations of each of the 10 aliquots (nal), and the target uncertainty (utrg), calculated using 20% TEa for each concentration. The results showed that the HSGC procedure produced good precision for all concentrations of the control material (Table 1).
Table 1.
Evaluation of HSGC procedure with 20% TEa (CLIA2019)
| Concentration | utrg | sr | sr/√(nal) (A) | utrg/3 (B) | Conclusion |
|---|---|---|---|---|---|
| Low | 5.60 | 0.87 | 0.02 | 1.87 | Good precision1 |
| Medium | 13.4 | 0.61 | 0.02 | 4.47 | |
| High | 26.8 | 1.31 | 0.03 | 8.93 |
1 criterion (A) < (B) conclusion indicates ‘good precision’
Abbreviations: utrg represents the target uncertainty; sr is the standard deviation; and nal denotes the unit for measurement.
3.1.1 Homogeneity study
The minimum number of control materials was 3, or 10% of the batch. This study chose 10 bottles: sample numbers 2, 3, 6, 7, 9, 11, 13, 16, 18, 20. The results showed alcohol concentrations of 28.00% (SD = 0.87%), 66.74% (SD = 0.61%), and 134.17% (SD = 1.31%), and are summarized in Table 2. None of the data showed trends or outliers (P > 0.05). The developed control materials were homogeneous—p = 0.3736, 0.9013, and 0.0672 for 28, 67, and 134 mg%, respectively—and SUbb reported 0.2810, 0.3402, and 0.9042 mg%, respectively. These data are shown in Table 3.
Table 2.
Alcohol concentration in control materials analysed by HSGC at day 1
| Low concentration (mg%) | Medium concentration (mg%) | High concentration (mg%) | |||||||
|---|---|---|---|---|---|---|---|---|---|
| Bottle no. | 1st | 2nd | average | 1st | 2nd | average | 1st | 2nd | average |
| 2 | 28.12 | 27.88 | 28.00 | 66.06 | 67.79 | 66.93 | 135.49 | 134.43 | 134.96 |
| 3 | 27.47 | 28.05 | 27.76 | 67.05 | 67.42 | 67.24 | 133.59 | 134.49 | 134.04 |
| 6 | 27.81 | 28.07 | 27.94 | 65.75 | 66.29 | 66.02 | 133.03 | 133.92 | 133.48 |
| 7 | 27.91 | 28.22 | 28.07 | 66.25 | 66.70 | 66.48 | 133.26 | 134.85 | 134.06 |
| 9 | 27.74 | 28.74 | 28.24 | 66.47 | 66.80 | 66.64 | 133.37 | 135.23 | 134.30 |
| 11 | 28.14 | 30.53 | 29.34 | 67.71 | 66.19 | 66.95 | 132.37 | 135.06 | 133.72 |
| 13 | 27.97 | 28.98 | 28.48 | 66.72 | 66.91 | 66.82 | 134.02 | 135.05 | 134.54 |
| 16 | 27.20 | 27.23 | 27.22 | 65.72 | 67.57 | 66.65 | 133.89 | 134.82 | 134.36 |
| 18 | 26.87 | 29.20 | 28.04 | 67.29 | 66.40 | 66.85 | 134.20 | 133.16 | 133.68 |
| 20 | 27.01 | 26.94 | 26.98 | 66.72 | 66.96 | 66.84 | 134.64 | 134.50 | 134.57 |
| Average | 28.00 | 66.74 | 134.17 | ||||||
| SD | 0.87 | 0.61 | 1.31 | ||||||
| Assign Value | 28 | 67 | 134 | ||||||
Table 3.
Standard uncertainty from inhomogeneity (SUbb)
| Concentration | Source of variation | SS | df | MS | F | p-value | SUbb (mg%) | Ubb of conc. (%) |
|---|---|---|---|---|---|---|---|---|
| Low | Between groups | 7.5976 | 9 | 0.8442 | 1.2301 | 0.3736 | ±0.2810 | 1.00 |
| Within groups | 6.8623 | 10 | 0.6862 | |||||
| Total | 14.4599 | 19 | (ubb) | |||||
| Medium | Between groups | 1.9173 | 9 | 0.2130 | 0.4416 | 0.9013 | ±0.3402 | 0.51 |
| Within groups | 5.1758 | 10 | 0.5176 | |||||
| Total | 7.0931 | 19 | (u’bb) | |||||
| High | Between groups | 23.2574 | 9 | 2.5842 | 2.7235 | 0.0672 | ±0.9042 | 0.67 |
| Within groups | 9.4883 | 10 | 0.9488 | |||||
| Total | 32.7457 | 19 | (ubb) |
Abbreviations: SS represents the sum of squares; df denotes the degrees of freedom; MS indicates the mean squares; F is the F ratio; SUbb is the standard uncertainty due to inhomogeneity; Ubb is the uncertainty due to inhomogeneity.
3.1.2 Characterization study
The SUchar of the control materials was evaluated without the unweighted mean or laboratory uncertainties, by referring to the SDM results with assigned values of 28, 67, and 134 mg% for low, medium, and high concentrations, respectively (Table 2). The SUchar showed a minimum of 67 mg% (SUchar = ±0.1032 mg%) and a maximum of 134 mg% (SUchar = ±0.3595 mg%), as outlined in Table 4.
Table 4.
Standard uncertainty due to characterization (SUchar)
| Concentration | Average (mg%) | SD (mg%) | SDM (mg%) | SUchar (mg%) | Uchar of conc. (%) |
|---|---|---|---|---|---|
| Low | 28.00 | 0.65 | 0.2054 | 0.2054 | 0.73 |
| Medium | 66.74 | 0.33 | 0.1032 | 0.1032 | 0.15 |
| High | 134.17 | 1.14 | 0.3595 | 0.3595 | 0.27 |
Abbreviations: SD is standard deviation; SDM is the standard deviation mean; and SUchar is the standard uncertainty due to characterisation; Uchai, is the uncertainty due to characterisation.
3.1.3 Stability study
Storage-controlled materials remained acceptable with no significant change for any concentration over 3 months (p > 0.05). The minimum and maximum SUIts values were 28 mg% (SUIts = 0.6030 mg%) and 134 mg% (SUIts = 1.5417 mg%) (Table 5), respectively.
Table 5.
Standard uncertainty due to long-term instability (SUits)
| Duration | Concentration (mg%) | ||
|---|---|---|---|
| Low | Medium | High | |
| 0 day | 28.00 | 66.74 | 134.17 |
| 7 days | 28.14 | 68.42 | 135.40 |
| 14 days | 28.15 | 66.58 | 133.07 |
| 1 month | 27.39 | 64.95 | 131.87 |
| 2 months | 27.47 | 65.73 | 132.94 |
| 3 months | 28.68 | 66.47 | 134.00 |
| SUIts(mg%) | 0.6030 | 1.3921 | 1.5417 |
| UIts of conc. (%) | 2.15 | 2.08 | 1.15 |
Abbreviations: SUIts is the standard uncertainty due to long-term instability; UIts is the uncertainty due to long-term instability.
3.1.4 Expanded uncertainty
The Ux of developed control materials were calculated with a 95% CI (coverage factor = 2). Results showed that Ux = 1.3925, 2.8736, and 1.8231 mg% for concentrations of 28, 67, and 134 mg%, respectively (Table 6).
Table 6.
Expended uncertainty of developed control materials (Ux).
| Source of SU | Concentration (mg%) | ||
|---|---|---|---|
| Low | Medium | High | |
| SUbb | 0.2810 | 0.3402 | 0.9042 |
| SUchar | 0.2054 | 0.1032 | 0.3595 |
| SUIts | 0.6030 | 1.3921 | 1.5417 |
| Combined uncertainty (Uc) | 0.6962 | 1.4368 | 1.8231 |
| Ux (mg%) | 1.3925 | 2.8736 | 1.8231 |
| Ux of conc. (%) | 4.97 | 4.29 | 2.72 |
Abbreviations: SUbb is the standard uncertainty due to inhomogeneity; SUchar is the standard uncertainty due to characterisation; SUIts is the standard uncertainty due to long-term instability; Uc is the combined uncertainty; Ux is the expended uncertainty.
3.2 Application in breath-alcohol analysers from Bangkok Metropolitan Police Station
The Bangkok Metropolitan Police Station routinely uses breath-alcohol analysers. Our control materials were tested on 70 instruments which were grouped according to the time after the latest calibration: < 2 months (1), 2–4 months (2), and > 4 months (3) (Table 7). The results from applying the developed control materials showed precision and %CV which were minimum for group (1)—67 mg% (CV = 2.90%)—and maximum for group (3)—28 mg% (CV = 14.24%)—illustrated in Figure 1(a). The accuracy is shown as |%BIAS| which was also at its minimum in group (1) at 28 mg% (| BIAS | = 4.23%) and at its maximum in group (3), 28 mg% (| BIAS | =12.70%). This is summarized in Figure 1(b). The TE was also calculated for each analyser; minimum TE was found in group (1) at 134 mg% (TE=8.60%) and the maximum was found in group (3) at28 mg% (TE=26.94%), outlined in Figure 1(c). Notably, the calculated TE showed that only the SD-400Touch instruments in groups (1) and (2) met the 20% CLIA2019 TEa standard.
Table 7.
Breath-alcohol analysers grouped according to time after latest calibration. This structuring is applied to the data in Figure 1
| Model | Time after latest calibration/analyser (n) | sum | ||
|---|---|---|---|---|
| < 2 months (1) | 2-4 months (2) | > 4 months (3) | ||
| SD-400Touch | 30 | 25 | 15 | 70 |
Figure 1.
Results from the application of control materials to breath-alcohol analysers, grouped according to time after latest calibration: (a) %CV in each group and concentration; (b) |%BIAS| in each group and concentration; (c) %TE in each group and concentration
4. DISCUSSION
Quality control plays an important role in the reporting of reliable results from medical laboratories. For this reason, quality control is critical for preventing inaccuracy or imprecision in the results of all tests—even breath-alcohol testing performed by police officers on motorists at sideroads. Ensuring that breath-alcohol analysers report values correctly is challenging. This is because the reference materials are expensive, and calibration is scheduled in six month intervals.
According to ISO/IEC 17043:2010, the ‘general requirements for proficiency testing’ describe qualifications in procedures and reference materials used for determining result quality [15], [17], [19]. The MU of the reference materials was determined as per ISO Guide 35:2017 specifying three causes of uncertainty: inhomogeneity, characterisation, and long-term instability, which may be evaluated to expand the uncertainty [15],[17].
In this study, the reference material was used as a control to determine the quality status of each breath-alcohol analyser. We developed the control material by diluting ethyl alcohol 99.99% (Certified Reference Material grade) in deionised water in three concentrations and then sealing the samples with parafilm, aluminium sheet, plastic screw caps and placing them in storage at 25±2 °C and 50 ±5% humidity for 3 months. This accessible procedure could substitute for the more expensive reference materials currently in use. The %TE for each concentration was found to be 4.97%, 4.29% and 2.72% in control material concentrations of 28, 67, and 134 mg% respectively. Our study did not differ from other studies in which TE = 4.72%, 4.72%, and 4.27% was found in alcohol reference material concentrations of 46.6, 50.8, and 56.3 mg% [20].
The results of applying the developed control materials in 70 police-issued breath-alcohol analysers revealed the TE to be acceptable only when the most recent calibration was performed less than four months ago, according to the 20% TEa standard outlined in CLIA2019. The further the instruments are removed from their latest calibration date, past the four-month mark, increased the imprecision of the analysers. Additionally, multiple confounding factors were found, including the service life of each instrument and the experience of the users.
A limitation of this study is that our control materials were applied only in the Bangkok area and must still be tested for commutability. The confounding factors (service life, user competency) also remain unexplored.
5. CONCLUSIONS
Our control materials were developed by employing a designed protocol and storage conditions that met the criteria of CLIA2019. We determined the TEa limit as per the specifications of ISO Guide 35:2017.
These materials could be used to routinely evaluate the quality of breath-alcohol analysers for more reliable results.
Acknowledgments
We thank the Bangkok Metropolitan Police Station for allowing us to use the data and Breath-alcohol analyzer for this research, and Language advisor from Faculty of allied Health Sciences for improving the use of English in the manuscript.
Funding for open access charge: University of Thammasat University/Thailand.
Footnotes
Funding
This research has received financial support from the Thammasat University Research Unit in Developing of Quality Control Materials for Medical Laboratory.
Credit authorship contribution statement
Narisa Kengtrong Bordeerat: Conceptualization, Data curation, Methodology, Resources, Formal analysis, Investigation.
Krittin Chumsawat: Investigation, Visualization, Data curation, Methodology, Resources, Formal analysis Writing – original draft.
Somsak Fongsupa: Conceptualization, Formal analysis, Methodology.
Sudawadee Kongkhum: Visualization, Writing – review and editing.
Pramote Sriwanitchrak: Writing – review and editing.
Authors’ disclosures declarations
No known conflicts of interest are associated with this publication.
REFERENCES
- 1.Organization, W.H., Global status report on alcohol and health 2018. 2019: World Health Organization. [Google Scholar]
- 2.Fell J.C., Voas R.B., The effectiveness of reducing illegal blood alcohol concentration (BAC) limits for driving: evidence for lowering the limit to. 05 BAC. Journal of safety research, 2006. 37(3): p. 233-243. [DOI] [PubMed] [Google Scholar]
- 3.Accidental on road data. Office of the Permanent Secretary 2022; Available from: https://datagov.mot.go.th/dataset/roadaccident/resource/960813aa-f410-4356-bd5a-10f63c8e4ce0?inner_span=True.
- 4.Royal Gazette, The Land Traffic Act Update. 1979; Available from: http://www.thailawforum.com/land-traffic-act/
- 5.Ministerial regulations. The Land Traffic Act Update. 2017. [Google Scholar]
- 6.Jung Y., et al. , Smartphone-based colorimetric analysis for detection of saliva alcohol concentration. Applied optics, 2015. 54(31): p. 9183-9189. [DOI] [PubMed] [Google Scholar]
- 7.Wang X., Yee S., Carey P., An integrated array of multiple thin-film metal oxide sensors for quantification of individual components in organic vapor mixtures. Sensors and Actuators B: Chemical, 1993. 13(1-3): p. 458-461. [Google Scholar]
- 8.Pérez-Ponce A., Garrigues S., de La Guardia M. Vapour generation–fourier transform infrared direct determination of ethanol in alcoholic beverages. Analyst, 1996. 121(7): p. 923-928. [Google Scholar]
- 9.Modjtahedi A., Amirfazli A., Farhad S., Low catalyst loaded ethanol gas fuel cell sensor. Sensors and Actuators B: Chemical, 2016. 234: p. 70-79. [Google Scholar]
- 10.Rahman M.R., et al. , The application of power-generating fuel cell electrode materials and monitoring methods to breath alcohol sensors. Sensors and Actuators B: Chemical, 2016. 228: p. 448-457. [Google Scholar]
- 11.Allan J.T., Geoffrey H.L., Easton E.B., The effect of the gas diffusion layer on the performance of fuel cell catalyst layers in ethanol sensors. Sensors and Actuators B: Chemical, 2018. 254: p. 120-132. [Google Scholar]
- 12.Gullberg R.G., Estimating the measurement uncertainty in forensic breath-alcohol analysis. Accreditation and quality assurance, 2006.11(11): p. 562-568. [Google Scholar]
- 13.Dubowski K.M., Quality assurance in breath-alcohol analysis. Journal of analytical toxicology, 1994. 18(6): p. 306-311. [DOI] [PubMed] [Google Scholar]
- 14.Gullberg R., Methodology and quality assurance in forensic breath alcohol analysis. Forensic science review, 2000.12(1-2): p. 49-68. [PubMed] [Google Scholar]
- 15.Guide I., 35 (2017) Reference materials—guidance for characterization and assessment of homogeneity and stability. International Organization for Standardization (ISO), Geneva, 2017. 114. [Google Scholar]
- 16.Westgard J.O. 2019. : CLIA Proposed Changes to PT Acceptable Limits. Available from: https://www.west-gard.com/2019-clia-changes.htm.
- 17.Botha A, et al. Outline for the revision of ISO Guide 35. Accreditation and Quality Assurance, 2013. 18(2): p. 115-118. [Google Scholar]
- 18.Bartels R, et al. , A Perspective on a Quality Management System for AI/ML-Based Clinical Decision Support in Hospital Care. Frontiers in Digital Health, 2022: p. 119. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Trapmann S, et al. , The new International Standard ISO 17034: general requirements for the competence of reference material producers. Accreditation and Quality Assurance, 2017. 22(6): p. 381-387. [Google Scholar]
- 20.Bunsoong J, Bunsoong T., Preparation of standard alcohol for quality control testing in alcohol breath testing. BULLETIN OF THE DEPARTMENT OF MEDICAL SCIENCES, 2020. 62(2): p. 96-105. [Google Scholar]