Machines Learning to Detect Obstructive Sleep Apnea in Children. Are We There Yet?

Obstructive sleep apnea (OSA) is common in children, estimated to occur in 1.2–5.7% (1), and is known to be associated with adverse neurocognitive, behavioral, and health-related consequences (2–4). Considering these adverse effects, OSA screening is recommended by the American Academy of Pediatrics, with polysomnography recommended for children who snore and have at least one other symptom that may be related to OSA (1). Unfortunately, pediatric polysomnography is not readily available in many areas, including underserved communities, and is further compounded by a paucity of board-certified pediatric specialists in sleep medicine (5). Considering such issues with access to pediatric sleep care, there is a need to develop alternative screening tools involving readily available diagnostic testing to detect OSA in children. Prior screening tools have included questionnaires (6, 7), biomarkers (8, 9), and varied approaches to nocturnal pulse oximetry (10, 11). Promising biomarkers have been identified, but have not yet been validated in clinical practice (8, 9). Questionnaires have shown a wide range of sensitivity and specificity (6, 7), and similarly, attempts to use nocturnal pulse oximetry have had limited success (12).

In this issue of the Journal, Hornero and colleagues (pp. 1591–1598) take a new approach to an old screening tool (13). Specifically, they used an innovative approach of machine learning to generate a neural network algorithm to detect OSA from overnight pulse oximetry tracings. Most prior attempts involved manual interpretation of nocturnal pulse oximetry as a screening tool for OSA (11). Such a machine learning approach enabled the authors to evaluate thousands of potential algorithms to optimize a screening algorithm. There are other strengths to the approach undertaken by Hornero and colleagues, including an impressive number of patients (n = 4191) from 13 countries with varied pulse oximetry acquisition techniques. Such an approach increases the generalizability of the findings and should be commended (13).

The algorithm developed by the authors showed improved test characteristics compared with earlier manual techniques for predicting the presence of OSA. At various apnea–hypopnea index (AHI) thresholds, sensitivity ranged from 68% to 84%, and specificity ranged from 53% to 94%. Similar to prior screening tools, test performance was best for predicting the presence of moderate to severe OSA.

The real-world application of any new diagnostic tool is vital for improved patient and process outcomes through successful implementation and uptake into practice. On the basis of their results, the authors suggest a diagnostic protocol for applying this tool in clinical practice. Children with an estimated AHI lower than 1 based on overnight pulse oximetry are unlikely to have moderate or severe sleep apnea and may not require polysomnography. Children with a machine-estimated AHI of 5 or more per hour are likely to have at least mild OSA by polysomnography (AHI ≥1), and could be referred to treatment directly without undergoing polysomnography. Children with a machine-estimated AHI of 1–5 per hour would be considered indeterminate and can be referred for polysomnography. Based on the author’s findings, approximately 1 in 20 children would be incorrectly diagnosed with OSA and potentially treated unnecessarily. In adults, the treatment would most likely involve positive airway pressure therapy that poses little to no harm in the presence of a false-positive test. However, adenotonsillectomy is the most common treatment for OSA in children. This would mean that there is a greater risk associated with a false-positive test considering that 1 in 20 children could potentially be exposed to the risk for surgical complications unnecessarily. Needless to say, the real-world implementation of a clinical management pathway that incorporates a machine learning tool needs to be tested for potential benefits versus harms in implementation trials (14).

The machine learning tool could also miss the diagnosis of OSA in approximately 1 in 20 children with a machine-estimated AHI <1 per hour. Hornero and colleagues correctly recognized that sensitivity could potentially be improved by clinical correlation (13). Specifically, children who report significant symptoms of OSA that screen negative may benefit from being referred for polysomnogram, whereas children with minimal symptoms and a negative pulse oximetry would likely not need further evaluation. For example, a child who snores and falls asleep daily during school should be referred for polysomnogram even with an estimated AHI lower than 1 on nocturnal oximetry. Meanwhile, a snoring child with tonsillar hypertrophy without other significant symptoms of OSA (daytime sleepiness, behavior problems, etc.) would not need a polysomnogram if nocturnal pulse oximetry estimated an AHI lower than 1 per hour. The machine learning screening algorithm and proposed clinical protocol could conceivably be integrated into pulse oximetry software. Oximetry reports could then not only provide data such as an oxygen desaturation index but also suggest further evaluation and management for OSA. This could facilitate better detection of previously undiagnosed OSA by reducing barriers to testing.

Future work is needed to validate the authors’ findings on home-based oximetry considering that the current study used oximetry recorded during the course of a technician-attended polysomnography. Such differences in the setting of implementation of novel tools or approaches are vital for the success of such innovation. Such differences in the setting of data collection have been considered a limitation of other limited channel approaches for detecting OSA (15), but lends itself nicely to be tested by hybrid clinical-effectiveness/implementation trials (14, 16).

Machine learning–based algorithms that use only information readily available from the clinical record have been shown to be useful screening tools for OSA in adults (17). In the current study, machine learning enhanced the test performance of isolated nocturnal pulse oximetry, but the authors did not evaluate a combined approach, using oximetry results in combination with clinical data, questionnaires, or urinary biomarkers. A prior attempt to combine nocturnal pulse oximetry with clinical data has had limited success (12), but it is possible that the use of a combinatorial approach of machine learning with clinical information could be even more successful and is of particular importance in mild OSA.

The use of machine learning and predictive analytics across various platforms is vital for implementation efforts aimed at bringing billions of dollars of research findings to the bedside (14). Our eagerness to implement these modern tools needs to be balanced with a careful understanding of the setting and real-world circumstances of the implementation efforts. Hornero and colleagues have taken an important step toward automated detection of OSA in children (13). The real-world implementation of the clinical prediction pathway incorporating the machine learning technique proposed by Hornero and colleagues awaits further testing (13).