An Automated Scoring of Clinical Asthma Score: Proof of Concept and the Future Possibility

To the Editor:

The use of high-intensity clinical data from the electronic medical record (EMR) has been getting increasingly popular, especially in intensive care medicine, where a large amount of data are continuously generated as the changing condition of patients is tracked by the minute (1, 2). However, the analysis of those data to explore a clinical causal relationship or the impact of an intervention can be particularly misleading when the severity of illness is not rigorously examined and controlled (3, 4).

The modified Wood’s Clinical Asthma Score (mWCAS), generated with oxygen saturation level, presence of expiratory wheezing, inspiratory breath sounds, use of accessory muscles, and cerebral function/status (Table 1), is used to assess the severity of respiratory distress in children with bronchiolitis (5). An mWCAS greater than 3 is an indication of moderate-to-severe respiratory distress (6). In this study, we aimed to demonstrate a proof of concept of an algorithm to generate an automated mWCAS (A-mWCAS) for critically ill children and to validate it with manually computed mWCAS (M-mWCAS).

TABLE 1.

Modified Wood’s Clinical Asthma Score Five-Item Scoring

Item	0	0.5	1	2
Spo2	Spo2 ≥ 95% in room air	95% > Spo2 > 90% in room air	Spo2 ≥ 90% with Fio2 > 21%	Spo2 < 90% with Fio2 > 21%
Expiratory wheezing	None	Mild (+)	Moderate (++)	Marked (+++)
Inspiratory breath sounds	Normal	Slightly decreased	Decreased	Absent
Use of accessories muscles	None	Mild (+)	Moderate (++)	Maximal (+++)
Cerebral status	Normal	Agitated when disturbed	Depressed/agitated	Markedly depressed/coma

This retrospective study included all infants under 2 years old with a clinical diagnosis of bronchiolitis, ventilated with noninvasive neurally adjusted ventilatory assist (NIV-NAVA), in a Canadian tertiary PICU, between October 2016 and June 2018. The study was approved by the ethics committee of the Sainte-Justine Hospital. The algorithm, written with Python 3.7, was directly connected to the EMR (IntelliSpace Critical Care and Anesthesia, Version F.01; Philips, Eindhoven, The Netherlands). The first step of the developed algorithm consisted of the automatic extraction of the five main required items, as well as the Fio₂, from the EMRs of the PICU, using structured query language queries. Then, each item was given a score based on the rules listed in Table 1. This was possible because all the items were explicitly represented in the EMR in drop-down menus, except for the cerebral status. In the EMR, we have 18 different cerebral statuses in a drop-down menu. These statuses were grouped into four categories, based on Table 1, by a group of pediatric intensivists. After that, the attributed scores were added together to obtain the A-mWCAS. M-mWCAS was computed retrospectively by a clinical expert who reviewed the patients’ medical charts and manually extracted the score’s items in a completely independent and blind way from the automatic algorithm. Both the A-mWCAS and the M-mWCAS were therefore based on data collected by multiple clinicians (respiratory therapists or nurses), but a single “rater” (one machine vs one clinical expert) was involved in each strategy.

We included 64 infants with a median (25–75th percentile) age of 52 months (32–92 mo). Manual and automatic scores were calculated for all the included patients at eight different times from 2 hours before to 24 hours after NIV-NAVA application. Scores were calculated only when all the required score elements were available. The median number of scores assessment per patient was 4 (3–5). Overall, 256 pairs of A-mWCAS and M-mWCAS were generated. Cohen kappa coefficients were applied to estimate the agreements between the two scores. The overall kappa score was 0.71 (95% CI, 0.64–0.77), in which we observed exact agreement for the 78.5% pairs and agreement with a difference in score of less than or equal to 0.5 in 96% (Fig. 1). The Pearson correlation coefficient was R² = 0.90. A sensitivity analysis was conducted in a subgroup of 57 patients with the same number of three ratings (total of 171 scores), with similar results: kappa score of 0.71, exact agreement in 75.4% pairs, and 95% of pairs with a difference less than 0.5.

Figure 1.

Agreement table showing the actual agreement between the 256 automated modified Wood’s Clinical Asthma Score (mWCAS) and manually computed mWCAS.

We also examined the agreement for each score component, in which we found a very good agreement in the oxygen saturation (kappa score 0.91), the expiratory wheeze (kappa score 0.79), the inspiratory breath sounds (kappa score 0.91), and the use of accessories muscles (kappa score 0.89). However, we observed a poor agreement in the cerebral status with a kappa score of 0.51. We assumed that this stemmed from the difference in the ways of our data collection; the clinical evaluation of the cerebral status is relatively subjective and might vary among care team members (i.e., doctor, nurse, and respiratory therapist). The automated score likely decreases this variability by consistently using the same source of score (respiratory therapist). On the other hand, the clinical expert scored the cerebral status for the M-mWCAS by referring to other potentially relevant information such as the notes from the bedside nurses or physicians.

There are limitations in our study, including the small sample size, the retrospective nature of the study, and the single-center dataset. Further prospective studies are warranted to confirm the validity of this approach. Nonetheless, this first clinical report establishes the proof of concept of an algorithm to generate an automated respiratory distress score in the pediatric population. In conclusion, the A-mWCAS could provide an objective, fast, robust, and reliable assessment of the severity of the respiratory distress in a large electronic database. This will be a key parameter to control when analyzing large clinical respiratory datasets. It will also be an essential real-time component in future clinical decision support systems being developed to optimize the management of children with respiratory distress.

ACKNOWLEDGMENTS

We thank Drs. Gabriel Masson and Michael Sauthier for their contribution to this study in data preparation, and Dr. Philippe Jouvet for his contribution in conceiving the study. We want to thank the Fonds de Recherche du Québec Santé (FRQS), the Respiratory Health Network of the FRQS, and TransMedTech Institute for providing funds.