Development of a Global Respiratory Severity Score for Respiratory Syncytial Virus Infection in Infants

Abstract

Background.

Respiratory syncytial virus (RSV) infection in infants has recognizable clinical signs and symptoms. However, quantification of disease severity is difficult, and published scores remain problematic. Thus, as part of a RSV pathogenesis study, we developed a global respiratory severity score (GRSS) as a research tool for evaluating infants with primary RSV infection.

Methods.

Previously healthy infants <10 months of age with RSV infections representing the spectrum of disease severity were prospectively evaluated. Clinical signs and symptoms were collected at 3 time points from hospitalized infants and those seen in ambulatory settings. Data were also extracted from office, emergency department, and hospital records. An unbiased data-driven approach using factor analysis was used to develop a GRSS.

Results.

A total of 139 infants (84 hospitalized and 55 nonhospitalized) were enrolled. Using hospitalization status as the output variable, 9 clinical variables were identified and weighted to produce a composite GRSS. The GRSS had an area under the receiver operator curve of 0.961. Construct validity was demonstrated via a significant correlation with length of stay (r = 0.586, P < .0001).

Conclusions.

Using routine clinical variables, we developed a severity score for infants with RSV infection that should be useful as an end point for investigation of disease pathogenesis and as an outcome measure for therapeutic interventions.

Respiratory syncytial virus (RSV) ushers in yearly outbreaks of respiratory disease and is responsible for significant morbidity in infancy, causing a range of illnesses, from mild upper respiratory illness (URI) to severe lower respiratory tract infection (LRTI) primarily manifesting as bronchiolitis or pneumonia [1]. Bronchiolitis due to RSV is one of the top 3 diagnoses leading to hospitalization during the first year of life, and RSV is estimated to cause between 52–82000 bronchiolitis-related and 22–44000 pneumonia-related hospitalizations annually in the United States [2]. In addition, recurrent wheezing episodes in the first decade following primary infection highlights the potential long-term effects of RSV infection [3, 4].

Despite the relative consistency of the signs and symptoms of RSV infection in young infants with primary infection, quantifying disease severity in individual infants remains elusive. Several severity scoring systems have been described, although there is no consensus on the most clinically useful and appropriate clinical and laboratory measures to include in a score [5–10]. Reasons for the lack of consistency in scoring systems include heterogeneity in the scope and purpose of the scores, as well as the age groups included. Some scoring systems for use in infants were developed by including only subjects with more-severe disease who were undergoing evaluation in emergency departments or already hospitalized and were created with the intent to predict need for oxygen use, hospitalization, or intensive care [6, 11–15]. Other scoring systems were developed to measure changes in specific signs of RSV disease, as well as overall illness severity, over relatively brief periods as an outcome measure to assess various therapeutic interventions. In each, a score is calculated at a single time point or over a relatively brief interval by summing points assigned for prespecified clinical variables such as respiratory rate, chest wall retractions, wheezing, and oxygen saturation (SaO₂) measurements. In many cases, variables are selected and weights assigned to each variable only on the basis of expert opinion [5]. However, none of these scoring systems have been developed to grade the overall severity of RSV infection over the course of the illness [16–18], and no score has been fully validated with questions regarding the utility of the various scores remaining [5, 7, 10].

A severity score that reflects the full spectrum of disease severity, encompassing both systemic and respiratory parameters, over the course of the illness would be very useful as an outcome measure in research aimed at determining the predictive value of early clinical parameters or novel biomarkers, as well as for use in clinical trials of antiviral agents or other therapeutics. Therefore, as part of a study of RSV disease pathogenesis (the Assessing Predictors of Infant RSV Effects and Severity [AsPIRES] study), we developed a global respiratory severity score (GRSS), using clinical data routinely collected during the course of illness of young infants with primary RSV infection.

METHODS

Study Participants

To capture the complete spectrum of disease severity, from mild illness involving outpatients to severe illness involving inpatients, we enrolled RSV-infected infants from 3 cohorts. All infants were born after 1 May of the previous winter and were <10 months of age at the time of RSV disease, ensuring that they would all be experiencing primary infection. The first cohort included infants hospitalized with RSV disease, while the second cohort included infants recruited at birth and actively monitored for RSV infection during their first winter, with only those acquiring RSV infection included in this report. The third cohort comprised infants seen in outpatient pediatric offices and emergency departments with respiratory symptoms suggestive of RSV infection. Infants in the third cohort were all originally discharged to home. The study encompassed 3 winters (during October 2012–April 2015) at 5 locations in Rochester, New York. Hospitalized infants were enrolled at the University of Rochester Medical Center’s (URMC’s) Golisano Children’s Hospital and Rochester General Hospital (RGH). Outpatients were enrolled at the Elmwood Pediatric Group practice and at the Pediatric Clinics and Pediatric Emergency Departments at URMC and RGH. Birth cohort infants were enrolled at URMC and RGH at the time of their birth hospitalization. All subjects were previously healthy full-term infants (gestation duration, ≥36 weeks) born after May 1 of the previous winter. Infants were excluded if they were hospitalized only for apnea; had any high-risk conditions, such as congenital cardiac disease, neurologic conditions, chronic aspiration, immunosuppression, malignancy, inability to complete the study; or qualified for palivizumab prophylaxis. The Research Subject Review Board of the University of Rochester and RGH approved the study, and all parents provided written informed consent.

Study Protocol and Procedures

RSV infection was identified by reverse transcription–polymerase chain reaction on nasal swabs collected at the time of hospitalization, during an ambulatory visit, or at a home visit. Following confirmation of RSV infection, all study subjects underwent a standard evaluation at 3 time points over 4 weeks. The first study visit occurred within 24 hours of hospitalization or diagnosis of RSV infection, the second visit occurred 12–16 days after illness onset, and the final visit occurred on illness day 25–32. For outpatients, illness symptoms were reported by the parent at each visit, including the presence and severity of nasal discharge or congestion, the presence of fever or lethargy, and whether the child had difficulty breathing, wheezing, cyanosis, apnea, or poor feeding. A study physician or nurse performed a physical examination at each study visit, regardless of location. The physical examination included measurement of heart rate, respiratory rate, and temperature; room-air pulse oximetry; determination of whether wheezing, rales/rhonchi, retractions, cyanosis, apnea, lethargy, and poor air movement were present; and evaluation of general appearance. Except for general appearance, which was graded from 0 to 3 (for well, mild, moderate, or severe, respectively), a value of 1 was assigned if a finding was present, and a value of 0 was assigned if a finding was absent. After the examination, biological samples were obtained, including nasal swab, nasal wash, nasal brush, and buccal swab specimens and a blood sample collected by venipuncture.

In addition, emergency department, hospital, and office records were reviewed for infants evaluated in those locations. The presence or absence of each of the clinical signs and symptoms was recorded, as well as the worst room air SaO₂ that was identified at any time during the illness, and these data were included in the analysis. Length of hospital stay (LOS), maximal oxygen support, duration of supplemental oxygen use, duration of intravenous fluid receipt, need for pediatric intensive care unit (PICU) stay, and duration of ventilator support were also recorded.

Age-Adjusted Maximum Respiratory Rate

Maximum respiratory rate is an important clinical variable used in our statistical analyses and is known to have an association with respiratory diseases. Since the normal range of respiratory rate is age dependent, with rates declining with older age, we adjusted this variable by using linear regression analysis of recently published normative data derived from >3800 infants and children [19]. Using these data, we found that the rate (slope) of the negative association between respiratory rate and age varies from the first to the fourth quartile of respiratory rates. On average, the median respiratory rate declines 0.63 breaths/minute with each month of age, but at the 95th centile it declines by 0.83 breaths/month. Therefore, we adjusted the maximum respiratory rate by subtracting the measured maximum respiratory rate by 0.80 breaths/month of age (the maximum age adjusted respiratory rate).

Statistical Analysis

The primary aim was to use an unbiased analytic approach to develop a weighted global respiratory severity score that best captured the information from clinical variables associated with the physical signs of RSV-associated LRTI obtained from the full time course of the illness. Following development, we tested the construct validity of the score by evaluating how well the GRSS predicted LOS, a previously accepted marker of disease severity. As a further test of construct validity, we applied our method to the clinical data obtained only up to and including the first full study visit so that all subjects would have an approximately equal period of observation.

First, marginal analyses were used to identify clinical variables that were significantly associated with the output variable (hospitalization versus nonhospitalization). For continuous variables (maximum age-adjusted respiratory rate and worst recorded SaO₂), a 2-sample Welch t test was used to determine whether there were significant mean differences between the hospitalized and nonhospitalized groups; for categorical variables, the Fisher exact test was used.

Next, we replaced missing values (9 of 1529 data points [0.6% of all values]) with the sample mean of the corresponding variables computed from available cases (mean imputation) and standardized the data so that every clinical variable had a mean of 0 and a variance of 1. After imputation and standardization, we performed a factor analysis based on principal component analysis and varimax rotation to identify the main mode of variation in the data [20]. Two factors were identified, and a logistic regression analysis was used to identify the relationship between these 2 factors and hospitalization. A likelihood ratio test determined that only the first factor was significantly associated with hospitalization. This factor was then linearly transformed to have a range between 0 and 10; the result was the global respiratory severity score (GRSS). The Pearson correlation test was used to test the significant association between the GRSS and the LOS.

All analyses were performed with SAS (version 9.3; SAS Institute, Cary, NC) and the R programming language (version 3.2; R Foundation for Statistical Computing, Vienna, Austria).

RESULTS

A total of 139 RSV-infected infants were enrolled: 78 at the time of hospitalization, 19 from the birth cohort (of whom 4 were hospitalized during their illness), and 42 seen in an ambulatory setting (26 in pediatric offices and 16 in the emergency department, of whom 2 were hospitalized during the illness). Thus, there were 84 hospitalized and 55 nonhospitalized RSV-infected infants for analysis. The subjects were sociodemographically diverse, but the cohorts were generally similar in characteristics, with only the mean age of the hospitalized infants significantly lower than that for the nonhospitalized infants (Table 1). Average household size was 4.1 individuals, with an average of 1.7 children per household.

Table 1.

Characteristics of the 139 Infants in the Study Population at Enrollment, by Cohort

Characteristic	Nonhospitalized (n = 55)	Hospitalized (n = 84)	P
Age at infection, mo
Mean ± SE	4.1 ± 0.3	2.8 ± 0.2	< .001
Range	1.1–8.9	0.5–9.4
Gestational age, wk (range)			.26
Mean ± SE	39.1 ± 0.2	38.8 ± 0.1
Range	36–42	36–41
Male sex	25 (45)	41 (49)	.73
White	34 (62)	65 (77)	.06
Hispanic	9 (16)	11 (13)	.63
Cesarean section delivery	13 (24)	27 (32)	.34
Birth weight, kg			.82
Mean ± SE	3.3 ± 0.1	3.4 ± 0.1
Range	1.8–5.3	1.7–4.9
Maternal age, y			.08
Mean ± SE	29.8 ± 0.7	28.1 ± 0.7
Range	19–41	17–44

Data are no. (%) of infants, unless otherwise indicated.

Abbreviation: SE, standard error.

Hospitalized and nonhospitalized infants differed significantly in most clinical parameters determined by physical examination (Table 2). Only the presence of apnea was not statistically different between the 2 groups. Eleven infants (13%) were admitted to the PICU, including 6 (7%) who required invasive mechanical ventilation. The average LOS (±standard error [SE]) for hospitalized infants was 4.2 ± 0.6 days, with a median of 2.3 days. The first study visit occurred at a mean duration (±SE) after illness onset of 4.7 ± 0.21 days for hospitalized children and 4.6 ± 0.28 days for the nonhospitalized cohort.

Table 2.

Clinical Findings Among the 139 Infants in the Study Population, by Cohort

Parameter	Nonhospitalized (n = 55)	Hospitalized (n = 84)	P
Overall appearance
Well	17 (31)	1 (1)
Mildly ill	36 (65)	51 (61)	< .001
Moderately ill	2 (4)	28 (33)
Severely ill	0 (0)	4 (5)
Wheezing	15 (28)	62 (74)	< .001
Rales/rhonchi	8 (15)	62 (74)	< .001
Retractions	18 (33)	78 (93)	< .001
Cyanosis	0 (0)	7 (8)	.04
Apnea	0 (0)	3 (4)	.28
Lethargy	1 (2)	25 (30)	< .001
Poor air movement	1 (2)	17 (21)	< .001
Maximum respiratory rate, breaths/ min	47 ± 1.5	63 ± 1.8	< .001
Worst SaO2 in room air, %	98 ± 0.3	86 ± 0.8	< .001
Maximum FIO2	0.21 ± 0.0	0.33 ± 0.22	< .001
Duration of O2 receipt, d	NA	3.1 ± 0.5
Duration of intravenous fluid receipt, d	NA	2.3 ± 0.4
Current PICU stay	NA	11 (13)
Mechanical ventilation receipt	NA	6 (7)
Duration of hospital stay, d, median	NA	2.3

Data are no. (%) of infants or mean value ± standard error, unless otherwise indicated.

Abbreviations: FIO₂, fraction of inspired O₂; NA, not applicable; PICU, pediatric intensive care unit; SaO₂, arterial blood O₂ saturation.

To ensure content validity, we used the marginal analysis to select 9 clinical variables in the development of an unbiased severity score, including general appearance; presence of wheezing, rales, retractions, cyanosis, lethargy, or poor air movement; maximal age-adjusted respiratory rate; and worst room-air SaO₂. As a first step, factor analysis was used to assess the optimal discrimination between hospitalized and nonhospitalized infants. Two factors were identified (factor 1 and factor 2) and were able to explain 26% and 15% of proportional variance, respectively. In the absence of a gold standard to define severe disease, we chose hospitalization status as the output variable in the development of the GRSS. The relationship between hospitalization status and factor 1 and factor 2 is shown in a principal component analysis in Figure 1. Logistic regression identified the optimal discrimination between hospitalized and nonhospitalized infants by these factors, and based on the likelihood ratio test, factor 1 was highly significant (P = .001), while factor 2 was nonsignificant (P = .38). Therefore, factor 1 was selected as the sole predictor for the output variable of hospitalization, standardized so that it ranged from 0 (least severe) to 10 (most severe), and termed the “global respiratory severity score.” The final equation for calculating the GRSS is shown in Table 3, as is the weight calculated for each of the 9 variables (β_j). We used this equation and included imputed values for the 7 subjects with missing variables (Table 3) to calculate a GRSS for all subjects, with the distribution of scores shown in Figure 2. By use of a score of 3.5 as a threshold for predicting hospitalization, only 14 of 139 subjects were misclassified; 6 hospitalized infants had a GRSS of ≤3.5 (range, 1.4–3.4), and 8 nonhospitalized infants had scores of >3.5 (range, 3.6–5.3). Review of the medical records from the hospitalized subjects who were misclassified (n = 6) found that 3 were <4 weeks of age and admitted with fever and presumed sepsis, with one admitted because the primary care physician was concerned about croup and another admitted because of maternal concern about caring for the child at home. Misclassified infants in the nonhospitalized group (n = 8) were generally older, ranging in age from 6 weeks to 8.9 months (mean, 4.6 months), only 2 were <3.5 months of age, and all had a room-air SaO₂ of ≥95% despite having wheezing and/or retractions detected on physical examination. The receiver operating characteristic (ROC) curve provides an excellent area under the curve (AUC) of 0.961 for the GRSS (Figure 3). Of note, when calculated separately by using the same cutoff (3.5), the AUC was 0.962 and 0.959 for infants ≤3 and >3 months of age, respectively.

Figure 1.

The association between hospitalization status and the 2 leading factors derived from a factor analysis. Open circles represent nonhospitalized subjects, and solid circles represent hospitalized subjects. Factor 1 and factor 2 are the top 2 factors identified by principal component analysis and varimax rotation. A subsequent logistic regression analysis determined that only factor 1 was significantly associated with hospitalization; thus the global respiratory severity score is calculated on the basis of factor 1.

Table 3.

Mathematical Equation Constructed by Factor Analysis to Calculate the Global Respiratory Severity Score (GRSS)

$$\text{GRSS is}={\displaystyle \sum _{j=1}^9{X}_j{\beta }_j+c,c=\mathrm{6.46.}}$$

Here, Xj, where j = 1, 2, …, 9, denotes the clinical variables used in the analysis, and βj denotes the weights assigned to these variables as listed in the table below. Also listed in this table are mean imputation values for each variable that were applied to the 7 subjects with missing data.

Variable	β j	Imputed Valuea
General appearance	0.719	1.144
Wheezing	0.936	0.558
Rales/rhonchi	1.150	0.507
Retractions	1.360	0.691
Cyanosis	0.812	0.050
Lethargy	0.475	0.190
Poor air movement	0.735	0.134
Worst room air SaO2	–0.078	90.360
Maximum respiratory rate	0.034	57.015

Abbreviation: SaO₂, arterial blood O₂ saturation.

^aImputed values were defined as the mean value calculated using data from all subjects with available values. Mean imputation values for each variable were used for the 7 subjects with missing data.

Figure 2.

Histogram of the global respiratory severity score (GRSS) for 139 infants. We used a score of 3.5 as the threshold to predict hospitalization (represented by the vertical broken line) and found that 14 infants were misclassified (shown in light grey shaded bars), of whom 6 with a GRSS ≤3.5 were hospitalized, and 8 with a GRSS of >3.5 were not hospitalized.

Figure 3.

Receiver operating characteristic (ROC) curve of the respiratory severity score in discriminating hospitalized from nonhospitalized subjects. Gray-shaded areas are 95% confidence intervals computed from 2000 stratified bootstrap replications. The area under the ROC curve is 0.961.

Next, we assessed the construct validity of the GRSS by analyzing LOS for the hospitalized infants because LOS has been used by others as a gauge of RSV disease severity and was not used in training the severity score in the factor analysis (Figure 4). The Pearson correlation coefficient between the calculated GRSS and LOS was 0.586 (P < .0001). Of note, the slope of the correlation between GRSS and LOS declined sharply after day 6 (β = 0.061), compared with the slope on or before day 6 (β = 0.628). Additionally, we compared the GRSS for children admitted to the PICU to that for hospitalized children not admitted to the PICU. The mean GRSS (±SE) for 11 children admitted to the PICU was 8.5 ± 0.34, compared with 5.7 ± 0.19 for the 73 infants admitted to the general pediatric wards (P < .0001).

Figure 4.

The relationship between the global respiratory severity score (GRSS) and length of stay (LOS) for hospitalized subjects. Open circles represent nonhospitalized subjects shown as references (not included in correlation analyses), solid circles represent hospitalized subjects with a LOS of ≤6 days, and triangles represent hospitalized subjects with a LOS of >6 days.

Finally, we applied the severity score equation to all subjects, using only data collected on or before the first full study visit, to determine how the score would function at diagnosis of RSV disease and to standardize data collection. We used the same 9 variables and found that the GRSS ROC still had an AUC of 0.925, with a slightly lower Pearson correlation coefficient with LOS (0.45, P < .0001). Nineteen of 139 infants (8 nonhospitalized and 11 hospitalized) were now misclassified. They included the same 8 nonhospitalized infants and 4 of the hospitalized infants who were misclassified using data from the entire clinical course.

DISCUSSION

During the AsPIRES study of RSV disease pathogenesis in young infants, we needed an outcome measure of disease severity for analysis of host biologic measurements and viral data, and none of the existing severity scores adequately satisfied our needs. Use of single parameters, such as hypoxia or need for hospitalization, have been used to categorize respiratory disease severity, although there is no consensus of their use as a gold standard [5]. In addition, although various severity scores using several components of the physical examination and laboratory data, such as SaO₂, have been published, they generally only provide a severity measure over a relatively short time frame. Thus, we sought to develop a comprehensive respiratory severity score, the GRSS, using data collected during the course of illness. To accomplish this, we used routinely measured clinical variables commonly associated with RSV infection in infants. However, unlike most other scores, such as the Tal, the modified Tal, or the RDAI scores [12, 16, 18], that use preselected variables in calculating the score, we used a novel unbiased analytic approach to select variables and assign optimal weights to each.

We used hospitalization status as the output variable for development of the GRSS because we believe it was the best choice in the absence of a recognized gold standard for respiratory disease severity. Supporting this choice, we found that, in the majority of misclassified subjects, the decision about whether to hospitalize could be readily explained on the basis clinical and social factors and was not necessarily related to respiratory symptoms noted on physical examination. In infants who were hospitalized because of fever and young age, their low GRSS indicated mild respiratory illness and was confirmed by the findings in the medical record. Supporting the content validity of the GRSS, we found a significant correlation between the hospitalized infants and LOS, another commonly used measure of disease severity. The diminished correlation after day 6 of illness might suggest that infants hospitalized with a high GRSS who stay beyond day 6 remain in the hospital for a combination of RSV-related and non–RSV-related complications. The mean GRSS of infants who required PICU care was significantly greater than that for hospitalized infants on the general care wards, further supporting the content validity of the score.

Strengths of this study include the prospective design with a sociodemographically diverse population that spanned the spectrum of disease severity, the unbiased novel analytic approach to score development, performance of both content and construct validation by using an analytic approach to choose variables to include in the score, evaluation of the GRSS against LOS, and comparison of the GRSS of infants cared for in the PICU to that for those in the general pediatric wards. Although hospitalized infants were somewhat overrepresented in the study, we did include a moderate number of subjects from the birth cohort with upper respiratory tract signs and symptoms who required no medical visits for clinical care and infants with moderate illness seeking ambulatory medical care. In addition, we did not attempt to grade the degree of various physical signs such as wheezing or retraction, as often done with other complex scoring systems, and thus reduced concern about interobserver variation. Finally, by applying a unique data-driven analytic strategy of marginal analysis followed by factor analysis and logistic regression to select variables and derive weights for each selected variable, we were able to combine the variables into a single summary statistic (the GRSS). Although a relatively large number of variables are used, the equation to calculate the GRSS is readily adaptable for a computer-generated score and can be accessed at our website (available at: https://rprc.urmc.rochester.edu/app/AsPIRES/RSV-GRSS/).

Nevertheless, there are limitations. First, this scoring system was not developed to predict outcomes in individual patients or for measuring changes in severity over brief periods but was an attempt to quantify overall illness severity. In addition, we have not yet validated the GRSS prospectively or in a separate cohort, the overall sample size was modest, and we limited our population to infants with primary RSV infection less than 10 months of age. Also, not all subjects completed all study visits after enrollment or had equal days of medical records from which to extract clinical data. This is most relevant for the mildly ill subjects, who had few or no office visits and may have missed a study visit. To assess the potential impact of this limitation, we also validated the approach used to develop the GRSS by using only data that were collected up to and including the first full study visit. This analysis showed that, although a few more infants were misclassified, the AUC of the ROC curve was still very good and was consistent with the typical time course of RSV illness, which generally peaks in severity about 4–5 days after illness onset, a time very close to the first study visit evaluation for most subjects. It is possible that the use of several members of the study team to examine subjects and extract data from medical records may have introduced variation into the data collection. However, Gajdos et al [11] found that interobserver reliability for the parameters we assessed is very high, especially when not attempting to grade the degree of each parameter, even when many observers from different healthcare professions participate. Interrater reliability has also been shown to be high when using more-complicated scoring systems that do require gradations in physical findings, such as the Tal and modified Tal scores, suggesting that trained medical and research professionals provide similar assessments of ill children [8, 12]. Feldman et al [9] developed a severity score based on data obtained during the first 24 hours after enrollment, using a combination of study visits and chart data extraction, that provided significant separation of upper respiratory tract infection from LRTI and hospitalized from nonhospitalized status. However, it should be noted that, in their study, a modified Tal score was used and included subjects with all causes of respiratory tract infection.

In conclusion, we developed a respiratory illness severity score for use in a broad spectrum of RSV disease in full-term previously healthy infants during their primary RSV infection. We intend to use this research tool to study disease pathogenesis in this population, although it may also be useful as an outcome measure for identification of biomarkers to predict severe disease or in the study of therapeutic agents such as antivirals.

Notes

Acknowledgments. We thank the children and families, for their participation; and the study coordinator, Gerry Lofthus, PhD, and study nurses, Melissa Bowman, RN, Cindy MacDonald, RN, Mary Criddle, RN, and Doreen Francis, RN, for recruitment and evaluation of the study participants.

Financial support. This work was supported by the National Institute of Allergy and Infectious Diseases (contract HHSN272201200005C).

Potential conflicts of interest. All authors: No reported conflicts. All authors have submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Conflicts that the editors consider relevant to the content of the manuscript have been disclosed.