Use of the ReSVinet Scale for parents and healthcare workers in a paediatric emergency service: a prospective study

Abstract

Introduction

Most scales for acute respiratory infection (ARI) are limited to healthcare worker (HCW) use for clinical decision-making. The Respiratory Syncytial Virus network (ReSVinet) Scale offers a version for parents that could potentially help as an early warning system.

Objective

To determine whether or not the ReSVinet Scale for ARI in infants can be reliably used by HCWs and parents in an emergency service.

Methods

A prospective study was done of infants with ARI who were admitted to a paediatric emergency room to assess the ReSVinet Scale when used by faculty (paediatric doctor-professors), residents (doctors doing their first specialty in paediatrics) and parents. Spearman’s correlation and a weighted kappa coefficient were used to measure interobserver agreement. Internal consistency was also tested by Cronbach’s alpha test.

Results

Overall, 188 patients, 58% male, were enrolled. A Spearman’s correlation of 0.92 for faculty and resident scoring and 0.64 for faculty or resident and parent scoring was found. The weighted kappa coefficients were 0.78 for faculty versus residents, 0.41 for faculty versus parents, and 0.41 for residents versus parents. Cronbach’s alpha test was 0.67 for faculty, 0.62 for residents and 0.69 for parents.

Conclusion

There was good correlation in the ReSVinet scores between health professionals when used in the paediatric emergency area. Agreement between parents and health professionals was found to be more variable. Future studies should focus on finding ways to improve its reliability when used by parents before the scale is used in the emergency room.

What is known about the subject?

• Acute respiratory infection is an important cause of morbidity and mortality in children under 2 years of age.

• Absence of a universally available scale to enable parents to measure severity so they can determine when to bring the child for medical assessment is a major limitation.

What this study adds?

• There was good correlation of scores between health professionals.

• There was more variation in scores between parents and health professionals.

Introduction

Acute respiratory infection (ARI) is an important cause of morbidity and mortality in children under 2 years of age. Although bronchiolitis, which is caused by the respiratory syncytial virus, occurs most frequently (80%),¹ there are many respiratory syndromes that affect different areas of the respiratory tract and range from mild upper airway to severe and complicated lower airway diseases.² There are multiple aetiological factors in this group, most of them viral,³ although bacteria and fungi can also cause ARI. Diagnosis is based on clinical findings, hence the importance of being well informed regarding the signs and symptoms of the different respiratory diseases affecting children under 2 years of age in order to reduce mortality through accurate diagnosis and early intervention.^4–6

Every year 2.2 million children worldwide die from ARI. Reducing mortality in children under 5 years of age to less than 25 for every 1000 live births is one of the sustainable development goals for 2030.² Mortality in children under 5 has dropped from 12.7 million in 1990 to 5.6 million in 2016 as a result of advances achieved over the past 25 years.^{7 8} However, this reduction still falls short of the millennium goals. ARI is one of the primary causes of diseases in infants, and local beliefs regarding different diseases and use of home remedies or over-the-counter medications may delay care, thus increasing disease-related complications.^8–11

To date many scales have been developed to measure ARI severity. In all cases, respiratory rate is the most frequent item (81%), followed by the presence of abnormal breathing sounds (78%),¹² which is the basis for most of these scales designed for healthcare workers (HCW).¹³ The absence of a universally available scale for parents to measure severity so that they can determine when to bring the child in for medical assessment is a major limitation.^{13 14} A tool that can be used by parents or non-medical personnel to assess ARI severity needs to be validated.^{12 15}

The Respiratory Syncytial Virus network (ReSVinet) scale was developed and submitted for evaluation by the Galician Pediatric Research Network and the FIVE research group with collaboration of 90 paediatricians working in outpatient, inpatient and intensive care settings.¹⁵ The scale is designed to assess global clinical severity in patients with ARI and may be used with ambulatory and hospitalised patients without the need for direct medical assessment. The scale can be used by HCWs with information derived from medical records and by non-medical personnel (parents) by providing them with a simple tool for determining the severity of their child’s condition.¹⁵ This tool consists of the following seven parameters: (1) feeding intolerance, (2) medical intervention, (3) respiratory difficulty, (4) respiratory frequency, (5) apnoea, (6) general condition and (7) fever. The various parameters are assigned point values from 0 to 3, except for fever (from 0 to 2), which result in a total potential score of 20 points (table 1).¹⁵ The scale was validated by testing it on hospitalised children in three hospitals in Spain, but it has never been applied in a paediatric emergency room. The main objective of this study is to determine if the ReSVinet Scale for assessing ARI severity in infants admitted to emergency at a general hospital in Bogotá, Colombia is reliable when used by HCW and parents.

Table 1.

ReSVinet Scale*

Item	0 point	1 point	2 points	3 points
Feeding intolerance	No.	Mild: decreased appetite and/or isolated vomits with cough.	Partial: frequent vomits with cough, rejected feed but able to tolerate fluids sufficiently to ensure hydration.	Total: oral intolerance or absolute rejection of oral feed, not able to guarantee adequate hydration orally. Required nasogastric and/or intravenous fluids.
Medical intervention	No.	Basic: nasal secretions aspiration, physical examination, trial of nebulised bronchodilators, antipyretics.	Intermediate: oxygen therapy required. Complementary examinations were needed (chest X-rays, blood gases, hematimetry, etc). Maintained nebulised therapy with bronchodilators.	High: required respiratory support with positive pressure (either non-invasive in continuous positive airway pressure (CPAP), bilevel positive airway pressure (BiPAP) or high-flow oxygen; or invasive through endotracheal tube).
Respiratory difficulty	No.	Mild: not in basal situation but does not appear severe. Wheezing only audible with stethoscope, good air entrance. If modified Wood-Downes, Wang score or any other respiratory distress score is applied, it indicates mild severity.	Moderate: makes some extra respiratory effort (intercostal and/or tracheosternal retraction). Presented expiratory wheezing audible even without stethoscope, and air entrance may be decreased in localised areas. If modified Wood-Downes, Wang score or any other respiratory distress score is applied, it indicates moderate severity.	Severe: respiratory effort is obvious. Inspiratory and expiratory wheezing and/or clearly decreased air entry. If modified Wood-Downes, Wang score or any other respiratory distress score is applied, it indicates high severity.
Respiratory frequency	Normal <2 months: 40–50 bpm. 2–6 months: 35–45 bpm. 6–12 months: 30–40 bpm. 12–24 months: 25–35 bpm. 24–36 months: 20–30 bpm.	Mild or occasional tachypnoea: presented episodes of tachypnoea, well tolerated, limited in time by self-resolution or response to secretion aspiration or nebulisation.	Prolonged or recurrent tachypnoea: tachypnoea persisted or recurred despite secretion aspiration and/or nebulisation with bronchodilators.	Severe alteration: severe and sustained tachypnoea. Very superficial and quick breath rate. Normal/low breath rate with obvious increased respiratory effort and/or mental status affected. Orientative rates of severe tachypnoea: <2 months: >70 bpm; 2–6 months: >60 bpm; 6–12 months: >55 bpm; 12–24 months: >50 bpm; 24–36 months: >40 bpm.
Apnoea	No.			Yes. At least one episode of respiratory pause medically documented or strongly suggested through anamnesis.
General condition	Normal.	Mild: not in basal situation, child was mildly uncomfortable but does not appear to be in a severe condition, not impress of severity. Parents are not alarmed. Could wait in the waiting room or even stay at home.	Moderate: patient looks ill and will need medical examination and eventually further complementary examinations and/or therapy. Parents are concerned. Cannot wait in the waiting room.	Severe: agitated, apathetic, lethargic. No need for medical training to realise severity. Parents are very concerned. Immediate medical evaluation and/or intervention were required.
Fever	No.	Yes, mild central temperature <38.5°C.	Yes, moderate central temperature >38.5°C.

Reproduced from Justicia-Grande et al.15

*The original article also contains the version for parents.

BiPAP, bilevel positive airway pressure; bpm, breaths per minute; CPAP, continuous positive airway pressure; ReSVinet, Respiratory Syncytial Virus network.

Materials and methods

An observational, descriptive cohort study was done on patients under 2 years of age admitted to the emergency room at our general hospital in Bogotá, Colombia that provides medical services for workers with mandatory insurance. The study included patients presenting signs and symptoms of any type of ARI (rhinopharyngitis, croup, bronchiolitis, wheezing episodes or pneumonia). Parents were invited to participate by filling out/completing the scale and those who completed it were included. The study was done between late November 2017 and May 2018 (high season for respiratory diseases). Patients were only approached if the parents spoke Spanish and did not have significant comorbidities, for example, heart disease, chronic lung disease, anatomical lung malformation, neurological diseases, severe malnutrition, any immune deficiency or cancer. The authors of the scale gave us authorisation and provided the original version in Spanish. The language and content of the scale were reviewed by all attending physicians in our paediatric department, who determined the suitability for Colombian clinicians and parents. A convenience random sample of 10 parents were chosen on their understanding of the scale and the only difficulty found was with the term apnoea. Thus, a specific explanation of the term was given to parents before they filled out the scale.

Patients were identified by the treating physicians, who then contacted the researchers. The scale was completed at the same time by parents, paediatric doctor-professors (hereinafter faculty) and doctors doing their first specialty in paediatrics (hereinafter residents) after the parents consented to participate. Follow-up on discharged patients was done by telephone on day 10 after discharge to ask about new incidents that could lead to a return visit, if they required a readmission to another hospital or if they were continuing treatment at home. Patients admitted to the hospital were followed until discharge or at 30 days using the clinical records to determine length of stay, need for admission to the paediatric intensive care unit (PICU) or use of antibiotics. All parents were contacted again by phone after 30 days in order to determine if a fatal event had occurred following the ARI. The follow-up was carried out on day 10 after the onset of the disease in an effort to detect the higher incidence of respiratory failure and admission to the PICU and the follow-up on day 30 to detect recovery from the disease or mortality in most patients with lower respiratory tract infection.

Statistical analysis

The following parameters were taken into consideration for sample size: type I error, 0.05; type II error, 0.20; null hypothesis test value, 0.7; and population correlation coefficient, 0.82, as described in the article by Justicia-Grande et al,¹⁵ for a final sample size of 191 patients. A univariate analysis was done and the median (p50) and IQR (p25–p75) were used for quantitative variables. Qualitative variables were reported in terms of percentages and absolute numbers. Total scores for faculty, residents and parents are reported, as are percentages by item for each level of severity. Comparison between total scores was done using the Mann-Whitney U test and analysis by individual items by ordinal logistic regression.

For internal consistency, a Cronbach’s alpha test was calculated with the expectation of reaching a threshold of at least 0.7. A Spearman’s correlation coefficient was used to test the strength and direction of the correlations between observers. Interobserver agreement for the total scale and for each item was also calculated using weighted kappa coefficients for groups (faculty-resident, faculty-parent, resident-parent). A value of 0.01–0.2 was determined to have slight, 0.21–0.40 fair, 0.41–0.60 moderate, 0.61–0.80 substantial and 0.81–1 almost perfect agreement, respectively.¹⁶ Each item was also compared by categories of severity as selected by parent, faculty or resident using an ordered logistic regression. The Stata V.13 software package was used for statistical analysis as well as for calculation of the sample size and graphics. Cronbach’s alpha coefficients were analysed using R V.4.0.2statistical software package.

Patient and public involvement

Patients and/or the public were not involved in the design, or conduct, or reporting or dissemination plans of this research.

Results

We initially enrolled 191 patients, but when the statistical analysis was done, two cases that did not meet the age criteria were found and one was lost to initial follow-up. Table 2 shows their demographic and clinical characteristics. The median age was 10 (IQR 4.5–16) months, and 110 were boys (58.5%). ARI diagnoses included 40.4% rhinopharyngitis, 28.7% bronchiolitis, 17.1% croup, 12.2% wheezing episodes and 1.6% pneumonia. The median days between first symptoms and consultation was 3.5 (IQR 2–7). Of the total number of patients, 29.3% were hospitalised, 35.6% stayed for a short-term observation (less than 24 hours) and 35.1% were discharged after initial assessment. The median hospital length of stay was 3 (IQR 2–5). Only 2.7% were admitted to the PICU and 4.8% received antibiotics. No deaths in this cohort were reported at the 30-day follow-up call in 182 cases (96.8%). Six patients did not respond to the last follow-up phone call.

Table 2.

Clinical and demographic characteristics

Variable	Total (N=188)
	n (%)
Age in months, median (IQR)	10 (4.5–16)
0–3	40 (21.3)
4–12	66 (35.1)
>12	82 (43.6)
Gender, male	110 (58.5)
Diagnosis
Rhinopharyngitis	76 (40.4)
Bronchiolitis	54 (28.7)
Croup	32 (17.1)
Wheezing	23 (12.2)
Pneumonia	3 (1.6)
Time from first symptoms in days, median (IQR)	3.5 (2–7)
1–7	143 (76)
8–14	24 (12.8)
15–30	21 (11.2)
Treatment decision
Discharged	66 (35.1)
Short-term observation (<24 hours)	67 (35.6)
Hospital admission	55 (29.3)
Viral isolation tested	58 (30.8)
Negative	35 (18.6)
RSV	17 (9)
Adenovirus	3 (1.6)
Parainfluenza 1	2 (1.1)
Adenovirus and RSV	1 (0.5)
Length of stay in days, median (IQR)	3 (2–5)
0–7	178 (94.7)
8–14	8 (4.3)
15–30	2 (1)
Antibiotics prescribed	9 (4.8)
PICU admission	5 (2.7)
30-day mortality, n=182	0
Return visit before 10 days	25 (13.3)
Discharged	10 (5.3)
Short-term observation (<24 hours)	15 (7.9)
Hospital admission	0

n, absolute number or decimal; PICU, paediatric intensive care unit; RSV, respiratory syncytial virus.

Scores on the ReSVinet Scale

All the observers completed the ReSVinet Scale. The median total scale score was 5 (IQR 4–7) for faculty, 6 (IQR 4–7) for residents and 6 (IQR 4–8) for parents. When total scores were compared, no significant differences were found (faculty vs residents p=0.66, faculty vs parents p=0.18, residents vs parent p=0.33). Analysis by individual items showed statistical differences between parents and either faculty or residents in medical intervention, respiratory difficulty and frequency (p<0.01). For respiratory difficulty and rate, parents scored respiratory difficulty higher by choosing mild, while HCWs chose normal. Parents scored medical intervention lower than HCWs (figure 1).

Figure 1.

Respiratory Syncytial Virus network (ReSVinet) Scale percentage by item.

A look at Cronbach’s alpha for total internal consistency of the scale revealed 0.67 (95% CI 0.6 to 0.74) for faculty, 0.62 (95% CI 0.54 to 0.71) for residents and 0.69 (95% CI 0.62 to 0.75) for parents, respectively. Overall Spearman’s coefficient between the faculty and residents was 0.92 (95% CI 0.90 to 0.95), and 0.64 (95% CI 0.55 to 0.73) between faculty or residents and parents (figures 2–4). The weighted kappa coefficients were then examined and substantial agreement between faculty and residents (0.78, 95% CI 0.76 to 0.80), and moderate agreement between faculty and parents (0.41, 95% CI 0.36 to 0.47) and between residents and parents (0. 41, 95% CI 0.32 to 0.49) were found. Agreement for each item is shown in table 3.

Figure 2.

Scatter and fitted plot of the ReSVinet score: faculty versus parents. ReSVinet, Respiratory Syncytial Virus network.

Figure 3.

Scatter and fitted plot of ReSVinet score: residents versus parents. ReSVinet, Respiratory Syncytial Virus network.

Figure 4.

Scatter and fitted plot of ReSVinet score: faculty versus residents. ReSVinet, Respiratory Syncytial Virus network.

Table 3.

Interobserver agreement for each item on the scale (weighted kappa coefficient)

Item	Faculty vs residents (95% CI)	Faculty vs parents (95% CI)	Residents vs parents (95% CI)
Feeding intolerance	0.72 (0.64 to 0.77) Substantial	0.45 (0.42 to 0.49) Moderate	0.44 (0.35 to 0.46) Moderate
Medical intervention	0.88 (0.82 to 0.89) Almost perfect	0.07 (0.00 to 0.12) Slight	0.26 (0.23 to 0.32) Fair
Respiratory difficulty	0.76 (0.73 to 0.85) Substantial	0.28 (0.22 to 0.36) Fair	0.26 (0.23 to 0.35) Fair
Respiratory frequency	0.86 (0.74 to 0.91) Almost perfect	0.08 (0.02 to 0.13) Slight	0.08 (0.05 to 0.17) Slight
Apnoea	*	*	*
General condition	0.76 (0.71 to 0.82) Substantial	0.25 (0.25 to 0.30) Fair	0.34 (0.28 to 0.37) Fair
Fever	0.99 (0.99 to 1) Almost perfect	0.82 (0.74 to 0.88) Almost perfect	0.82 (0.78 to 0.87) Almost perfect
Total	0.78 (0.76 to 0.80) Substantial	0.41 (0.36 to 0.47) Moderate	0.41 (0.32 to 0.49) Moderate

*Not scored due to non-occurrence of symptom within the observation period of the faculty or resident group.

Discussion

This study shows that the ReSVinet Scale has very good reliability when used by HCWs for assessing ARI in the paediatric emergency setting, although it was lower when comparing use by parents and HCWs. Substantial interobserver agreement was found between faculty and paediatric residents, but agreement between HCWs and parents was moderate due to differences in scoring medical intervention, respiratory difficulty and frequency items.

Advances intended to reduce morbidity and mortality in paediatric patients have taken place around the world over the past few years. Several studies show variations in the way ARI is diagnosed and treated in the paediatric population.¹⁷ The ReSVinet was designed to determine the severity of acute lower respiratory tract infections in infants. This study included both lower and upper ARIs in the search for a more practical clinical application for infants with any significant respiratory disease. Moreover, because the study was a prospective concurrent enrolment, an attempt was made to avoid memory biases which may arise from the use of clinical records.

Significant variability was found in the individual elements of the scale, particularly in medical intervention and respiratory frequency and difficulty, where agreement between parents and HCWs was found to be slight or fair. When assessing respiratory rate, measurements should ideally be taken for 60 s and with a calm patient. In the case of children in an emergency room, this may be difficult to assess.¹⁸ Moreover, at younger ages, it can be more complex to assess as it varies based on the patient’s own conditions (fever, crying, pain, hunger and anxiety from being in an unfamiliar environment).¹⁹ Other studies have shown poor agreement between parents and HCWs regarding detection of respiratory distress, with kappa values between 0.26 and 0.36.²⁰ We believe another way of measuring signs of respiratory distress, such as frequent reassessments (every 3 hours), may make it easier to identify an abnormal respiratory pattern. Up to 7% of patients may develop intercostal retractions during periodic monitoring.²¹ Other authors have used automatic respiratory rate counters; however, they are not universally available and require cost-effectiveness studies.^{22 23} It has also been previously shown that the sensitivity and specificity of caregivers is insufficient to recognise tachypnoea and signs of respiratory distress in respiratory diseases and pneumonia.²⁴

On the subject of medical intervention, some parents chose no treatment in our study and this caused serious disagreement. When the original scale was evaluated, it was used before discharge. In our study, the parents were asked to complete the scale in the emergency room (to avoid any memory bias) right after medical evaluation was done and a course of action was defined. Three observers filled it out at the same time to ensure that they were evaluating the same conditions instead of conditions that had changed. New studies should evaluate whether this scale requires any prior training (prior to using it) or a specific guide that explains how to score individual points in greater detail on the scale in the parents’ version. Other factors included in the study were the best time to ask parents to fill out the scale, their level of education and the perceptions of parents and patients. Some parents do not consider being evaluated by a physician as some kind of medical ‘intervention’, despite the act being described in the original ReSVinet Scale.

Agreement regarding the item ‘general condition’ was fair, but there was significant variability in the score. The explanation may be that the sample used was mainly for upper respiratory infection, and parents could overestimate their scores and lose objectivity when going to the emergency room.^{25 26} General condition is subjective and varies between parents and medical examiners since going from home to a hospital environment may cause changes in their comfort, state of mind (calm, irritable, scared) and their well-being (normal, feverish or sick). Therefore, it will be necessary to try to standardise these conditions when examining patients. It has been shown that there is a growing tendency to visit paediatric emergency rooms for reasons such as perceived severity of symptoms, availability of continuous care that allows for assessment within a short period of time and perceived benefit of having been assessed by a paediatrician.^{11 27}

There was better agreement regarding feeding intolerance and almost perfect agreement regarding fever. This is because both are easy to measure objectively since they are quite clear on the scale for parental use, thermometers are widely available in our environment, and even the detection of fever by palpation has shown suitable sensitivity—88% (95% CI 79% to 93%).^{28 29}

Cronbach’s alpha was low and a possible theory for this is that it could be a result of assessing different respiratory diagnoses. Since the initial design was only for bronchiolitis, the result may be lower when various diseases with different symptoms are assessed. However, a larger sample of all diseases would be required to determine this. A universal scale for all ARIs is, in our opinion, a far better option than individual scales for each respiratory disease, since the literature shows that most scales that comply with Cronbach’s alpha requirements are homogeneous and assess patients suffering from only one specific cause of respiratory distress.

Regarding the lower agreement with parents, several variables may explain these findings: (1) parents’ expectations from the point of view of a healthcare setting where patients were recruited (the original study was done with hospitalised patients,¹⁵ while this study was done in the emergency room); (2) the higher percentage of rhinopharyngitis in this study versus bronchiolitis in the original study¹⁰; and (3) Bogota’s high altitude (2600 m above sea level) can change the perception of symptoms, but this has not been carefully studied.^{28 29}

One limitation of the study is that only three patients were diagnosed with pneumonia. Pneumonia is a leading cause of death in developing countries. Therefore, it would be relevant to have a representative sample of this condition. Another limitation is the low number of patients admitted to the PICU. Finally, another possible bias is the inclusion of families with mandatory insurance and only those who speak Spanish. If avoidable deaths in infants are to be reduced, improved access to healthcare services must be a priority.²³ The results of this study give rise to new hypotheses for studies designed to validate individual elements of the ReSVinet Scale that may be easy for parents to complete. Additional studies involving lower respiratory tract infections are also needed.

Conclusion

The ReSVinet Scale was found to have substantial reliability when used by HCWs in the paediatric emergency area in assessing ARI. In our population, agreement between scoring by HCWs and that done by parents was moderate, and slight agreement was found for medical intervention and respiratory rate. The scale has been shown to be promising and reliable when used by healthcare personnel to assess the severity of ARI in children under 2 years of age. Future studies should focus on finding ways to enable parents to use it reliably for two individual items so that it can be used in the emergency room.

Data availability statement

Data are available upon reasonable request. Data are not in a repository. Data are deidentified participant data.

Ethics statements

Patient consent for publication

Not required.

Ethics approval

The study was approved by the institutional review board and the hospital ethics review board (Comité de Ética en Investigación en Seres Humanos-CEISH-Hospital de San José).