Wheeze as an Adverse Event in Pediatric Vaccine and Drug Randomized Controlled Trials: A Systematic Review

Abstract

Introduction

Wheeze is an important sign indicating a potentially severe adverse event in vaccine and drug trials, particularly in children. However, there are currently no consensus definitions of wheeze or associated respiratory compromise in randomized controlled trials (RCTs).

Objective

To identify definitions and severity grading scales of wheeze as an adverse event in vaccine and drug RCTs enrolling children <5 years and to determine their diagnostic performance based on sensitivity, specificity and inter-observer agreement.

Methods

We performed a systematic review of electronic databases and reference lists with restrictions for trial settings, English language and publication date ≥ 1970. Wheeze definitions and severity grading were abstracted and ranked by a diagnostic certainty score based on sensitivity, specificity and inter-observer agreement.

Results

Of 1,205 articles identified using our broad search terms, we identified 58 eligible trials conducted in 38 countries, mainly in high-income settings. Vaccines made up the majority (90%) of interventions, particularly influenza vaccines (65%). Only 15 trials provided explicit definitions of wheeze. Of 24 studies that described severity, 11 described wheeze severity in the context of an explicit wheeze definition. The remaining 13 studies described wheeze severity where wheeze was defined as part of a respiratory illness or a wheeze equivalent. Wheeze descriptions were elicited from caregiver reports (14%), physical examination by a health worker (45%) or a combination (41%). There were 21/58 studies in which wheeze definitions included combined caregiver report and healthcare worker assessment. The use of these two methods appeared to have the highest combined sensitivity and specificity.

Conclusion

Standardized wheeze definitions and severity grading scales for use in pediatric vaccine or drug trials are lacking. Standardized definitions of wheeze are needed for assessment of possible adverse events as new vaccines and drugs are evaluated.

Introduction

Wheeze is an important clinical sign of a potential adverse respiratory event in pediatric drug and vaccine trials. Reports of wheeze in children participating in influenza vaccine trials have raised particular concern of causal associations between vaccine receipt and adverse respiratory events [1–4]. However, wheeze definitions vary and range from transient audible breath sounds with no associated respiratory symptoms to the presence of severe respiratory distress. This is a missed opportunity for consistent case verification and data comparability within and across trials and products.

During the pre-licensure evaluation of the Ann-Arbor backbone live attenuated influenza vaccine (LAIV), a safety signal for wheeze was identified in children <2 years and among children with a past history of wheeze and/or asthma [2]. No wheeze signal was identified for the Leningrad- backbone LAIV, but the clinical development of this product preceded the Ann-Arbor backbone LAIV and wheezing was not directly solicited as an adverse event in these trials [5]. Because the burden of influenza is greatest in children <2 years of age [6], determining the safety of LAIV vaccines in this age group is a priority [7]. In order to conduct safety and efficacy trials of the Leningrad-backbone LAIV and other vaccines in this population, a consistent definition of wheeze is needed.

To inform the development of a standardized definition of wheeze to be used in clinical trials of Leningrad-backbone LAIV in young children as well as other trials of vaccines or drugs, we conducted a systematic review to identify definitions and severity grading of wheeze as an adverse event in these settings. In addition, we assigned scores and ranks based on the diagnostic certainty of wheeze definitions to inform recommendations to a Brighton Collaboration convened working group.

Methods

Search Strategy

We conducted an electronic literature search on October 28, 2014, applying the Patient Population, Intervention, Comparator, Outcome, Timing and Setting (PICOTS) framework [8] outlined in Supplementary Materials Table 1. We considered studies as the population and case definitions as the outcomes. We searched the following clinical databases: MEDLINE, EMBASE, Web of Science, Scopus, CINAHL Plus, the Cochrane Library databases and WHO Global Health Library. The search was restricted to English language publications and trials published after 1970. We searched ‘gray literature’ databases, conference abstracts and manually reviewed reference lists of selected publications and records recommended by experts to encompass a broad range of the available literature. Wheezing definitions from the protocols of two ongoing studies, and one recently published study were also included [9–11]. Case reports, case series, cross-sectional, case-control, cohort or quasi-experimental studies were excluded. Our search strategy is in Supplementary Tables 2 and 3. We merged our search results into EndNote (Thomson Reuters, New York, NY) and removed duplicate citations.

Population

We included vaccine and drug randomized controlled trials (RCTs) reporting wheezing or respiratory signs, symptoms and diseases that were considered “wheeze equivalents” including asthma and bronchiolitis as an adverse event. We included studies conducted in inpatient, outpatient, or community settings. We excluded studies that did not enroll children < 5 years of age. World Bank country income economy classification at the time the study was conducted was used as a proxy for socioeconomic status and medical resource availability.

Data Extraction

Two independent reviewers (D.M., S.K.) screened titles and abstracts of all citations to identify potentially eligible studies (first screen). The second screen consisted of full-text review of studies selected by either reviewer in the first screen with agreement required for final study eligibility and inclusion in the systematic review. Disagreements on study inclusion were resolved by discussions between the reviewers or by the decision of a third reviewer (D.J.H). A log of excluded studies and reasons for exclusion was kept and is available upon request. The two reviewers independently extracted data using a standardized form and resolved any discrepancies by consensus.

Outcomes

We identified study-specific wheeze definitions and wheeze severity grading scales. If wheeze was not described, definition(s) of pre-defined wheeze equivalents (rhonchi, bronchiolitis, bronchitis, asthma, reactive airway disease and respiratory hypersensitivity) were assessed. We pre-specified subgroups by age (< 2 years, ≥ 2 years), history of wheeze/asthma, comorbidity status, and categories of wheeze definitions (specific wheeze definition, wheeze assessed without an explicit definition and other wheeze equivalents).

When possible, we evaluated each included study to determine whether and to what extent U.S. National Institutes of Health, Division of AIDS (DAIDS) severity grades [12] were assessed in the context of wheeze or wheeze equivalents. The DAIDS severity grading for adverse event definitions is a widely used severity grading scale for respiratory events in clinical trials. This grading system for adverse events related to dyspnea or respiratory distress in patients <14 years defines grade 1 as “wheezing OR minimal increase in respiratory rate for age”, grade 2 as “nasal flaring OR intercostal retractions OR pulse oximetry 90–95%”, grade 3 as “dyspnea at rest causing inability to perform usual social and functional activities OR pulse oximetry <90%”, and grade 4 as “respiratory failure with ventilation support indicated”.

In addition, we ranked wheeze definitions by a diagnostic certainty score, developed for the purpose of this review. The diagnostic certainty score is based on performance in three categories: sensitivity, specificity, and inter-observer agreement (high, moderate, low). Each category grade is converted to a numeric score (high=2, moderate=1 and low=0), and summed to create a diagnostic certainty score with a range from 0–6 points. The diagnostic certainty score is based on the detection of any wheeze, regardless of severity or clinical importance. We assumed that formalization and/or verification of clinician diagnosed wheeze improved specificity and reproducibility, e.g. use of an algorithm, second clinician confirmation, computer-assisted techniques, and severity assessment. We made assumptions that: 1) Physicians and healthcare workers assessing wheeze were well-trained, 2) Caregivers identify symptoms of respiratory illness without the aid of a stethoscope and 3) Active surveillance is likely to pick up more wheeze cases including those that are not severe, in comparison to healthcare worker or caregiver assessments alone.

Analysis

We analyzed the identified outcomes by the following procedures: 1) Enumeration of studies grouped by outcome type (wheeze defined, wheeze equivalent(s) defined, and wheeze or wheeze equivalent(s) assessed without explicit definition); 2) Classification of wheeze and wheeze equivalent by assessor type and qualifications, method of wheeze detection, timing and operationalization of assessment; 3) Enumeration of studies providing data on wheeze and wheeze equivalent severity; 4) Determination of ability to assign study measures of severity to a DAIDS grade; and 5) Ranking of wheeze definitions based on the proposed diagnostic certainty score.

Results

We identified 1,205 citations from an electronic database search, of which 335 studies were selected for full text review. A total of 58 studies were included in the final synthesis as depicted in the PRISMA flow chart (Fig. 1).

Figure 1.

PRISMA Flow Chart

Wheeze Description Statistics

Approximately 70% of trials were conducted in North America (23), Europe (17) and Australia (7) as illustrated in Figure 2. Less than 20% of included trials were from low income and lower middle income countries. Most (80%) of the trials in this systematic review included children < 2 years. In studies including children ≥2 years, 90% excluded children with a history of wheeze. All studies that included children < 2 years of age excluded children with a history of wheeze. The majority (90%) of study interventions were vaccines, with the remainder being intervention trials for Fluticasone, Immunoglobulin, Sublingual immunotherapy, Lactobacillus, M. phlei, mattress covers and dust mite control. Of the 51 included vaccine trials, 33 involved influenza vaccines (65%) with the remainder vaccines against bacteria (20%), viruses other than influenza (12%), Mycobacterium tuberculosis (2%), and Plasmodium falciparum (2%). Influenza vaccine was the most common intervention among the pediatric drug and vaccine trials identified.

Figure 2. Geographical Location of All Trial Sites.

Note: The boundaries and names shown and the designations used on this map do not imply the expression of any opinion whatsoever on the part of the World Health Organization concerning the legal status of any country, territory, city or area or of its authorities, or concerning the delimitation of its frontiers or boundaries.

A wheeze definition was provided in 26% of the trials (Supplementary Table 4a) and 45% of the trials assessed wheeze as a sign or part of a respiratory illness without an explicit definition (Supplementary Table 4b). Wheeze was not assessed in almost one-third of trials (29%) but other wheeze equivalents were described (Supplementary Table 4c). Supplementary Tables 4a, 4b, and 4c provide results stratified by age (children <2 years and children ≥2 years). The characteristics of the wheeze definitions are summarized in Table 1. Most (69%) studies involved health worker assessments. Almost half the studies that involved health workers did not specify their level of training: 52% in studies including children < 2 years of age and 25% in those excluding this age group. One out of 3 trials either had wheeze assessed by a health worker or by a combination of a caregiver and health worker.

Table 1.

Characteristics of Wheeze Definitions in Trial Settings That Include Children < 5 Years

CHARACTERISTICS OF WHEEZE DEFINITIONS	FREQUENCY (Supplementary Table 4a studies)
ASSESSOR QUALIFICATIONS
Caregiver (Parent/Guardian)	21 (7)
Health Worker	40 (9)
HEALTH WORKER QUALIFICATIONS
Physician	10 (5)
Study nurse	3 (0)
Health worker (description not provided)	27 (7)
NUMBER OF ASSESSORS
1 assessor – Caregiver only	5 (1)
1 assessor – Health Worker only	20 (7)
> 1 assessor–Both Caregiver and Health Worker	21 (4)
HEALTH WORKER EXAMINATION DETAILS
Wheeze audible without a stethoscope	5 (4)
Wheeze on auscultation	9 (6)
Characteristic findings of wheeze on auscultation	4 (4)
Bronchodilator response	1 (0)
Additional tests: pulmonary function testa/plethysmographyb	3 (1)
TIMING OF WHEEZE OR WHEEZE EQUIVALENT
Reference made to timing of wheeze or wheeze equivalent	7 (4)
Within minutes/hours	1 (1)
Within days	1 (0)
Within weeks	1 (0)
Within months	3 (2)
Within years	1 (1)
OPERATIONALIZATION OF ASSESSMENT
Questionnaire	4 (1)
Diaries/checklists/symptom score cards	9 (4)
Telephone interview/contact	3 (0)
Home visits	5 (1)
Facility based patient presentation	40 (11)
ICD-9 codes for asthma/reactive airway disease/medically attended acute respiratory illness	2 (1)

Supplementary Table 4A studies: An explicit wheeze definition is provided.

Of the 58 studies, few reported on audible wheeze without a stethoscope (7%), wheeze on auscultation (14%), and detailed auscultation findings (7%). These characteristics were more often reported in trials that excluded children <2 years (25% of studies in older populations). Four studies provided detailed auscultation findings, 2 of which are ongoing studies [9,10], and included children 24–59 months [3]. Only one study documented bronchodilator response. Additional tests in a few studies included pulmonary function tests and plethysmography (Table 2). Reference to timing of wheeze or wheeze equivalents as an adverse event was made in 7 of the 58 studies identified (12%), and spanned minutes, days, weeks, months and years (Table 1). In influenza vaccine studies, the spectrum of timing of wheeze or wheeze equivalents included 30 minutes to 42 days after the intervention (Supplementary Table 5). Some studies contacted caregivers via telephone (5%) or home visits (5%), but the majority of studies were facility/site based (69%).

TABLE 2.

PROPOSED RANKING OF DIAGNOSTIC CERTAINTY FOR WHEEZE DEFINITIONS IN PEDIATRIC VACCINE AND DRUG TRIALS

Wheeze Definition	Sensitivity	Specificity	Inter rater agreement	Diagnostic Certainty (range 0–6)	Example Study
COMBINED CAREGIVER REPORT AND HEALTH WORKER ASSESSMENT
Caregiver report using diary or other memory aid, active surveillance, and follow-up with clinician. Clinician diagnosed wheeze verified by computer assisted technique/2 clinicians/algorithmic approach.	High	High	High	6	Not used in a study
Caregiver report using diary or other memory aid, active surveillance, and follow-up with clinician	High	High	Moderate	5	Not used in a study
Active surveillance for wheeze via phone calls, followed by visit by study nurse	High	High	Moderate	5	Piedra, 1996[31]
HEALTH WORKER ASSESSMENT
Clinician diagnosed wheeze verified by computer assisted technique	Moderate	High	High	5	Not used in a study
Wheeze verified by two physicians	Moderate	High	High	5	Not used in a study
Clinician diagnosed wheeze employing an algorithmic approach	Moderate	High	High	5	Not used in a study
Medically attended wheeze -defined as presence of wheezing on a physical examination conducted by a health care provider, for a patient with signs of respiratory distress [severity assessment]	Moderate	High	High	5	Belshe, 1982 [32]
Daily clinical observation	Moderate	High	Moderate	4	Wright, 1976 [13]
Physician verified wheeze (no required severity)	Moderate	High	Moderate	4	Custovic, 2002 [33]
Surveillance of medical visits to hospitals and emergency rooms for wheeze asthma and other respiratory symptoms	Moderate	High	Moderate	4	Bergen, 2004 [2]
Clinical observation immediately after vaccine administration	Low	High	Moderate	3	Greenhawt, 2012 [34]
Retrospective review of ICD-9 codes for asthma or reactive airway disease	Low	High	Moderate	2	Gaglani 2008 [4]
CAREGIVER REPORT
Asthma symptoms assessed using ISAAC questionnaire by parent	High	Low	High	4	Kiraly, 2013 [35]
Caregiver report using diary or other memory aid, and active surveillance via daily or weekly telephone calls	High	Moderate	Moderate	4	Esposito 2003 [36]
Daily active surveillance for respiratory symptoms including wheeze by telephone by study staff	High	Moderate	Moderate	4	Anderson, 1992 [14]
Caregiver report using a diary or other memory aid	Moderate	Low	Low	1	Rose, 2010 [37]
Caregiver recording of any serious adverse events and passive surveillance	Low	Low	Low	0	Tam, 2007 [15]

Scoring: High=2, Moderate=1, Low=0. Add scores across all three components of diagnostic certainty. Higher scores= higher diagnostic certainty. All scores are for assessing any wheeze. Scores assumed healthcare workers are well trained.

Severity Assessment of Wheeze

Only 28% of studies described the severity of respiratory system findings and even fewer studies (7%) documented severity assessment specific to wheeze outcomes. While all studies among older populations (>2 years old) included descriptions of severity assessment specific to wheeze, wheeze-specific severity assessments were only provided in 25% of trials that included children <2 years (Supplementary Table 5). Descriptions of severity included frequency of wheeze episodes, features of respiratory distress including: tachypnea, lower chest wall indrawing, grunting, inability to breastfeed/drink/talk; hypoxia evidenced by clinical cyanosis, pulse oximetry or blood gas analysis; hospitalization: seeking an unscheduled hospital visit, “medically attended” or life threatening; and those associated with death (Figure 2 and Supplementary Table 6). In addition, when the DAIDS severity grading was applied to studies reporting wheeze severity, trials that excluded children <2 years more frequently included wheeze descriptions that allowed detection of all 4 grades of severity (75%), as compared to studies that included children <2 years (0%).

Diagnostic Certainty

Table 2 presents our ranking of diagnostic certainty based on sensitivity, specificity and reproducibility of different wheeze assessments. The highest scoring wheeze classification used a combination of parental report of any respiratory illness (using a diary or other memory aid), active surveillance, (examination by a clinician) and verification (using a computer assisted technique, a second clinician or an algorithmic approach). No studies included in this systematic review used a combination of all three components (report, active surveillance and verification). Six additional wheeze definitions were given a diagnostic certainty score of 5, including the use of daily clinical observation [13] and daily active telephone surveillance. [14]. The lowest ranked definition was the combination of parental recording of any serious adverse event and passive surveillance [15].

Discussion

This systematic review of existing wheeze definitions and severity grading scales in pediatric vaccine and drug intervention trials including children < 5 years demonstrates several important gaps in the current literature. First, wheeze as an adverse event has been almost exclusively assessed in trials from high resource settings. In addition, a standardized definition and assessment of wheeze is lacking. Finally, wheeze severity as a proxy for respiratory compromise is inconsistently assessed in spite of the availability of a commonly used severity grading system.

Definition

The heterogeneity in wheeze definitions identified in this systematic review may reflect the different etiologies of wheeze. For this review, we were specifically interested in acute onset diffuse airway narrowing consistent with bronchiolitis, asthma or reactive airway disease. However, wheeze is non-specific and may be due to other acute pathology, including the presence of foreign bodies, mucous plugs, and pulmonary edema, as well as chronic obstructive airway disease. The diverse definitions also reflect the wide spectrum of clinical presentation of obstructive airway disease. Timing within the context of the definition of wheeze as an adverse event in the pediatric drug and vaccine trials varied widely from minutes, days, weeks, months to years; even within same type of intervention as shown in the influenza vaccine trials. This raises the question on what the optimal timing of assessing wheeze as an adverse event is, which may depend on the etiology. In future clinical trials, it may also be important to come to a consensus on what the best way of framing (for example acute or delayed wheeze) and further defining timing in relation to wheeze definitions.

Severity

Few studies in this review described an assessment of wheeze severity and there was wide variability in how wheeze severity is captured and described. Classifications provided were diverse, including components of the severity classification of pneumonia by WHO [16] and asthma by the Global Initiative for Asthma (GINA) [17]. When the DAIDS classification for respiratory adverse events in children less than 14 years was applied, very few studies were able to detect all 4 grades of respiratory distress severity. Definitions of wheeze as an adverse event benefit from the development of a clear and consistent severity grading reflecting degrees of respiratory compromise.

Although we used DAIDS grading in determining the adequacy of individual study wheeze severity assessment, this classification system may not be ideal for grading severity of wheeze in trials, given the potential for differential adverse events in children, particularly those <2 years. In addition, DAIDS grade 3 includes inability to perform usual social activities, which may require more specific details for children, such as the inability to drink/breastfeed. Of note, recently completed LAIV trials [9,11,18] defined wheezing illness in their protocols and included a spectrum of severity ranging from mild to life threatening.

Another challenge for defining and grading diseases with reactive airways is that wheeze may be absent in the most severe presentations due to airflow limitation. Despite these challenges, wheeze is an important sign of respiratory illness, has been associated with certain vaccine receipt in the past, and suitable for vaccine and drug safety monitoring. This systematic review highlights the need for a consistent, easy to implement, accurate definition that is correlated with clinical illness to be used in pediatric trials.

Diagnostic certainty score

The diagnostic certainty score developed for this study facilitated assessment of available definitions and scoring systems (Table 4) and may be useful to guide the development of a standardized case definition. Wheeze in articles included in this review was assessed by caregivers and/or healthcare workers. Caregiver report of respiratory symptoms or abnormalities may be more sensitive when compared to health worker assessment, as it captures wheeze and non-wheeze signs, but may lack specificity [19–21]. Respiratory questionnaires and the use of videos have been validated to improve the accuracy and reliability of caregiver-reported wheeze [22,23]. In contrast, we expect that wheeze identified on examination by healthcare workers will be more specific than caregiver assessment, but may miss community cases and have decreased sensitivity. Data on the health worker assessment of wheeze in children from observational studies in reference to a health worker of a higher level of training or computer assisted techniques is also conflicting[24,25]. Health worker training and formal standardization of assessments of dyspneic wheezing children [26] and verification by multiple healthcare workers may improve diagnostic accuracy and inter-observer reliability. Modern computer assisted techniques may also help assess wheeze and provide objectivity currently lacking in existing methods [27,28]. Conversely, an objective approach to wheeze assessment may require a simple algorithmic approach to lung sounds in children regardless of terminology[29,30].

Our results suggest that where resources permit, a three-stage assessment of wheeze that includes detection of acute respiratory illness by caregivers, confirmation of wheeze upon clinical examination by healthcare workers, and wheeze validation using specific tools or additional confirmation by another health worker optimizes sensitivity and specificity of wheezing illness as an adverse event in pediatric trials.

While this review had several strengths, including the use of a broad search strategy to capture relevant studies documenting wheeze definitions and severity grading, there were also some limitations. Only English language publications were assessed, which may have contributed to some language bias. Trials involving drugs that were not indexed as therapeutic use, therapy or treatment outcome may have been omitted. Although included studies may not have reported details of wheeze definitions in the manuscripts, they may have been available to the investigators, potentially leading to misclassification of wheeze definitions and severity assessment in our report.

Recommendations

A standard definition of wheeze that includes a severity scale of respiratory compromise would be useful for clinical trials and post-licensure vaccine and drug safety surveillance of pediatric populations. Wheeze may indicate the presence of serious illness and should be detected early in clinical trials. With new drugs and vaccines in the pipeline targeted for use in low-income countries, there is a need for a standardized, consensus definition that can be easily operationalized. In addition, we recommend that any wheeze definition should leverage a three-stage assessment design to improve sensitivity, specificity and inter-observer agreement. Including caregiver report of wheeze and other symptoms of respiratory illness and active surveillance with health care worker clinical evaluation and verification of clinician diagnosed wheeze would likely optimize both sensitivity and specificity, leading to higher diagnostic certainty. Finally, wheeze assessment should be implemented by trained personnel. Wheeze can be difficult to differentiate from other respiratory signs and symptoms, particularly in children. Well-trained caregivers and staff, conversant in identifying wheeze in children will be needed to accurately detect wheeze in the context of a clinical trial. We believe that training health workers may improve inter-observer agreement and provide consistency in case ascertainment and data collection across sites and studies.

Figure 3. Frequency Chart of Severity Components of Wheeze Definitions.

Supplementary Table 4A studies: An explicit wheeze definition is provided

Supplementary Table 4B studies: Wheeze is described as part of a respiratory illness without an explicit definition

Supplementary Table 4C studies: Wheeze equivalents (e.g. asthma, bronchiolitis) descriptions are provided