Abstract
Objective
This study aimed to validate the Erasmus Guillain–Barré Syndrome Respiratory Insufficiency Score in the International Guillain–Barré Syndrome Outcome Study cohort, and to improve its performance and region‐specificity.
Methods
We examined data from the first 1,500 included patients, aged ≥6 years and not ventilated prior to study entry. Patients with a clinical variant or mild symptoms were also included. Outcome was mechanical ventilation within the first week from study entry. Model performance was assessed regarding the discriminative ability (area under the receiver operating characteristic curve) and the calibration (observed vs predicted probability of mechanical ventilation), in the full cohort and in Europe/North America and Asia separately. We recalibrated the model to improve its performance and region‐specificity.
Results
In the group of 1,023 eligible patients (Europe/North America n = 842, Asia n = 104, other n = 77), 104 (10%) required mechanical ventilation within the first week from study entry. Area under the curve values were ≥0.80 for all validation subgroups. Mean observed proportions of mechanical ventilation were lower than predicted risks: full cohort 10% versus 21%, Europe/North America 9% versus 21%, and Asia 17% versus 23%. After recalibration, predicted risks for the full cohort and Europe/North America corresponded to observed proportions.
Interpretation
This prospective, international cohort study validated the Erasmus Guillain–Barré Syndrome Respiratory Insufficiency Score, and showed that the model can be used in the full spectrum of Guillain–Barré syndrome patients. In addition, a more accurate, region‐specific version of the model was developed for patients from Europe/North America. ANN NEUROL 2022;91:521–531
Guillain–Barré syndrome (GBS) is a postinfectious inflammatory disease of the peripheral nervous system that is frequently complicated by respiratory insufficiency. Approximately 10–30% of all patients with GBS require mechanical ventilation during the disease course. 1 Respiratory failure in GBS often develops insidiously, without traditional signs of respiratory compromise. Delayed intubation may lead to aspiration and a subsequent increased risk of pneumonia, which is associated with a worse outcome. 2 , 3 Early prediction of respiratory insufficiency in GBS patients is important to correctly triage patients to the appropriate level of care (ie, general ward, high care unit, or intensive care unit [ICU]) and to prevent complications associated with delayed intubation. Previous studies identified various risk factors for respiratory insufficiency in GBS, including factors related to the disease progression rate, severity of muscle weakness, nerve conduction study (NCS) parameters, respiratory function tests, infection serology, liver enzymes, and antiganglioside antibodies. 2 , 4 , 5 , 6 , 7 , 8 , 9 , 10 , 11 , 12 The Erasmus Guillain–Barré Syndrome Respiratory Insufficiency Score (EGRIS) is a prediction model that estimates the risk of respiratory failure—defined by the need for mechanical ventilation within the first week from hospital admission—in individual patients with GBS. 5 EGRIS predictions are based on 3 clinical factors that are determined at hospital admission: the time from onset of weakness to admission, presence of facial and/or bulbar weakness, and the severity of muscle weakness defined by the Medical Research Council (MRC) sum score (Table 1). The EGRIS total score ranges from 0 to 7, which corresponds to an estimated risk of respiratory failure within the first week ranging from 1 to 90%. Results from previous single country studies already showed differences in the clinical presentation, disease course, subtypes, and outcome of GBS among countries. 13 , 14 , 15 , 16 , 17 This regional variation was recently confirmed by our study describing the first 1,000 patients included in the International Guillain–Barré Syndrome Outcome Study (IGOS). 18 The EGRIS has been developed with data from a Dutch GBS cohort, but is currently used in GBS patients from all around the world. 5 Until now, validation has only been performed in 2 smaller Asian cohorts. 19 , 20 Therefore, this study aimed to validate the EGRIS in the IGOS cohort to define its performance in an international GBS population. The second aim was to further improve model performance by applying region‐specific adjustments to the EGRIS.
TABLE 1.
EGRIS Scoring System 5
| Predictor | Categories | Score |
|---|---|---|
| Time from onset of weakness to hospital admission, days | >7 | 0 |
| 4–7 | 1 | |
| ≤3 | 2 | |
| Facial and/or bulbar weakness at hospital admission | Absent | 0 |
| Present | 1 | |
| MRC sum score at hospital admission | 51–60 | 0 |
| 41–50 | 1 | |
| 31–40 | 2 | |
| 21–30 | 3 | |
| ≤20 | 4 | |
| EGRIS total score | 0–7 |
EGRIS = Erasmus Guillain–Barré Syndrome Respiratory Insufficiency Score; MRC = Medical Research Council.
Patients and Methods
Dataset for External Validation
For this external validation study, we used data from the first 1,500 patients included in IGOS, an ongoing prospective multicenter cohort study on GBS, in which all variants and subtypes of GBS are represented. 21 Patients were enrolled between May 2012 and April 2017 in 155 hospitals from 19 countries: Argentina, Australia, Bangladesh, Belgium, Canada, China, Denmark, France, Germany, Greece, Italy, Japan, Malaysia, the Netherlands, South Africa, Spain, Taiwan, United Kingdom, and USA. IGOS was approved by the review board of the Erasmus University Medical Center, Rotterdam, the Netherlands (MEC‐2011‐477), and the local institutional review boards of participating hospitals or universities. Written informed consent was obtained from all patients or their legal representatives.
For validation of the EGRIS, we included all patients with GBS or its variants who had been enrolled in IGOS within 2 weeks from the onset of weakness. 22 , 23 Patients in whom the diagnosis was altered during the 1‐ to 3‐year follow‐up were excluded. We also excluded patients under 6 years, because the MRC scores cannot be assessed reliably in young children, and patients from Bangladesh, as most of these patients do not receive specific immunotherapy and facilities for supportive care (including ventilatory support) are limited in Bangladesh. Finally, we excluded patients who were admitted to the hospital before the onset of weakness and patients who were ventilated prior to study entry. Patients in whom mechanical ventilation was started on the same day as the entry assessment were retained in the analysis.
Statistical Analysis
Predictive Performance
Because study entry is the first data collection time point in IGOS, we used "MRC sum score at entry" and "facial and/or bulbar weakness at entry" to calculate the EGRIS score, and defined outcome as "the need for mechanical ventilation within the first week from study entry." Some patients were first admitted to another hospital before they were transferred to an IGOS‐participating center. For these patients, we used the date of the first hospital admission to define the time from onset of weakness to admission. We assessed model performance by determining the discrimination and calibration. Discrimination is the ability of the model to distinguish between patients who need and do not need mechanical ventilation and is quantified by the area under the receiver operating characteristic (ROC) curve. The ROC curve provides the sensitivity (ie, true‐positive rate) of a model at different probability thresholds plotted against 1 − specificity (ie, false‐positive rate). The area under the ROC curve (AUC) ranges from 0.5 (discriminative ability equal to flipping a coin) to 1 (perfect discrimination), and represents the probability that in a random pair of patients, one who was ventilated and one who was not ventilated, the EGRIS is higher in the patient who was ventilated. We calculated 2 types of AUC values: the “external validation AUC” and the “refitted AUC.” The external validation AUC defines the discriminative ability of the original EGRIS model (with its original regression coefficients) in the IGOS cohort. This external validation AUC was compared with the AUC value in the EGRIS development cohort. A similar AUC value, or a minimal change as compared to the development AUC, would indicate that the original EGRIS model can also be applied to a more diverse cohort of GBS patients. The refitted AUC provides the discriminative ability of the EGRIS model with re‐estimated odds ratios based on the IGOS data. This measure provides the optimum discriminative ability that can be obtained with a model with these 3 clinical factors in the IGOS cohort. Calibration defines the accuracy of the model predictions by comparing the predicted probabilities with the observed frequencies of mechanical ventilation. Calibration curves were generated to graphically delineate the correspondence between the observed and predicted risks. In the case of perfect calibration, the curve would rest on the 45° diagonal, indicating that observed frequencies of mechanical ventilation are equal to predicted risks. 24 , 25
We determined model performance in the total group and in regional subgroups: Europe/North America (Eu/NA; including United Kingdom) and Asia, and compared this with model performance in the EGRIS development cohort. The subdivision into different regions was based on previously identified differences in the clinical presentation, disease course, and subtypes of GBS between various regions. 18 We compared the study design and patient characteristics of the development and validation cohorts, to explain potential differences in model performance. For external validation, we used the original regression formula with the EGRIS total score as a single predictor. We also assessed the predictive ability of the individual factors included in the EGRIS model and compared these between the development and regional validation cohorts.
Model Recalibration
To improve the accuracy of the model predictions (ie, the correspondence between the predicted values and those observed in the validation cohorts), we recalibrated the EGRIS model. With recalibration, systematic errors in model predictions can be corrected. For example, if predicted probabilities are systematically too low in the validation cohort, then recalibration increases all predicted probabilities. We used the “closed testing procedure” described in the paper by Vergouwe et al 26 to define the extent of updating that was required for the EGRIS model. This procedure compares 4 levels of updating, ranging from (1) no updating (ie, keeping the original model) to (4) full model revision (ie, re‐estimating all model coefficients), to identify the optimal updating method for the validation sample. The closed testing procedure was applied to the first imputation set, and showed that full revision of the model with re‐estimation of all regression coefficients did not significantly improve model performance. For recalibration of the EGRIS in this study, we applied correction factors to the original regression formula (intercept and coefficients), which is used to calculate the predicted probabilities. We corrected the regression formula that contained the EGRIS total score as single predictor. As per the closed testing procedure, we did not separately correct the coefficients of the individual factors included in the EGRIS total score, so their relative contribution to the score has remained the same. Therefore, this recalibration method only corrects the overall predicted probabilities, but does not change the discriminative ability. Average correction factors from the 10 imputation sets were used to recalibrate the model. 24 , 27 We used bootstrapping (with n = 500 bootstrap samples) to internally validate the recalibrated EGRIS model, using the validate function from the rms package in R. This bootstrapping procedure rederives the recalibrated EGRIS in each of the bootstrap samples and calculates the AUC value in the original dataset. The average AUC value from the models derived in the n = 500 bootstrap samples is compared to the AUC value of our recalibrated model to define the level of overfitting.
Missing Values
We used multiple imputation (n = 10) to impute missing values for the EGRIS predictors (R function: aregImpute). Calibration curves were based on data from the first imputation set. Data were analyzed using SPSS Statistics version 24 and R Studio version 3.6.1. (R packages: Hmisc, rms, devtools, CalibrationCurves).
Results
From the IGOS‐1500 cohort, we excluded patients with an alternative diagnosis (n = 85, 6%; of whom 53 had chronic inflammatory demyelinating polyneuropathy), patients with a protocol violation (n = 34, 2%), and patients for whom no data were entered at all (n = 7, 0.5%). From the remaining cohort of 1,374 patients, we excluded the Bangladeshi patients (n = 203, 15%) and patients aged <6 years or with missing age (n = 44, 3%). Of the remaining 1,133 patients, 52 patients (5%) were ventilated prior to study entry, 52 (5%) patients were admitted to the hospital before the onset of weakness, 7 patients (0.6%) had missing values for the date of onset of weakness or the date of hospital admission, and 5 patients (0.4%) had a missing start date of mechanical ventilation. All these patients were also excluded. For validation of the EGRIS, 1,023 patients remained in the analysis (Fig 1), of whom 121 (12%) required mechanical ventilation at some point during follow‐up (Table 2). Patients were included in the following countries: Argentina (n = 40), Australia (n = 9), Belgium (n = 19), Canada (n = 22), China (n = 12), Denmark (n = 104), France (n = 29), Germany (n = 50), Greece (n = 12), Italy (n = 114), Japan (n = 62), Malaysia (n = 25), the Netherlands (n = 112), South Africa (n = 28), Spain (n = 96), Taiwan (n = 5), United Kingdom (n = 139), and USA (n = 145). In total, 0.6% of the data points (126/20,610) were missing for the EGRIS predictors, which were imputed by multiple imputation.
FIGURE 1.
Study population. The sum of the exclusions in the second and third box is higher than the total number of exclusions at the corresponding step because of overlap in patient characteristics; that is, 6 patients with age < 6 years were included in Bangladesh, 5 patients who were ventilated prior to study entry were also admitted before the onset of weakness, and 1 patient with missing start date of mechanical ventilation was also admitted before the onset of weakness. EGRIS = Erasmus Guillain–Barré Syndrome Respiratory Insufficiency Score; Eu/NA = Europe/North America; IGOS = International Guillain–Barré Syndrome Outcome Study; val. = validation.
TABLE 2.
Characteristics of the Patients in the EGRIS Development and IGOS Validation Cohorts
| Predictors and Outcome | IGOS Validation Full Cohort, n = 1,023 | Development Cohort, 5 n = 565 |
|---|---|---|
| Age, years | 53 (39–66) | NA a |
| Time onset weakness > 7 days | 107 (11%) | 157 (28%) |
| Time to hospital admission = 4–7 days | 280 (27%) | 219 (39%) |
| Time to hospital admission ≤ 3 days | 636 (62%) | 189 (34%) |
| MRC sum score at entry | ||
| 51–60 | 454/1,017 (45%) | 127 (23%) |
| 41–50 | 329/1,017 (32%) | 250 (44%) |
| 31–40 | 126/1,017 (12%) | 106 (19%) |
| 21–30 | 57/1,017 (6%) | 53 (9%) |
| ≤ 20 | 51/1,017 (5%) | 29 (5%) |
| Facial and/or bulbar weakness at entry | 379/1,022 (37%) | 170 (30%) |
| GBS disability score at entry | ||
| ≤2 | 301/1,016 (30%) | 33 (6%) |
| >2 | 715/1,016 (70%) | 532 (94%) |
| GBS variant | ||
| Sensorimotor | 641/973 (66%) b | NA |
| Pure motor | 146/973 (15%) b | NA |
| MFS | 81/973 (8%) b | 18 (3%) |
| MFS‐GBS overlap | 57/973 (6%) b | NA |
| Other | 48/973 (5%) b | NA |
| MV during follow‐up | 121 (12%) | 128 (23%) |
| MV within the first week of admission | 104 (10%) | 110 (20%) |
| IVIg/PE | 931 (91%) | 95% c |
This table provides an overview of the characteristics of the patients in the EGRIS development cohort and the IGOS validation dataset. Numbers are provided as median (IQR) or n (%), unless stated otherwise.
The EGRIS development cohort contained data from 5 different studies. The median age of the patients was derived from the separate articles describing these studies: (1) study 1–3, median age (IQR) in years: 52 (33–66), 27 , 28 , 30 ; (2) study 4: median age (95% confidence interval) in years: 46 (23–76) 29 ; (3) study 5: median age (IQR) in years: 50 (35–63). 31
For the IGOS validation cohort, we used GBS variants at visit week 2 as classified by the local treating neurologist. If the week 2 variant was missing, we used the variant at week 1 or study entry. Other GBS variants include the pharyngeal–cervical–brachial variant, pure sensory GBS, ataxic variant, and Bickerstaff brainstem encephalitis.
This proportion was deduced from the separate articles describing the 5 studies that were included in the EGRIS development cohort. This number provides an approximation of the proportion of patients who were treated in the development cohort, as the exact numbers could not be retrieved.
EGRIS = Erasmus Guillain–Barré Syndrome Respiratory Insufficiency Score; GBS = Guillain–Barré syndrome; IGOS = International Guillain–Barré Syndrome Outcome Study; IQR = interquartile range; IVIg = intravenous immunoglobulin; MFS = Miller Fisher syndrome; MRC = Medical Research Council; MV = mechanical ventilation; NA = not applicable/available; PE = plasma exchange.
Characteristics of the EGRIS Development Cohort and IGOS Validation Cohorts
The characteristics of the EGRIS development cohort and the IGOS validation cohort are provided in Table 2 and Supplementary Tables S1 and S2. The EGRIS development cohort contained data from 5 different studies, including 2 randomized controlled trials, 28 , 29 2 pilot studies, 30 , 31 and 1 observational study. 32 Most of the patients in the development cohort were included in Dutch centers, although a minority were included in Germany or Belgium. Two‐thirds of the IGOS patients were admitted to the hospital within 3 days from the onset of weakness, as compared to one‐third in the EGRIS development cohort. The proportion of severely affected patients (as indicated by the inability to walk unaided at study entry) was 94% in the EGRIS development cohort and 70% in the IGOS validation cohort. The IGOS validation cohort included data on the full spectrum of GBS clinical variants, whereas variants were excluded from the EGRIS development cohort, except for 18 patients with Miller Fisher syndrome (MFS). In the IGOS cohort, 121 (12%) patients required mechanical ventilation at some point during follow‐up, and the time to start of ventilation ranged from 0 to 33 days. Ten percent of the IGOS patients already required mechanical ventilation within the first week from study entry, versus 20% in the EGRIS development cohort (see Table 2, Supplementary Table S1).
Discriminative Ability
Validation of the original EGRIS model in the IGOS cohort showed an AUC value (95% confidence interval [CI]) of 0.86 (0.80–0.91) in the full IGOS cohort, 0.86 (0.80–0.93) in the Eu/NA subgroup, and 0.80 (0.62–0.91) in Asia. The external validation AUC values were comparable to the development AUC of 0.84 (Fig 2). Refitted AUC values for the full cohort and Eu/NA subgroup were similar to the AUC values that were derived upon external validation of the original model. For the Asian cohort, the refitted AUC value (95% CI) was slightly higher than the external validation AUC: 0.86 (0.72–0.93) versus 0.80 (0.62–0.91; see Fig 2). We also assessed the predictive ability of each of the individual factors included in the EGRIS model (Table 3). The predictive ability of the MRC sum score and facial and/or bulbar weakness was similar between the EGRIS development and IGOS validation cohorts. Disease progression rate (ie, the time in days between the onset of weakness and hospital admission) was a strong predictor in the EGRIS development cohort, but odds ratios were not significant for the full IGOS cohort and Eu/NA subgroup (see Table 3). Because of the small sample size of the Asian cohort (especially the small number of events; only 18 patients needed mechanical ventilation within the first week), we could not determine the predictive ability of the individual factors in this subgroup.
FIGURE 2.
Discrimination upon external validation. The area under the receiver operating characteristic curve (AUC) value is a measure for the discriminative ability of a prediction model, ranging from 0.5 (flipping a coin) to 1.0 (perfect discrimination). For the Erasmus Guillain–Barré Syndrome Respiratory Insufficiency Score (EGRIS), this represents the ability of the model to distinguish between patients who need and do not need mechanical ventilation. The external (Ext.) validation AUC = the discriminative ability of the original EGRIS model in the International Guillain–Barré Syndrome Outcome Study (IGOS) cohort. Refitted AUC = the discriminative ability of the model after refitting, in other words, re‐estimation of the odds ratio based on the IGOS data. The refitted AUC provides the optimum discriminative ability that can be obtained with these 3 clinical factors in the IGOS dataset. The dotted line represents the AUC value in the EGRIS development cohort. CI = confidence interval; Eu/NA = Europe/North America.
TABLE 3.
Effects of the Individual Predictors Included in the EGRIS Model
| Predictor | Validation, OR (95% CI) | Development, OR (95% CI) | |
|---|---|---|---|
| Full Cohort | Eu/NA | ||
| Time from onset of weakness to hospital admission (days) | |||
| >7 | Ref | Ref | Ref |
| 4–7 | 0.5 (0.1–1.9) | 0.3 (0.1–1.6) | 2.6 (1.2–5.7) |
| ≤3 | 2.8 (0.9–8.1) | 2.3 (0.7–8.0) | 7.6 (3.5–16.6) |
| Facial and/or bulbar weakness at admission a | |||
| Absent | Ref | Ref | Ref |
| Present | 4.6 (2.8–7.4) | 3.5 (2.0–6.0) | 3.5 (2.1–6.0) |
| MRC sum score at admission a | |||
| 51–60 | Ref | Ref | Ref |
| 41–50 | 3.9 (1.9–8.4) | 5.0 (2.0–12.7) | 3.8 (1.4–10.4) |
| 31–40 | 9.1 (4.0–20.8) | 12.7 (4.6–34.7) | 8.0 (2.8–22.6) |
| 21–30 | 22.3 (9.4–53.0) | 32.7 (11.5–93.1) | 27.1 (9.0–81.6) |
| ≤20 | 30.9 (12.8–74.4) | 35.9 (12.5–102.8) | 40.5 (11.7–139.4) |
Values at study entry in the IGOS validation cohorts.
CI = confidence interval; EGRIS = Erasmus Guillain–Barré Syndrome Respiratory Insufficiency Score; Eu/NA = Europe/North America; MRC = Medical Research Council; OR = odds ratio; Ref = reference.
Calibration
In all 3 validation cohorts, the observed proportion of patients who needed mechanical ventilation within the first week from study entry was lower than the predicted risk based on the EGRIS model (Figs 3 and 4). After adjustment of the original regression formula (intercept and coefficient)—the updating approach that was most appropriate based on the closed testing procedure—the correspondence between the predicted probabilities and observed frequencies improved for the full cohort and Eu/NA subgroup (see Fig 4). Due to the small sample size and wide 95% CI around the calibration curve for the Asian cohort, it was not possible to recalibrate the model for this subgroup. Internal validation of the recalibrated EGRIS for European and North American patients (EGRIS‐Eu/NA) by bootstrapping showed an AUC of 0.862, indicating that there was no overfitting.
FIGURE 3.
Observed probabilities versus predicted risks. Mean observed proportions of mechanical ventilation (MV) within 1 week in the International Guillain–Barré Syndrome Outcome Study validation cohorts versus predicted risks based on the Erasmus Guillain–Barré Syndrome Respiratory Insufficiency Score model are shown. Eu/NA = Europe/North America.
FIGURE 4.
Calibration curves: original and after recalibration. This figure provides the calibration curves for the original (left) and recalibrated (right) Erasmus Guillain–Barré Syndrome Respiratory Insufficiency Score (EGRIS) model, for the full International Guillain–Barré Syndrome Outcome Study cohort, Europe/North America (Europe/North America), and Asia. Observed probabilities of mechanical ventilation (y‐axis) are plotted against predicted risks based on the EGRIS model (x‐axis). The dotted lines represent perfect calibration (ie, predicted risks are equal to observed frequencies). The gray‐shaded areas are 95% confidence intervals around the calibration curves. NA = not applicable.
Discussion
This study validated the EGRIS in a GBS cohort with patients from 18 countries, including all disease severities and GBS clinical variants. The model was able to distinguish between patients at high and low risk for mechanical ventilation as indicated by the high AUC values (≥0.8). In all regions, the risk of mechanical ventilation was overestimated by the EGRIS, that is, the predicted probabilities were higher than the observed proportions of mechanical ventilation. Recalibration improved the correspondence between the predicted and observed risks, and enabled us to develop a more accurate, region‐specific version for patients from Europe and North America (EGRIS‐Eu/NA).
Our findings are in line with previous studies that validated the EGRIS in Japan and Malaysia. 19 , 20 Both studies assessed the discriminative ability of the model by comparing EGRIS scores between patients who did and did not require mechanical ventilation within the first week of admission. EGRIS scores were significantly higher for patients who required mechanical ventilation. The study by Tan et al also provided an AUC value for the group of severely affected (GBS disability score ≥ 3) GBS patients (without MFS), which was similar to the AUC value in our Asian cohort (0.786). 19 , 20 Model calibration was not described in these studies but could be deduced from the reported results. In both studies, the risk of mechanical ventilation was underestimated by the EGRIS model (Yamagishi et al: predicted probability 13%, observed 17%; Tan et al: predicted probability 23%, observed 44%). These results confirm that the EGRIS can be used in Asia to identify GBS patients at high risk for developing respiratory failure, as indicated by the high AUC values. Model calibration in Asia varies between studies, which may be explained by differences in the clinical settings and selection of patients. Assessment of model performance in a larger Asian cohort may provide a better estimate of model calibration in Asian GBS patients, and will enable the development of a region‐specific version. Until that time, we recommend using the original, validated EGRIS in Asia, but want to emphasize that attention should be paid to differences between predicted and observed outcomes when the EGRIS is applied in clinical practice, especially in situations where specific cutoffs for predicted probabilities are used to guide decision‐making.
In the current study, only 10% of the patients required mechanical ventilation within the first week (and 12% during overall follow‐up), which is lower than reported in most previous studies. This low frequency is in part explained by the selection of a specific subgroup of GBS patients for this validation study, as in the cohort including the Bangladeshi patients and patients ventilated prior to study entry (n = 1,034), the proportion requiring ventilation was 16% within the first week (and 18% overall). Another possible explanation is the study design of IGOS, which allowed the inclusion of all patients with GBS, including milder or variant forms, in contrast to previous studies investigating cohorts from trials or admitted to the ICU. This also was illustrated by a recent meta‐analysis of 34 studies on respiratory insufficiency in GBS, which included data from both observational studies and trials in severely affected patients, and showed that the prevalence of mechanical ventilation varied from 7% to 65%. 1 In addition, when we focused on the IGOS patients who were admitted to the ICU (n = 222, 22%), we found that 101 (45%) of these patients required ventilation within the first week.
The EGRIS model systematically overestimated the risk of respiratory insufficiency, which may be explained by various factors. First, the EGRIS was developed in a cohort of patients with mostly severe forms of GBS and high risks of respiratory failure as compared to the validation cohort. The original EGRIS was probably influenced by this higher a priori risk of respiratory failure in the development cohort, even though the model includes predictors related to disease severity. Second, most patients in the EGRIS development cohort participated in trials and probably have been monitored and treated more strictly than the patients in the validation cohort, which was based on observational data. In addition, the guidelines for monitoring and start of ventilation may differ between countries. These differences in monitoring and treatment protocols also may have influenced the decision to start ventilation. Third, there is a marked regional variation of GBS. Several factors previously have been associated with the risk of respiratory failure in GBS, and their occurrence may differ between the development and validation cohort. Examples include the type of preceding infection, the NCS subtype, and the target of the immune response. 8 , 10 , 11 , 12 Because these factors were not tested in both the development and validation cohort, their prognostic value will need to be defined in future studies. When we assessed the effect of the individual predictors included in the EGRIS model, we found that the time from onset of weakness to hospital admission was not significantly associated with the risk of mechanical ventilation in the IGOS cohort. This finding is explained by the categories that were used for this variable (≤3 days, 4–7 days, >7 days), because when we included time to admission as a continuous variable (instead of a categorical variable), in a regression model with the same 3 predictors, we did find a significant effect in the IGOS cohort. Nonetheless, the discriminative ability of the model in the IGOS cohort did not change by including time to admission as either a continuous or a categorical variable, and therefore we kept the categories as originally specified for the EGRIS model.
How can these results be applied in clinical practice? The validated EGRIS can be applied in all adult patients with GBS, including mild cases and clinical variants. At hospital admission, the EGRIS scoring system (see Table 1) can be used to calculate the EGRIS based on the time from onset of weakness to hospital admission, the presence of facial and/or bulbar weakness, and the severity of limb weakness as defined by the MRC sum score. The predicted probability of mechanical ventilation for an individual patient with GBS can be determined based on the calculated EGRIS (Fig 5). To predict the risk of respiratory insufficiency for GBS patients from Europe and North America, we recommend using the recalibrated EGRIS (EGRIS‐Eu/NA). For patients from other regions (including Asia), we recommend using the original EGRIS that was validated in the current study. The EGRIS is also available as an online tool that can be accessed at https://gbstools.erasmusmc.nl/prognosis-tool/0/0. The predicted probabilities of respiratory failure that are provided by this online tool are now based on the original EGRIS, but we will update this tool based on the results of this study. In practice, clinicians can use the EGRIS to early identify GBS patients at highest risk of developing respiratory insufficiency within the first week of admission, to provide them with the appropriate level of care and prevent complications from delayed or emergency intubation. Without the EGRIS model, clinicians would only be able to provide general information on the risk of respiratory insufficiency based on reported prevalences from large population studies. In contrast, by using the EGRIS, the risk of respiratory insufficiency can be further stratified for individual patients based on clinical information that can be easily obtained at hospital admission.
FIGURE 5.
Predicted probabilities of mechanical ventilation within 1 week according to the recalibrated Erasmus Guillain–Barré Syndrome Respiratory Insufficiency Score (EGRIS) Europe/North America (Eu/NA) model. This figure provides the predicted probabilities of the need for mechanical ventilation within the first week from hospital admission based on the EGRIS (scores 0–7). Probability graphs are based on the original EGRIS model (red line) and the recalibrated model for the Eu/NA subgroup (EGRIS‐Eu/NA; green line). Dashed and gray areas around the curves represent the 95% confidence intervals. The EGRIS model can be applied to all patients with Guillain–Barré syndrome (GBS), including mild cases (GBS disability score ≤ 2) and GBS variants. The EGRIS total score can be calculated based on the scoring system provided in Table 1. With the EGRIS total score and the probability graphs provided above, one can deduce the predicted probability of the need for mechanical ventilation for an individual patient with GBS. To predict the need for mechanical ventilation within the first week in Eu/NA GBS patients, the probability graph based on the recalibrated model can be used: EGRIS‐Eu/NA (green line). For predictions in GBS patients from countries outside Europe: North America, the probability graph based on the original validated EGRIS model can be used (red line). EGRIS = Erasmus Guillain–Barré Syndrome Respiratory Insufficiency Score; MV = mechanical ventilation.
This study has several limitations. First, part of the IGOS‐1500 cohort had to be excluded for this validation study because we could not calculate the EGRIS (ie, children <6 years old, patients admitted before the onset of weakness, or patients with missing data for the EGRIS predictors) or because patients were already ventilated before study entry. As MRC scores are difficult to determine in young children, additional studies should be performed to identify alternative predictors that can be used instead of the MRC sum score to predict the risk of respiratory failure in children with GBS. Furthermore, in clinical practice, routine examination does not always include assessment of all individual muscles included in the MRC sum score. Several previous studies have shown an association between weakness in selected proximal muscles and respiratory failure in GBS, 4 , 6 , 33 and further studies should be performed to determine whether the EGRIS could be simplified by the inclusion of individual muscles scores instead of the MRC sum score. Second, when the EGRIS model is applied in practice, it is important to realize that neither the original model nor the recalibrated EGRIS‐Eu/NA provide the “gold standard” for the prediction of respiratory failure in GBS, but model performance may differ depending on the clinical setting and patient population. Therefore, especially in settings where specific cutoff values for predicted probabilities are used to drive decision‐making, it will remain important to pay attention to differences between predicted and observed risks. Validation is a continuous process, and additional studies should be performed to validate the original, but also the recalibrated EGRIS‐Eu/NA in new GBS cohorts.
In conclusion, this study validated the EGRIS in an international GBS cohort, and showed that the model can be applied to the full spectrum of GBS patients. In addition, a region‐specific version was developed for patients from European and North American countries.
Author Contributions
A.Y.D., C.W., H.F.L., B.I., N.P., Y.Y., S.K., M.M.D., W.W., N.K., K.C.G., and B.C.J. contributed to the conception and design of the study, the acquisition and analysis of data, and to drafting the text and preparing the figures. All local investigators in the IGOS Consortium were directly responsible for the extensive data collection in IGOS. To be included in the IGOS Consortium, these local investigators had to enroll a minimum number of patients in the study and had to cooperate in extensive data quality controls. Members of the IGOS Consortium can be found in Supplementary Table S3. Specifically, we would like to acknowledge local investigators who included ≥5 patients in the IGOS‐1500 cohort (investigators 1–65 in Supplementary Table S3).
Potential Conflicts of Interest
Nothing to report.
Supporting information
TABLE S1 Characteristics of the Patients in the EGRIS Development Cohort and Regional Validation Cohorts of IGOS
This table provides an overview of the characteristics of the patients in the Erasmus Guillain–Barré Syndrome Respiratory Insufficiency Score (EGRIS) development cohort and the International Guillain–Barré Syndrome Outcome Study (IGOS) validation datasets from Europe/North America and Asia. Numbers are provided as median (IQR) or n (%), unless stated otherwise.
aThe EGRIS development cohort contained data from 5 different studies. The median age of the patients was derived from the separate articles describing these studies: (1) combined cohort of 3 studies, median age (IQR) in years: 52 (33–66) 28 , 29 , 31 ; (2) median age (95% confidence interval) in years: 46 (23–76) 30 ; (3) median age (IQR) in years: 50 (35–63). 32
bFor the IGOS validation cohort, we used GBS variants at visit week 2 as classified by the local treating neurologist. If the week 2 variant was missing, we used the variant at week 1 or study entry. Other GBS variants include pharyngeal–cervical–brachial variant, pure sensory GBS, ataxic variant, Bickerstaff brainstem encephalitis, et cetera.
cThis proportion was deduced from the separate articles describing the 5 studies that were included in the EGRIS development cohort. This number provides an approximation of the proportion of patients who were treated in the development cohort, as the exact numbers could not be retrieved.
Eu/NA = Europe/North America; GBS = Guillain–Barré syndrome; IQR = interquartile range; MFS = Miller Fisher syndrome; MRC = Medical Research Council; MV = mechanical ventilation; NA = not applicable/available.
TABLE S2. Study Design, Inclusion Criteria, and Exclusion Criteria of the Studies Included in the EGRIS Development Cohort and International Guillain–Barré Syndrome Outcome Study Validation Cohort
*The Erasmus Guillain–Barré Syndrome Respiratory Insufficiency Score (EGRIS) development cohort included data from 5 studies. Four of these 5 studies had similar inclusion and exclusion criteria, and are therefore reported in one column. From studies 1, 2, and 3, 20 patients were excluded because they were ventilated prior to referral to one of the participating hospitals. From studies 4 and 5, 3 patients were excluded because ventilation was started before referral to a trial hospital. The full EGRIS development cohort included 565 patients.
TABLE S3. The IGOS Consortium
*All coauthors mentioned for this paper are also part of the IGOS Consortium, but fulfil additional criteria that justify authorship.
Acknowledgments
The IGOS study is funded by the GBS‐CIDP Foundation International, gain, Erasmus University Medical Center, Glasgow University, CSL Behring, Grifols, Annexon, and Hansa.
We thank the patients who participated in this long‐term follow‐up study, and K. Duong for her extensive work in the data management of the IGOS clinical database.Several authors of this publication are members of the Netherlands Neuromuscular Center and the European Reference Network for Rare Neuromuscular Diseases.
Contributor Information
Bart C. Jacobs, Email: b.jacobs@erasmusmc.nl.
the IGOS Consortium:
Zhahirul Islam, Quazi Deen Mohammad, Thomas Harbo, Soren H. Sindrup, Govindsinh Chavada, Amy Davidson, Hugh J. Willison, Carlos Casasnovas, Kathleen Bateman, James A.L. Miller, Pieter A. van Doorn, Bianca van den Berg, Christine Verboon, Joyce Roodbol, Sonja E. Leonhard, Samuel Arends, Linda Luijten, Luana Benedetti, Satoshi Kuwabara, Peter Van den Bergh, Soledad Monges, Girolama A. Marfia, Nortina Shahrizaila, Giuliana Galassi, Yann Pereon, Jan Bürmann, Krista Kuitwaard, Ruud P. Kleyweg, Cintia Marchesoni, María J. Sedano Tous, Luis Querol, Yuzhong Wang, Eduardo Nobile‐Orazio, Simon Rinaldi, Angelo Schenone, Julio Pardo, Frederique H. Vermeij, Helmar C. Lehmann, Volkan Granit, Guido Cavaletti, Gerardo Gutiérrez‐Gutiérrez, Fabio A. Barroso, Leo H. Visser, Hans D. Katzberg, Efthimios Dardiotis, Shahram Attarian, Anneke J. van der Kooi, Filip Eftimov, Paul W. Wirtz, Johnny P.A. Samijn, H. Jacobus Gilhuis, Robert D.M. Hadden, James K.L. Holt, Kazim A. Sheikh, Summer Karafiath, Michal Vytopil, Giovanni Antonini, Thomas E. Feasby, Catharina G. Faber, J.C.H.M. (Hans) Kramers, Mark Busby, Rhys C. Roberts, Nicholas J. Silvestri, Raffaella Fazio, Gert W. van Dijk, Marcel P.J. Garssen, Jan G.G.M. Verschuuren, R.A.C. Hughes, D.R. Cornblath, H.P. Hartung, L.C. de Koning, M. van Woerkom, M. Mandarakas, R.C. Reisin, S.W. Reddel, P. Ripellino, S.T. Hsieh, J.M. Addington, S. Ajroud‐Driss, H. Andersen, U.A. Badrising, I.R. Bella, T.E. Bertorini, R. Bhavaraju‐Sanka, M. Bianco, P. Brand, T.H. Brannagan, Chiara Briani, S. Butterworth, C.C. Chao, S. Chen, K.G. Claeys, M.E. Conti, J.S. Cosgrove, M.C. Dalakas, M.A. Derejko, C. Dornonville de la Coeur, A. Echaniz‐Laguna, C. Fokke, T. Fujioka, E.A. Fulgenzi, T. García‐Sobrino, J.M. Gilchrist, J.M. Goldstein, N.A. Goyal, Y.A.M. Grimbergen, S. Grisanti, L. Gutmann, I. Illa, J.V. Holbech, M. Htut, K. Jellema, I. Jericó Pascual, M.C. Jimeno‐Montero, K. Kaida, M. Khoshnoodi, K. Kimpinski, M. Kuwahara, L. Kiers, N. Kokubun, J.Y. Kwan, S.S. Ladha, L. Landschoff Lassen, V. Lawson, E. Lee Pan, M.P.T. Lunn, A. Magot, H. Manji, C. Márquez Infante, L. Martín‐Aguilar, E. Martinez Hernandez, G. Mataluni, M.G. Mattiazzi, C.J. McDermott, G.D. Meekins, G. Morís de la Tassa, J. Mozzoni, C. Nascimbene, V. Nedkova, R.J. Nowak, P. Orizaola Balaguer, M. Osei‐Bonsu, R.M. Pascuzzi, V. Prada, M.T. Pulley, I. Rojas‐Marcos, S.A. Rudnicki, G.M. Sachs, M. Samukawa, L. Santoro, L. Schwindling, Y. Sekiguchi, C.L. Sommer, Y. Song, B. Stein, A.M. Stino, C‐Y. Tan, H. Tankisi, P.T. Twydell, P. van Damme, T. van der Ree, R. van Koningsveld, J.D. Varrato, L. Zhou, and S. Zivkovic
References
- 1. Green C, Baker T, Subramaniam A. Predictors of respiratory failure in patients with Guillain‐Barre syndrome: a systematic review and meta‐analysis. Med J Aust 2018;208:181–188. [DOI] [PubMed] [Google Scholar]
- 2. Cheng BC, Chang WN, Chang CS, et al. Predictive factors and long‐term outcome of respiratory failure after Guillain‐Barre syndrome. Am J Med Sci 2004;327:336–340. [DOI] [PubMed] [Google Scholar]
- 3. Orlikowski D, Sharshar T, Porcher R, et al. Prognosis and risk factors of early onset pneumonia in ventilated patients with Guillain‐Barre syndrome. Intensive Care Med 2006;32:1962–1969. [DOI] [PubMed] [Google Scholar]
- 4. Sharshar T, Chevret S, Bourdain F, Raphael JC. French Cooperative Group on Plasma Exchange in Guillain‐Barré Syndrome. Early predictors of mechanical ventilation in Guillain‐Barre syndrome. Crit Care Med 2003;31:278–283. [DOI] [PubMed] [Google Scholar]
- 5. Walgaard C, Lingsma HF, Ruts L, et al. Prediction of respiratory insufficiency in Guillain‐Barre syndrome. Ann Neurol 2010;67:781–787. [DOI] [PubMed] [Google Scholar]
- 6. Kannan Kanikannan MA, Durga P, Venigalla NK, et al. Simple bedside predictors of mechanical ventilation in patients with Guillain‐Barre syndrome. J Crit Care 2014;29:219–223. [DOI] [PubMed] [Google Scholar]
- 7. Wu X, Li C, Zhang B, et al. Predictors for mechanical ventilation and short‐term prognosis in patients with Guillain‐Barre syndrome. Crit Care 2015;19(1):310. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Durand MC, Porcher R, Orlikowski D, et al. Clinical and electrophysiological predictors of respiratory failure in Guillain‐Barre syndrome: a prospective study. Lancet Neurol 2006;5:1021–1028. [DOI] [PubMed] [Google Scholar]
- 9. Lawn ND, Fletcher DD, Henderson RD, Wolter TD, Wijdicks EF. Anticipating mechanical ventilation in Guillain‐Barre syndrome. Arch Neurol 2001;58 (6):893–898. [DOI] [PubMed] [Google Scholar]
- 10. Durand MC, Lofaso F, Lefaucheur JP, et al. Electrophysiology to predict mechanical ventilation in Guillain‐Barre syndrome. Eur J Neurol 2003;10:39–44. [DOI] [PubMed] [Google Scholar]
- 11. Visser LH, van der Meche FG, Meulstee J, et al. Cytomegalovirus infection and Guillain‐Barre syndrome: the clinical, electrophysiologic, and prognostic features. Dutch Guillain‐Barre Study Group. Neurology 1996;47:668–673. [DOI] [PubMed] [Google Scholar]
- 12. Kaida K, Kusunoki S, Kanzaki M, et al. Anti‐GQ1b antibody as a factor predictive of mechanical ventilation in Guillain‐Barre syndrome. Neurology 2004;62:821–824. [DOI] [PubMed] [Google Scholar]
- 13. Bogliun G, Beghi E, Italian GBS Registry Study Group . Incidence and clinical features of acute inflammatory polyradiculoneuropathy in Lombardy, Italy, 1996. Acta Neurol Scand 2004;110:100–106. [DOI] [PubMed] [Google Scholar]
- 14. Islam Z, Jacobs BC, van Belkum A, et al. Axonal variant of Guillain‐Barre syndrome associated with campylobacter infection in Bangladesh. Neurology 2010;74:581–587. [DOI] [PubMed] [Google Scholar]
- 15. Liu S, Xiao Z, Lou M, et al. Guillain‐Barre syndrome in southern China: retrospective analysis of hospitalised patients from 14 provinces in the area south of the Huaihe River. J Neurol Neurosurg Psychiatry 2018;89:618–626. [DOI] [PubMed] [Google Scholar]
- 16. Lyu RK, Tang LM, Cheng SY, et al. Guillain‐Barre syndrome in Taiwan: a clinical study of 167 patients. J Neurol Neurosurg Psychiatry 1997;63:494–500. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17. Mitsui Y, Kusunoki S, Arimura K, et al. A multicentre prospective study of Guillain‐Barre syndrome in Japan: a focus on the incidence of subtypes. J Neurol Neurosurg Psychiatry 2015;86:110–114. [DOI] [PubMed] [Google Scholar]
- 18. Doets AY, Verboon C, van den Berg B, et al. Regional variation of Guillain‐Barre syndrome. Brain 2018;141:2866–2877. [DOI] [PubMed] [Google Scholar]
- 19. Yamagishi Y, Suzuki H, Sonoo M, et al. Markers for Guillain‐Barre syndrome with poor prognosis: a multi‐center study. J Peripher Nerv Syst 2017;22:433–439. [DOI] [PubMed] [Google Scholar]
- 20. Tan CY, Razali SNO, Goh KJ, Shahrizaila N. The utility of Guillain‐Barre syndrome prognostic models in Malaysian patients. J Peripher Nerv Syst 2019;24:168–173. [DOI] [PubMed] [Google Scholar]
- 21. Jacobs BC, van den Berg B, Verboon C, et al. International Guillain‐Barre syndrome outcome study: protocol of a prospective observational cohort study on clinical and biological predictors of disease course and outcome in Guillain‐Barre syndrome. J Peripher Nerv Syst 2017;22:68–76. [DOI] [PubMed] [Google Scholar]
- 22. Asbury AK, Cornblath DR. Assessment of current diagnostic criteria for Guillain‐Barre syndrome. Ann Neurol 1990;27:S21–S24. [DOI] [PubMed] [Google Scholar]
- 23. Sejvar JJ, Kohl KS, Gidudu J, et al. Guillain‐Barre syndrome and Fisher syndrome: case definitions and guidelines for collection, analysis, and presentation of immunization safety data. Vaccine 2011;29:599–612. [DOI] [PubMed] [Google Scholar]
- 24. Steyerberg EW. Clinical prediction models. New York, NY: Springer, 2009. [Google Scholar]
- 25. Vergouwe Y, Moons KG, Steyerberg EW. External validity of risk models: use of benchmark values to disentangle a case‐mix effect from incorrect coefficients. Am J Epidemiol 2010;172:971–980. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26. Vergouwe Y, Nieboer D, Oostenbrink R, et al. A closed testing procedure to select an appropriate method for updating prediction models. Stat Med 2017;36:4529–4539. [DOI] [PubMed] [Google Scholar]
- 27. Janssen KJ, Moons KG, Kalkman CJ, et al. Updating methods improved the performance of a clinical prediction model in new patients. J Clin Epidemiol 2008;61:76–86. [DOI] [PubMed] [Google Scholar]
- 28. van der Meche FG, Schmitz PI. A randomized trial comparing intravenous immune globulin and plasma exchange in Guillain‐Barre syndrome. Dutch Guillain‐Barre Study Group. N Engl J Med 1992;326:1123–1129. [DOI] [PubMed] [Google Scholar]
- 29. van Koningsveld R, Schmitz PI, Meche FG, et al. Effect of methylprednisolone when added to standard treatment with intravenous immunoglobulin for Guillain‐Barre syndrome: randomised trial. Lancet 2004;363:192–196. [DOI] [PubMed] [Google Scholar]
- 30. Garssen MP, van Koningsveld R, van Doorn PA, et al. Treatment of Guillain‐Barre syndrome with mycophenolate mofetil: a pilot study. J Neurol Neurosurg Psychiatry 2007;78:1012–1013. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31. Dutch Guillain‐Barre Study Group . Treatment of Guillain‐Barre syndrome with high‐dose immune globulins combined with methylprednisolone: a pilot study. Ann Neurol 1994;35:749–752. [DOI] [PubMed] [Google Scholar]
- 32. Ruts L, Drenthen J, Jongen JL, et al. Pain in Guillain‐Barre syndrome: a long‐term follow‐up study. Neurology 2010;75:1439–1447. [DOI] [PubMed] [Google Scholar]
- 33. Walgaard C, Lingsma HF, van Doorn PA, et al. Tracheostomy or not: prediction of prolonged mechanical ventilation in Guillain‐Barre syndrome. Neurocrit Care 2017;26:6–13. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
TABLE S1 Characteristics of the Patients in the EGRIS Development Cohort and Regional Validation Cohorts of IGOS
This table provides an overview of the characteristics of the patients in the Erasmus Guillain–Barré Syndrome Respiratory Insufficiency Score (EGRIS) development cohort and the International Guillain–Barré Syndrome Outcome Study (IGOS) validation datasets from Europe/North America and Asia. Numbers are provided as median (IQR) or n (%), unless stated otherwise.
aThe EGRIS development cohort contained data from 5 different studies. The median age of the patients was derived from the separate articles describing these studies: (1) combined cohort of 3 studies, median age (IQR) in years: 52 (33–66) 28 , 29 , 31 ; (2) median age (95% confidence interval) in years: 46 (23–76) 30 ; (3) median age (IQR) in years: 50 (35–63). 32
bFor the IGOS validation cohort, we used GBS variants at visit week 2 as classified by the local treating neurologist. If the week 2 variant was missing, we used the variant at week 1 or study entry. Other GBS variants include pharyngeal–cervical–brachial variant, pure sensory GBS, ataxic variant, Bickerstaff brainstem encephalitis, et cetera.
cThis proportion was deduced from the separate articles describing the 5 studies that were included in the EGRIS development cohort. This number provides an approximation of the proportion of patients who were treated in the development cohort, as the exact numbers could not be retrieved.
Eu/NA = Europe/North America; GBS = Guillain–Barré syndrome; IQR = interquartile range; MFS = Miller Fisher syndrome; MRC = Medical Research Council; MV = mechanical ventilation; NA = not applicable/available.
TABLE S2. Study Design, Inclusion Criteria, and Exclusion Criteria of the Studies Included in the EGRIS Development Cohort and International Guillain–Barré Syndrome Outcome Study Validation Cohort
*The Erasmus Guillain–Barré Syndrome Respiratory Insufficiency Score (EGRIS) development cohort included data from 5 studies. Four of these 5 studies had similar inclusion and exclusion criteria, and are therefore reported in one column. From studies 1, 2, and 3, 20 patients were excluded because they were ventilated prior to referral to one of the participating hospitals. From studies 4 and 5, 3 patients were excluded because ventilation was started before referral to a trial hospital. The full EGRIS development cohort included 565 patients.
TABLE S3. The IGOS Consortium
*All coauthors mentioned for this paper are also part of the IGOS Consortium, but fulfil additional criteria that justify authorship.