Skip to main content
PLOS One logoLink to PLOS One
. 2023 Aug 22;18(8):e0289267. doi: 10.1371/journal.pone.0289267

Validation of the accuracy of the modified World Federation of Neurosurgical Societies subarachnoid hemorrhage grading scale for predicting the outcomes of patients with aneurysmal subarachnoid hemorrhage

Tuan Anh Nguyen 1,2, Ton Duy Mai 2,3,4, Luu Dang Vu 5,6, Co Xuan Dao 2,4,7,*, Hung Manh Ngo 4,8,9, Hai Bui Hoang 2,10, Tuan Anh Tran 4,5,6, Trang Quynh Pham 9,11, Dung Thi Pham 12, My Ha Nguyen 13, Linh Quoc Nguyen 1,2, Phuong Viet Dao 2,3,4, Duong Ngoc Nguyen 1, Hien Thi Thu Vuong 14, Hung Dinh Vu 10, Dong Duc Nguyen 8, Thanh Dang Vu 15, Dung Tien Nguyen 3, Anh Le Ngoc Do 1,2, Cuong Duy Nguyen 16, Son Ngoc Do 2,4,7, Hao The Nguyen 9,11, Chi Van Nguyen 1,2, Anh Dat Nguyen 1,2, Chinh Quoc Luong 1,2,4
Editor: Martin Kieninger17
PMCID: PMC10443875  PMID: 37607172

Abstract

Background

Evaluating the prognosis of patients with aneurysmal subarachnoid hemorrhage (aSAH) who may be at risk of poor outcomes using grading systems is one way to make a better decision on treatment for these patients. This study aimed to compare the accuracy of the modified World Federation of Neurosurgical Societies (WFNS), WFNS, and Hunt and Hess (H&H) Grading Scales in predicting the outcomes of patients with aSAH.

Methods

From August 2019 to June 2021, we conducted a multicenter prospective cohort study on adult patients with aSAH in three central hospitals in Hanoi, Vietnam. The primary outcome was the 90-day poor outcome, measured by a score of 4 (moderately severe disability) to 6 (death) on the modified Rankin Scale (mRS). We calculated the areas under the receiver operator characteristic (ROC) curve (AUROCs) to determine how well the grading scales could predict patient prognosis upon admission. We also used ROC curve analysis to find the best cut-off value for each scale. We compared AUROCs using Z-statistics and compared 90-day mean mRS scores among intergrades using the pairwise multiple-comparison test. Finally, we used logistic regression to identify factors associated with the 90-day poor outcome.

Results

Of 415 patients, 32% had a 90-day poor outcome. The modified WFNS (AUROC: 0.839 [95% confidence interval, CI: 0.795–0.883]; cut-off value≥2.50; PAUROC<0.001), WFNS (AUROC: 0.837 [95% CI: 0.793–0.881]; cut-off value≥3.5; PAUROC<0.001), and H&H scales (AUROC: 0.836 [95% CI: 0.791–0.881]; cut-off value≥3.5; PAUROC<0.001) were all good at predicting patient prognosis on day 90th after ictus. However, there were no significant differences between the AUROCs of these scales. Only grades IV and V of the modified WFNS (3.75 [standard deviation, SD: 2.46] vs 5.24 [SD: 1.68], p = 0.026, respectively), WFNS (3.75 [SD: 2.46] vs 5.24 [SD: 1.68], p = 0.026, respectively), and H&H scales (2.96 [SD: 2.60] vs 4.97 [SD: 1.87], p<0.001, respectively) showed a significant difference in the 90-day mean mRS scores. In multivariable models, with the same set of confounding variables, the modified WFNS grade of III to V (adjusted odds ratio, AOR: 9.090; 95% CI: 3.494–23.648; P<0.001) was more strongly associated with the increased risk of the 90-day poor outcome compared to the WFNS grade of IV to V (AOR: 6.383; 95% CI: 2.661–15.310; P<0.001) or the H&H grade of IV to V (AOR: 6.146; 95% CI: 2.584–14.620; P<0.001).

Conclusions

In this study, the modified WFNS, WFNS, and H&H scales all had good discriminatory abilities for the prognosis of patients with aSAH. Because of the better effect size in predicting poor outcomes, the modified WFNS scale seems preferable to the WFNS and H&H scales.

Introduction

Subarachnoid hemorrhage (SAH) is often a devastating clinical event with a substantial mortality rate, a high rate of morbidity among survivors, and a healthcare burden [1, 2]. The most cause of spontaneous SAH is a ruptured aneurysm. The degree of neurologic impairment and the extent of subarachnoid bleeding at the time of admission are the most important predictors of neurologic complications and outcomes [3, 4]. Moreover, appropriate therapy for an aneurysmal SAH is often dependent upon the severity of the hemorrhage involved. Therefore, the significance and vitality of obtaining the SAH severity grade as soon as the presentation and stabilization of the patient allow that cannot be underestimated. In order to standardize the clinical classification of patients with SAH, several grading systems are utilized upon the initial evaluation–most notable of which are the grading systems proposed by Hunt and Hess (H&H) and that of the World Federation of Neurological Surgeons (WFNS) [5, 6].

To date, no grading systems are completely free from interobserver variability. For the H&H scale, the classifications are arbitrary, some of the terms are vague, and some patients may present with initial features that defy placement within a single grade [7]. As a result, the interobserver agreement for the H&H scale is poor [8, 9]. A systematic review of SAH grading scales also found conflicting data regarding the utility of the H&H scale for prognosis [7]. Furthermore, it is unclear if there are significant differences in outcome for adjacent H&H grades [1013]. Unlike the H&H scale, the WFNS scale uses objective terminology to assign grades [7]. Because the WFNS scale requires only an assessment of the Glasgow coma scale (GCS) and motor function, it may be easier to administer than the H&H scale. However, the lack of formal validation of the WFNS scale might lead to occasional overlap between grades (particularly between grades II and III), where the outcomes predicted by the assigned grade may not differ substantially [11, 14, 15]. As a result, the interobserver variability for the WFNS scale is still moderate [9]. Additionally, a systematic review of SAH grading scales also found conflicting data regarding the prognostic power of the WFNS grades [7]. Therefore, making more accurate initial predictions of outcome after SAH remains a challenge.

Stratification of patients is usually done in accordance to their clinical SAH grade prior to undergoing aneurysm repair during the acute phase [1624]. Although it is very unlikely the SAH grading scale would yield a 100% accurate prediction of outcomes, in order to achieve the maximum intended effect, the ideal SAH grading can be obtained via the following prerequisites: 1) the scale must be easy to apply during the acute phase of the disease; 2) it must be free from observer variability; 3) there must be a significant correlation between patient outcome and the grading scale; 4) two adjacent grades must have significantly different outcomes [25]. Removing the presence of motor deficit, the WFNS had proposed a modification to the original WFNS scale in conjunction with the Japan Neurosurgical Society. In two studies, this modified WFNS scale is seen with a much more promising discriminatory value compared with the original WFNS scale, but broader validation studies are required [15, 26, 27].

The aim of this study was to determine the relationship between the grades on the modified WFNS, WFNS, and H&H scales and the actual outcome and to compare the prognostic accuracy of these scales.

Methods

Source of data

This multicenter prospective observational study is the major update of our published previous study [2830], which collected data on patients with aneurysmal SAH consecutively admitted to the three national tertiary hospitals (Vietnam-Germany Friendship, Bach Mai, and Hanoi Medical University Hospital) in Hanoi, Vietnam, between August 2019 and August 2020, to investigate the rate of poor outcomes and associated factors from aneurysmal SAH in the country [28]. To determine the relationship between the grades on the modified WFNS, WFNS, and H&H scales and the actual outcome and to compare the prognostic accuracy of these scales, we continued to collect data on these patients consecutively admitted to these three hospitals between September 2020 and June 2021. We then merged the data sets from two stages of data collection for the three hospitals. These hospitals are designated central hospitals in northern Vietnam by the Ministry of Health of Vietnam; the first is a surgical hospital with 1,500 beds, the second is a large general hospital with 3,200 beds, and the last is a small hospital with 580 beds. Each participating hospital had at least two representatives (i.e., fully trained clinicians or surgeons) who were a part of the study team. Participation was voluntary and unfunded. All patients received a follow-up till death in the hospital or within 30 or 90 days of ictus and had clinic visits or phone contacts on days 30th and 90th after ictus for the modified Rankin Scale (mRS) assessments, mRS ranges from 0 (no disability) to 6 (death) [31], and evaluation of complications (e.g., chronic hydrocephalus).

Participants

This study included all patients (aged 18 years or older) presenting with aneurysmal SAH to the three central hospitals within 4 days of ictus. We defined a case of aneurysmal SAH as a person who had the presence of blood visible on a head computed tomography (CT) scan (or in case the CT scan was negative, the presence of xanthochromia in the cerebral spinal fluid) in combination with an aneurysm confirmed on CT or digital subtraction angiography (DSA) [4]. We excluded patients for whom the GCS on admission was unable to be scored (e.g., patients intubated and under sedation before arrival at the central hospital) or patients who became lost at 90 days of follow-up during the study. In the case of aphasia, patients were classified according to the clinically possible GCS scores derived from their eye and motor scores [32, 33]. When different possible verbal scores placed patients in different categories, these patients were excluded.

All patients were managed following the American Heart Association (AHA)/American Stroke Association (ASA) guidelines for the management of aneurysmal SAH [4]. Aneurysm repair with endovascular coiling or surgical clipping was performed as early as possible and immediately if rebleeding occurred. The decision to treat the cerebral aneurysms was made based on the discretion of the physician in charge of the patients and the availability of endovascular coiling or neurosurgical clipping, which depended on the participating hospital and the financial situation (either insurance or patient self-pay).

Data collection

The data for each study patient were recorded from the same unified samples (case record form). A case record form (CRF) was adopted across the study sites to collect the common variables. Data were entered by a researcher or investigator into the study database via EpiData Entry software (EpiData Association, Denmark, Europe), which was used for simple or programmed data entry and data documentation that could prevent data entry errors or mistakes. We also checked the data for implausible outliers and missing fields and contacted hospital representatives for clarification. Patient identifiers were not entered into the database to protect the patients’ confidentiality.

Outcome measures

The primary outcome of this study was poor neurological function (poor outcome) on day 90th after ictus, which was defined as mRS scores of 4 (moderately severe disability) to 6 (death) [34, 35]. We also examined the following secondary outcomes: poor outcome on day 30th after ictus, 30- and 90-day mortality rates, and incidence rate of complications.

Predictor measures

We defined exposure variables as SAH grading scales (i.e., the PAASH, WFNS, and H&H grading scales) at the time of admission to the hospital. Based on the admission GCS, we divided patients into the five categories of the WFNS grading scale ranging from grade I (GCS score of 15) to V (GCS scores of 3 to 6), of which focal deficits make up 1 additional grade for patients with a GCS score of 14 or 13 [6], and into the five categories of the modified WFNS ranging from grade I (GCS score of 15) to V (GCS scores of 3 to 6), regardless of the presence of neurologic deficits [15]. Based on the clinical condition on admission, we also classified patients into the five severity groups according to the H&H grading scale, which consists of five grades ranging from minimally symptomatic to coma [5]. All data elements required for calculating the GCS score and for classifying patients according to the modified WFNS, WFNS, or H&H grading scale at the time of admission to the hospital were prospectively assessed and collected on the same unified CRF by a fully trained clinician or surgeon of the participating hospitals and then were entered by a researcher or investigator into a study database via the EpiData Entry software for later analysis.

We determined confounding factors as variables collected on the same unified CRF by a fully trained clinician or surgeon. The CRF included variables based on the unruptured intracranial aneurysm (UIA) and SAH work group (WG) recommendations [36], such as information on:

  • (i) Medical histories (e.g., stroke, UIA, etc.), clinical presentation (e.g., GCS and focal neurological signs).

  • (ii) Admission head CT scan (e.g., presence of SAH, intraventricular hemorrhage (IVH) or intracerebral hemorrhage (ICH), and Fisher scale) and follow-up head CT scan during hospitalization (e.g., presence of SAH, IVH or ICH) or on days 30th and 90th after ictus (e.g., the presence of chronic hydrocephalus). We also collected data on the aneurysm site and aneurysm size from DSA or multi-slice CT (MSCT) angiography scan.

  • (iii) Surgical and endovascular interventions (i.e., surgical clipping or endovascular coiling), rescue therapies (e.g., surgical hematoma evacuation, defined as any surgical procedure evacuating epidural, subdural, intraventricular, or intraparenchymal hematoma, such as decompressive craniotomy, open craniotomy, or minimally invasive surgery; external ventricular drain (EVD) placement; ventriculoperitoneal (VP) shunt), and intensive care unit (ICU) therapies (e.g., mechanical ventilation).

  • (iv) Neurological complications (e.g., rebleeding, which included bleeding into the subarachnoid space, intracerebral, intraventricular, or subdural compartments; delayed cerebral ischaemia (DCI), hydrocephalus). Rebleeding from a ruptured aneurysm was classified into two subtypes: early or late rebleeding. We defined early or late rebleeding as rebleeding occurring in the hospital before or after an aneurysm repair, respectively.

  • (v) Clinical time course (e.g., time from ictus to hospital arrival, length of hospitalization).

  • (vi) We also collected data on demographics (i.e., sex, age) and system variables, which are available as an online supplement of a previously published paper [28].

Sample size

In the present study, poor neurological function on day 90th after the ictus served as the primary outcome. We, therefore, used the formula to determine the minimal sample size for estimating a population proportion with a confidence level of 95%, a confidence interval (margin of error) of ±4.7% and an assumed population proportion of 39.1%, based on the rate of poor neurological function on day 90th after the ictus (39.1%) reported in a previously published study [37]. Therefore, we should have at least 415 patients in our sample. Because of this, our sample size was sufficient and reflected a normal distribution.

n=z2xp^1-p^ε2

where:

z is the z score (z score for a 95% confidence level is 1.96)

ε is the margin of error (ε for a confidence interval of ± 4.7% is 0.047)

p^ is the population proportion (p^ for a population proportion of 39.1% is 0.391)

n is the sample size

Statistical analyses

We used IBM® SPSS® Statistics 22.0 (IBM Corp., Armonk, United States of America) and Analyse-it statistical software (Analyse-it Software, Ltd., Leeds, United Kingdom) for data analysis. We report the data as numbers (no.) and percentages (%) for categorical variables and medians and interquartile ranges (IQRs) or means and standard deviations (SDs) for continuous variables. Furthermore, comparisons were made between poor and good outcomes on day 30th and 90th after ictus for each variable using the Chi-squared test or Fisher’s exact test for categorical variables and the Mann–Whitney U test, Kruskal–Wallis test, or one-way analysis of variance for continuous variables.

Odds ratios (ORs) for a poor outcome on days 30th and 90th after ictus with 95% confidence intervals (CIs) were calculated for each grade of the SAH grading scales (i.e., the modified WFNS, WFNS, and H&H scales) with a univariable logistic regression model, with grade I taken as the reference. In all of the SAH grading scales, significant intergrade differences concerning the outcome (mean mRS scores) on days 30th and 90th after ictus that were determined using the Kruskal–Wallis H test with the Dunn-Bonferroni principle as a post hoc analysis.

We converted from descriptive SAH grading scales to numerical SAH grading scales in ascending order (S1 Table in S1 File). Receiver operator characteristic (ROC) curves were plotted, and the areas under the ROC curve (AUROC) were calculated to determine the discriminatory ability of all SAH grading scales for the prognosis of the patients upon admission. The cut-off value of each SAH grading scale was determined by ROC curve analysis and defined as the cut-off point with the maximum value of Youden’s index (i.e., sensitivity + specificity—1). Based on the cut-off value of each SAH grading scale, we assigned the patients to two severity groups: either the grade that was less than the cut-off value or another that was greater than or equal to the cut-off value. We also performed a pairwise comparison among the AUROCs of the SAH grading scales for predicting the poor outcome on days 30th and 90th after ictus by using the Z-statistics.

We assessed the factors associated with 90-day poor outcomes using logistic regression analysis. To reduce the number of predictors and the multicollinearity issue and resolve the overfitting, we used different methods to select variables as follows: (a) we put all variables (including exposure and confounding factors) of demographics, baseline characteristics, clinical and laboratory characteristics, neuroimaging findings, clinical time course, treatments, and complications into the univariable logistic regression model; (b) we selected variables if the p value was <0.05 in the univariable analysis between the good and poor outcomes on day 90th after ictus, as well as those that are clinically crucial, to put in the multivariable logistic regression model. These variables included demographics (i.e., age), risk factors for aneurysmal SAH (i.e., hypertension), comorbidities (i.e., diabetes mellitus), initial neuroimaging findings (i.e., location of blood within the subarachnoid space, the occurrence of IVH, and ICH, and aneurysm location), the severity of the aneurysmal SAH on admission (i.e., the grade of modified WFNS, WFNS, or H&H scale that was either greater than or equal to the cut-off value), treatments (i.e., aneurysm repair, nimodipine, surgical hematoma evacuation, EVD), and complications (i.e., rebleeding, DCI, acute hydrocephalus, and pneumonia). Using a stepwise backwards elimination method, we started with the full multivariable logistic regression model that included the selected variables. This method then deleted the variables stepwise from the full model until all remaining variables were independently associated with the risk of 90-day poor outcomes in the final model. Similarly, we used these methods of variable selection and analysis for assessing factors associated with 30-day poor outcomes. For examining the effect size of each grade of the SAH grading scales, in combination with confounding factors, for predicting the 30- and 90-day poor outcomes, we replaced the severity variable with each SAH grading scale (i.e., the modified WFNS, WFNS, or H&H scale, with grade I taken as the reference) in the multivariable logistic regression models, with the same set of confounding variables. We presented the odds ratios (ORs) and 95% confidence intervals (CIs) in the univariable logistic regression model and the adjusted ORs (AORs) and 95% CIs in the multivariable logistic regression model.

For all analyses, the significance levels were two-tailed, and we considered P < 0.05 to be statistically significant.

Ethical issues

The Hanoi Medical University (Approval number: 3335/QĐ-ĐHYHN), Vietnam-Germany Friendship Hospital (Approval number: 818/QĐ-VĐ; Research code: KH04.2020), and Bach Mai Hospital (Approval number: 3288/QĐ-BM; Research code: BM_2020_1247) Scientific and Ethics Committees approved this study. This study was conducted according to the principles of the Declaration of Helsinki. The Vietnam-Germany Friendship Hospital Scientific and Ethics Committees waived written informed consent for this non interventional study, and public notification of the study was made by public posting. The authors who performed the data analysis kept the datasets in password-protected systems, and we only present anonymized data.

Results

Data on 415 eligible patients with aneurysmal SAH were submitted to the study database (Fig 1 and Table 1), in which there were few missing data.

Fig 1. Flowchart of the study design and assessment occasions.

Fig 1

(Abbreviations: “discharged to die”, defined as patients were in grave condition or dying and were classified with a modified Rankin Scale score of 5 (severe disability) at the time of discharge; DCI: delayed cerebral ischemia; GCS: Glasgow Coma Scale; H&H: Hunt and Hess grading scale; poor outcome: defined as modified Rankin Scale scores of 4 (moderately severe disability) to 6 (death); SAH: subarachnoid hemorrhage; WFNS: World Federation of Neurosurgical Societies grading scale).

Table 1. Baseline characteristics and outcomes of patients with aneurysmal subarachnoid haemorrhage.

Variables N (%)
All cases 415
Baseline characteristics
 Age (year), median (IQR) 57.0 (48.0–67.0)
 Gender (male) 198 (47.7)
 Modified WFNS scale
  Grade I 204 (49.2)
  Grade II 38 (9.2)
  Grade III 24 (5.8)
  Grade IV 99 (23.9)
  Grade V 50 (12.0)
 WFNS scale
  Grade I 204 (49.2)
  Grade II 48 (11.6)
  Grade III 14 (3.4)
  Grade IV 99 (23.9)
  Grade V 50 (12.0)
 H&H scale
  Grade I 45 (10.8)
  Grade II 168 (40.5)
  Grade III 62 (14.9)
  Grade IV 48 (11.6)
  Grade V 92 (22.2)
Outcomes
 Poor outcomes
  30 days after ictus 138 (33.3)
  90 days after ictus 133 (32.0)
 Deaths
  30 days after ictus 89 (21.4)
  90 days after ictus 97 (23.4)

Abbreviations: H&H, Hunt and Hess; IQR, interquartile range; WFNS, World Federation of Neurological Surgeons.

See S2 to S9 Tables in S1 File for additional information.

Baseline characteristics and clinical outcomes

Of the total patients, 198/415 (47.7%) were men, and the median age was 57.0 (IQR: 48.0–67.0) (Table 1). Overall, 32.0% (133/415) of patients with aneurysmal SAH had a poor outcome on day 90th after ictus, 23.4% (97/415) of whom died within 90 days of ictus (Table 1). The baseline characteristics, management, complications, and outcomes of the patients were compared between patients who had a good outcome and patients who had a poor outcome on days 30th and 90th after ictus, as shown in S2 to S9 Tables in S1 File.

Overall prognostic performance of the SAH grading scales

Figs 2 and 3 show the overall performances of the SAH grading scales for predicting the poor outcome, of which the modified WFNS (AUROC: 0.839 [95% CI: 0.795–0.883]; cut-off value≥2.50; sensitivity: 82.7%; specificity: 77.7%; PAUROC<0.001), the WFNS (AUROC: 0.837 [95% CI: 0.793–0.881]; cut-off value≥3.5; sensitivity: 75.9%; specificity: 83.0%; PAUROC<0.001), and the H&H scales (AUROC: 0.836 [95% CI: 0.791–0.881]; cut-off value≥3.5; sensitivity: 72.2%; specificity: 84.4%; PAUROC<0.001) all had good discriminatory abilities for predicting the 90-day poor outcome (Fig 3). There were also the good discriminatory abilities of the SAH grading scales for predicting the 30-day poor outcome on day 30th after ictus, as shown in Fig 2 and S10 Table in S1 File.

Fig 2. The overall prognostic performance of the SAH grading scales for the poor outcomes on day 30th after ictus: The area under the ROC curves of the modified WFNS (AUROC: 0.839 [95% CI: 0.796–0.883]; cut-off value: ≥2.5; sensitivity: 82.6%; specificity: 78.7%; PAUROC <0.001), the WFNS (AUROC: 0.836 [95% CI: 0.793–0.880]; cut-off value: ≥3.5; sensitivity: 75.4%; specificity: 83.8%; PAUROC <0.001), and the H&H scales (AUROC: 0.839 (95% CI: 0.795–0.883); cut-off value: ≥3.5; sensitivity: 71.7%; specificity: 85.2%; PAUROC <0.001) for predicting the poor outcomes on day 30th after ictus in patients with aneurysmal SAH.

Fig 2

(Abbreviations: AUROC: areas under the receiver operating characteristic curve; H&H: Hunt and Hess; poor outcome: defined as a modified Rankin Scale [mRS] score of 4 to 6; ROC, receiver operating characteristic; SAH: subarachnoid hemorrhage; WFNS, World Federation of Neurological Surgeons).

Fig 3. The overall prognostic performance of the SAH grading scales for the poor outcomes on day 90th after ictus: The area under the ROC curves of the modified WFNS (AUROC: 0.839 [95% CI: 0.795–0.883]; cut-off value: ≥2.5; sensitivity: 82.7%; specificity: 77.7%; PAUROC <0.001), the WFNS (AUROC: 0.837 [95% CI: 0.793–0.881]; cut-off value: ≥3.5; sensitivity: 75.9%; specificity: 83.0%; PAUROC <0.001), and the H&H scales (AUROC: 0.836 [95% CI: 0.791–0.881]; cut-off value: ≥3.5; sensitivity: 72.2%; specificity: 84.4%; PAUROC <0.001) for predicting the poor outcomes on day 90th after ictus in patients with aneurysmal SAH.

Fig 3

(Abbreviations: AUROC: areas under the receiver operating characteristic curve; H&H: Hunt and Hess; poor outcome: defined as a modified Rankin Scale [mRS] score of 4 to 6; ROC, receiver operating characteristic; SAH: subarachnoid hemorrhage; WFNS, World Federation of Neurological Surgeons).

Table 2 shows the differences between the AUROC curves among different test-pairwise, of which the AUROCs for predicting the 90-day poor outcome did not differ significantly between the modified WFNS and WFNS scales (AUROC difference: 0.002; 95% CI: -0.001–0.005; Z-statistic: 1.25; p = 0.211), the modified WFNS and H&H scales (AUROC difference: 0.003; 95% CI: -0.015–0.021; Z-statistic: 0.27; p = 0.786), and the WFNS and H&H scales (AUROC difference: 0.001; 95% CI: -0.017–0.018; Z-statistic: 0.07; p = 0.947). For predicting the poor outcome on day 30th after ictus, there were no significant differences between the AUROC curves among different test-pairwise, as shown in Table 2.

Table 2. Pairwise comparisons of AUROC of the modified WFNS, WFNS, and H&H scales for predicting the poor outcome (mRS of 4 to 6) after ictus in patients with aneurysmal SAH.

Comparison AUROC difference (95% CI) SE Z -statistic p-value
Poor outcome on day 30th after ictus
Modified WFNS and WFNS 0.003 (0.000–0.006) 0.0019 1.54 0.123
Modified WFNS and H&H 0.000(-0.018–0.018) 0.0107 0.01 0.992
WFNS and H&H 0.003(-0.014–0.020) 0.0104 0.27 0.787
Poor outcome on day 90th after ictus
Modified WFNS and WFNS 0.002(-0.001–0.005) 0.0018 1.25 0.211
Modified WFNS and H&H 0.003(-0.015–0.021) 0.0109 0.27 0.786
WFNS and H&H 0.001(-0.017–0.018) 0.0106 0.07 0.946

Abbreviations: AUC, the area under the curve; AUROC, the area under the receiver operating characteristic; H&H, Hunt and Hess scale; Modified WFNS, Modified World Federation of Neurosurgical Societies scale; SAH, subarachnoid hemorrhage; SE, standard error; WFNS, World Federation of Neurosurgical Societies scale.

Differences between the clinical outcomes of the adjacent grades

There were no significant differences in the mean 90-day mRS scores among all adjacent grades of the SAH grading scales except between grades IV and V (3.75 [SD: 2.46] vs 5.24 [SD: 1.68], p = 0.026, respectively) of the modified WFNS scale, between grades IV and V (3.75 [SD: 2.46] vs 5.24 [SD: 1.68], p = 0.026, respectively) of the WFNS scale, and between grades IV and V (2.96 [SD: 2.60] vs 4.97 [SD: 1.87], p<0.001, respectively) of the H&H scale (Table 3). Differences among the mean 30-day mRS scores of adjacent grades of the three SAH grading scales is available in Table 3.

Table 3. Comparison of outcomes between the intergrades of the subarachnoid hemorrhage grading scales.

SAH grading scale Day 30th after ictus Day 90th after ictus
mRS, mean (SD) No. of patients p-valuea mRS, mean (SD) No. of patients p-valuea
Modified WFNS scale
 I 0.69 (1.50) 204 NA 0.63 (1.55) 204 NA
 II 1.18 (2.02) 38 >0.999 1.00 (2.07) 38 >0.999
 III 2.63 (2.50) 24 0.372 2.29 (2.65) 24 0.641
 IV 3.86 (2.36) 99 0.432 3.75 (2.46) 99 0.073
 V 5.26 (1.60) 50 0.051 5.24 (1.68) 50 0.026
WFNS scale
 I 0.69 (1.50) 204 NA 0.63 (1.55) 204 NA
 II 1.63 (2.29) 48 0.096 1.40 (2.35) 48 0.916
 III 2.14 (2.41) 14 >0.999 1.86 (2.51) 14 >0.999
 IV 3.86 (2.36) 99 0.403 3.75 (2.46) 99 0.159
 V 5.26 (1.60) 50 0.051 5.24 (1.68) 50 0.026
H&H scale
 I 0.98 (1.79) 45 NA 0.87 (1.87) 45 NA
 II 0.65 (1.45)b 168 >0.999 0.59 (1.50)b 168 >0.999
 III 2.10 (2.44) 62 0.391 1.89 (2.49) 62 0.588
 IV 3.06 (2.47) 48 0.306 2.96 (2.60) 48 0.211
 V 5.03 (1.76) 92 0.001 4.97 (1.87) 92 <0.001

a Probability values were obtained by comparing the mean mRS score of a given grade with that of the mRS score just above it (nonparametric test by Dunn’s multiple comparisons).

b The grade of the H&H scale shows the reversed rank order of the mean mRS scores.

Abbreviations

H&H: Hunt and Hess; mRS: modified Rankin Scale; NA: not applicable; No.: number; SAH: subarachnoid hemorrhage; SD: standard deviation; WFNS: World Federation of Neurological Surgeons.

Associations between the grading scales and clinical outcomes

In the univariable logistic regression analyses, we found that most grades of the SAH grading scales, with grade I taken as the reference, were significantly associated with the increased risk of the 30- and 90-day poor outcomes, except for associations between grade II of the modified WFNS scale and the increased risk of the 30-day (OR: 1.475; 95% CI: 0.515–4.226; p = 0.469) and the 90-day poor outcome (OR: 1.566; 95% CI: 0.544–4.509; p = 0.406), and between grade II of the H&H scale and the increased risk of the 30-day (OR: 0.784; 95% CI: 0.269–2.287; p = 0.656) and the 90-day poor outcome (OR: 0.727; 95% CI: 0.247–2.139; p = 0.563) (Table 4).

Table 4. Odds ratio for a poor outcome (mRS of 4 to 6) for the SAH grading scales.

SAH grading scale Poor outcome on day 30th after ictus Poor outcome on day 90th after ictus
N mRS: 4–6, no. (%) OR (95% CI) p-value N mRS: 4–6, no. (%) OR (95% CI) p-value
Modified WFNS scale
 I 204 19 (13.8) reference <0.001 204 18 (13.5) reference <0.001
 II 38 5 (3.6) 1.475 (0.515–4.226) 0.469 38 5 (3.8) 1.566 (0.544–4.509) 0.406
 III 24 10 (7.2) 6.955 (2.720–17.784) <0.001 24 9 (6.8) 6.200 (2.380–16.154) <0.001
 IV 99 61 (44.2) 15.630 (8.391–29.117) <0.001 99 58 (43.6) 14.618 (7.803–27.383) <0.001
 V 50 43 (31.2) 59.812 (23.648–151.281) <0.001 50 43 (32.3) 63.476 (24.947–161.511) <0.001
WFNS scale
 I 204 19 (9.3) reference <0.001 204 18 (8.8) reference <0.001
 II 48 11 (22.9) 2.895 (1.272–6.587) 0.011 48 10 (20.8) 2.719 (1.164–6.350) 0.021
 III 14 4 (28.6) 3.895 (1.114–13.621) 0.033 14 4 (28.6) 4.133 (1.177–14.520) 0.027
 IV 99 61 (61.6) 15.630 (8.391–29.117) <0.001 99 58 (58.6) 14.618 (7.803–27.383) <0.001
 V 50 43 (86.0) 59.812 (23.648–151.281) <0.001 50 43 (86.0) 63.476 (24.947–161.511) <0.001
H&H scale
 I 45 5 (11.1) reference <0.001 45 5 (11.1) reference <0.001
 II 168 15 (8.9) 0.784 (0.269–2.287) 0.656 168 14 (8.3) 0.727 (0.247–2.139) 0.563
 III 62 19 (30.6) 3.535 (1.206–10.358) 0.021 62 18 (29.0) 3.273 (1.112–9.631) 0.031
 IV 48 23 (47.9) 7.360 (2.478–21.860) <0.001 48 22 (45.8) 6.769 (2.277–20.121) 0.001
 V 92 76 (82.6) 38.000 (12.973–111.306) <0.001 92 74 (80.4) 32.889 (11.362–95.201) <0.001

Abbreviations

CI: confidence interval; H&H: Hunt and Hess; mRS: modified Rankin Scale; N: total number of patients for each grade; no.: number; OR: odds ratio; SAH: subarachnoid hemorrhage; SD: standard deviation; WFNS: World Federation of Neurological Surgeons.

See S11 to S16 Tables in S1 File for additional information.

When we added each SAH grading scale, with grade I taken as the reference, to the multivariable logistic regression models, with the same set of confounding variables, for predicting the 30- and 90-day poor outcomes (S11 to S16 Tables in S1 File), we found that most grades of the modified WFNS, WFNS, and H&H scales were independently associated with the increased risk of the 90-day poor outcome, except for grade II (AOR: 0.990; 95% CI: 0.140–6.977; p = 0.992) of the modified WFNS scale, grades II (AOR: 2.097; 95% CI: 0.493–8.914; p = 0.316) and III (AOR: 6.198; 95% CI: 0.946–40.582; p = 0.057) of the WFNS scale, and grade II (AOR: 4.574; 95% CI: 0.683–30.654; p = 0.117) of the H&H scale (S14 to S16 Tables in S1 File). Associations between the grades of the modified WFNS, WFNS, and H&H scales and the risk of the 30-day poor outcome are available in S11 to S13 Tables in S1 File.

The risk factors for poor outcomes

In the multivariable logistic regression models (Tables 5 and 6, S17 to S20 Tables in S1 File), with the same set of confounding variables, a modified WFNS grade of III to V (AOR: 9.090; 95% CI: 3.494–23.648; P<0.001) (Table 6) was more strongly associated with the increased risk of the 90-day poor outcome, compared to a WFNS grade of IV to V (AOR: 6.383; 95% CI: 2.661–15.310; P<0.001) (S19 Table in S1 File) and an H&H grade of IV to V (AOR: 6.146; 95% CI: 2.584–14.620; P<0.001) (S20 Table in S1 File). Similarly, we also found that a modified WFNS grade of III to V (AOR: 10.516; 95% CI: 4.092–27.026; P<0.001) (Table 5) was a more strongly independent predictor of the 30-day poor outcome, compared to a WFNS grade of IV to V (AOR: 6.879; 95% CI: 2.884–16.408; P<0.001) (S17 Table in S1 File) and an H&H grade of IV to V (AOR: 7.475; 95% CI: 3.202–17.449; P<0.001) (S18 Table in S1 File). Other factors were independently associated with the risk of the 30-day and 90-day poor outcome in the multivariable logistic regression models with different exposure variables are available in Tables 5 and 6 and S17 to S20 Tables in S1 File.

Table 5. Factors associated with the 30-day poor outcome (mRS of 4 to 6) in patients with aneurysmal subarachnoid hemorrhage.

Factors Univariable logistic regression analysesa Multivariable logistic regression analysisb
OR 95% CI for OR p-value AOR 95% CI for AOR p-value
Lower Upper Lower Upper
Demographics
 Age ≥ 60 years 2.407 1.586 3.654 <0.001 2.921 1.281 6.661 0.011
Risk factors of aneurysmal SAH
 Hypertension 2.056 1.354 3.122 0.001 NA NA NA NA
Comorbidities
 Diabetes mellitus 3.751 1.669 8.433 0.001 NA NA NA NA
Neuroimaging findings on admission
 Location of blood within the subarachnoid space:
  Basal cistern 2.685 1.728 4.173 <0.001 NA NA NA NA
  Sylvian fissure 2.981 1.125 7.900 0.028 NA NA NA NA
  Interhemispheric fissure 1.960 1.207 3.183 0.007 NA NA NA NA
  Interpeduncular fossa 3.254 1.995 5.308 <0.001 2.559 1.070 6.122 0.035
  Suprasellar cistern 1.929 1.221 3.047 0.005 NA NA NA NA
  Ambient cistern 3.054 1.902 4.906 <0.001 NA NA NA NA
  Quadrigeminal cistern 5.141 3.270 8.085 <0.001 NA NA NA NA
 IVH 3.013 1.847 4.914 <0.001 NA NA NA NA
 ICH 1.860 1.142 3.029 0.013 NA NA NA NA
 Aneurysm locations
  PCoA aneurysm 0.554 0.299 1.026 0.061 NA NA NA NA
  VA aneurysm 3.341 1.265 8.820 0.015 NA NA NA NA
Severity of aneurysmal SAH on admission
 Modified WFNS grade of III to Vc 17.551 10.374 29.693 <0.001 10.516 4.092 27.026 <0.001
Aneurysm repairs and other treatments
 Aneurysm repairs:
  No aneurysm repair Reference <0.001 Reference <0.001
  Endovascular coiling 0.009 0.003 0.027 <0.001 0.009 0.002 0.032 <0.001
  Surgical clipping 0.020 0.007 0.059 <0.001 0.011 0.003 0.040 <0.001
 Surgical hematoma evacuationd 2.438 1.297 4.583 0.006 NA NA NA NA
 EVDe 3.537 1.846 6.777 <0.001 4.594 1.595 13.228 0.005
 Nimodipine 0.076 0.028 0202 <0.001 NA NA NA NA
Complications
 Rebleeding 7.868 2.537 24.396 <0.001 19.734 3.879 100.399 <0.001
 DCI 7.316 2.848 18.792 <0.001 20.914 5.331 82.049 <0.001
 Acute hydrocephalus 3.134 2.034 4.830 <0.001 NA NA NA NA
 Pneumonia 4.091 2.295 7.292 <0.001 2.826 1.122 7.119 0.028
Constant 0.741 0.681

aEach variable of the demographics, risk factors for aneurysmal SAH, comorbidities, initial clinical, neuroimaging and laboratory characteristics, the severity of aneurysmal SAH (i.e., modified WFNS scale) on admission, treatments, and complications was analyzed in the univariable logistic regression model and was considered in the multivariable logistic regression model if the P-value was <0.05 in univariable logistic regression analysis, as well as clinically crucial factors.

bAll selected variables were included in the multivariable logistic regression model with the stepwise backward elimination method. Variables, then, were deleted stepwise from the full model until all remaining variables were independently associated with poor outcomes.

c The grades of the modified WFNS scale which were higher than or equal to the cut-off value.

d Surgical hematoma evacuation was defined as any surgical procedure evacuating epidural, subdural, intraventricular, or intraparenchymal hematoma, such as decompressive craniotomy, open craniotomy, or minimally invasive surgery.

e The reason for the EVD insertion was the complication of acute hydrocephalus and others (See S5 and S9 Tables in S1 File for additional information).

Abbreviations: AOR, adjusted odds ratio; CI, confidence interval; DCI, delayed cerebral ischemia; EVD: external ventricular drainage; ICH, intracerebral hemorrhage; IVH, intraventricular hemorrhage; mRS, modified Rankin Scale; NA, not available; OR, odds ratio; PCoA, posterior communicating artery; SAH, subarachnoid hemorrhage; VA, vertebral artery; WFNS, World Federation of Neurosurgical Societies.

See S17 to S20 Tables in S1 File for additional information.

Table 6. Factors associated with the 90-day poor outcome (mRS of 4 to 6) in patients with aneurysmal subarachnoid hemorrhage.

Factors Univariable logistic regression analysesa Multivariable logistic regression analysisb
OR 95% CI for OR p-value AOR 95% CI for AOR p-value
Lower Upper Lower Upper
Demographics
 Age ≥ 60 years 2.581 1.691 3.939 <0.001 3.554 1.524 8.287 0.003
Risk factors of aneurysmal SAH
 Hypertension 2.133 1.400 3.250 <0.001 NA NA NA NA
Comorbidities
 Diabetes mellitus 3.986 1.772 8.968 0.001 NA NA NA NA
Neuroimaging findings on admission
 Location of blood within the subarachnoid space:
  Basal cistern 2.718 1.738 4.251 <0.001 NA NA NA NA
  Sylvian fissure 2.811 1.060 7.454 0.038 NA NA NA NA
  Interhemispheric fissure 2.063 1.257 3.385 0.004 NA NA NA NA
  Interpeduncular fossa 3.206 1.953 5.264 <0.001 NA NA NA NA
  Suprasellar cistern 1.882 1.186 2.986 0.007 NA NA NA NA
  Ambient cistern 2.990 1.852 4.829 <0.001 NA NA NA NA
  Quadrigeminal cistern 4.414 3.431 8.545 <0.001 2.665 1.139 6.234 0.024
 IVH 2.793 1.711 4.559 <0.001 NA NA NA NA
 ICH 1.879 1.151 3.068 0.012 NA NA NA NA
 Aneurysm locations
  PCoA aneurysm 0.590 0.318 1.094 0.094 NA NA NA NA
  VA aneurysm 3.542 1.341 9.356 0.011 NA NA NA NA
Severity of aneurysmal SAH on admission
 Modified WFNS grade of III to Vc 16.625 9.790 28.233 <0.001 9.090 3.494 23.648 <0.001
Aneurysm repairs and other treatments
 Aneurysm repairs:
  No aneurysm repair Reference <0.001 Reference <0.001
  Endovascular coiling 0.011 0.004 0.030 <0.001 0.009 0.002 0.032 <0.001
  Surgical clipping 0.023 0.009 0.062 <0.001 0.014 0.004 0.049 <0.001
 Surgical hematoma evacuationd 2.111 1.123 3.971 0.020 NA NA NA NA
 EVDe 3.382 1.773 6.453 <0.001 5.365 1.852 15.544 0.002
 Nimodipine 0.071 0.027 0.191 <0.001 NA NA NA NA
Complications
 Rebleeding 6.133 2.138 17.594 0.001 22.913 4.560 115.135 <0.001
 DCI 6.245 2.538 15.365 <0.001 16.210 4.279 61.412 <0.001
 Acute hydrocephalus 3.134 2.028 4.842 <0.001 NA NA NA NA
 Pneumonia 4.022 2.263 7.148 <0.001 3.375 1.306 8.722 0.012
Constant 0.731 0.631

aEach variable of the demographics, risk factors for aneurysmal SAH, comorbidities, initial clinical, neuroimaging and laboratory characteristics, the severity of aneurysmal SAH (i.e., modified WFNS scale) on admission, treatments, and complications was analyzed in the univariable logistic regression model and was considered in the multivariable logistic regression model if the P-value was <0.05 in univariable logistic regression analysis, as well as clinically crucial factors.

bAll selected variables were included in the multivariable logistic regression model with the stepwise backward elimination method. Variables, then, were deleted stepwise from the full model until all remaining variables were independently associated with poor outcomes.

c The grades of the modified WFNS scale which were higher than or equal to the cut-off value.

d Surgical hematoma evacuation was defined as any surgical procedure evacuating epidural, subdural, intraventricular, or intraparenchymal hematoma, such as decompressive craniotomy, open craniotomy, or minimally invasive surgery.

e The reason for the EVD insertion was the complication of acute hydrocephalus and others (See S5 and S9 Tables in S1 File for additional information).

Abbreviations: AOR, adjusted odds ratio; CI, confidence interval; DCI, delayed cerebral ischemia; EVD: external ventricular drainage; ICH, intracerebral hemorrhage; IVH, intraventricular hemorrhage; mRS, modified Rankin Scale; NA, not available; OR, odds ratio; PCoA, posterior communicating artery; SAH, subarachnoid hemorrhage; VA, vertebral artery; WFNS, World Federation of Neurosurgical Societies.

See S17 to S20 Tables in S1 File for additional information.

Discussion

The present study revealed that nearly one-third of patients with aneurysmal SAH had poor outcomes on days 30th and 90th after ictus (33.3% and 32.0%, respectively), over one-fifth of whom died within 30 and 90 days of ictus (21.4% and 23.4%, respectively) (Fig 1 and Table 1). The modified WFNS, WFNS, and H&H scales all had good discriminatory ability concerning the prognosis of patients on days 30th and 90th after ictus (Figs 2 and 3, S10 Table in S1 File), with no significant differences between the AUROC curves among different test-pairwise (Table 2). There were only significant differences in the 30-day mean mRS scores between grades IV and V of the H&H scale and in the 90-day mean mRS scores between grades IV and V of the modified WFNS, the WFNS and the H&H scale (Table 3). In the univariable logistic regression analyses, with grade I taken as the reference, the modified WFNS scale did not show more gradual increases in OR for the 30- and 90-day poor outcome, in ascending grades, compared to the WFNS and H&H scales (Table 4). In the multivariable logistic regression analyses, with the same set of confounding variables, the modified WFNS scale, however, showed more gradual increases in AOR for the 30- and 90-day poor outcome, in ascending grades, compared to the WFNS and H&H scales (S11 to S16 Tables in S1 File). Moreover, in the multivariable logistic regression analyses, with the same set of confounding variables, a modified WFNS grade of III to V (Tables 5 and 6) was more strongly associated with the increased risk of the 30- and 90-day poor outcome compared to a WFNS grade of IV to V (S17 to S18 in S1 File) and an H&H grade of IV to V (S19 to S20 in S1 File).

In our study, the rate of the 90-day poor outcome was lower than the rate reported in a previously published study (43.6%) [37]. The mortality rates of our patients on days 30th and 90th after ictus were also lower than rates reported in previous studies (22–25% and 25–29%, respectively) [38, 39]. These findings might be due to several advances in the past decades in the medical diagnosis and care of patients with aneurysmal SAH, including medical advances, new systems of standardized care in neurocritical care units, and new surgical and endovascular techniques for aneurysmal SAH [4, 40]. Additionally, as unique to patients with aneurysmal SAH in Vietnam, many are not transferred to a central hospital and are relegated to death in the local hospitals as well as dying outside of the hospital [41]. Therefore, these particular differences in poor outcomes or deaths might be accounted for because our cohort is likely to be highly selected. These differences might also be attributed to our study only including patients presenting to the participating hospitals within four days after ictus and excluding patients for whom the admission GCS was unable to be scored (e.g., patients intubated and under sedation before arrival at the central hospital). Thus, these factors have resulted in an implicit selection bias and an enrolment and inclusion incompletion of patients in the study database. As a result, our cohort is likely to be underestimated in the poor outcome and mortality rates.

Although there were no significant differences between the AUROC curves among different test-pairwise, our modified WFNS, WFNS, and H&H scales all had good discriminatory ability concerning the prognosis of patients on days 30th and 90th after ictus. However, our study showed only significant differences in the 30-day mean mRS scores between grades IV and V of the H&H scale and in the 90-day mean mRS scores between grades IV and V of the modified WFNS, the WFNS and the H&H scale. To date, there is no universally accepted scale to assess the clinical condition of these patients at the time of admission [7, 42]. Although the WFNS and H&H scales are both widely used in clinical practice and research reports, the lack of formal validation of the WFNS scale might lead to occasional overlap between grades [11, 14, 15], and the interobserver agreement for the H&H scale is poor [8, 9]. The WFNS scale has two main advantages over the GCS alone. It compresses the GCS into five grades, which may create greater intergrade differences in outcome. It includes the presence of a focal motor deficit axis. However, the amount of additional prognostic power derived from adding this axis is unknown [7]. Our study showed no significant differences in the 30- and 90-day outcomes (mean mRS scores) among all adjacent grades of the WFNS scale except for the significant differences in the 90-day mean mRS scores between grades IV and V. A previous study also showed that the differences in the outcomes between grades II and III failed to reach statistical significance on the WFNS scale [15]. These findings might be due to the lack of formal validation of the WFNS scale, which might lead to occasional overlap between grades (particularly between grades II and III), where the outcomes predicted by the assigned grade may not differ substantially [11, 14]. The present study also showed no significant differences in the 30- and 90-day outcomes (mean mRS scores) between the adjacent grades of the H&H scale except for between grades IV and V. Although the H&H scale is easy to administer, the classifications are arbitrary, some of the terms are vague (e.g., drowsy, stupor, and deep coma), and some patients may present with initial features that defy placement within a single grade [7]. As an example, a rare presentation of SAH may include severe headache (i.e., grade II), normal level of consciousness, and severe hemiparesis (i.e., grade IV). In such cases, the clinician must subjectively decide which of the presenting features is the most important for determining the grade. Therefore, our findings might be accounted for by the poor interobserver agreement and might also contribute to the conflicting data regarding the utility of the H&H scale for prognosis [8, 9, 11]. The modified WFNS scale is very easy to apply and is based solely on the GCS, which has better outcome predictability [15, 26, 27]. Unlike the originally suggested modified WFNS scale, for which the 90-day mean mRS scores of each grade differs from that of adjacent grades with a statistical significance except between grades III and IV [15], the present study showed a significant difference in the 90-day mean mRS scores only between grades IV and V of the modified WFNS scale. This variation might be because of the differences concerning the time points of GCS evaluation as well as assessment of SAH grading scale (i.e., after initial stabilization vs upon admission) between the two studies. A retrospective cohort study has shown that the outcome after aneurysmal SAH was best predicted by assessing WFNS grade after neurocritical stabilization and neurosurgical/neuro-interventional procedures [43]. The fact that significant differences in the 30- and 90-day mean mRS scores between the adjacent grades were not observed more clearly in the modified WFNS scale compared to those in the WFNS and the H&H scales in our study suggests that the modified WFNS scale maybe not preferable to the WFNS and H&H scales.

In our univariable logistic regression analyses, with grade I taken as the reference, the modified WFNS scale did not show more gradual increases in OR for the 30- and 90-day poor outcome, in ascending grades, compared to the WFNS and H&H scales. When we added each SAH grading scale (i.e., modified WFNS, WFNS, or H&H scale) to the multivariable logistic regression model, with the same set of confounding variables, for predicting the 30- and 90-day poor outcome, we found a more gradual increase in AOR of the modified WFNS scale, in ascending grades, compared to those of the WFNS and H&H scales, which might be due to the same or more number of grades, that was independently associated with the increased risk of poor outcome, was observed in the modified WFNS scale compared to the WFNS and the H&H scale. Previous critiques identified a caution problem with ORs [4448], and a recent literature review has raised this issue again [49]; namely, (i) there is no single OR; instead, any estimated OR is conditional on the data and the model specification; (ii) ORs should not be compared across different studies using different samples from different populations; and (iii) nor should they be compared across models with different sets of confounding variables [49]. However, we used the univariable logistic regression analyses, with the same grade (i.e., I) taken as the reference, and the multivariable logistic regression analyses, with the same set of confounding variables, to determine the relationships among the grades on the modified WFNS, WFNS, and H&H scales and the actual outcomes. Therefore, a more gradual increase in effect size (i.e., AOR) of the modified WFNS scale, in ascending grades, for predicting the poor outcome in our multivariable logistic regression analyses suggests that the modified WFNS may be preferable to the WFNS and H&H scales.

Although the advances in diagnostic and treatment strategies for aneurysmal SAH have substantially improved the outcomes of hospitalized patients in recent decades [5053]. predicting the outcome of aneurysmal SAH remains a problematic issue. The clinical condition can vary during the acute phase, and complications occurring during the clinical course and treatments rendered can influence the outcome [28, 54, 55]. In the present study, complications (e.g., rebleeding, DCI, pneumonia) also accounted for a substantial proportion of patients with aneurysmal SAH and contributed significantly to a high rate of poor outcomes. Nevertheless, grading patients with SAH on admission is crucial for clinical and research purposes. Most grading systems are used in practice to standardize the clinical classification of patients with SAH based only on the initial neurologic examination and the appearance of blood on the initial head CT [5, 6, 25, 56]. Therefore, a scale applied upon admission will never give a 100% perfect prediction for the outcome. Although our modified WFNS, WFNS, and H&H scales all had good discriminatory ability concerning the prognosis of patients on days 30th and 90th after ictus, there were no significant differences between the AUROC curves among different test-pairwise. However, a modified WFNS grade of III to V, a WFNS grade of IV to V and an H&H grade of IV to V were independent predictors of the 30- and 90-day poor outcome. Because of the better effect size (i.e., AOR) of modified WFNS grade of III to V for predicting the 30- and 90-day poor outcome, the modified WFNS scale was preferable to the WFNS and the H&H scales.

An advantage of the present study was data from the multicenter, which had little missing data (S21 Table in S1 File). However, the present study has some limitations as follows: Firstly, our data are from a selected population of cases that were mainly transferred to the three highest-level public sector hospitals in Vietnam. Therefore, the number of patients with aneurysmal SAH is likely to be considerably higher. Secondly, there is a lack of interobserver agreement, since we were unable to provide multiple clinicians for each of the patients’ initial evaluations. Finally, this study only included patients presenting to the participating hospitals within 4 days of ictus and excluded patients for whom admission GCS was unable to be scored (e.g., patients intubated and under sedation before arrival at the central hospital). These factors might have resulted in an implicit selection bias and an incomplete enrolment and inclusion of patients in the database. Differences in figures found between Vietnam and other nations might be accounted for by these factors above.

Conclusions

This study investigated a selected cohort of patients with aneurysmal SAH, a high rate of poor outcomes and a high mortality rate presented to central hospitals in Vietnam. The modified WFNS, WFNS, and H&H scales all had good discriminatory abilities for the prognosis of patients with aneurysmal SAH. Because of the better effect size in predicting poor outcomes, the modified WFNS scale seems preferable to the WFNS and H&H scales.

Supporting information

S1 File

(PDF)

Acknowledgments

We thank all staff of the Center for Emergency Medicine, the Stroke Center, the Department of Neurosurgery, and the Radiology Centre at Bach Mai Hospital for their support with this study. We thank all staff of the Neurosurgery Center at Vietnam-Germany Friendship Hospital for their support with this study, as well as the Emergency and Critical Care Department at Hanoi Medical University Hospital. We also thank the staff of the Faculty of Public Health at Thai Binh University of Medicine and Pharmacy for their support and statistical advice. Finally, we thank Miss Truc-Cam Nguyen from Stanford University, Stanford, California, the United States of America, and Miss Mai Phuong Nguyen from the Hotchkiss School, Lakeville, Connecticut, the United States of America, for their support of our manuscript.

Data Availability

All relevant data are within the paper and its Supporting information files.

Funding Statement

The author(s) received no specific funding for this work.

References

  • 1.Lawton MT, Vates GE. Subarachnoid Hemorrhage. The New England journal of medicine. 2017;377(3):257–66. doi: 10.1056/NEJMcp1605827 . [DOI] [PubMed] [Google Scholar]
  • 2.Macdonald RL, Schweizer TA. Spontaneous subarachnoid haemorrhage. Lancet. 2017;389(10069):655–66. Epub 2016/09/18. doi: 10.1016/S0140-6736(16)30668-7 . [DOI] [PubMed] [Google Scholar]
  • 3.Suarez JI. Diagnosis and Management of Subarachnoid Hemorrhage. Continuum (Minneapolis, Minn). 2015;21(5 Neurocritical Care):1263–87. Epub 2015/10/02. doi: 10.1212/CON.0000000000000217 . [DOI] [PubMed] [Google Scholar]
  • 4.Connolly ES Jr., Rabinstein AA, Carhuapoma JR, Derdeyn CP, Dion J, Higashida RT, et al. Guidelines for the management of aneurysmal subarachnoid hemorrhage: a guideline for healthcare professionals from the American Heart Association/american Stroke Association. Stroke; a journal of cerebral circulation. 2012;43(6):1711–37. Epub 05/03. doi: 10.1161/STR.0b013e3182587839 [DOI] [PubMed] [Google Scholar]
  • 5.Hunt WE, Hess RM. Surgical risk as related to time of intervention in the repair of intracranial aneurysms. Journal of neurosurgery. 1968;28(1):14–20. Epub 1968/01/01. doi: 10.3171/jns.1968.28.1.0014 . [DOI] [PubMed] [Google Scholar]
  • 6.The World Federation of Neurological Surgeons (WFNS) Committee. Report of World Federation of Neurological Surgeons Committee on a Universal Subarachnoid Hemorrhage Grading Scale. Journal of neurosurgery. 1988;68(6):985–6. Epub 1988/06/01. doi: 10.3171/jns.1988.68.6.0985 . [DOI] [PubMed] [Google Scholar]
  • 7.Rosen DS, Macdonald RL. Subarachnoid hemorrhage grading scales: a systematic review. Neurocritical care. 2005;2(2):110–8. Epub 2005/09/15. doi: 10.1385/NCC:2:2:110 . [DOI] [PubMed] [Google Scholar]
  • 8.Lindsay KW, Teasdale GM, Knill-Jones RP. Observer variability in assessing the clinical features of subarachnoid hemorrhage. Journal of neurosurgery. 1983;58(1):57–62. Epub 1983/01/01. doi: 10.3171/jns.1983.58.1.0057 . [DOI] [PubMed] [Google Scholar]
  • 9.Degen LA, Dorhout Mees SM, Algra A, Rinkel GJ. Interobserver variability of grading scales for aneurysmal subarachnoid hemorrhage. Stroke; a journal of cerebral circulation. 2011;42(6):1546–9. Epub 2011/04/30. doi: 10.1161/STROKEAHA.110.601211 . [DOI] [PubMed] [Google Scholar]
  • 10.Oshiro EM, Walter KA, Piantadosi S, Witham TF, Tamargo RJ. A new subarachnoid hemorrhage grading system based on the Glasgow Coma Scale: a comparison with the Hunt and Hess and World Federation of Neurological Surgeons Scales in a clinical series. Neurosurgery. 1997;41(1):140–7; discussion 7–8. Epub 1997/07/01. doi: 10.1097/00006123-199707000-00029 . [DOI] [PubMed] [Google Scholar]
  • 11.Hirai S, Ono J, Yamaura A. Clinical grading and outcome after early surgery in aneurysmal subarachnoid hemorrhage. Neurosurgery. 1996;39(3):441–6; discussion 6–7. Epub 1996/09/01. doi: 10.1097/00006123-199609000-00002 . [DOI] [PubMed] [Google Scholar]
  • 12.Proust F, Hannequin D, Langlois O, Freger P, Creissard P. Causes of morbidity and mortality after ruptured aneurysm surgery in a series of 230 patients. The importance of control angiography. Stroke; a journal of cerebral circulation. 1995;26(9):1553–7. Epub 1995/09/01. doi: 10.1161/01.str.26.9.1553 . [DOI] [PubMed] [Google Scholar]
  • 13.Ogilvy CS, Carter BS. A proposed comprehensive grading system to predict outcome for surgical management of intracranial aneurysms. Neurosurgery. 1998;42(5):959–68; discussion 68–70. Epub 1998/05/20. doi: 10.1097/00006123-199805000-00001 . [DOI] [PubMed] [Google Scholar]
  • 14.Gotoh O, Tamura A, Yasui N, Suzuki A, Hadeishi H, Sano K. Glasgow Coma Scale in the prediction of outcome after early aneurysm surgery. Neurosurgery. 1996;39(1):19–24; discussion -5. Epub 1996/07/01. doi: 10.1097/00006123-199607000-00005 . [DOI] [PubMed] [Google Scholar]
  • 15.Sano H, Satoh A, Murayama Y, Kato Y, Origasa H, Inamasu J, et al. Modified World Federation of Neurosurgical Societies subarachnoid hemorrhage grading system. World neurosurgery. 2015;83(5):801–7. Epub 2014/12/24. doi: 10.1016/j.wneu.2014.12.032 . [DOI] [PubMed] [Google Scholar]
  • 16.Asano T, Takakura K, Sano K, Kikuchi H, Nagai H, Saito I, et al. Effects of a hydroxyl radical scavenger on delayed ischemic neurological deficits following aneurysmal subarachnoid hemorrhage: results of a multicenter, placebo-controlled double-blind trial. Journal of neurosurgery. 1996;84(5):792–803. Epub 1996/05/01. doi: 10.3171/jns.1996.84.5.0792 . [DOI] [PubMed] [Google Scholar]
  • 17.Auer LM. Acute operation and preventive nimodipine improve outcome in patients with ruptured cerebral aneurysms. Neurosurgery. 1984;15(1):57–66. Epub 1984/07/01. doi: 10.1227/00006123-198407000-00011 . [DOI] [PubMed] [Google Scholar]
  • 18.Bell BA, Miller JD, Neto NG, O’Neill P, Laughton LM. Effect of naloxone on deficits after aneurysmal subarachnoid hemorrhage. Neurosurgery. 1985;16(4):498–501. Epub 1985/04/01. . [PubMed] [Google Scholar]
  • 19.Haley EC Jr., Kassell NF, Alves WM, Weir BK, Hansen CA. Phase II trial of tirilazad in aneurysmal subarachnoid hemorrhage. A report of the Cooperative Aneurysm Study. Journal of neurosurgery. 1995;82(5):786–90. Epub 1995/05/01. doi: 10.3171/jns.1995.82.5.0786 . [DOI] [PubMed] [Google Scholar]
  • 20.Ljunggren B, Brandt L, Säveland H, Nilsson PE, Cronqvist S, Andersson KE, et al. Outcome in 60 consecutive patients treated with early aneurysm operation and intravenous nimodipine. Journal of neurosurgery. 1984;61(5):864–73. Epub 1984/11/01. doi: 10.3171/jns.1984.61.5.0864 . [DOI] [PubMed] [Google Scholar]
  • 21.Nibbelink DW, Torner JC, Henderson WG. Intracranial aneurysms and subarachnoid hemorrhage. A cooperative study. Antifibrinolytic therapy in recent onset subarachnoid hemorrhage. Stroke; a journal of cerebral circulation. 1975;6(6):622–9. Epub 1975/11/01. doi: 10.1161/01.str.6.6.622 . [DOI] [PubMed] [Google Scholar]
  • 22.Ohta T, Kikuchi H, Hashi K, Kudo Y. Nizofenone administration in the acute stage following subarachnoid hemorrhage. Results of a multi-center controlled double-blind clinical study. Journal of neurosurgery. 1986;64(3):420–6. Epub 1986/03/01. doi: 10.3171/jns.1986.64.3.0420 . [DOI] [PubMed] [Google Scholar]
  • 23.Säveland H, Ljunggren B, Brandt L, Messeter K. Delayed ischemic deterioration in patients with early aneurysm operation and intravenous nimodipine. Neurosurgery. 1986;18(2):146–50. Epub 1986/02/01. doi: 10.1227/00006123-198602000-00005 . [DOI] [PubMed] [Google Scholar]
  • 24.Shibuya M, Suzuki Y, Sugita K, Saito I, Sasaki T, Takakura K, et al. Effect of AT877 on cerebral vasospasm after aneurysmal subarachnoid hemorrhage. Results of a prospective placebo-controlled double-blind trial. Journal of neurosurgery. 1992;76(4):571–7. Epub 1992/04/01. doi: 10.3171/jns.1992.76.4.0571 . [DOI] [PubMed] [Google Scholar]
  • 25.Takagi K, Tamura A, Nakagomi T, Nakayama H, Gotoh O, Kawai K, et al. How should a subarachnoid hemorrhage grading scale be determined? A combinatorial approach based solely on the Glasgow Coma Scale. Journal of neurosurgery. 1999;90(4):680–7. Epub 1999/04/08. doi: 10.3171/jns.1999.90.4.0680 . [DOI] [PubMed] [Google Scholar]
  • 26.Sano H, Inamasu J, Kato Y, Satoh A, Murayama Y, Diseases WC, et al. Modified world federation of neurosurgical societies subarachnoid hemorrhage grading system. Surgical neurology international. 2016;7(Suppl 18):S502–S3. doi: 10.4103/2152-7806.187491 . [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Fung C, Inglin F, Murek M, Balmer M, Abu-Isa J, Z’Graggen WJ, et al. Reconsidering the logic of World Federation of Neurosurgical Societies grading in patients with severe subarachnoid hemorrhage. Journal of neurosurgery. 2016;124(2):299–304. Epub 2015/09/19. doi: 10.3171/2015.2.JNS14614 . [DOI] [PubMed] [Google Scholar]
  • 28.Luong CQ, Ngo HM, Hoang HB, Pham DT, Nguyen TA, Tran TA, et al. Clinical characteristics and factors relating to poor outcome in patients with aneurysmal subarachnoid hemorrhage in Vietnam: A multicenter prospective cohort study. PloS one. 2021;16(8):e0256150. Epub 2021/08/14. doi: 10.1371/journal.pone.0256150 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Nguyen TA, Vu LD, Mai TD, Dao CX, Ngo HM, Hoang HB, et al. Predictive validity of the prognosis on admission aneurysmal subarachnoid haemorrhage scale for the outcome of patients with aneurysmal subarachnoid haemorrhage. Scientific reports. 2023;13(1):6721. doi: 10.1038/s41598-023-33798-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Nguyen TA, Mai TD, Vu LD, Dao CX, Ngo HM, Hoang HB, et al. Factors related to intracerebral haematoma in patients with aneurysmal subarachnoid haemorrhage in Vietnam: a multicentre prospective cohort study. BMJ open. 2023;13(4):e066186. doi: 10.1136/bmjopen-2022-066186 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.van Swieten JC, Koudstaal PJ, Visser MC, Schouten HJ, van Gijn J. Interobserver agreement for the assessment of handicap in stroke patients. Stroke; a journal of cerebral circulation. 1988;19(5):604–7. Epub 1988/05/01. doi: 10.1161/01.str.19.5.604 . [DOI] [PubMed] [Google Scholar]
  • 32.Juvela S. Risk factors for impaired outcome after spontaneous intracerebral hemorrhage. Archives of neurology. 1995;52(12):1193–200. Epub 1995/12/01. doi: 10.1001/archneur.1995.00540360071018 . [DOI] [PubMed] [Google Scholar]
  • 33.Prasad K, Menon GR. Comparison of the three strategies of verbal scoring of the Glasgow Coma Scale in patients with stroke. Cerebrovascular diseases (Basel, Switzerland). 1998;8(2):79–85. Epub 1998/04/21. doi: 10.1159/000015822 . [DOI] [PubMed] [Google Scholar]
  • 34.Luong CQ, Nguyen AD, Nguyen CV, Mai TD, Nguyen TA, Do SN, et al. Effectiveness of Combined External Ventricular Drainage with Intraventricular Fibrinolysis for the Treatment of Intraventricular Haemorrhage with Acute Obstructive Hydrocephalus. Cerebrovascular Diseases Extra. 2019;9(2):77–89. doi: 10.1159/000501530 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.Gaberel T, Gakuba C, Fournel F, Le Blanc E, Gaillard C, Peyro-Saint-Paul L, et al. FIVHeMA: Intraventricular fibrinolysis versus external ventricular drainage alone in aneurysmal subarachnoid hemorrhage: A randomized controlled trial. Neuro-Chirurgie. 2019;65(1):14–9. Epub 2019/01/15. doi: 10.1016/j.neuchi.2018.11.004 . [DOI] [PubMed] [Google Scholar]
  • 36.Suarez JI, Sheikh MK, Macdonald RL, Amin-Hanjani S, Brown RD Jr., de Oliveira Manoel AL, et al. Common Data Elements for Unruptured Intracranial Aneurysms and Subarachnoid Hemorrhage Clinical Research: A National Institute for Neurological Disorders and Stroke and National Library of Medicine Project. Neurocritical care. 2019;30(Suppl 1):4–19. Epub 2019/05/16. doi: 10.1007/s12028-019-00723-6 . [DOI] [PubMed] [Google Scholar]
  • 37.van Heuven AW, Dorhout Mees SM, Algra A, Rinkel GJ. Validation of a prognostic subarachnoid hemorrhage grading scale derived directly from the Glasgow Coma Scale. Stroke; a journal of cerebral circulation. 2008;39(4):1347–8. Epub 2008/03/01. doi: 10.1161/STROKEAHA.107.498345 . [DOI] [PubMed] [Google Scholar]
  • 38.Mackey J, Khoury JC, Alwell K, Moomaw CJ, Kissela BM, Flaherty ML, et al. Stable incidence but declining case-fatality rates of subarachnoid hemorrhage in a population. Neurology. 2016;87(21):2192–7. Epub 2016/10/23. doi: 10.1212/WNL.0000000000003353 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39.Øie LR, Solheim O, Majewska P, Nordseth T, Müller TB, Carlsen SM, et al. Incidence and case fatality of aneurysmal subarachnoid hemorrhage admitted to hospital between 2008 and 2014 in Norway. Acta neurochirurgica. 2020;162(9):2251–9. Epub 2020/07/01. doi: 10.1007/s00701-020-04463-x [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 40.Picetti E, Barbanera A, Bernucci C, Bertuccio A, Bilotta F, Boccardi EP, et al. Early management of patients with aneurysmal subarachnoid hemorrhage in a hospital with neurosurgical/neuroendovascular facilities: a consensus and clinical recommendations of the Italian Society of Anesthesia and Intensive Care (SIAARTI)—part 2. Journal of Anesthesia, Analgesia and Critical Care. 2022;2(1):21. doi: 10.1186/s44158-022-00049-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 41.Do SN, Luong CQ, Pham DT, Nguyen CV, Ton TT, Pham TTN, et al. Survival after out-of-hospital cardiac arrest, Viet Nam: multicentre prospective cohort study. Bulletin of the World Health Organization. 2021;99(1):50–61. Epub 10/28. doi: 10.2471/BLT.20.269837 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42.Dengler NF, Sommerfeld J, Diesing D, Vajkoczy P, Wolf S. Prediction of cerebral infarction and patient outcome in aneurysmal subarachnoid hemorrhage: comparison of new and established radiographic, clinical and combined scores. European journal of neurology. 2018;25(1):111–9. Epub 2017/09/25. doi: 10.1111/ene.13471 . [DOI] [PubMed] [Google Scholar]
  • 43.Giraldo EA, Mandrekar JN, Rubin MN, Dupont SA, Zhang Y, Lanzino G, et al. Timing of clinical grade assessment and poor outcome in patients with aneurysmal subarachnoid hemorrhage. Journal of neurosurgery. 2012;117(1):15–9. Epub 2012/05/01. doi: 10.3171/2012.3.JNS11706 . [DOI] [PubMed] [Google Scholar]
  • 44.Allison PD. Comparing Logit and Probit Coefficients Across Groups. Sociological Methods & Research. 1999;28(2):186–208. doi: 10.1177/0049124199028002003 [DOI] [Google Scholar]
  • 45.Mood C. Logistic Regression: Why We Cannot Do What We Think We Can Do, and What We Can Do About It. European Sociological Review. 2009;26(1):67–82. doi: 10.1093/esr/jcp006 [DOI] [Google Scholar]
  • 46.Gail MH, Wieand S, Piantadosi S. Biased estimates of treatment effect in randomized experiments with nonlinear regressions and omitted covariates. Biometrika. 1984;71(3):431–44. doi: 10.1093/biomet/71.3.431 [DOI] [Google Scholar]
  • 47.Yatchew A, Griliches Z. Specification Error in Probit Models. The Review of Economics and Statistics. 1985;67(1):134–9. doi: 10.2307/1928444 [DOI] [Google Scholar]
  • 48.Mroz TA, Zayats YV. Arbitrarily Normalized Coefficients, Information Sets, and False Reports of “Biases” in Binary Outcome Models. The Review of Economics and Statistics. 2008;90(3):406–13. doi: 10.1162/rest.90.3.406 [DOI] [Google Scholar]
  • 49.Norton EC, Dowd BE. Log Odds and the Interpretation of Logit Models. Health services research. 2018;53(2):859–78. doi: 10.1111/1475-6773.12712 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 50.Nieuwkamp DJ, Setz LE, Algra A, Linn FHH, de Rooij NK, Rinkel GJE. Changes in case fatality of aneurysmal subarachnoid haemorrhage over time, according to age, sex, and region: a meta-analysis. The Lancet Neurology. 2009;8(7):635–42. Epub 06/06. doi: 10.1016/S1474-4422(09)70126-7 . [DOI] [PubMed] [Google Scholar]
  • 51.Khan AU, Dulhanty L, Vail A, Tyrrell P, Galea J, Patel HC. Impact of specialist neurovascular care in subarachnoid haemorrhage. Clinical neurology and neurosurgery. 2015;133:55–60. Epub 2015/04/04. doi: 10.1016/j.clineuro.2015.03.006 . [DOI] [PubMed] [Google Scholar]
  • 52.Pickard JD, Murray GD, Illingworth R, Shaw MD, Teasdale GM, Foy PM, et al. Effect of oral nimodipine on cerebral infarction and outcome after subarachnoid haemorrhage: British aneurysm nimodipine trial. Bmj. 1989;298(6674):636–42. Epub 1989/03/11. doi: 10.1136/bmj.298.6674.636 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 53.Molyneux AJ, Kerr RSC, Yu L-M, Clarke M, Sneade M, Yarnold JA, et al. International subarachnoid aneurysm trial (ISAT) of neurosurgical clipping versus endovascular coiling in 2143 patients with ruptured intracranial aneurysms: a randomised comparison of effects on survival, dependency, seizures, rebleeding, subgroups, and aneurysm occlusion. Lancet (London, England). 2005;366(9488):809–17. doi: 10.1016/S0140-6736(05)67214-5 . [DOI] [PubMed] [Google Scholar]
  • 54.Lord AS, Fernandez L, Schmidt JM, Mayer SA, Claassen J, Lee K, et al. Effect of rebleeding on the course and incidence of vasospasm after subarachnoid hemorrhage. Neurology. 2012;78(1):31–7. Epub 2011/12/16. doi: 10.1212/WNL.0b013e31823ed0a4 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 55.Rosengart AJ, Schultheiss KE, Tolentino J, Macdonald RL. Prognostic factors for outcome in patients with aneurysmal subarachnoid hemorrhage. Stroke; a journal of cerebral circulation. 2007;38(8):2315–21. Epub 2007/06/16. doi: 10.1161/STROKEAHA.107.484360 . [DOI] [PubMed] [Google Scholar]
  • 56.Fisher CM, Kistler JP, Davis JM. Relation of cerebral vasospasm to subarachnoid hemorrhage visualized by computerized tomographic scanning. Neurosurgery. 1980;6(1):1–9. Epub 1980/01/01. doi: 10.1227/00006123-198001000-00001 . [DOI] [PubMed] [Google Scholar]

Decision Letter 0

Martin Kieninger

31 Jan 2023

PONE-D-22-32313Validation of the accuracy of the modified World Federation of Neurosurgical Societies subarachnoid haemorrhage grading scale for predicting the outcomes of patients with aneurysmal subarachnoid haemorrhagePLOS ONE

Dear Dr. Luong,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Mar 17 2023 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

  • A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

  • A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

  • An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Martin Kieninger

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. We noted in your submission details that a portion of your manuscript may have been presented or published elsewhere. Please clarify whether this conference proceeding or publication was peer-reviewed and formally published. If this work was previously peer-reviewed and published, in the cover letter please provide the reason that this work does not constitute dual publication and should be included in the current manuscript.

3. Please include a caption for figure 2.

4. Please upload a copy of Figure 2, to which you refer in your text on page 20. If the figure is no longer to be included as part of the submission please remove all reference to it within the text.

5. Please include captions for your Supporting Information files at the end of your manuscript, and update any in-text citations to match accordingly. Please see our Supporting Information guidelines for more information: http://journals.plos.org/plosone/s/supporting-information.

Additional Editor Comments:

I would like to apologize again for the long wait. However, the diametrically different assessments of Reviewer 1 and Reviewer 2 made a third review necessary.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: I Don't Know

Reviewer #3: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: This is a very interesting study concerning the validation of modified World Federation of Neurosurgical Societies (WFNS) subarachnoid haemorrhage grading scale in predicting the 90-day poor outcome defined as modified Rankin Scale scores of 4 to 6. The validation process was also correlated to WFNS and H&H scores. The result showed more accurate than those in WFNS and H&H scores. The manuscript is well written and should be published.

Reviewer #2: Thank you for submitting your manuscript.

Your multi center analysis of the 3 SAH grades (WFNS, mWFNS and Hunt and Hess) was thoroughly performed.

I however do have some comments.

1. It is stated that the WFNS is more complex to administer than the H&H scales because it requires the GCS and the motor function. First, I am not sure how it is more complicated than the H&H, since the GCS is a standard validated objective score that all neurosurgeon, and even emergency medicine doctors are all comfortable with. The H&H grade is very subjective. In addition, the WFNS score requires the GCS and the presence of a motor deficit only, which isn’t to complicated.

2. How many patients were excluded because of the initial GCs was not scored? Looking at the chart of the referring hospital or even the ambulance chart would yield the GCS.

3. Why were patients lost at 90 day? If surgical or even treated by embolization, were follow ups not performed?

4. The management of the aneurysmal SAH, let it be surgical or endovascular treatment, if very variable. The treatment option may play an important role in the outcome of the patient. Not having a standardized treatment option in these patients, and looking at the outcome is not adequate. This is a main caveat of the study.

5. It is stated that 415 patients presented to the study sites. How many were excluded because of the inadequate documentation of GCS? And how many and why where they lost to follow-up?

6. The conclusion that the mWFSN score was strongest associated with an increased risk of poor outcome at 30/90 days of ictus is a strong statement. This needs to be revised accordingly to the limits of the study.

7. Although their data do support their conclusion, I do not believe that this has any clinical impact, nor does it help in the clinical decision making of the surgeon /radiologist.

Although interesting, the data inclusion criteria is very weak.

Reviewer #3: The authors present the results of a prospective trial in patients with aneurysmal SAH, attempting to analyze the predictive value of SAH scales for poor outcome. This is an interesting study that carries high importance for the field. However, there are a number of aspects that need to be addressed before publication.

• How long was the median follow-up time?

• Were there any changes of patient status after 90 days ?

• What was the outcome at discharge?

• In the participants and treatment section, it is mentioned that “In the case of aphasia, patients were classified according to the clinically possible GCS scores derived from their eye and motor scores. How exactly is the “eye score” performed, and how was it implemented in the GS scale?

• How were missing data handled?

• Since the applied SAH grading systems are on an ordinal scale niveau, how meaningful is the approach of calculating ROC, which usually requires continuous data as the diagnostic input variable? For example, formulating a resulting cut-off value of 2.5 for poor outcomes in the modified WFNS scale is clinically challenging to implement. Please elaborate.

• Comparing odd`s ratios between groups and models has been criticized, in fact, this approach has recently been rejected as an adequate method in this context. Several authors have argued that odds ratios will change if variables are added to the model, even if those additional variables are independent from the other variables. This concern particularly applies to the analysis of differences between clinical outcomes of the adjacent grades. Please comment on this critical aspect.

• Clinically, there are a number of variables that may influence the results of this analysis. In particular, was there an influence of aneurysm location and the frequency of vasospasm / delayed ischemia? How many patients received decompressive craniectomy?

• The strongest risk factor for poor outcome was the modified WFNS score of 3-5, which translates into an initial GCS score of 3-13. This variability makes the application of such a “risk - factor” somewhat challenging to implement in the clinical setting.

• To my best knowledge, the modified WFNS score appears to carry a better discriminatory value for good outcomes compared to the original WFNS score. Do the authors find similar results for the patients with a good outcome?

• Since the initial SAH grading is influenced by acute hydrocephalus, how many patients with acute hydrocephalus improved after implantation of an intraventricular drain? Did these patients show a different result regarding the SAH scales ad their prediction of poor outcomes?

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

Reviewer #3: No

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

PLoS One. 2023 Aug 22;18(8):e0289267. doi: 10.1371/journal.pone.0289267.r002

Author response to Decision Letter 0


31 Mar 2023

Professor Martin Kieninger

Academic Editor

PLOS ONE

March 17, 2023

Dear Prof. Martin Kieninger,

On behalf of all authors, I am resubmitting herewith our revised manuscript entitled “Validation of the accuracy of the modified World Federation of Neurosurgical Societies subarachnoid haemorrhage grading scale for predicting the outcomes of patients with aneurysmal subarachnoid haemorrhage” (PONE-D-22-32313R1).

We sincerely appreciate the kind comments and points raised by the Editors and by the Reviewers. We have carefully considered all comments and suggestions and revised our manuscript following each of these points. These comments have enabled us to substantially improve our manuscript. We hope that Editor will find our revised manuscript suitable for publication in PLOS ONE.

We confirm that this work is original and has not been published elsewhere nor is it currently under consideration for publication elsewhere. All authors have read, approved the manuscript, and agreed to authorship and order of authorship for this manuscript, and all authors have the appropriate permissions and rights to the reported data.

We have provided our point-by-point responses to the comments of the Editors and the Reviewers attached.

We thank you for your kind consideration of this submission.

Sincerely yours,

Chinh Quoc Luong, MD., PhD.

Center for Emergency Medicine,

Bach Mai Hospital,

No. 78, Giai Phong, Phuong Mai ward, Dong Da district, Hanoi 100000, Vietnam

Email: luongquocchinh@gmail.com

We thank the Editors and the Reviewers for the valuable comments and suggestions that greatly helped us to improve the contents of this paper. In what follows, we will use the boldface to indicate comments from the Editors and the Reviewers, the standard font face for our responses and we highlighted in yellow the modifications that we did to the manuscript.

RESPONSE TO EDITORS

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Our answer:

We thank you for the positive feedback. We have carefully considered the Reviewers' comments and suggestions and have revised our manuscript following each of these points.

Please submit your revised manuscript by Mar 17 2023 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Our answer:

Thank you for this comment. We have submitted the revised manuscript on time.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

Our answer:

Thank you for this comment. We have included a rebuttal letter, a marked-up copy of the manuscript, and an unmarked version of the revised manuscript when submitting our revised manuscript.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

Our answer:

Thank you for this comment. We do not have any changes in our financial disclosure.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

Our answer:

Thank you for this comment. Laboratory protocol does not apply to our study, but study protocol does. Our study protocol has included in the Methods section.

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

Our answer:

Thank you for this comment. We have ensured that our manuscript meets PLOS ONE's style requirements, including those for file naming.

2. We noted in your submission details that a portion of your manuscript may have been presented or published elsewhere. Please clarify whether this conference proceeding or publication was peer-reviewed and formally published. If this work was previously peer-reviewed and published, in the cover letter please provide the reason that this work does not constitute dual publication and should be included in the current manuscript.

Our answer:

This multicenter prospective observational study is the major update of our published previous study.[1] Moreover, we presented a portion of this manuscript as an abstract poster at the 14th World Stroke Congress organized by the World Stroke Organization, which took place in Singapore between 26 and 29 October 2022,[2] and only the poster abstract was peer-reviewed and formally published in the conference proceeding of the 14th World Stroke Congress.[3] However, we performed additional experiments or collected additional data that were not a part of the study from the published articles. We also presented new data in this submission that were not previously presented in the published articles. Therefore, this work does not constitute dual publication and should be included in the current manuscript.

[1] Luong CQ, Ngo HM, Hoang HB, Pham DT, Nguyen TA, Tran TA, Nguyen DN, Do SN, Nguyen MH, Vu HD, Vuong HTT, Mai TD, Nguyen AQ, Le KH, Dao PV, Tran TH, Vu LD, Nguyen LQ, Pham TQ, Dong HV, Nguyen HT, Nguyen CV, Nguyen AD. Clinical characteristics and factors relating to poor outcome in patients with aneurysmal subarachnoid hemorrhage in Vietnam: A multicenter prospective cohort study. PLoS One. 2021 Aug 13;16(8):e0256150. doi: 10.1371/journal.pone.0256150.

[2] Linh Quoc Nguyen, Tuan Anh Nguyen, Ton Duy Mai, et al. Validation of the accuracy of the modified World Federation of Neurosurgical Societies subarachnoid haemorrhage grading scale for predicting the outcomes of patients with aneurysmal subarachnoid haemorrhage. Paper abstract presented at: 14th World Stroke Congress; 26-29 October 2022, 2022; Singapore.

[3] 14th World Stroke Congress, Singapore, 26-29 October 2022. International Journal of Stroke 2022; 17: 3-288.

3. Please include a caption for figure 2.

Our answer:

Thank you for this comment. We have included a caption for Figure 2.

4. Please upload a copy of Figure 2, to which you refer in your text on page 20. If the figure is no longer to be included as part of the submission please remove all reference to it within the text.

Our answer:

Thank you for this comment. We have uploaded a copy of Figure 2.

5. Please include captions for your Supporting Information files at the end of your manuscript, and update any in-text citations to match accordingly. Please see our Supporting Information guidelines for more information: http://journals.plos.org/plosone/s/supporting-information.

Our answer:

Thank you for this comment. We have included captions for the Supporting Information files at the end of the manuscript and have updated any in-text citations to match accordingly.

Additional Editor Comments:

I would like to apologize again for the long wait. However, the diametrically different assessments of Reviewer 1 and Reviewer 2 made a third review necessary.

Our answer:

Thank the Editor and Reviewers so much for their support and for taking the time to leave their excellent reviews.

RESPONSE TO REVIEWERS

Reviewers' comments:

Reviewer #1:

This is a very interesting study concerning the validation of modified World Federation of Neurosurgical Societies (WFNS) subarachnoid haemorrhage grading scale in predicting the 90-day poor outcome defined as modified Rankin Scale scores of 4 to 6. The validation process was also correlated to WFNS and H&H scores. The result showed more accurate than those in WFNS and H&H scores. The manuscript is well written and should be published.

Our answer:

We thank you for the positive feedback. We also thank the Reviewer for taking the time to review our manuscript.

Reviewer #2: Thank you for submitting your manuscript.

Your multi center analysis of the 3 SAH grades (WFNS, mWFNS and Hunt and Hess) was thoroughly performed.

I however do have some comments.

Our answer:

We thank you for the positive feedback. We have carefully considered the Reviewer’s comments and suggestions and have revised our manuscript following each of these points.

1. It is stated that the WFNS is more complex to administer than the H&H scales because it requires the GCS and the motor function. First, I am not sure how it is more complicated than the H&H, since the GCS is a standard validated objective score that all neurosurgeon, and even emergency medicine doctors are all comfortable with. The H&H grade is very subjective. In addition, the WFNS score requires the GCS and the presence of a motor deficit only, which isn’t to complicated.

Our answer:

Thank you for pointing this out. We have rewritten the paragraph appropriately, as follows:

"Because the WFNS scale requires only an assessment of the Glasgow coma scale (GCS) and motor function, it may be easier to administer than the H&H scale." (Page 5, Lines 113-115)

2. How many patients were excluded because of the initial GCs was not scored? Looking at the chart of the referring hospital or even the ambulance chart would yield the GCS.

Our answer:

Thank you for pointing this out. We would also like to thank the Reviewer for this valuable comment and suggestion that could help us improve data collection and quality for our current study project. We will consider looking at the chart of the referring hospital or even the ambulance chart rather than that only upon admission to the participating hospitals to yield the Glasgow coma score (GCS).

In the present study, because there was a lack of electronic health record systems in our participating hospitals, all data were prospectively collected on the same unified case record forms (CRF) by representatives/investigators (i.e., fully trained clinicians and surgeons) and were entered into a database via the EpiData Entry software after the completion of data collection for later analysis. Data on patients for whom the initial GCS was unable to be scored or on patients who became lost at both 30- and 90-day follow-up visits would not be entered into a database by investigators. Although no patients became lost at 30 and 90 days of follow-up during the present study, we did not have data on how many patients for whom investigators could not score the initial GCS and excluded them from the present study. Thus, this factor has resulted in an implicit selection bias and an enrolment and inclusion incompletion of patients in the study database. As a result, our cohort is likely to be underestimated in the incidence rate of poor outcomes and deaths. We have provided further discussion concerning the limitations of the present study in the Discussion section (Pages 27-28, Lines 461-477; Page 32, Lines 564-575).

3. Why were patients lost at 90 day? If surgical or even treated by embolization, were follow ups not performed?

Our answer:

Thank you for this comment. In our study, all patients received follow-up visits or phone contacts up to 90 days post-enrolment. Because poor neurological function on day 90th after the ictus served as the primary outcome, data on patients who became lost at 90-day follow-up visits or phone contacts would not be entered into a database by investigators. However, no patients became lost at 90 days of follow-up during the present study.

4. The management of the aneurysmal SAH, let it be surgical or endovascular treatment, if very variable. The treatment option may play an important role in the outcome of the patient. Not having a standardized treatment option in these patients, and looking at the outcome is not adequate. This is a main caveat of the study.

Our answer:

Thank you for this comment. This study aimed to determine the relationship between the grades on the modified World Federation of Neurosurgical Societies (WFNS), WFNS, and Hunt and Hess (H&H) scales and the actual outcomes and to compare the accuracy of these scales in predicting the outcomes of patients with aneurysmal subarachnoid hemorrhage (SAH), regardless of the method of aneurysm repairs. We defined the primary and secondary outcomes as the 90- and 30-day poor neurological function, respectively. We determined the exposure variables as the modified WFNS, WFNS and H&H scales. All data elements required for calculating these scales at the time of admission were prospectively collected on a case record form (CRF) and entered into a database via the EpiData Entry software for later analysis. We also determined confounding factors as the variables of the baseline and clinical characteristics, the neuroimaging findings, the management, and the complications. However, we did not yet present the Methods section to reader in an intelligible fashion. Therefore, we have streamlined the items following the TRIPOD statement - the TRIPOD checklist - for reporting a study developing or validating a multivariable prediction model for diagnosis or prognosis.[1]

[1] Moons KG, Altman DG, Reitsma JB, Ioannidis JP, Macaskill P, Steyerberg EW, Vickers AJ, Ransohoff DF, Collins GS. Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): explanation and elaboration. Ann Intern Med. 2015 Jan 6;162(1):W1-73. doi: 10.7326/M14-0698. PMID: 25560730.

5. It is stated that 415 patients presented to the study sites. How many were excluded because of the inadequate documentation of GCS? And how many and why where they lost to follow-up?

Our answer:

Thank you for pointing this out. In the present study, because there was a lack of electronic health record systems in our participating hospitals, all data were prospectively collected on the same unified case record forms (CRF) by representatives/investigators (i.e., fully trained clinicians and surgeons) and were entered into a database via the EpiData Entry software after the completion of data collection for later analysis. Data on patients for whom the initial GCS was unable to be scored or on patients who became lost at both 30- and 90-day follow-up visits would not be entered into a database by investigators. Although no patients became lost at 30 and 90 days of follow-up during the present study, we did not have data on how many patients for whom investigators could not score the initial GCS and excluded them from the present study. We have reworded the paragraph stating that “415 patients presented to the study sites” as follows:

"Data on 415 eligible patients with aneurysmal SAH were submitted to the study database (Fig. 1 and Table 1), in which there were few missing data." (Page 14, Lines 310-311)

6. The conclusion that the mWFSN score was strongest associated with an increased risk of poor outcome at 30/90 days of ictus is a strong statement. This needs to be revised accordingly to the limits of the study.

Our answer:

Thank you for pointing this out. We have reworded the Conclusion sections in the Abstract and the main text as follows:

“In this study, the modified WFNS, WFNS, and H&H scales all had good discriminatory abilities for the prognosis of patients with aneurysmal SAH. Because of the better effect size in predicting poor outcomes, the modified WFNS scale seems preferable to the WFNS and H&H scales.” (Page 4, Lines 80-83)

“This study investigated a selected cohort of patients with aneurysmal SAH, a high rate of poor outcomes and a high mortality rate presented to central hospitals in Vietnam. The modified WFNS, WFNS, and H&H scales all had good discriminatory abilities for the prognosis of patients with aneurysmal SAH. Because of the better effect size in predicting poor outcomes, the modified WFNS scale seems preferable to the WFNS and H&H scales.” (Page 32, Lines 578-582)

7. Although their data do support their conclusion, I do not believe that this has any clinical impact, nor does it help in the clinical decision making of the surgeon /radiologist.

Although interesting, the data inclusion criteria is very weak.

Our answer:

Thank the Reviewer so much for taking the time to leave an excellent review.

Reviewer #3:

The authors present the results of a prospective trial in patients with aneurysmal SAH, attempting to analyze the predictive value of SAH scales for poor outcome. This is an interesting study that carries high importance for the field. However, there are a number of aspects that need to be addressed before publication.

Our answer:

We thank you for the positive feedback. We have carefully considered the Reviewer’s comments and suggestions and have revised our manuscript following each of these points.

• How long was the median follow-up time?

Our answer:

Thank you for this comment. In the present study, all patients received a follow-up visit or phone contact till death in the hospital or within 30 or 90 days after symptom onset of aneurysmal subarachnoid hemorrhage (SAH) and had clinic visits or phone contacts on days 30th and 90th after ictus. We have clarified this issue in the Methods section (Page 7, Lines 154-158). Therefore, the median follow-up time was 30 (IQR: 15 - 20) days.

• Were there any changes of patient status after 90 days ?

Our answer:

In our study, all patients received a follow-up visit or phone contact till death in the hospital or within 30 or 90 days after symptom onset of aneurysmal subarachnoid hemorrhage (SAH) and had clinic visits or phone contacts on days 30th and 90th after ictus. Therefore, data on changes in patient status beyond over 90 days after onset was unavailable in our study.

• What was the outcome at discharge?

Our answer:

Thank you for this comment. In the present study, outcomes at hospital discharge were poor neurological function (poor outcome, defined as mRS of 4 to 6), as shown in S5 Table in S1 File.

• In the participants and treatment section, it is mentioned that “In the case of aphasia, patients were classified according to the clinically possible GCS scores derived from their eye and motor scores. How exactly is the “eye score” performed, and how was it implemented in the GS scale?

Our answer:

Thank you for this comment.

In the present study, all data elements required for calculating the subarachnoid hemorrhage (SAH) grading scales at the time of admission were prospectively collected on a case record form (CRF) and entered into a database via the EpiData Entry software for later analysis.

In the case of aphasia (3.2%; 10/415; kindly see S3 Table in S1 File for additional information), patients were classified according to the clinically possible Glasgow coma scale (GCS) scores derived from their eye and motor scores. For this purpose, the verbal scoring strategy in these patients was replaced by the median verbal score of patients with similar eye-motor scores but with lesions of the non-dominant hemisphere.[30],[31] This strategy involves the following steps: (a) First, A consecutive series of patients are assessed using the GCS. No verbal score is given to those with aphasia; (b) Second, All combinations of eye and motor scores for the patients without aphasia and with lesions of the non-dominant hemisphere are tabulated. The median verbal score is then determined for each combination; and (c) Finally, In patients with aphasia, the verbal score is imputed with the median verbal score of the patients without aphasia with lesions of the non-dominant hemisphere but with the same eye and motor scores.

We have cited the paragraph concerning the verbal scoring strategy in patients with missing verbal scores to two relevant references (Page 8, Line 169; Page 38, Lines 712-717), as follows:

References

[30] Juvela S. Risk factors for impaired outcome after spontaneous intracerebral hemorrhage. Arch Neurol. 1995 Dec;52(12):1193-200. doi: 10.1001/archneur.1995.00540360071018. PMID: 7492294.

[31] Prasad K, Menon GR. Comparison of the three strategies of verbal scoring of the Glasgow Coma Scale in patients with stroke. Cerebrovasc Dis. 1998 Mar-Apr;8(2):79-85. doi: 10.1159/000015822. PMID: 9548004.

• How were missing data handled?

Our answer:

Thank you for this comment. To minimize missing data, we performed the following steps: (a) First, we recorded data for each study patient in the same unified samples (case record form). A case record form was adopted across the study sites to collect the common variables; (b) Second, we submitted the data to the study database via EpiData Entry software, which was used for simple or programmed data entry and data documentation that could prevent data entry errors or mistakes, after the completion of data collection for later analysis; and (c) Finally, we checked the data for implausible outliers and missing fields and contacted hospital representatives for clarification.

As a result, there were few missing data in our study (S21 Table in S1 File). Therefore, we did not use any ways of handling missing values.

• Since the applied SAH grading systems are on an ordinal scale niveau, how meaningful is the approach of calculating ROC, which usually requires continuous data as the diagnostic input variable? For example, formulating a resulting cut-off value of 2.5 for poor outcomes in the modified WFNS scale is clinically challenging to implement. Please elaborate.

Our answer:

Thank you for this comment. In the present study, we converted from descriptive subarachnoid hemorrhage (SAH) grading scales to numerical SAH grading scales in ascending order (kindly see S1 Table in S1 File for additional information). Because of this, the cut-off value of each SAH grading scale was determined by receiver operator characteristic (ROC) curve analysis and defined as more or equal to the cut-off point with the maximum value of Youden’s index (i.e., sensitivity + specificity - 1). However, we have clarified this issue in the Statistical analyses section (Page 12, Lines 257-265) and throughout the manuscript, as follows:

"We converted from descriptive SAH grading scales to numerical SAH grading scales in ascending order (S1 Table in S1 file). Receiver operator characteristic (ROC) curves were plotted, and the areas under the ROC curve (AUROC) were calculated to determine the discriminatory ability of all SAH grading scales for the prognosis of the patients upon admission. The cut-off value of each SAH grading scale was determined by ROC curve analysis and defined as the cut-off point with the maximum value of Youden’s index (i.e., sensitivity + specificity - 1). Based on the cut-off value of each SAH grading scale, we assigned the patients to two severity groups: either the grade that was less than the cut-off value or another that was greater than or equal to the cut-off value." (Page 12, Lines 257-265)

• Comparing odd`s ratios between groups and models has been criticized, in fact, this approach has recently been rejected as an adequate method in this context. Several authors have argued that odds ratios will change if variables are added to the model, even if those additional variables are independent from the other variables. This concern particularly applies to the analysis of differences between clinical outcomes of the adjacent grades. Please comment on this critical aspect.

Our answer:

Thank you for this comment. The Reviewer is right that previous critiques have identified a caution problem with odds ratios,[1-5] and Edward and Bryan (2018) have recently raised this issue again.[6] Allison (1999) explained why odds ratios cannot be compared across samples.[1] Mood (2010) extended this work nicely to show that odds ratios cannot be interpreted as absolute effects, nor can they be compared across models or across groups within models.[2] Several authors have pointed out that odds ratios will change if variables are added to the model, even if those additional variables are independent of the other variables (Gail, Wieand, and Piantadosi 1984;[3] Yatchew and Griliches, 1985;[4] Allison 1999;[1] Mood 2010[2]). Mroz and Zayats (2008) also discussed the effect of omitted variables on the interpretation of odds ratios in logit models. Overall, a recent literature review (Edward and Bryan, 2018) has shown that there is no single odds ratio; instead, any estimated odds ratio is conditional on the data and the model specification.[6] Odds ratios should not be compared across different studies using different samples from different populations.[6] Nor should they be compared across models with different sets of explanatory variables.[6] Therefore, we have removed comparisons of odds ratios for poor outcomes among the intergrades of the subarachnoid hemorrhage (SAH) grading scales. Moreover, we have further discussed the comparison of odds ratios between grades for predicting poor outcomes in the Discussion section. (Pages 30-31, Lines 523-544)

References

[1] Allison PD. Comparing Logit and Probit Coefficients Across Groups. Sociological Methods & Research. 1999;28(2):186-208. doi: 10.1177/0049124199028002003.

[2] Mood C. Logistic Regression: Why We Cannot Do What We Think We Can Do, and What We Can Do About It. European Sociological Review. 2009;26(1):67-82. doi: 10.1093/esr/jcp006.

[3] Gail MH, Wieand S, Piantadosi S. Biased estimates of treatment effect in randomized experiments with nonlinear regressions and omitted covariates. Biometrika. 1984;71(3):431-44. doi: 10.1093/biomet/71.3.431.

[4] Yatchew A, Griliches Z. Specification Error in Probit Models. The Review of Economics and Statistics. 1985;67(1):134-9. doi: 10.2307/1928444.

[5] Mroz TA, Zayats YV. Arbitrarily Normalized Coefficients, Information Sets, and False Reports of “Biases” in Binary Outcome Models. The Review of Economics and Statistics. 2008;90(3):406-13. doi: 10.1162/rest.90.3.406.

[6] Norton EC, Dowd BE. Log Odds and the Interpretation of Logit Models. Health services research. 2018;53(2):859-78. doi: https://doi.org/10.1111/1475-6773.12712.

• Clinically, there are a number of variables that may influence the results of this analysis. In particular, was there an influence of aneurysm location and the frequency of vasospasm / delayed ischemia? How many patients received decompressive craniectomy?

Our answer:

Thank you for this comment. In the present study, we found that vertebral artery (VA) aneurysm and delayed cerebral ischemia (DCI) were significantly associated with poor outcomes on day 30th and 90th after ictus in the univariable logistic regression analyses. However, the multivariable logistic regression analysis showed only DCI was an independent predictor of poor outcomes on day 30th and 90th after ictus. We have added these variables to the multivariable logistic regression model (Tables 5 and 6, and S17 to S20 Tables in S1 File). In our study, surgical hematoma evacuation was defined as any surgical procedure evacuating epidural, subdural, intraventricular, or intraparenchymal hematoma, such as decompressive craniotomy, open craniotomy, or minimally invasive surgery. Of the total patients, 10.6% (44/415) received surgical hematoma evacuation. However, data on the reasons for surgical hematoma evacuation was unavailable in the present study. We have clarified this issue in the Methods section (Page 10, Lines 218-221) and have added these variables to S5 Table in S1 File.

• The strongest risk factor for poor outcome was the modified WFNS score of 3-5, which translates into an initial GCS score of 3-13. This variability makes the application of such a “risk - factor” somewhat challenging to implement in the clinical setting.

Our answer:

Thank you for this comment. In the present study, based on the cut-off value (≥2.5) of modified WFNS (Figs. 2 and 3), we assigned the patients to two severity groups: either the modified WFNS grade of I to II or another grade of III to V. In the multivariable logistic regression model, a modified WFNS grade of III to V was an independent predictor of the poor outcome (Tables 5 and 6). However, when we added the originally-suggested modified WFNS scale, with grade I taken as the reference, to the multivariable logistic regression model, with the same set of confounding variables, we found a gradual increase in adjusted odds ratio (AOR) of the modified WFNS scale, in ascending grades, for predicting the poor outcome (S11 and S14 in S1 File). These findings mean that a modified WFNS grade of III to V was an independent predictor of poor outcome, of which a higher modified WFNS grade was associated with a higher risk of poor outcome. Therefore, this variability does not make applying a “risk factor” challenging to implement in clinical settings (Kindly see Table 1 for additional information).

• To my best knowledge, the modified WFNS score appears to carry a better discriminatory value for good outcomes compared to the original WFNS score. Do the authors find similar results for the patients with a good outcome?

Our answer:

Thank you for this comment. To date, all SAH grading scales have been developed for predicting poor outcomes, in ascending grades, such as the Hunt and Hess (H&H) grading scale, World Federation of Neurosurgical Societies (WFNS) grading scale, or modified WFNS grading scale. It means that a higher grade on the SAH grading scales was associated with a higher risk of poor outcomes, and vice versa. A previously published study shows that the modified WFNS and the original WFNS scale both had good discriminatory ability concerning the prognosis of patients (either good or poor outcome) on day 90th after ictus, with the AUROC value of the modified WFNS scale that was significantly greater than those of the original WFNS scale.[1] In the present study, although the modified WFNS and the original WFNS scale both had good discriminatory ability concerning the prognosis of patients (either good or poor outcome) on day 90th after ictus (Fig. 3), there were no significant differences between the AUROC values of these scales (Table 2). This variation might be because of the differences concerning the outcome measures (i.e., the good outcome, defined as mRS score ≤1 in [1] vs defined as mRS score ≤3 in our study) between the two studies.

References

[1] Sano H, Satoh A, Murayama Y, Kato Y, Origasa H, Inamasu J, Nouri M, Cherian I, Saito N; members of the 38 registered institutions and WFNS Cerebrovascular Disease & Treatment Committee. Modified World Federation of Neurosurgical Societies subarachnoid hemorrhage grading system. World Neurosurg. 2015 May;83(5):801-7. doi: 10.1016/j.wneu.2014.12.032. Epub 2014 Dec 20. PMID: 25535068.

• Since the initial SAH grading is influenced by acute hydrocephalus, how many patients with acute hydrocephalus improved after implantation of an intraventricular drain? Did these patients show a different result regarding the SAH scales ad their prediction of poor outcomes?

Our answer:

Thank you for this comment. In our study, 32.8% (136/415) of patients with aneurysmal subarachnoid hemorrhage (SAH) had a complication of acute hydrocephalus, and 10.4% (43/414) received external ventricular drain (EVD). However, data on the reason for the insertion of an EVD was acute hydrocephalus which accounted for only 7.7% (32/414) of patients with aneurysmal SAH, and only 135 patients with aneurysmal SAH complicated by acute hydrocephalus was it recorded if an EVD was inserted or not, as shown in Table below. Of 135 patients with aneurysmal SAH complicated by acute hydrocephalus (Table below), 23.7% (32/135) of patients received an EVD; only 34.4% (11/32) of whom had the 30-day or 90-day good outcome, defined as a mRS score of 0 to 3. The table below also shows no significant difference in the admission severity and outcomes between patients who received an EVD and patients who did not. Therefore, the present study revealed that EVD did not improve poor outcomes in patients with aneurysmal SAH complicated by acute hydrocephalus, which might not impact the discriminatory ability of SAH grading scales for predicting poor outcomes.

Table: The admission severity and the outcome among patients with aneurysmal subarachnoid hemorrhage complicated by acute hydrocephalus

(See 'Response to Reviewers' file attached)

Thank the Editors and Reviewers so much for taking the time to leave their excellent reviews!

Sincerely yours,

Chinh Quoc Luong, MD., PhD.

Center for Emergency Medicine,

Bach Mai Hospital,

No. 78, Giai Phong, Phuong Mai ward, Dong Da district, Hanoi 100000, Vietnam

Email: luongquocchinh@gmail.com

On behalf of all authors

Attachment

Submitted filename: Response to Reviewers.pdf

Decision Letter 1

Martin Kieninger

9 Jun 2023

PONE-D-22-32313R1Validation of the accuracy of the modified World Federation of Neurosurgical Societies Subarachnoid Hemorrhage Grading Scale for predicting the outcomes of patients with aneurysmal subarachnoid hemorrhagePLOS ONE

Dear Dr. Luong,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Jul 24 2023 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

  • A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

  • A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

  • An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Martin Kieninger

Academic Editor

PLOS ONE

Journal Requirements:

Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #3: All comments have been addressed

Reviewer #4: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #3: Yes

Reviewer #4: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #3: Yes

Reviewer #4: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #3: Yes

Reviewer #4: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #3: Yes

Reviewer #4: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #3: The authors have in fact carefully addressed all aspects I have raised in my review. I congratulate the authors for this important study

Reviewer #4: Dear Dr. Luong. You and the manuscript writing team have made a great and highly precious work. I am really impressed by the study methodology as compared to the original WFNS, HH and MWFNS comparative reviews. The inclusion of all radiologic and epidemiological and clinical data, the transparent way of sample calculation and cut-off estimation as well as the logistic regression analysis were highly precise and well-performed. However, I would like the inclusion of 2 items to enhance the value of this great work:

1- inclusion of PAASH scale in the calculation for ROC analysis curve and the cut-off prognostication table. the 5-category Prognosis on Admission of Aneurysmal Subarachnoid Hemorrhage (PAASH) grading scale has been shown by S.M. Dorhout Mees et al [DOI: 10.1161/STROKEAHA.107.498345] to show a more gradual increase of OR in ascending categories as compared to WFNS scale and as comparable to modified WFNS. In your work, your stated that " the modified WFNS scale did not show more gradual increases in OR for the 30- and 90-day poor outcome, in ascending grades, compared to the WFNS and H&H scales " , so inclusion of PAASH gradual OR prognostication to this context will render your manuscript more comprehensive and recognizable. Additionally, the mRS 4-5 were defined by you as a poor outcome, though some papers include mRS 3 as well. An explanation for the situation of mRS 3 patient categories will be highly valuable. These notes don't demerit your valuable work. Rather, they ensure more worldwide recognition of it. Thank you.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #3: No

Reviewer #4: Yes: Mohamed Mostafa

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

PLoS One. 2023 Aug 22;18(8):e0289267. doi: 10.1371/journal.pone.0289267.r004

Author response to Decision Letter 1


24 Jun 2023

Professor Martin Kieninger

Academic Editor

PLOS ONE

June 24, 2023

Dear Prof. Martin Kieninger,

On behalf of all authors, I am resubmitting herewith our revised manuscript entitled “Validation of the accuracy of the modified World Federation of Neurosurgical Societies Subarachnoid Hemorrhage Grading Scale for predicting the outcomes of patients with aneurysmal subarachnoid hemorrhage” (PONE-D-22-32313R2).

We sincerely appreciate the kind comments and points raised by the Editors and by the Reviewers. We have carefully considered all comments and suggestions and revised our manuscript following each of these points. These comments have enabled us to substantially improve our manuscript. We hope that Editor will find our revised manuscript suitable for publication in PLOS ONE.

We confirm that this work is original and has not been published elsewhere nor is it currently under consideration for publication elsewhere. All authors have read, approved the manuscript, and agreed to authorship and order of authorship for this manuscript, and all authors have the appropriate permissions and rights to the reported data.

We have provided our point-by-point responses to the comments of the Editors and the Reviewers attached.

We thank you for your kind consideration of this submission.

Sincerely yours,

Chinh Quoc Luong, MD., PhD.

Center for Emergency Medicine,

Bach Mai Hospital,

No. 78, Giai Phong, Phuong Mai ward, Dong Da district, Hanoi 100000, Vietnam

Email: luongquocchinh@gmail.com

We thank the Editors and the Reviewers for the valuable comments and suggestions that greatly helped us to improve the contents of this paper. In what follows, we will use the boldface to indicate comments from the Editors and the Reviewers, the standard font face for our responses and we highlighted in yellow the modifications that we did to the manuscript.

RESPONSE TO EDITORS

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Our answer:

We thank you for the positive feedback. We have carefully considered the Reviewers' comments and suggestions and have revised our manuscript following each of these points.

Please submit your revised manuscript by Jul 24 2023 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Our answer:

Thank you for this comment. We have submitted the revised manuscript on time.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

Our answer:

Thank you for this comment. We have included a rebuttal letter, a marked-up copy of the manuscript, and an unmarked version of the revised manuscript when submitting our revised manuscript.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

Our answer:

Thank you for this comment. We do not have any changes in our financial disclosure.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

Our answer:

Thank you for this comment. Laboratory protocol does not apply to our study, but study protocol does. Our study protocol has included in the Methods section.

We look forward to receiving your revised manuscript.

Kind regards,

Martin Kieninger

Academic Editor

PLOS ONE

Our answer:

Thank the Editor and Reviewers so much for their support and for taking the time to leave their excellent reviews.

Journal Requirements:

Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

Our answer:

Thank you for this comment. We have reviewed the reference list, which is complete and correct. There are not any retracted articles on the list.

RESPONSE TO REVIEWERS

Reviewers' comments:

Reviewer #3: The authors have in fact carefully addressed all aspects I have raised in my review. I congratulate the authors for this important study

Our answer:

Thank the Reviewer so much for taking the time to leave his/her excellent review.

Reviewer #4: Dear Dr. Luong. You and the manuscript writing team have made a great and highly precious work. I am really impressed by the study methodology as compared to the original WFNS, HH and MWFNS comparative reviews. The inclusion of all radiologic and epidemiological and clinical data, the transparent way of sample calculation and cut-off estimation as well as the logistic regression analysis were highly precise and well-performed. However, I would like the inclusion of 2 items to enhance the value of this great work:

Our answer:

Thank the Reviewer so much for taking the time to leave his/her excellent review. We have carefully considered the Reviewer’s comments and suggestions and have revised our manuscript following each of these points.

1- inclusion of PAASH scale in the calculation for ROC analysis curve and the cut-off prognostication table. the 5-category Prognosis on Admission of Aneurysmal Subarachnoid Hemorrhage (PAASH) grading scale has been shown by S.M. Dorhout Mees et al [DOI: 10.1161/STROKEAHA.107.498345] to show a more gradual increase of OR in ascending categories as compared to WFNS scale and as comparable to modified WFNS.

Our answer:

Thank you for this valuable comment and suggestion. This study is part of an ongoing research project that aims to develop and validate the accuracy of several subarachnoid hemorrhage (SAH) grading scales based on admission Glasgow Coma Score (GCS) for predicting the outcomes of patients with aneurysmal SAH. With the latest updated dataset, we have performed further analysis and found that the 5-category Prognosis on Admission of Aneurysmal Subarachnoid Hemorrhage (PAASH) grading scale showed a more gradual increase of odds ratio in ascending categories compared to the Hunt and Hess (H&H), World Federation of Neurological Surgeons (WFNS), and modified WFNS grading scales. We will report the result in another paper.

In your work, your stated that " the modified WFNS scale did not show more gradual increases in OR for the 30- and 90-day poor outcome, in ascending grades, compared to the WFNS and H&H scales ", so inclusion of PAASH gradual OR prognostication to this context will render your manuscript more comprehensive and recognizable. Additionally, the mRS 4-5 were defined by you as a poor outcome, though some papers include mRS 3 as well. An explanation for the situation of mRS 3 patient categories will be highly valuable. These notes don't demerit your valuable work. Rather, they ensure more worldwide recognition of it. Thank you.

Our answer:

Thank you for your valuable comment and suggestion. This study is part of an ongoing research project that aims to develop and validate the accuracy of several subarachnoid hemorrhage (SAH) grading scales based on admission Glasgow Coma Score (GCS) for predicting the outcomes of patients with aneurysmal SAH. In this study, we defined good outcomes as a modified Rankin Scale (mRS) score of 0 to 3, which is also used in previously published studies.[1],[2] However, with the latest updated dataset, we will define excellent outcomes as an mRS score of 0 to 2 and report the result in another paper.

[1] Hoh BL, Ko NU, Amin-Hanjani S, Hsiang-Yi Chou S, Cruz-Flores S, Dangayach NS, Derdeyn CP, Du R, Hänggi D, Hetts SW, Ifejika NL, Johnson R, Keigher KM, Leslie-Mazwi TM, Lucke-Wold B, Rabinstein AA, Robicsek SA, Stapleton CJ, Suarez JI, Tjoumakaris SI, Welch BG. 2023 Guideline for the Management of Patients With Aneurysmal Subarachnoid Hemorrhage: A Guideline From the American Heart Association/American Stroke Association. Stroke. 2023 May 22. doi: 10.1161/STR.0000000000000436.

[2] Post R, Germans MR, Tjerkstra MA, Vergouwen MDI, Jellema K, Koot RW, Kruyt ND, Willems PWA, Wolfs JFC, de Beer FC, Kieft H, Nanda D, van der Pol B, Roks G, de Beer F, Halkes PHA, Reichman LJA, Brouwers PJAM, van den Berg-Vos RM, Kwa VIH, van der Ree TC, Bronner I, van de Vlekkert J, Bienfait HP, Boogaarts HD, Klijn CJM, van den Berg R, Coert BA, Horn J, Majoie CBLM, Rinkel GJE, Roos YBWEM, Vandertop WP, Verbaan D; ULTRA Investigators. Ultra-early tranexamic acid after subarachnoid haemorrhage (ULTRA): a randomised controlled trial. Lancet. 2021 Jan 9;397(10269):112-118. doi: 10.1016/S0140-6736(20)32518-6.

Thank the Editors and Reviewers so much for taking the time to leave their excellent reviews!

Sincerely yours,

Chinh Quoc Luong, MD., PhD.

Center for Emergency Medicine,

Bach Mai Hospital,

No. 78, Giai Phong, Phuong Mai ward, Dong Da district, Hanoi 100000, Vietnam

Email: luongquocchinh@gmail.com

On behalf of all authors

Attachment

Submitted filename: Response to Reviewers.docx

Decision Letter 2

Martin Kieninger

17 Jul 2023

Validation of the accuracy of the modified World Federation of Neurosurgical Societies Subarachnoid Hemorrhage Grading Scale for predicting the outcomes of patients with aneurysmal subarachnoid hemorrhage

PONE-D-22-32313R2

Dear Dr. Dao,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Martin Kieninger

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #4: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #4: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #4: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #4: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #4: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #4: I thank editors for their valuable consideration of my review points and would lije to express my apologies for late response due to urgent circumstances. Thank you for sharing me this spectacular chance.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #4: Yes: Mohamed Mostafa

**********

Acceptance letter

Martin Kieninger

11 Aug 2023

PONE-D-22-32313R2

Validation of the accuracy of the modified World Federation of Neurosurgical Societies Subarachnoid Hemorrhage Grading Scale for predicting the outcomes of patients with aneurysmal subarachnoid hemorrhage

Dear Dr. Dao:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Martin Kieninger

Academic Editor

PLOS ONE

Associated Data

    This section collects any data citations, data availability statements, or supplementary materials included in this article.

    Supplementary Materials

    S1 File

    (PDF)

    Attachment

    Submitted filename: Response to Reviewers.pdf

    Attachment

    Submitted filename: Response to Reviewers.docx

    Data Availability Statement

    All relevant data are within the paper and its Supporting information files.


    Articles from PLOS ONE are provided here courtesy of PLOS

    RESOURCES