Prognostic models for prolonged disorders of consciousness: an integrative review

Abstract

Disorders of consciousness (DoC) are acquired conditions of severe altered consciousness. During the past decades, some prognostic models for DoC have been explored on the basis of a variety of predictors, including demographics, neurological examinations, clinical diagnosis, neurophysiology and brain images. In this article, a systematic review of pertinent literature was conducted. We identified and evaluated 21 prognostic models involving a total of 1201 DoC patients. In terms of the reported accuracies of predicting the prognosis of DoC, these 21 models vary widely, ranging from 60 to 90%. Using improvement of consciousness level as favorable outcome criteria, we performed a quantitative meta-analysis, and found that the pooled sensitivity and specificity of the hybrid model that combined more than one technique were both superior to those of any single technique, including EEG and fMRI at the tasks and resting state. These results support the view that any single technique has its own advantages and limitations; and the integrations of multiple techniques, including diverse brain images and different paradigms, have the potential to improve predictive accuracy for DoC. Then, we provide methodological points of view and some prospects about future research. Totally, in comparison to a great many diagnostic methods for the DoC, the research of prognostic models is sparse and preliminary, still largely in its infancy with many challenges and opportunities.

Electronic supplementary material

The online version of this article (10.1007/s00018-020-03512-z) contains supplementary material, which is available to authorized users.

Introduction

Severe brain injury can lead to impaired consciousness. Some patients quickly recover consciousness from acute injuries, while some fall into the extending state of unawareness of self and environment. When such state has lasted for more than 28 days, the patient can be diagnosed as prolonged disorders of consciousness (DoC). The etiologies of DoC include trauma, stroke, anoxia and so on, thus it is a complex, heterogeneous and acquired condition. An estimated 100,000–300,000 patients with prolonged DoC are living in United States [1]. Prevalence figures in European vary from 0.2 to 6.1 patients per 100,000 members of the population [2]. No reliable nationwide data are available on the prevalence of DoC in China; but it is commonly thought that incidence and prevalence of patients with prolonged DoC is progressively increasing, paradoxically, thanks to improvements of medical resuscitation techniques. To date, the DoC is believed to be related to the disturbances of the brainstem, thalamus and cortex, but its pathophysiological mechanisms is not yet clear [3]. Because of impaired responsiveness, the DoC patients cannot communicate functionally or behave purposefully. They remain bedridden and are completely dependent on others for care. The social, economic, and ethical consequences are, therefore, tremendous.

For a long time, the DoC patients have been considered to have a universally poor prognosis, so there is a general pessimism that meaningful recovery is unlikely for DoC patients [4]. Fortunately, the findings from recent clinical studies herald a potential philosophy shift: the long-time outcome cannot be considered as definitive for DoC patients [1]. Further progression of clinical conditions beyond the classical temporal limits of 6 months after a non-traumatic and 12 months after a traumatic brain injury can be no longer considered exceptional. The outcomes of DoC patients have improved over the last two decades [5]. For example, researchers suggest that 10–24% of vegetative state patients, i.e., a subcategory of DoC who are usually considered to have a poor prognosis, can regain consciousness sometimes years after the brain injury [6]. Spontaneous recovery from prolonged DoC occurs more regularly than previously believed. Specifically, some patients evolve from vegetative state/unresponsive wakefulness syndrome (VS/UWS) into minimally conscious state (MCS) [7], regain consciousness [8–10], and stay independent at home as well as even return to school or work [11, 12]. Totally, increasing follow-up reports demonstrate the potential for functional recovery in prolonged DoC. A recent guideline committee, therefore, suggested that the term “permanent” should be abandoned when discussing the prognosis of DoC [13].

An accurate prognostication is important not only for the patient and the family, but also for the clinicians and caregivers. The medical community is often confronted with prognostic counseling of families of DoC patients. Prognostic models can help clinicians more effectively translate aggregate statistical data into care decisions for individual patients. On the other hand, although very few treatment options exist for patients with DoC at present, recent studies suggest that some patients with prolonged DoC might benefit from therapeutic interventions, even years after the injury [14]. Furthermore, recent pilot studies have proposed new therapeutic interventions [15–17]. Before using all of these therapeutic interventions, clinicians first need to determine if the patient is a suitable candidate, and thus the prognostication has been an indispensable step.

To date, the prognostications for DoC patients have still depended on expert observation of the patient’s behavior over a sufficient period of time. However, behavioral assessments are inevitably subjective and vulnerable to a variety of personal interferences [18]. More importantly, the DoC patients share the common impaired consciousness, but have many different causes and are associated with several neuropathological processes and different severities. The association between clinical symptoms and the underlying biological substrates is inconsistent and variable at the individual level, even when the behavioral symptoms are very similar. Clinicians and scientists have, therefore, been seeking an evidence-based prognostication for DoC at the individual level [19, 20]. The high clinical demands have led to efforts to develop prognostic models, which utilize one or more predictors to predict an estimate of specific functional recovery, for example, conscious recovery, within a specific period. During the past decades, some prognostic models have been explored on the basis of the features of patient’s demographics, neurological examinations, clinical diagnosis, neurophysiology and brain images. These models presented different predictive accuracies for detecting a patient with prolonged DoC at baseline who would have clinically favorable outcome at follow-up.

To determine the prognostic effects of the various techniques, we review recent advances in the prognostic models for patients with DoC. First, a systematic review of pertinent literature is conducted. Then, using a quantitative meta-analysis, we report the pooled sensitivities and specificities of the various prognostic techniques. Finally, we provide methodological points of view and some prospects about future research.

Methods

Search methods for identification of studies

As suggested in previous research [21], this review defined “prognostic model” as that which converts one or more predictors to an estimate of functional recovery within a specific period at an individual level. An initial search was conducted on the 24 August 2019 across the databases (i.e., all databases in the Web of Science collections: Web of Science Core Collection; Chinese Science Citation Database; Current Contents Connect; Derwent Innovations Index; KCI-Korean Journal Database; MEDLINE^®; Russian Science Citation Index; and SciELO Citation Index) using the following search terms: (“prognos*” OR “predict*” OR “outcome”) AND (“disorders of consciousness” OR “unresponsive” OR “vegetative state” OR “minimally conscious state” NOT “Coma”). The search language was set to English. Publication time was limited after 2002, because the consensus-based criteria for diagnosing MCS were published in 2002 [22]. As the next step, since outcome prediction was not the main aim of some studies but rather a by-production of other analyses (for example, diagnosis), the review authors consulted recent systematic reviews on DoC [1, 23, 24]. That is, the review authors screened and cross-referenced these review articles for the studies that identified the patient with prolonged DoC who would achieve conscious improvement after a period of time.

This process of search was updated on the 6 January 2020 to identify additional articles published after the initial search.

Data collection and analysis

Selection of studies

One author (MS) performed the first assessment of the search results to remove the duplicate studies. Next, two authors (MS and YY) independently reviewed the remaining titles and abstracts for potentially eligible studies for full paper review. Then, the same two authors independently assessed full manuscripts against the exclusion criteria. The exclusion criteria included: (1) were not studies of diagnostic test accuracy (i.e., did not identify the patients with prolonged DoC at baseline who would have clinically favorable outcome at follow-up); (2) the total sample size of involved patients was less than 20, since too few samples distort the evaluation of predictive accuracy (for example, sensitivity and specificity); (3) fully using patients’ survival instead of functional improvement as the prognostic criterion; (4) focused on the patients in the acute or sub-acute stage after brain injury (i.e., enrollment time < 28-day post-injury). There have been a lot of prognostic models that emphasize on the acute phases (for example, within 1 week), especially, for trauma and cardiac arrest. Since DoC is diagnosed after at least 28-day post-injury, this review has not included these acute or sub-acute brain injury studies. Where necessary, a third review author (JH) resolved disagreements that the other two were not able to resolve through discussion. We present a PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) flow diagram to outline the number of studies at each decision-making point for including studies in this review, as shown in Fig. 1.

Fig. 1.

Study flow diagram

Data extraction and management

One author (MS) extracted the following data from the enrolled articles using a pre-determined extraction form.

Bibliographic details of paper:

• Author, title, year and journal

Basic clinical details:

• Number of samples

• Diagnoses for patients

Details of the prognostic model:

• Modeling method

• Features used in the prognostic model

• Criterion of the prognostications

• Validation method

Details of the outcome:

• Assessments of outcome, including the name of tests and the duration of follow-up

We tried to create a 2 × 2 table for each prognostic model according to the information described in the published article. Where a published article did not present all relevant data for creating a 2 × 2 table, we still retained the study in the list of the models, but did not include it in any meta-analysis. Notably, some of the published articles involved more than one prognostic models [25–28]; but since these models under study were developed using the same population, the reported accuracies were not independent. Therefore, only the model with the highest accuracy was presented and summarized.

Assessment of methodological quality

We assessed the methodological quality of each enrolled study using the QUADAS-2 tool [29], which consists of four domains: (1) participant selection; (2) index test; (3) reference standard; (4) participant flow and timing. Two independent raters (MS, YY) independently performed the QUADAS-2 assessment, resolving disagreement by further review and discussion, with potential to involve a third review author (JH) as arbiter if necessary. We assessed each domain in terms of risk of bias, with the first three domains also considered in terms of applicability concerns.

Statistical analysis and data synthesis

The prognostic models were divided into seven subcategories based on the type of techniques and paradigms, including “neurological examinations”, “EEG: tasks” (i.e., EEG signal responses to stimulus or event-related potential), “EEG: resting state” (i.e., EEG signal oscillations at the resting state), “FDG-PET: resting state” (i.e., ¹⁸F-FDG PET metabolism at the resting state), “fMRI: tasks” (i.e., fMRI signal responses to stimulus or followed by command), “fMRI: resting state” (i.e., fMRI signal oscillations at the resting state), and “hybrid model” (i.e., combining more than one type of technique).

Here, we defined the improvement of consciousness level as a “favorable prognosis”. In detail, the criteria included any of the following two outcome assessments: (1) the increase in Glasgow Outcome Score (GOS) or extended Glasgow Outcome Score (GOS-E); (2) the improvement in DoC diagnostic category.

For each model involved in the meta analysis, we calculated the summary statistics, namely sensitivity, specificity and diagnostic odds ratio (DOR). As usual, sensitivity was defined as (true positives)/(true positives + false negatives), specificity as (true negatives)/(true negatives + false positives), and DOR as (true positives × true negatives)/(false positives × false negatives). If a 2 × 2 table contained zeroes, we added 0.5 to all counts in the table to calculate an approximation of the DOR [30]. In addition, the 95% confidence intervals (95% CI) for each individual model were calculated. Next, we conducted an exploratory univariate meta-analysis for DOR with the R package “meta”. Then, we calculated the pooled sensitivity and specificity for each of the seven subcategories using the R package “bamdit” (i.e., Bayesian meta-analysis of diagnostic test data). As a bivariate meta-analysis method, the “bamdit” enables calculation of summary estimates while accounting for variation within and between models and any potential correlation between sensitivity and specificity [31].

Investigations of heterogeneity

In the univariate meta-analysis for DOR, the between-study statistical heterogeneity was assessed using I² and the Cochrane Q test on the basis of the random-effects analysis. Next, for the prognostic subcategories with moderate heterogeneity (i.e., I² > 50%), we visually examined the forest plots of sensitivity and specificity to explore the potential sources of heterogeneity.

Assessment of reporting bias

Publication bias is the most well known reporting bias. On one hand, we performed systematic electronic searches, and additionally consulted recent systematic reviews on DoC to retrieve as many eligible studies as possible for inclusion in this review. On the other hand, we calculated the Deeks’ funnel plot asymmetry test to give an estimation of publication bias for this review. In fact, there is still a lot of uncertainty about the determinants and extent of publication bias for test accuracy studies. Traditionally used methods for assessing funnel plot asymmetry, such as Begg’s test and Egger’s test, are not recommended for test accuracy reviews, and thus a more appropriate method for detecting funnel plot asymmetry in test accuracy reviews is the Deeks’ test [32–34]. Although the Deeks’ test has low power when there is heterogeneity in test accuracy reviews [35], we calculated this test to give an estimation of publication bias. Specifically, we performed a test for the correlation coefficient between log diagnostic odds ratios (DOR) and inverse root of effective sample sizes (ESS), and a P value less than 0.1 was considered statistically significant.

Results

Results of the search

We conducted our electronic literature searches in August 2019 and January 2020, which, respectively, yielded 130 and additional 15 records. Following our initial review of titles and abstracts, we retrieved 101 full-text papers to assess for eligibility against our exclusion criteria. Of these, we excluded 77 publications, largely because the studies were not diagnostic test accuracy studies, or the number of the involved patients was less than 20, or the studies focused on acute or sub-acute patients. Thus, we included 24 publications in this systematic review. These studies are shown in Table 1, which provides the following information: outcome assessments; sample size; prognostication target; model type; validation method; features used as input in the prognostic model; and accuracy. Since the three of the 24 studies did not provide enough information to calculate the 2 × 2 table for sensitivity and specificity, the remaining 21 prognostic models were quantitatively evaluated in the meta-analysis.

Table 1.

Prognostic models for prolonged DoC

Study	Outcome assessments	Sample size	Prognostication	Model type	Validation method	Features	Accuracy	TP	FP	FN	TN
Neurological examinations
Dolce et al. [42]	GOS score at 50, 100 and 180 days after brain injury	333 VS/UWS	Good (i.e. GOS 4-5) or negative (i.e. GOS 1-2) at each time point	CART J48 decision tree	Tenfold cross validation	Twenty-two neural reflexes	Accuracy ranging from 74–83% for each time point: no reports for sensitivity or specificity	–	–	–	–
Billeri et al. [36]	CRS-R and GOS score after 18- month follow-up	19 VS/UWS, 18 MCS	Responders (i.e. improvement in GOS and/or DoC category) or non-responders	Thresholding	No cross validation	Short-latency afferent inhibition	Accuracy = 83.8%, Sensitivity = 66.7%, Specificity = 100%	12	0	6	19
EEG: tasks
Logi et al. [37]	Behavioral-based observations or reports after 5 months of EEG recording	44 VS/UWS, 6 Coma	Recovery of consciousness (i.e. able to carry out simple orders or perform reproducible and sustained purposeful behavior) or not	EEG reactivity pattern present or not	No cross validation	EEG reactivity to external auditory and painful stimulus	Accuracy = 76%, Sensitivity = 68.7%, Specificity = 88.9%	9	7	0	27
Estraneo et al. [27]	Clinical criteria and CRS-R at 24 months postonset	43 VS/UWS	Responsive (MCS or better) or unresponsive (VS)	Somatosensory evoked potentials present or not	No cross validation	Median nerve somatosensory evoked potentials	Accuracy = 83.7%, Sensitivity = 100%, Specificity = 79.4%	22	2	10	16
Steppacher et al. [28]	CRS score and the Barthel Index after 2–14 years of brain injury	48 VS/UWS, 39 MCS	Recovery of consciousness (i.e. functionally communicate) or not	N400 present or not	No cross validation	N400	Accuracy = 80.4%, Sensitivity = 46.8%, Specificity = 98.3%	14	1	16	56
Bagnato et al. [43]	CRS-R score after 3 months of EEG recording	57 VS/UWS, 44 MCS	Improved CRS-R score or not	Amplitude–Frequency–Reactivity (AFR) scale	No cross validation	EEG amplitude, frequency, and reactivity to forced eye opening	*Estimated accuracy = 60.4%, sensitivity = 48.4%, estimated specificity = 79.5%	30	8	32	31
Li et al. [25]	Modified GOS score after 1 year	12 VS/UWS, 9 MCS	Improved GOS score or not	EEG reactivity pattern present or not	No cross validation	EEG reactivity to thermal stimulation	Accuracy = 85.7%, sensitivity = 90.0%, specificity = 81.8%	9	2	1	9
Scarpino et al. [44]	CRS-R score after about 3–4 months of EEG recording	61 VS/UWS, 41 MCS	Improved DoC category or not	Reactivity pattern present or not	No cross validation	EEG presence of reactivity to auditory and/or noxious stimuli	Accuracy = 76.5%, sensitivity = 71.7%, specificity = 81.6%	38	9	15	40
EEG: resting state
Sara et al. [45]	GOS-E after 6 months of EEG recording	38 VS/UWS	Improved (i.e. GOS-E > 2) or unimproved (GOS-E ≤ 2, i.e. still VS or dead)	Mean EEG approximate entropy ≥ 0.8 or not	No cross validation	EEG approximate entropy	Accuracy = 94.7%, Sensitivity = 84.6%, Specificity = 100%	11	0	2	25
Sitt et al. [46]	CRS-R score after 42 days of EEG recording	71 VS/UWS	Improved (i.e. to MCS or better) or unimproved (i.e. still VS)	ROC curve analysis	No cross validation	Normalized power in the theta frequency band	Accuracy = 66.2%, Sensitivity = 72.7%, Specificity = 63.3%	16	18	6	31
Schorr et al. [47]	CRS-R score after 12 months of EEG recording	58 VS/UWS	Improved (i.e. to MCS or better) or unimproved (i.e. still VS or dead)	ROC curve analysis	No cross validation	Parietal coherence in theta band	Accuracy = 77.6%, Sensitivity = 72.7%, Specificity = 78.7%	8	10	3	37
Chennu et al. [48]	GOS-E score after 1 year follow-up	61 DoC patients	Positive outcome (i.e. GOS-E > 2) or negative outcome (i.e. GOS-E ≤ 2)	SVM	Fourfold cross validation	Modularity and clustering coefficients in EEG delta band	Accuracy = 82%, Sensitivity = 92%, Specificity = 64%	36	8	3	14
Stefan et al. [38]	CRS-R after average 1 year follow-up	39 VS/UWS	Improved (i.e. to MCS or better) or unimproved (i.e. still VS or dead)	Generalized linear model, and binary classification based on ROC curve	Tenfold cross validation	Frequency of microstate A in the 2–20 Hz frequency band, path length obtained from thresholding alpha coherence, and clustering coefficient obtained from thresholding alpha coherence	*Estimated accuracy = 87.2%, sensitivity = 80%, estimated specificity = 89.7%	8	3	2	26
FDG-PET: resting state
Stender et al. [26]	GOS-E score after 1 year follow-up	26 VS/UWS, 76 MCS	Conscious (i.e. GOS-E > 2) or unconscious (i.e. GOS-E ≤ 2).	Thresholding	No cross validation	18F-FDG PET bilateral metabolism of the associative frontoparietal cortex	Accuracy = 74%, sensitivity = 67%, specificity = 92%	51	25	2	24
fMRI: tasks
Coleman et al. [49]	CRS-R score after 6 months of fMRI scanning	22 VS/UWS	Progressed to MCS or not	Activation pattern present or not	No cross validation	Response to speech stimuli or semantic aspects of speech	Accuracy = 95.4%, sensitivity = 87.5%, specificity = 100%	7	0	1	14
Vogel et al. [50]	CRS-R score after 40-480 days of fMRI scanning	10 VS, 12 MCS	Improved DoC category or unimproved	Activation pattern present or not	No cross validation	Activation in the mental imagery task	Accuracy = 74%, sensitivity = 96.2%, specificity = 49.0%	11	1	3	7
Wang et al. [51]	CRS-R score after 12 months of fMRI scanning	39 VS/UWS	Improved (i.e. to MCS or better) or unimproved (i.e. still VS or dead)	Activation pattern present or not	No cross validation	fMRI response to subject’s own name in associate auditory cortex or not	Accuracy = 74%, Sensitivity = 75%, Specificity = 73.9%	12	6	4	17
fMRI: resting state
Wu et al. [39]	GOS score after three months follow-up	18 VS/UWS, 14 coma	Awaken (i.e. GOS > 2) or unawaken (i.e. GOS ≤ 2).	SVM	Leave-one-out cross validation	Functional connectivity strength in each voxel across the whole brain, and then feature selection	Accuracy = 81.25%, Sensitivity = 88.89%, Specificity = 71.43%	16	4	2	10
Qin et al. [40]	GOS score after three months follow-up	50 VS/UWS	Emerged from UWS (i.e. GOS > 2) or remained in UWS (i.e. GOS ≤ 2).	Thresholding	No cross validation	Functional connectivity between posterior cingulate cortex (PCC) and left lateral parietal cortex (LLPC)	Accuracy = 74%, Sensitivity = 81%, Specificity = 69%	17	9	4	20
Hybrid model
Xu et al. [52]	CRS-R score at 12 months after traumatic brain injury	58 DoC	MCS (or better) group or unconscious (VS and brain death) group	Binary logistic regression	No cross validation	Somatosensory evoked potentials, GCS	Accuracy = 85.7%, Sensitivity = 87.5%, Specificity = 84.4%	19	3	3	33
Kang et al. [53]	GOS score after 1 year brain injury	56 VS/UWS	recovery of awareness (GOS > 2 or MCS) or unrecovery of awareness (GOS ≤ 2)	Manual designed scale	No cross validation	Motor response, type of brain injury, EEG reactivity to the noxious stimulation, sleep spindles and N20	Accuracy = 85.7%, Sensitivity = 87.5%, Specificity = 84.4%	21	5	3	27
Song et al. [41]	GOS score after at least 1 year follow-up	Training dataset: 46 VS/UWS, 17MCS Testing dataset 1: 20 VS/UWS, 5 MCS Testing dataset 2: 16 VS/UWS, 8 MCS	Recovery of consciousness (GOS > 2) or unrecovery of consciousness (GOS ≤ 2)	Partial least square regression, and binary classification based on ROC curve	Cross validation in training dataset, and independent testing in two testing datasets	Functional connectivity patterns using resting state fMRI, as well as patient’s etiology, age, and duration of injury	Training dataset: accuracy = 92%, sensitivity = 85%, specificity = 94% Testing dataset 1: accuracy = 88%, sensitivity = 83%, specificity = 92% Testing dataset 2: accuracy = 88%, sensitivity = 100%, specificity = 83%	28	7	4	73
Lucca et al. [54]	CRS-R scores till to 8 weeks follow-up	123 VS/UWS, 57 MCS	Emergence from DoC or not (i.e. still VS or MCS)	Cox regression	Independent testing	Age, sex, etiology, ICU days and MCS/VS state	C-index = 0.708: no reports for sensitivity or specificity	–	–	–	–
Steppacher et al. [55]	The ability of functional communication at 2–15 years after the DoC patients’ discharge	59 VS/UWS, 43 MCS	Functionally communicate or not	Logistic regression	No cross validation, but test the stability across different analytic strategies	Diagnosis, N400, P300	AIC = 74.66: no reports for sensitivity or specificity	–	–	–	–

TP true positives, FP false positives, FN false negatives, TN true negatives

*Use Youden index to define the cut-off

Methodological quality of included studies

We assessed methodological quality using the QUADAS-2 tool. The overall methodological quality of included study cohorts is summarized in Fig. 2. The review authors’ judgments about each methodological quality item for each study included in the meta-analysis are shown in Supplementary Material 1.

Fig. 2.

Risk of bias and applicability concerns graph. The methodological quality of each included study was assessed using the QUADAS-2 tool, which consists of four domains: (1) participant selection; (2) index test; (3) reference standard; (4) participant flow and timing. The review author’s judgements about each domain presented as percentages across the included studies

Assessment of publication bias

Using the Deeks’ funnel plot asymmetry test, we assessed the publication bias for this review, and found that the test of the correlation coefficient between log DOR and inverse root of ESS has P value > 0.1, as shown in Fig. 3. Therefore, the hypothesis that there is no publication bias cannot be rejected.

Fig. 3.

Deeks’ funnel plot asymmetry test to assess for publication bias. This method is based on the test of the correlation coeffcient between log diagnositic odds ratios and inverse root of effective sample sizes. The test of the correlation coeffcient has P value > 0.1, which suggests that the hypothesis that there is no publication bias cannot be rejected. The circle size is proportional to the sample size of the study. Different colors represent different techniques: pink for “Neurological examination”, yellow for “EEG: tasks”, red for “EEG: resting state”, green for “FDG-PET: resting state”, blue for “fMRI: tasks”, teal for “fMRI: resting state”, purple for “hybrid model”, respectively

Findings

The 21 prognostic models provided data eligible for inclusion in meta-analysis. Therefore, the total of 1201 patients with DoC was involved in the meta-analysis. We found that 43% of the total involved patients (i.e., 517 DoC patients) achieved “favorable prognosis”, i.e., the improvement of consciousness level, after a specific follow-up.

The results of the univariate meta-analysis for DOR are shown in Fig. 4. The DOR of all 21 prognostic models were 19.492 (95% CI 12.032–31.577) with moderate heterogeneity (I² = 44%, p = 0.02). After dividing the 21 prognostic models into the seven subcategories based on the type of techniques and paradigms, we noticed that there was still moderate heterogeneity for the subcategories “EEG: tasks” (I² = 51%, p = 0.07) and “EEG: resting state” (I² = 51%, p = 0.09). Potential sources of heterogeneity for these two subcategories might cover many factors, including study designs, features and modeling methods. Owing to an insufficient number of studies, we did not further perform moderator analysis to investigate the effects of potential source of heterogeneity on the meta-analysis.

Fig. 4.

Results of the univariate meta-analysis for DOR

In addition, we tested whether the predictive accuracy of the model and the sample size of involved patients improved with the year of publication, respectively. On one hand, we found that the DOR of the model did not improve along with the year of publication (robust regression, linear fitting formula is as follows: “DOR = − 2.240 × Year + 4546.475”, the P value for the t statistic of the hypothesis test that the slope coefficient was equal to zero or not was 0.431). On the other hand, we found that the sample size of involved patients in a single model showed an increasing trend along with the year of publication (robust regression, linear fitting formula is as follows: “Sample_size = 3.278 × Year − 6547.214”, the P value for the t-statistic of the hypothesis test that the slope coefficient was equal to zero or not was 0.181). These results are shown in Fig. 5.

Fig. 5.

Trends of the study characteristics of prognostic model for DoC along with the year of publication. a The DOR of the model; b the sample size of involved patients in a single model. The circle size is proportional to the sample size of the study. Different colors represent different techniques. The color correspondences are the same as Fig. 3

Individual technique accuracy

The results of the bivariate meta-analysis for the sensitivity and specificity are presented in Fig. 6. In addition, the Bayesian predictive surface by contours at 50%, 75% and 95% credibility levels is shown in Fig. 7.

Fig. 6.

Results of the bivariate meta-analysis: sensitivities, specificities, and 95% confidence intervals (95% CI)

Fig. 7.

Results of the bivariate analysis: Bayesian predictive surface by contours at different credibility levels. a Display of the predictive accuracy for each of 21 prognostic models. Each circle identifies the true positive rate vs. the false positive rate of each model. Different colors are used for different techniques and different sizes for sample sizes. b–f Bayesian predictive surface by contours at different credibility levels for the technique of “EEG: tasks”, “EEG: resting state”, “fMRI: tasks”, “fMRI: resting state”, and “Hybrid model”, respectively

Neurological examinations

Only one model based on the “neurological examinations” was included in the meta-analysis [36]. In Billeri et al., 2019 [36], the reported sensitivity was 66.7% (95% CI 44.9–88.4%); and the specificity was 100% (95% CI 100–100%).

EEG: tasks

Using the technique of “EEG: tasks”, six models were developed. The six models totally involved 404 DoC patients with 48.5% of them (i.e., 196 patients) achieving the improvement of consciousness level after a specific follow-up. Sensitivity ranged from 46.7 to 100%, and specificity from 79.4 to 98.3%. The pooled sensitivity was 65.1% (95% CI 54.7–68.7%), and the pooled specificity was 85.9% (95% CI 79.3–88.0%).

With inclusion of only data from two studies that were at low risk of bias (i.e., [27, 37]), the pooled sensitivity changed to 76.3% (95% CI 59.2–89.6%), and pooled specificity to 82.5% (95% CI 69.2–92.1%).

EEG: resting state

Five models were developed using the technique of “EEG: resting state”. The five models totally involved 267 DoC patients, and 35.6% of them (i.e., 95 patients) achieved the improvement of consciousness level at the follow-up. Sensitivity ranged from 72.7 to 92.3%, and specificity from 63.3 to 100%. The pooled sensitivity was 82.6% (95% CI 72.7 to 90.3%), and the pooled specificity was 78.7% (95% CI 69.7 to 86.6%).

Only one study was at low risk of bias [38]. In Stefan et al., 2018 [38], the reported sensitivity was 80.0% (95% CI 55.2 to 100%), and the specificity was 89.7% (95% CI 78.6 to 100%).

FDG-PET: resting state

Only one prognostic model was proposed based on the technique of “FDG-PET: resting state” [26]. In this model, 102 DoC patients were involved, and 52.0% of them (i.e., 53 patients) achieved consciousness recovery after 1-year follow-up. The reported sensitivity was 96.2% (95% CI 91.1–100%), while the specificity was 49.0% (95% CI 35.0–63.0%).

fMRI: tasks

Using the technique of “fMRI: tasks”, three models were developed. The three models totally involved 83 DoC patients, and 45.8% of them (i.e., 38 patients) achieved the improvement of consciousness level after a specific follow-up. Sensitivity ranged from 75.0 to 87.5%, and specificity from 73.9 to 100%. The pooled sensitivity was 78.6% (95% CI 63.5–90.3%), and the pooled specificity was 84.2% (95% CI 71.1–93.8%).

The three models were at high risk of bias, since selecting a test threshold for detecting brain activation to optimize sensitivity and specificity may lead to overestimation of the model’s performance.

fMRI: resting state

There were two models developed based on the technique of “fMRI: resting state”, which totally involved 82 patients with 47.6% of them (i.e., 39 patients) achieving the improvement of consciousness level after a specific follow-up. Sensitivity ranged from 81.0 to 88.9%, and specificity from 69.0 to 71.4%. The pooled sensitivity was 83.8% (95% CI 68.4–94.0%), and specificity was 69.7% (95% CI 52.5–83.8%).

The two models had high risk of bias (i.e., involving some patients in Coma [39] or selecting a specified threshold [40]).

Hybrid model

Three prognostic models based on the technique of “Hybrid model” were involved in the meta-analysis. The three models totally included 226 DoC patients, and 34.5% of the patients (i.e., 78 patients) achieved the improvement of consciousness level after a specific follow-up. Sensitivity ranged from 86.4 to 87.5%, and specificity from 84.4 to 91.3%. The pooled sensitivity was 86.9% (95% CI 77.5–93.7%), and pooled specificity was 89.4% (95% CI 82.7–94.4%). In addition, from Fig. 5, we can clearly see that, in synthesis, the “Hybrid model” is more sensitive and specific and less uncertain than the other models.

Only one study was at low risk of bias [41]. In Song et al. [41], the reported sensitivity was 87.5% (95% CI 76.0–99.0%), and the specificity was 91.3% (95% CI 85.1–97.4%).

Comparison of accuracy between EEG and fMRI

The EEG and fMRI have been among the most widely used brain imaging techniques in the research field of DoC. We pooled the models using the techniques “EEG: tasks” and “EEG: resting state” into a single category “EEG”; and on the other hand, we pooled the models using “fMRI: tasks” and “fMRI: resting state” into a single category “fMRI”. We found that the pooled “EEG” for the prediction of “favorable prognosis” had lower sensitivity (72.6%, with 95% CI 64.4–80.0%) in comparison to the pooled “fMRI” (81.7%, with 95% CI 71.5–89.9%), but higher specificity (82.9%, with 95% CI 77.2–87.9%) than the pooled “fMRI” (77.7%, with 95% CI 67.2–86.7%). These results are shown in Supplementary Material 2(A).

Comparison of accuracy between tasks and resting state

We pooled the models using the techniques “EEG: tasks” and “fMRI: tasks” into a single paradigm “tasks”, and then pooled the studies using the techniques “EEG: resting state” and “fMRI: resting state” into a single paradigm “resting state”. We found that the paradigm “tasks” for the prediction of “favorable prognosis” had lower sensitivity (68.9%, with 95% CI 59.7–77.7%) in comparison to the paradigm “resting state” (83.2%, with 95% CI 75.4–89.6%), but higher specificity (85.8%, with 95% CI 80.1–90.8%) than the “resting state” (76.7%, with 95% CI 68.5–84.0%). The details about these results are shown in Supplementary Material 2(B).

Discussion

Summary of main results

Our systematic review shows that some prognostic models have been proposed to early predict the prognosis of the DoC, but the predictive accuracies vary widely and range from 60 to 90%. Furthermore, the predictive accuracy (i.e., DOR) did not improve along with the year of publication, which suggests clinicians and scientists are doing some exploratory work in this area. The prognostic models for DoC are still in the developing phase.

Among these prognostic models, some used a single type of technique, while some integrated multiple techniques to propose the hybrid models. Although the evidence is limited, it suggests that the hybrid model could show a better predictive accuracy. We found that the “EEG” for the prediction of “favorable prognosis” had lower sensitivity but higher specificity in comparison to the “fMRI”; the paradigm “tasks” had lower sensitivity but higher specificity than the paradigm “resting state”. These results altogether support the view that any single technique has its own advantages and limitations, so any method when used alone carries a high risk of false prediction; and the integrations of multiple techniques, including diverse brain images and different paradigms, into one model have the potential to improve predictive accuracy.

Implications for future research

In our literature search for this review, we identified 24 prognostic models that met the following criteria: a classification or regression model to predict the prognosis of DoC; a minimum of 20 patients included; focusing on the chronic phase of DoC; and using functional improvement instead of survival as outcome assessments. Since the three of the 24 studies did not provide enough information to calculate the 2 × 2 table for sensitivity and specificity, only 21 prognostic models were quantitatively evaluated in the meta-analysis. Compared to a large number of diagnostic models for the DoC [126], the prognostic models are very few and sparse.

Second, prognostic model research has three main phases: model development (including prognostic factor research), validation, and investigations of impact in clinical practice. Unfortunately, most of these prognostic models are particularly lack of validations, which is likely to bring a high risk of overfitting (i.e., predictive performance is much poorer in new patients than expected from the development phase). The 17 of 21 models did not employ any validations; thus, it is difficult to evaluate the generalization performance of these models. The 3 of 21 models adopted cross validation, and only one study reported external validation, i.e., testing the model’s performance in another setting that differs in medical management teams as well as MRI scanners and scanning protocols [41].

Third, the criteria of follow-up and outcome assessments were not uniform and varied greatly. Some studies followed up for more than 1 year, while some studies reported the outcome at 3 months or even several weeks. In fact, the reported follow-up time is not equal to the time since injury. The time spent in DoC correlates with widespread brain atrophy and dysfunction. Although not much is known about the degradative process, a whole cascade of molecular processes is believed to take place, such as inflammation, apoptosis and necrosis in the neighborhood of the primary injury. Secondary brain injury could further influence the patient’s recovery. Consequently, prognostic analysis should be considered as a continuous process that needs updating and validations in contemporary series. Therefore, serial standardized evaluations to identify trends in the trajectory of recovery and more uniform outcome assessments should be paid more attention in the future research of prognostic models.

Fourthly, the modeling method is likely to be too simplified. Many of prognostic models used univariate analysis to estimate the predictive accuracy. Although this kind of univariate analysis simplified the analysis of relation between a single predictor and outcome, it would decrease the predictive accuracy of models. Predictors should be considered jointly rather than on their own, and can be combined in a multivariable prognostic model to quantify the risk for a particular outcome in individual patients. By combining individual predictors into a model, it will not only increase performance and generalizability, but also enhance the clinical applicability and interpretation of the model. In practice, some patients could have characteristics that affect the outcome in opposite directions. For example, for a 25-year-old DoC patient with anoxia, we would expect a favorable outcome based on age, but an unfavorable outcome based on the etiology of anoxia. Therefore, a good multivariable prognostic model will increase clinical utility, and be helpful to understand the effects of different factors in the prognosis. In addition, the vast majority of the current prognostic models are typically restricting their prediction to functional recovery as a dichotomous variable or not, shedding little light on the overall level of recovery. Notably, in these reviewed studies, the “favorable prognosis” was defined in varying ways, such as GOS score > 2, GOS-E > 2, or recovery of awareness (MCS or better).

Finally, almost all of the current prognostic models did not investigate the effects of treatment and rehabilitation. A DoC patient often receives more or less treatments and rehabilitations in clinical practice. Although the effects on the prognosis are not yet clear, these different treatments and rehabilitations could unpredictably influence a patient’s clinical evolution. Therefore, more caution is required when interpreting the performances of the prognostic models. It is expected that future prognostic models could involve and compare the effects of different interventions/care methods on the patient’s prognosis.

Strengths and weaknesses of the review

The presented 21 prognostic models were developed on the basis of different study designs and protocols. Thus, the processing protocols and parameters varied widely across these models. We organized the 21 prognostic models to seven subcategories based on the type of techniques and paradigms. Even so, there was still moderate heterogeneity for the subcategories “EEG: tasks” (I² = 51%, p = 0.07) and “EEG: resting state” (I² = 51%, p = 0.09). Owing to an insufficient number of studies, we did not further perform moderator analysis to investigate the effects of potential source of heterogeneity on the meta-analysis. More components and their effects will be included in future work.

A few of the studies involved more than one prognostic models and directly reported between-model comparisons [25–28]. Direct comparisons are likely to ensure an unbiased comparison, but such studies are very limited. Therefore, an indirect comparison was adopted in this review, and thus the difference in comparative accuracy is prone to confounding due to differences in patient and study characteristics.

Conclusions

A DoC is a heterogeneous mixture of different diseases or injuries. Prognostic models aim to assist (not replace) clinicians with their prediction of a DoC patient’s future outcome. Recent years have witnessed the progress of prognostic models, but the research is still preliminary and needs further efforts. Our results suggest that the hybrid prognostic model which integrates more than one technique and paradigms should show more promising accuracy. Reliable prognostic models for clinical practice are more likely to be obtained when they are: developed using a large, high quality dataset; based on a study protocol with a sound statistical analysis plan; validated in independent datasets obtained from different medical centers. This is a complex but worthwhile task.

Electronic supplementary material

Below is the link to the electronic supplementary material.