Mapping outcomes for recovery of consciousness in studies from 1986 to 2020: a scoping review protocol

Abstract

Introduction

Historically, heterogeneous outcome assessments have been used to measure recovery of consciousness in patients with disorders of consciousness (DoC) following traumatic brain injury (TBI), making it difficult to compare across studies. To date, however, there is no comprehensive review of clinical outcome assessments that are used in intervention studies of adults with DoC. The objective of this scoping review is to develop a comprehensive inventory of clinical outcome assessments for recovery of consciousness that have been used in clinical studies of adults with DoC following TBI.

Methods and analysis

The methodological framework for this review is: (1) identify the research questions, (2) identify relevant studies, (3) select studies, (4) chart the data, (5) collate, summarise and report results and (6) consult stakeholders to drive knowledge translation. We will identify relevant studies by searching the following electronic bibliographic databases: PubMed, Scopus, EMBASE, PsycINFO and The Cochrane Library (including Cochrane Database of Systematic Reviews, Cochrane Central Register of Controlled Trials and Cochrane Methodology Register). Criteria for article inclusion are published in the English-language, peer-reviewed studies of interventions aimed at facilitating recovery of consciousness among adults (> 18 years) with DoC following a severe TBI, published from January 1986 to December 2020. Articles meeting inclusion criteria at this stage will undergo a full text review. We will chart the data by applying the WHO International Classification of Functioning, Disability and Health Framework to identify the content areas of clinical outcome assessments. To support knowledge translation efforts, we will involve clinicians and researchers experienced in TBI care throughout the project from conceptualisation of the study through dissemination of results.

Ethics and dissemination

No ethical approval is required for this study as it is not determined to be human subjects research. Results will be presented at national conferences and published in peer-reviewed journals.

Trial registration number

CRD42017058383.

Strengths and limitations of this study.

• The proposed scoping review will result in a comprehensive catalogue of outcome assessments used in traumatic brain injury research aimed at facilitating recovery of consciousness among adults with disorders of consciousness (DoC).

• The outcome assessments will be grouped according to the WHO International Classification of Functioning, Disability and Health domains and subdomains to identify key trends and gaps in concepts of interest.

• To the authors’ knowledge, this will be the first study to identify whether the introduction of National Institute of Neurological Disorders and Stroke (NINDS) Common Data Elements (CDEs) influenced outcome assessment reporting among studies that received federal funding in the USA.

• Studies reporting US federal funding published after the introduction of NINDS CDEs may have been conducted prior to 2010 and therefore the authors may not have been strongly encouraged to use NINDS CDEs.

• It is possible that our search strategy will miss relevant studies; we will mitigate this risk by searching multiple databases and manually searching review articles and meta-analyses.

Introduction

Rationale

To date, there has been limited success in clinical trials for treatment of patients with severe traumatic brain injury (TBI) that result in disorders of consciousness (DoC).^1–3 Representing a continuum of impaired consciousness, DoC is based on a person’s ability to demonstrate arousal and/or awareness. The DoC continuum includes comatose, vegetative state/unresponsive wakefulness syndrome, minimally conscious state and emergence from the minimally conscious state.⁴ Recovery of consciousness for people with DoC following a severe TBI is uncertain and difficult to predict.^5–7 Accurate measurement of recovery of consciousness for people in DoC is essential for diagnosis and prognosis as well as determining the efficacy and effectiveness of interventions.^{5 8–10} To date, there has been no review of the range of clinical outcome assessments used in measuring recovery of consciousness.

Historically, measuring recovery of consciousness in clinical trials has involved a range of clinical outcome assessments measuring different concepts of interest (eg, response to pain, awareness), making it difficult to compare results across studies.^11–14 The National Institute of Neurological Disorders and Stroke (NINDS), part of the US National Institutes of Health (NIH), established a set of Common Data Elements (CDEs) for TBI in 2010 with the goal of promoting comparability of study findings. TBI researchers applying for US federal funding sources including NIH, Department of Defense, Department of Veteran’s Affairs are strongly encouraged to use NINDS CDEs for outcome measurement to improve comparability across trials. Further, a data repository for TBI research was created as a result of collaboration between NIH and the Federal Interagency Traumatic Brain Injury Research (FITBIR) Informatics System¹⁵; federally funded researchers may be required to submit their data to this repository in the future. This requirement provides additional incentive to use NINDS CDEs.^15–17

CDEs are categorised as core, basic or supplemental. The ‘core’ designation indicates data elements pertinent for all TBI studies. Basic CDEs are specific to studies of populations within TBI, such as ‘concussion/mild TBI’, ‘acute hospitalised’, ‘moderate/severe TBI: rehabilitation’ and ‘epidemiology’. Basic CDEs for ‘moderate/severe TBI: rehabilitation’ include, but are not limited to, pupil reactivity, death date and time, hospital discharge destination, and alteration of consciousness duration.¹⁸ Supplemental CDEs are optional and may be appropriate depending on the research question and scope.¹⁶ Only two supplemental CDEs are related to recovery of consciousness in adults: the Galveston Orientation Amnesia Test and JFK Coma Recovery Scale-Revised (CRS-R) (table 1).¹⁸

Table 1.

Examples of Common Data Elements

Type of CDE	Definition	Example of CDE
General core	Recommended for all NIH-funded studies: general	C00031: race expanded category
Disease-specific core	Recommended for all NIH-funded studies: disease specific (TBI)	C01001: Glasgow Coma Scale (GCS)—motor response scale
Basic*	Recommended for all TBI NIH-funded studies: specific to subdiseases (eg, epidemiology and moderate/severe: rehabilitation)	C07155: Disability Rating Scale Total Score
Supplemental	Recommended for NIH-funded studies: specific to study design or type of research	C07145: JFK Coma Recovery Scale-Revised—total score

*Basic CDEs are comparable to supplemental-highly recommended CDEs for other diagnostic categories.

CDEs, Common Data Elements; NIH, National Institutes of Health; TBI, traumatic brain injury.

Two studies have described the implementation of CDEs in TBI research.^{13 19} Yue et al described the implementation of CDEs for a multicentre prospective study and note recommendations for future data collection procedures as well as the success in transferring the data to FITBIR. Stead et al used CDEs to describe TBI patients in emergency departments and were able to compare results to several other published studies. Although the goal of the NINDS CDE project is to improve consistency and comparability across clinical studies of patients with DoC following severe TBI by encouraging more consistent use of clinical outcome assessments, there is currently no evidence to indicate whether this outcome has been achieved.

Objective

The primary objective of this scoping review is to develop a comprehensive inventory of clinical outcome assessments in clinical trials aimed at recovery of consciousness for patients with DoC after TBI. Secondary objectives are to examine the trends in primary outcomes over time and whether reporting of NINDS CDEs increased after their introduction in 2010 in studies that received US federal funding.

Methods and analysis

A scoping review is an appropriate method to achieve the stated objectives because we want to identify characteristics of clinical outcome assessments used to evaluate the recovery of consciousness following a severe TBI.²⁰ The scoping review will be conducted based on the Arksey and O’Malley²¹ methodological framework that has been refined by Levac et al.²² The methodological framework for this review will include: (1) identify the research questions, (2) identify relevant studies, (3) select studies, (4) chart the data, (5) collate, summarise and report results and (6) stakeholder engagement to drive knowledge translation.^{21 22}

Identify the research questions

Primary question

• What clinical outcome assessments have been used in published studies about recovery of consciousness for adults with severe TBI in states of disordered consciousness?

Secondary questions

• How have the outcomes assessments used to measure DoC in adults with severe TBI changed over time?

• Did the frequency of reporting clinical outcome assessments classified as NINDS CDEs change after their introduction in 2010 among federally funded studies in the USA?

Identify relevant studies

The search strategy was developed in collaboration with a research librarian. Our search terms are broad to identify all eligible studies. These search terms encompass three primary categories: severe TBI, recovery of consciousness, and outcomes.

Search terms

An in-depth outline of the full search strategy is reported in table 2.

Table 2.

Examples of the search strategy that will generate the articles to review for the research question

Database	Search terms	Customisation
Cochrane	((“traumatic brain injury”) OR (coma) OR (“persistent vegetative state”) OR (“minimally conscious state”) OR (“consciousness disorder*”) OR (“disorder* of consciousness”)) AND ((recovery) OR (“activities of daily living”) OR (awareness) OR (wakefulness)) AND ((“critical care outcome*”) OR (“treatment outcome*”) OR (“outcome assessment”) OR (evaluation) OR (assessment))	1987–2020, all publication types
Embase	((exp traumatic brain injury/ OR traumatic brain injur*.ti, ab.) OR (exp coma/ OR coma*.ti, ab.) OR (exp persistent vegetative state/ OR persistent vegetative state*.ti, ab.) OR (exp minimally conscious state/ OR minimally conscious state*.ti, ab.) OR (exp consciousness disorder/ OR consciousness disorder*.ti, ab. OR disorder* of consciousness.ti, ab.)) AND ((exp convalescence/ OR convalescence.ti, ab. OR recover*.ti, ab.) OR (exp daily life activity/ OR daily life activit*.ti, ab. OR activit* of daily living.ti, ab.) OR (exp awareness/ OR awareness.ti, ab.) OR (exp wakefulness/ OR wakefulness.ti, ab.)) AND ((exp critical care outcome/ OR critical care outcome*.ti, ab.) OR (exp treatment outcome/ OR treatment outcome*.ti, ab.) OR (evaluation*.ti, ab.) OR (exp outcome assessment/ OR assessment*.ti, ab.))	English, 1986–2020
PsycInfo	(SU (“traumatic brain injur*”) OR TI (“traumatic brain injur*”) OR AB (“traumatic brain injur*”) OR SU (coma*) OR TI (coma*) OR AB (coma*) OR SU (“persistent vegetative state*”) OR TI (“persistent vegetative state*”) OR AB (“persistent vegetative state*”) OR SU (“minimally conscious state*”) OR TI (“minimally conscious state*”) OR AB (“minimally conscious state*”) OR SU (“consciousness disorder*”) OR TI (“consciousness disorder*”) OR AB (“consciousness disorder*”) OR SU (“disorder* of consciousness”) OR TI (“disorder* of consciousness”) OR AB (“disorder* of consciousness”)) AND (SU (recover*) OR TI (recover*) OR AB (recover*) OR SU (“activit* of daily living”) OR TI (“activit* of daily living”) OR AB (“activit* of daily living”) OR SU (awareness) OR TI (awareness) OR AB (awareness) OR SU (wakefulness) OR TI (wakefulness) OR AB (wakefulness)) AND (SU (“critical care outcome*”) OR TI (“critical care outcome*”) OR AB (“critical care outcome*”) OR SU (“treatment outcome*”) OR TI (“treatment outcome*”) OR AB (“treatment outcome*”) OR SU (“outcome assessment*”) OR TI (“outcome assessment*”) OR AB (“outcome assessment*”) OR SU (evaluation*) OR TI (evaluation*) OR AB (evaluation*) OR SU (assessment*) OR TI (assessment*) OR AB (assessment*))	January 1987–31 December 2020, English only
PubMed	(Severe Traumatic Brain Injury [tiab] OR Brain Injuries, Traumatic [mesh] OR traumatic brain injury [tiab] OR coma, post-head injury [mesh] OR persistent vegetative state [mesh] OR minimally conscious state [tiab] OR consciousness disorders [mesh] OR disorders of consciousness [tiab]) AND (recovery [tiab] OR recovery of function [mesh] OR activities of daily living [mesh] OR awareness [mesh] OR awareness [tiab] OR wakefulness [mesh] OR wakefulness [tiab]) AND (Critical care outcomes [mesh] OR treatment outcome [mesh] OR “outcome assessment (health care)” [mesh] OR disability evaluation [mesh] OR evaluation [tiab] OR patient outcome assessment [mesh] OR assessment [tiab])	Humans, English, 1 January 1986–31 December 2020
Scopus	(TITLE-ABS-KEY (“traumatic brain injur*”) OR TITLE-ABS-KEY (coma*) OR TITLE-ABS-KEY (“persistent vegetative state*”) OR TITLE-ABS-KEY (“minimally conscious state*”) OR TITLE-ABS-KEY (“consciousness disorder*”) OR TITLE-ABS-KEY (“disorder* of consciousness”)) AND (TITLE-ABS-KEY (recover*) OR TITLE-ABS-KEY (“activit* of daily living”) OR TITLE-ABS-KEY (awareness) OR TITLE-ABS-KEY (wakefulness)) AND (TITLE-ABS-KEY (“critical care outcome*”) OR TITLE-ABS-KEY (“treatment outcome*”) OR TITLE-ABS-KEY (“outcome assessment*”) OR TITLE-ABS-KEY (evaluation*) OR TITLE-ABS-KEY (assessment*))	English

*Search dates will include 1 January 1986 to 31 December 2020.

Information sources

We will search the following electronic bibliographic databases: PubMed, Scopus, EMBASE, PsycINFO and The Cochrane Library (including Cochrane Database of Systematic Reviews, Cochrane Central Register of Controlled Trials, and Cochrane Methodology Register).

Synthesis of eligibility criteria

This review will include all published, peer-reviewed studies using an intervention/treatment to facilitate recovery of consciousness for adults (>18 years) with DoC following severe TBI (table 3).

Table 3.

Inclusion and exclusion criteria for the scoping review

Category	Inclusion criteria	Exclusion criteria
Language	English
Publication date range	January 1986 to December 2020	Before 1986
Participant sge	Participant age: >18 years of age At least one participant in the study was ≥18 years of age	All participants were under 18 years of age
Participant diagnosis	Participant diagnosis: disordered consciousness (DoC) following severe TBI DoC was established using a known assessment for evaluating states of consciousness such as the Coma Recovery Scale-Revised (CRS-R) or Glasgow Coma Scale<8 At least one participant in the study was diagnosed with DoC from a TBI	Participants had brain pathologies such as Alzheimer’s disease or non-traumatic brain injury, and/or were conscious, alert and oriented Participants had a Diagnostics and Statistical Manual of Mental Disorders (5th edition) diagnosis of psychiatric disorders
Intervention	Intervention aimed at facilitating recovery of consciousness	Purpose of intervention was not described as facilitating recovery of consciousness
Study design	All designs of primary, peer-reviewed studies including randomised control trials, observational studies, cohort studies, case–control studies, case series and case reports	Qualitative studies; meta-analyses, systematic reviews and scoping reviews

Publication date: January 1986 to December 2020.

Language: English.

TBI, traumatic brain injury.

Study design

This review will consider all designs of peer-reviewed studies including randomised control trials, observational studies, cohort studies, case–control studies, case series and case reports. Meta-analyses and review articles will be excluded.

Setting

This review will include intervention studies delivered in any setting to adults with DoC following a severe TBI. There is no restriction on the country of origin.

Participants

For a study to be included in this review, at least one participant in the study must have DoC following a severe TBI. A severe TBI resulting in DoC is defined as: (1) Glasgow Coma Scale score of 3–8¹² or (2) an assessment known for evaluating states of consciousness, such as the CRS-R.^{5 8} Studies will be excluded if all participants were under 18 years of age, had a Diagnostic and Statistical Manual of Mental Disorders (5th edition) diagnosis of a psychiatric disorder, had brain pathologies such as Alzheimer’s Disease or non-TBI, or were conscious, alert, and oriented. All non-human studies will be excluded.

Interventions

Examples of interventions to be included are medication, nutrition, rehabilitation therapy, non-invasive brain stimulation and surgery. Studies will be excluded if the purpose of the intervention/treatment provided was not described as facilitating recovery of consciousness.

Select studies

Following the search, each identified article will be uploaded to Endnote, a reference management system. Duplicate articles will be removed. Titles and abstracts will be screened by two independent reviewers to assess whether articles meet inclusion criteria (table 4). If studies are meta-analyses or reviews that are relevant to the research question, we will search the reference list. Articles that are included by the screening process will undergo a full text review. Two independent reviewers will read the full text articles to make a final determination of inclusion. Articles that do not meet inclusion criteria at this stage will be excluded from the final sample, with rationale documented. Discrepancies about inclusion of articles will be resolved through further discussion and/or input by a third reviewer.

Table 4.

Title and abstract review form

Questions
1. Is the article written in English?	□ Yes □ No
2. Is the article published after 1985?	□ Yes □ No
3. Is the article about human subjects?	□ Yes □ No
a. Are the human subject’s adults (≥18 years)	□ Yes □ No □ Unsure, requires full text review
b. Do the adults have a traumatic brain injury?	□ Yes □ No □ Unsure, requires full text review
c. Are the adults unconscious?	□ Yes □ No □ Unsure, requires full text review
4. Is the article about an intervention?	□ Yes □ No □ Unsure, requires full text review
a. Is the purpose of the intervention to facilitate recovery of consciousness?	□ Yes □ No □ Unsure, requires full text review
b. Is it a meta-analysis, scoping review or systematic review?	□ Yes → Exclude & search the reference list □ No

Chart the data

Data will be extracted from included articles by independent reviewers using a uniform data extraction tool developed for the study. A sample data extraction table is shown in box 1. Reviewers will use the Scottish Intercollegiate Guideline Network (SIGN) rating form to evaluate study quality.²³ Consistent with the SIGN protocol, case study designs will not be evaluated for quality; other studies’ methodological quality will be rated as high, acceptable, low or unacceptable-reject.²³ For each included article, data extraction will include details about the year of publication, funding source, study aims, study design, number of participants (including number lost to follow-up), recruitment, study completion rate, demographics (age, injury severity, days postinjury) of participants, clinical setting, specific intervention (including control conditions, if applicable), primary and secondary outcomes, timing and location of outcomes.

Box 1. Data extraction form for full text review.

Study information

Study title

Year

Funding source

Inclusion/exclusion criteria

Is the paper relevant to our research question, ‘What are the content areas of outcomes related to recovery of consciousness that have been used in clinical trials and/or intervention studies for adults with severe traumatic brain injury (TBI) in disorders of consciousness (DoC)?’ (ie, there are outcome measures for people in DoC following an intervention)

Inclusion criteria:

• Adults (>18 years) with primary diagnosis of severe TBI.

• Identified brain injury is noted to be severe by Glasgow Coma Scale of 8 or less.

• At least one of the study participants are in DoC following a TBI.

• Addressed outcome related to recovery of consciousness.

• Written in English.

Exclusion criteria:

• People with documented history of psychiatric illness (DSM criteria), and/or organic brain syndrome such as Alzheimer’s disease.

• All study participants are fully conscious.

• All study participants are <18 years of age.

• Study participants include non-traumatic brain injury only.

Study details

Study design

Sample/number of participants: include sample size and diagnoses (ie, DoC following TBI, stroke, anoxia)

Sample/demographics: age, injury severity, days postinjury (if reported)

Sample: the study’s inclusion criteria

Sample: the study’s exclusion criteria

Data collection procedures

Intervention characteristics (intervention(s), control condition(s), duration and protocol information)

Primary outcome measure

Context of use for primary outcome measure

Endpoint measure

Secondary outcome measures

Were outcome measures transformed? (Yes/No)

Timing of outcome measures

Results

Observed sample

Number of excluded participants

Number of participants lost to follow-up

Primary outcome (mean, proportion, other effect size index)

Statistical analyses (description of groups, comparison of groups)

Key findings

**Complete SIGN Quality Rating Based on Study Design

Collate, summarise and report information

Data analysis

We will transfer information from the data extraction forms into STATA14 to complete descriptive analyses. We will categorise studies based on sample size and report this information. We will also categorise studies into five groups (high, acceptable, low, unacceptable-reject or not rated) based on quality rating using SIGN criteria. We will examine whether sample size or quality rating biases results regarding frequency of clinical outcome assessment as well as utilisation of CDEs.

Conceptual framework and key concepts

WHO International Classification of Functioning, Disability and Health

Clinical outcome assessments will be categorised based on the WHO International Classification of Functioning, Disability and Health (ICF) framework using relevant concept of interest. This framework has two major components: functioning and disability which includes the domains of body function, body structure, and activities and participation that impact an individual’s daily life; and contextual factors which includes the domains of personal factors and environmental factors. Environmental factors consider the ‘physical, social and attitudinal environment in which people live and conduct their lives’.²⁴ Personal factors include age, gender and education; we will not apply this domain in classifying outcome assessments since these generally represent covariates rather than outcomes/endpoints.

Clinical outcome assessments will first be categorised into one of the four relevant WHO ICF domains (body structures, body functions, activities and participation, environmental factors) based on the concept of interest they are intended to measure. These categorisations will be mutually exclusive in that each outcome assessment will only be assigned to one domain. ICF domains can be further classified into subdomains.²⁴ We will also assign each outcome assessment to a relevant subdomain. Should an outcome assessment not fit into a WHO ICF domain, we will create an ‘Other’ domain. Once all outcome assessments are categorised into a domain, we will thematically analyse the outcome assessments in the ‘Other’ domain to determine if a new domain is needed. For example, previous literature argues for the inclusion of quality of life as a domain.²⁵

Common Data Elements

We will categorise outcome assessments as to whether they are an NINDS CDE for moderate/severe TBI. We will test the significance of the introduction for CDEs on outcome reporting before and after 2010 using a χ² test.

Presentation of results

Results will be presented via detailed quantitative and narrative summaries. First, we will present the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews (PRISMA-ScR) flow diagram demonstrating the inclusion of studies,^{26 27} including how many articles were retrieved from each database. We will also create an outcome map table that categorises outcome assessments by WHO ICF domain and subdomain. We will create two figures to display (1) the frequency of WHO ICF subdomains to show the gaps in the concepts of interest that outcome assessments address by domain and (2) the number and percent of studies that received US federal funding by year to show the proportion that used a CDE as a primary outcome. In addition, we will present a 2×2 table of CDE status and whether the publication was pre/post the introduction of CDEs.

Stakeholder engagement

Clinicians and researchers with extensive experience treating and studying recovery of consciousness following a TBI have been involved in the development of this scoping review protocol. We have formed the Recovery of Consciousness study team to continuously engage these stakeholders throughout the scoping review process, inclusive of study selection through dissemination of results.

Patient and public involvement

No patient involvement.

Ethics and dissemination

No ethical approval is required for this study as it is not determined to be human subjects research. Results will be presented at a national rehabilitation conference and submitted to a peer-reviewed journal for publication.

Reporting of protocol and study records

This study protocol and future reports will follow PRISMA-ScR guidelines for the publication of scoping reviews.²⁶

Ethics statements

Patient consent for publication

Not applicable.