Establishing a core outcome set for blunt cerebrovascular injury: an EAST modified Delphi method consensus study

Abstract

Objectives

Our understanding of blunt cerebrovascular injury (BCVI) has changed significantly in recent decades, resulting in a heterogeneous description of diagnosis, treatment, and outcomes in the literature which is not suitable for data pooling. Therefore, we endeavored to develop a core outcome set (COS) to help guide future BCVI research and overcome the challenge of heterogeneous outcomes reporting.

Methods

After a review of landmark BCVI publications, content experts were invited to participate in a modified Delphi study. For round 1, participants submitted a list of proposed core outcomes. In subsequent rounds, panelists used a 9-point Likert scale to score the proposed outcomes for importance. Core outcomes consensus was defined as >70% of scores receiving 7 to 9 and <15% of scores receiving 1 to 3. Feedback and aggregate data were shared between rounds, and four rounds of deliberation were performed to re-evaluate the variables not achieving predefined consensus criteria.

Results

From an initial panel of 15 experts, 12 (80%) completed all rounds. A total of 22 items were considered, with 9 items achieving consensus for inclusion as core outcomes: incidence of postadmission symptom onset, overall stroke incidence, stroke incidence stratified by type and by treatment category, stroke incidence prior to treatment initiation, time to stroke, overall mortality, bleeding complications, and injury progression on radiographic follow-up. The panel further identified four non-outcome items of high importance for reporting: time to BCVI diagnosis, use of standardized screening tool, duration of treatment, and type of therapy used.

Conclusion

Through a well-accepted iterative survey consensus process, content experts have defined a COS to guide future research on BCVI. This COS will be a valuable tool for researchers seeking to perform new BCVI research and will allow future projects to generate data suitable for pooled statistical analysis with enhanced statistical power.

Level of evidence

Level IV.

What is already known on this topic

• Although extensive research into blunt cerebrovascular injury (BCVI) has been conducted with heterogeneous outcome measures, the most important core outcome measures are not defined.

What this study adds

• The development of a core outcome set provides a foundation for future research teams to build on and collaborate over, increasing the yield of new research projects without limiting other outcomes beyond the core outcome set that authors would like to explore.

How this study might affecet research, practice, or policy

• This project will help facilitate scientific growth and contribute to improved study, management, and outcomes of BCVI.

Background

The management of blunt cerebrovascular injuries (BCVI) has evolved significantly from early descriptions.^{1 2} During the ensuing decades, researchers investigating BCVI have developed multiple screening tools,^{3 4} diagnostic tools,^{5 6} and treatments.^{7 8} Diagnosis of BCVI has been reported with ultrasound, CT angiography (CTA), MRI, and conventional angiography. Treatments include antiplatelet, anticoagulant, endovascular, and surgical therapies. The multiple permutations of screening, diagnosis, and treatment applied differently across studies have resulted in a heterogeneous literature on this topic, such that outcomes of various landmark trials are not easily compared between one another.

Identifying and evaluating an optimal treatment regimen is important, as BCVI is more common than previously thought. Although earlier reports from the 1990s reported a prevalence of <1% in patients sustaining major blunt trauma,^{9 10} more recent descriptions using formalized screening protocols report a prevalence of 1% to 3% of all blunt trauma and as high as 16% in specific high-risk presentations⁴; still, an estimated 20% of injuries are missed.¹¹ These injuries are also found in pediatric trauma patients, although this literature is more limited.^{12 13} Identification of BCVI is important, as BCVI-associated stroke rates as high as 64% have been observed without treatment^{10 14}; with treatment, stroke rates decrease to 3% to 12%.^{10 14–16} Among patients with a BCVI-associated stroke, mortality may reach 50%,¹⁴ with many surviving patients suffering severe neurological injury and/or institutionalization. Optimizing our understanding and management of this disease is a high-impact area of trauma research.

The low incidence of BCVI makes single-center studies difficult, and therefore multicenter research or data pooling (eg, meta-analysis) is needed to further advance our understanding and improve outcomes related to this injury. One means of facilitating data pooling is to develop a core outcome set (COS) to help define consensus endpoints of significant value.¹⁷ These core outcomes represent a minimum standard set of outcomes for future studies on this topic and serve as a guide for researchers without imposing a limit or cap on additional outcomes for study. Implementation of a COS into research helps maximize the number of studies which describe critical outcomes and thus allows for enhanced data pooling.

Several recent COS have been published in the field of trauma as researchers turn increasingly to collaborative and multi-institutional research. These include recently published COS guidelines on resuscitative endovascular balloon occlusion of the aorta¹⁸ and damage control laparotomy.¹⁹ We identified BCVI as a high-priority topic which would benefit from the development of a COS.

Methods

This BCVI COS study was conducted according to established standards described by the Core Outcome Set Standards for Development and Reporting and was registered with the COMET database.^20–22 The study was conducted under the oversight of an a priori designated steering committee. There were no perceived financial or ethical conflicts of interest between the steering committee members and the expert panel, and no funding was required for this project.

The steering committee first reviewed high-impact, peer-reviewed publications on BCVI from the Eastern Association for the Surgery of Trauma (EAST) Landmark Papers collection.²³ The first and last authors of these papers were invited to participate in the survey as expert panelists. Task force members and identified experts were invited to nominate additional peers with a known academic or clinical interest in BCVI. From the 28 invitations sent, an international and multidisciplinary panel of 15 experts (54% response rate) in the fields of trauma, neurosurgery, pediatric surgery, and vascular surgery was assembled. The panel was slightly larger than the reported ideal for Delphi panels, but due to its multidisciplinary nature we opted for broader inclusion.²⁴

In round 1, participants were asked to submit a free-form list of proposed core outcomes for further consideration. In round 2 and thereafter, the panelists were requested to score the submitted outcomes according to a Likert scale ranging from 1 to 9, based on the Grading of Recommendations Assessment, Development, and Evaluation scale.²⁵ Outcomes were presented to the panelists in a random order using a random number generator to minimize bias.²⁶ Scores of 1 to 3 represented less important outcomes and scores of 7 to 9 represented critically important outcomes. We defined a core outcome a priori as any item for which >70% of the panelists selected a score in the range of 7 to 9 and <15% of the panelists selected a score in the range of 1 to 3, consistent with prior COS publications.^{18 19 27} Items selected to score 1 to 3 by >15% of the panel in any round were excluded from further rounds. The Delphi process was planned to terminate either when all items were included or excluded, or when consensus no longer progressed between rounds on indeterminate items. The panelists were encouraged to submit new proposed outcomes at any point; these would be added to the subsequent rounds for panel consideration, and this process allowed an inclusive design to maximize the number of outcomes considered.

For each round after round 2, participants were provided with de-identified aggregate response data from the previous round presented as a histogram bar chart. Participants’ individual responses were provided only to that participant, reminding them of their outcome grading in the context of the group’s aggregate rating. Neither participants’ names, affiliations, or individual grading was shared with any other participant. This survey was conducted during a time of international medical crisis due to the COVID-19 pandemic and thus participants were sent up to three reminder emails for each round of the Delphi process to maximize participation.

Statistical analysis was performed using IBM SPSS V.28.0.0.0 software for Windows.²⁸ Assessment was performed using the intraclass correlation (ICC) function to assess intrarater test–retest reliability, in a two-way random-effects model, scoring for absolute agreement, with 95% CI.

Results

The Delphi process was completed in four rounds. In round 1 (starting August 24, 2021), requesting free-form submission of proposed core outcomes, 12 of 15 experts returned responses (80% response rate). After review by the task force and deletion of duplicate items, a list of 21 unique responses was compiled for further consideration in subsequent rounds (box 1). All non-duplicate submissions were distributed to the panel for consideration; this list included some measures which are not clinical outcomes. We allowed the panel to assess these non-outcome characteristics of value to further our discussion.

Box 1. Submissions from the panel for consideration as core outcomes.

• Time to diagnosis from hospital arrival.

• Incidence of postadmission symptom onset.

• Use of standardized screening tools.

• Imaging modality used for diagnosis.

• Worsening injury severity on interval imaging.

• Any vascular recanalization on interval imaging.

• Complete vascular recanalization on interval imaging.

• Stroke incidence, overall.

• Stroke incidence, by subtype (hemorrhagic, ischemic).

• Stroke incidence, stratified by treatment used (none, antiplatelet, anticoagulant, endovascular, surgical).

• Time from presentation to stroke.

• Incidence of pseudoaneurysm.

• Surgical or therapeutic delays attributed to BCVI therapy.

• Incidence of bleeding complications induced by therapy.

• Duration of anticoagulation/antiplatelet treatment use.

• Type of medical therapy (antiplatelet, anticoagulant).

• Incidence of endovascular management.

• Duration of follow-up.

• Postdischarge neurological outcomes.

• Destination of discharge.

• Overall mortality.

• Stroke incidence, before treatment initiation*.

In round 2 (September 23, 2021), the panelists were presented with the compiled list of proposed outcomes. All panelists participating in this round participated in all subsequent rounds without further attrition. In addition to scoring these 21 items according to our Likert scale, participants in this round submitted one additional outcome which had not been previously submitted, bringing the total number of outcomes under consideration to 22. One participant responded to only 19 of 21 items in this round; these two items were evaluated from a denominator of 11 rather than 12 experts. Of the 21 items in this round, 7 achieved the inclusion criteria: incidence of symptoms post admission, overall stroke incidence, stroke incidence stratified by treatment type, time to stroke, incidence of radiographic worsening of injury grade post admission, use of a standardized screening tool, and type of therapy used. Fourteen items, plus one new submission, remained for further consideration.

In round 3 (October 19, 2021), the panelists were presented only with the list of indeterminate items (15) along with de-identified aggregate results and a reminder of their individual scoring from the prior round. Of the 12 participants, 11 changed at least one score. In this round, four additional items met the inclusion criteria: incidence of overall mortality, incidence of stroke stratified by stroke type, incidence of stroke after initiating therapy, and total duration of therapy. Two items, destination of discharge and incidence of endovascular therapy use, scored >15% in the 1 to 3 range, resulting in exclusion, with nine items remaining indeterminate.

Round 4 (December 15, 2021) was conducted in the same manner as round 3. Of the 12 participants, 11 changed at least one score. Two items met the inclusion criteria: time to diagnosis and incidence of bleeding complications. Three items met the exclusion criteria: vascular recanalization rate on interval imaging, postdischarge neurological outcomes, and duration of follow-up reported. Four items failed to meet the inclusion or exclusion criteria and remained indeterminate. To determine the utility of proceeding to a fifth round of Delphi, we subjected the scores from those indeterminate results to an intraclass correlation analysis. The four indeterminate items demonstrated an ICC of 0.508 with a Cronbach’s alpha of 0.645. Having achieved a moderate agreement and recognizing the limited value of further rounds of deliberation,²⁴ the Delphi process was deemed complete.

The final list of 13 consensus items included 9 core outcomes and 4 non-outcome characteristics of value which the panel felt warranted publication for future study consideration (table 1).

Table 1.

Final set of included core outcomes and characteristics of value

Core outcomes	Non-outcome characteristics of value
Incidence of postadmission symptom onset	Time to BCVI diagnosis
Stroke incidence, overall	Use of standardized screening tool
Stroke incidence, stratified by subtype	Duration of treatment
Stroke incidence, before treatment initiation	Type of therapy used
Stroke incidence, stratified by treatment type
Time to stroke
Mortality, overall
Bleeding complications of therapy
Injury progression on radiographic follow-up

BCVI, blunt cerebrovascular injury.

Discussion

BCVI has come to the forefront of trauma care due to increased availability of high-quality CTA imaging and adoption of increased screening, including universal screening protocols.^{29 30} However, there is a lack of randomized controlled trials and an inability to pool analyses due to heterogeneity of outcomes data. To our knowledge, this is the first COS developed for BCVI. Through this iterative process, our international and multidisciplinary panel has identified the incidence of postadmission symptom onset, radiographic injury progression, overall stroke incidence, stroke incidence by subtype, stroke incidence associated with treatment use and treatment type, time to stroke, bleeding complications, and overall mortality as core outcomes. We have additionally identified time to diagnosis, use of standardized screening, and duration and type of therapy as data fields of value for future BCVI research. These outcomes and associated data fields of value are recommended by this panel for all future BCVI research, in addition to any other outcomes or descriptors selected by the research team. Use of these core items will ensure that critical outcome variables are recorded and measured in a consistent manner in future literature.

BCVI is a diagnosis that may present without initial symptoms, but tremendously elevates risk of stroke, which influences morbidity and mortality. Unsurprisingly, the majority of core outcomes identified by our panel focus on various aspects of stroke. Of the nine core outcomes, six specifically describe BCVI-associated strokes. Several of these items (postadmission symptom onset, stroke association with treatment use, and time to stroke) describe stroke timing and may influence our understanding of BCVI treatment timing and treatment importance in the context of other injuries. The remaining three items (overall stroke rate, stroke incidence by subtype, and stroke incidence associated with treatment type) may inform our understanding of therapeutic effectiveness and consequences, thereby facilitating a more nuanced understanding of BCVI treatment and timing beyond simple stroke incidence.

The three non-stroke outcomes identified by our panel (overall mortality, radiographic injury progression, and incidence of bleeding complications) are important considerations in this patient population. BCVI occurs predominantly in high-energy transfer polytrauma patients, whose injuries are not limited to BCVI. In this cohort, bleeding is a major contributor to mortality and influences patient care. Anticoagulants have been demonstrated to worsen traumatic brain injury,^{31 32} making treatment of BCVI with anticoagulant/antiplatelet agents a topic of concern.³³ Further, it has been reported that a majority of BCVI resolve quickly³⁴; patients with progressively worsening injuries may represent an important subpopulation at differential risk than patients whose injuries resolve. Inclusion of these non-stroke outcomes helps frame treatment needs and outcomes in the context of a patient with multiple competing clinical concerns and injuries.

Our panel identified four non-outcome data fields of importance for reporting. These items are not clinical outcomes, but represent study variables which may substantially influence outcomes and thus should be collected and reported in future BCVI studies. These data fields of importance reflect the heterogeneous clinical circumstances and decision-making in patients with BCVI. Multiple competing screening guidelines exist, with some centers relying on the Memphis criteria, Denver criteria, modifications of these, or offering universal screening to all trauma patients with significant mechanism of injury.²⁹ Similarly, there are parties who support treatment with acetylsalicylic acid,^{7 35} clopidogrel,^{7 36} full anticoagulation,^{36 37} or some combination of these treatments, resulting again in inconsistency of treatment type and prescribing patterns. Without reporting the screening, diagnosis, or treatment strategy, a clear and contextual understanding of outcomes is not possible and thus why our panelists felt strongly to include these results in this article.

The design of this study has multiple strengths. This consensus survey was conducted in an organized fashion using clear a priori definitions of inclusion and exclusion criteria, with established consensus building tools.^{24 25} Our cohort was international in nature, included both adult and pediatric experts, and represented a broad scope of experts from multiple clinical fields who are stakeholders in BCVI management. Another strength of the Delphi design is that there are not rigid time constraints, allowing panelists to give earnest consideration to their positions.²⁴

However, there are some limitations to consider. It would have been desirable to achieve strong agreement in the final round to demonstrate that the four remaining indeterminate outcomes would not achieve consensus. We think four rounds is an adequate attempt to achieve inter-rater consensus, and further deliberation appeared unlikely to move the panel from only moderate agreement to supermajority agreement in the absence of new information. Another limitation is that this study did not include patients in the panel, which may have provided insight into the values and patient-centered outcomes of interest for persons affected by this condition. However, the breadth of experiences in BCVI is broad, ranging from patients requiring no treatment to patients suffering devastating neurological injuries. Thus, it would be challenging to recruit a “typical” patient with experience in this disease process.

One challenge with developing a COS is that information may change over time which warrants updating. Like any guideline, a COS that is not adapted to new information may become outdated and limit its utility. Although this BCVI COS represents an expert consensus on core outcomes for today, consideration should be given toward interval reassessment and updates as future literature becomes available. Future researchers referring to this COS document should not be limited to reporting these core outcomes alone, but should be encouraged to use these core outcomes as a foundation which can be supplemented with additional outcomes as necessary for the intended study hypothesis. As future understanding evolves, newly understood outcomes of importance can be included along with these core outcomes without compromising data or study quality.

Conclusions

This COS for BCVI identified nine core outcomes which should be reported in future BCVI studies, as well as four data fields of value. Identification of these measures will optimize data collection and improve interoperability of future data sets for pooled research. In a field challenging to study such as BCVI, this COS tool will be valuable in improving our understanding of this disease process to improve patient outcomes.

Data availability statement

All data relevant to the study are included in the article or uploaded as supplementary information.

Ethics statements

Patient consent for publication

Not required.

Ethics approval

This study involves human participants as respondents to the survey. This study type is exempt under most medical ethics board from ethical review. Participants gave informed consent to participate in the study before taking part.