Developing Patient-centered Outcome Measures for Evaluating Vasospasm in Aneurysmal Subarachnoid Hemorrhage

Abstract

Rationale and Objectives

Patient-centered outcome measures have become an essential focus in research methodology in recent years. This may be particularly challenging in imaging research at the technology assessment level to incorporate patient-centeredness. A primary issue in this field is designing a reference standard that is applicable to the entire study population.

Materials and Methods

This important element is necessary for translation of findings into clinical practice. In our work, computed tomographic perfusion imaging is being evaluated as a new technology used in aneurysmal subarachnoid hemorrhage patients to detect cerebral vasospasm. We have developed a new reference standard employing a multistage hierarchical design incorporating both clinical and imaging criteria to determine a diagnosis of vasospasm.

Results

A flowchart of the reference standard levels is provided for illustration. The limitations and potential biases that may occur using this reference standard are discussed.

Conclusions

This reference standard will be applicable to the entire study population, including those with and without symptoms or further imaging with digital subtraction angiography.

In the past several years, there has been growing public interest in the management of health care, resource allocation, and improving patient outcomes. Patient involvement in health care issues has become a primary initiative by the consumer and government policy makers. It is our interest to incorporate patient-centeredness into research methodology using outcome measures. However, it may be challenging particularly in imaging research at the technology assessment level to incorporate patient-centeredness into research methodology. Littenberg (1) describes incorporating the overall medical, psychologic, and financial effects of the technology into assessing patient outcomes. This includes the unintended side effects and long-term morbidity and mortality. The selection of an appropriate reference standard that is representative of patient outcomes from a patients’ perspective is complex. The primary obstacle to overcome is to design a reference standard that is applicable to the entire population that the imaging study is intended for use in clinical practice. Analyses performed on a subgroup of the population due to limitations with the reference standard, such as only symptomatic patients receive the reference standard test, may yield important findings that are not translatable to the clinical population. This limitation is not always recognized when applying the literature in clinical practice for determining management and treatment decisions.

In our work, computed tomographic perfusion (CTP) is being evaluated as a new technology used in aneurysmal subarachnoid hemorrhage (A-SAH) patients to detect cerebral vasospasm. In the clinical care setting, CTP is intended for use on all patients with A-SAH, including patients with and without symptoms. It then becomes important to implement a reference standard in the research methodology that is applicable to the entire target population, including those with and without symptoms and those who will not have additional imaging performed. We have specifically developed a new reference standard to address these limitations using a multistage hierarchical design incorporating both clinical and imaging criteria for determination of vasospasm.

PATIENT POPULATION

Subarachnoid hemorrhage as a result of a ruptured intracranial aneurysmis a devastating condition, with an incidence approximating 10 per 100,000 persons annually, estimating 18,000 cases per year in North America (2). A-SAH is associated with as great as 67% patient fatality and 10%–20% long-term dependence in survivors (3). A serious complication of A-SAH is delayed cerebral vasospasm, which typically develops from 4 to 9 days after the hemorrhagic event in up to 70% of patients, and is a significant cause of morbidity and mortality in this patient population (4,5). The definition of cerebral vasospasm encompasses both clinical and imaging criteria as the development of delayed onset of ischemic neurologic deficits and the narrowing of the cerebral vessels documented by angiography or other imaging studies. Poor clinical outcome is associated with vasospasm leading to permanent neurologic deficit, stroke, and death.

Our goals in improving patient outcomes in the A-SAH population focus on early diagnosis and implementing effective treatment for vasospasm. Using CTP in this patient population has the ability to assess cerebral hemodynamic status on a capillary level. This physiologic information on cerebral blood flow (CBF), cerebral blood volume, and mean transit time may be able to assist in understanding the impairment in cerebral autoregulation that is thought to occur following A-SAH and the development of vasospasm. Advantages of CTP compared with other perfusion imaging modalities include its noninvasive technique, widespread availability, short acquisition time, easiness to perform, and limited patient contraindications. In addition, CTP has the potential to provide not only qualitative, but also quantitative information on these cerebral hemodynamic properties. Thus, it is important to determine the true test characteristics (sensitivity and specificity) of CTP for the detection of vasospasm.

Digital subtraction angiography (DSA) is considered the “gold standard” for confirming the presence of angiographic vasospasm, diagnosed by imaging findings demonstrating narrowing of large or medium sized intracranial arteries. This is often associated with diminished perfusion to the territory distal to the involved vessels. DSA is an invasive technique, which makes it a less desirable test, with a total complication rate of approximately 5% and a permanent stroke rate estimating 0.5%–1% (6,7). Patients with symptoms of vasospasm are more likely to have DSA performed compared to patients without symptoms because of the risks and potential complications associated with DSA. Given this selection bias, patients who undergo DSA evaluation are more likely to have a positive diagnosis of vasospasm. In our work, we are interested in implementing a reference standard that is applicable to all A-SAH patients to assess the diagnostic accuracy of CTP for detection of vasospasm.

REFERENCE STANDARD DESIGN

We have developed a multistage hierarchical system as a reference standard to determine vasospasm in A-SAH patients. All patients with A-SAH will enter through the reference standard system in a stepwise manner using clinical, imaging, and patient outcomes data to comprehensively determine a vasospasm diagnosis. Figure 1 is a flow diagram illustrating the multistage hierarchical design.

Figure 1.

Flowchart for reference standard to determine the study outcome of vasospasm diagnosis. DSA, digital subtraction angiography; HHH, hypertension, hypervolemia, hemodilution.

The primary level in the reference standard is DSA. Neuroradiologists review the DSA exam to record the degree of vessel narrowing and the vascular territories involved. No vasospasm is defined as no evidence of luminal narrowing; mild vasospasm is defined as less than 50% degree of luminal narrowing; moderate vasospasm is defined as 50%–75% degree of luminal narrowing; and severe vasospasm is defined as greater than 75% degree of luminal narrowing. The DSA performed at admission for initial diagnosis of aneurysm location or for the purpose of endovascular embolization for aneurysm treatment is not included in the reference standard for a “no vasospasm” diagnosis. This initial DSA exam is not sufficient evidence to assign a “no vasospasm” diagnosis for the study outcome because the development of vasospasm is unlikely to occur during this period. The period for delayed cerebral vasospasm falls in the range of 4 to 9 days after aneurysmal SAH, as mentioned previously, although earlier or later vasospasm may be observed. Therefore, baseline DSA exams that demonstrate findings of early vasospasm will be considered evidence for a vasospasm diagnosis. After aneurysm repair, DSA exams performed during the patient’s hospital course are included in the reference standard.

Patients who have not had a DSA during their hospitalization proceed to the secondary level in the reference standard. For some patients, DSA may not be favorable in patient care; therefore, the determination of a vasospasm diagnosis for these patients will be based on clinical or imaging outcomes data of each patient. The clinical criterion for assigning a vasospasm diagnosis is permanent neurologic deficit on clinical examination, distinct from the deficit at baseline produced by the initial aneurysmal SAH. The imaging criterion for the vasospasm outcome is delayed infarction present on follow-up computed tomography (CT) or magnetic resonance imaging (MRI) of the brain. Delayed infarction is defined as a new infarct on CT or MRI after day 4 that has not been present on the initial CT within 3 days after onset. This criterion allows for the exclusion of primary brain damage occurring during aneurysm rupture; complications of surgical intervention such as contusion, local brain edema, and infarction resulting from the occlusion of parent arteries; or perforators after aneurysm clipping. If a patient fits either or both of the criteria, they will be assigned a vasospasm diagnosis. However, if they did not undergo DSA and do not fit either criteria, they are given a “no vasospasm” diagnosis.

Patients without a DSA examination and who do not manifest clinical or imaging sequelae of vasospasm proceed to the tertiary level in the reference standard. This level is based on evaluating response-to-treatment in patients who receive medical HHH therapy to augment cerebral perfusion pressure. “Triple H” therapy stands for medically induced hypertension, hypervolemia, and hemodilution. Despite the lack of randomized controlled trials, medical HHH therapy has been shown to reverse delayed ischemic neurologic deficits in patients who have A-SAH (8). Those patients who demonstrate an improvement in symptoms or clinical examination after medical HHH therapy as determined by medical record review are considered responders to treatment and are assigned a vasospasm diagnosis. Patients who do not improve after treatment are evaluated further via chart review to determine another etiology for symptoms. Other causes that may account for onset of new symptoms in this population include hydrocephalus, re-hemorrhage from the aneurysm, postoperative infarction or contusion, infection or sepsis, and metabolic disturbances. Identification of another etiology to explain symptoms in this patient population allows for a diagnosis of “no vasospasm.” In patients without response to treatment and no other etiology identified, then further evaluation with DSA is performed and the patient will be classified at the primary level in the reference standard.

DISCUSSION

A review of the literature reveals that vasospasm is associated with altered or abnormal perfusion detected with different imaging modalities, such as positron emission tomography (9,10), single-photon emission computed tomography (11), Xenon-CT (9,11), MRI perfusion (12), and CT perfusion (13). CBF reduction after A-SAH has been shown to be a significant predictor of clinical outcome (14,15–17). In particular, Xenon-CT yields reliable information regarding quantitative CBF values in cerebral ischemia (9,11). Patients with CBF at or below 15 mL/100 g/min tended to develop stroke on follow-up CT (9,11). CBF and cerebral blood volume values using CTP were also significantly lower in patients with moderate to severe vasospasm (13). Overall, CTP data correlated well with positron emission tomographic and Xenon-CT results in several studies (9,18,19). However, in the clinical setting, CTP has many advantages over these other perfusion imaging modalities because of its rapid acquisition and ability to process data in an emergent fashion. In addition, CTP can be easily coupled with CT angiography to assess both brain perfusion and anatomic vessel narrowing during a single examination. Before instituting treatment for vasospasm, patients require a non-contrast head CT to exclude re-hemorrhage, cerebral edema, hydrocephalus, or stroke. CTP and CT angiography can be easily performed immediately after this noncontrast head CT; an ideal opportunity to assess brain perfusion and anatomic vessel narrowing without delaying patient treatment.

Given these important aspects, the accurate assessment of the test characteristics of CTP is valuable in translating these research findings into the clinical care setting. Incorporating patient-centeredness into research methodology for technology assessment can initially be addressed by implementing a reference standard that is applicable to the clinical population. In previous literature on the evaluation of vasospasm in A-SAH patients, imaging criteria using DSA for arterial narrowing or CT for delayed infarction has been used as the reference standard. For example, the accuracy of transcranial Doppler ultrasonography and single-photon emission CT in predicting vasospasm was based on DSA performed within 24 hours (20). Another study evaluating the potential use of CTP and CT angiography in the detection of vasospasm in patients with A-SAH was also based on DSA as the reference standard (21).

Using DSA as the sole standard for determining patient outcomes may bias accuracy estimates of CTP in our study. Only including patients who have had a DSA examination leads to a selection bias because patients with symptoms of vasospasm are more likely to have a DSA performed compared to subjects without symptoms. Subjects with symptoms are also more likely to have a vasospasm diagnosis with both DSA positive and CTP positive exams. Thereby, A-SAH patients with symptoms of vasospasm and DSA-positive and CTP-positive exams would be overrepresented in the analysis. DSA as a reference standard alone in this population would lead to inaccurate assessment of the CTP test characteristics and may bias results toward overestimating its diagnostic accuracy. An alternate approach is to subject all A-SAH patients to a DSA examination. However, the risks associated with DSA would not be justified, because major complications include embolus to a vascular territory, permanent neurologic deficit, stroke, and death reported in 0.1%–0.16% (22).

It is important to evaluate a test that will be used for medical decision making in a research population that is equivalent to the clinical population in which the test might be implemented. Doing otherwise may lead to a misunderstanding of the actual sensitivity and specificity of the test and thereby faulty subsequent patient management. Our proposed reference standard design is a practical and systematic method for determining the presence or absence of vasospasm, using both clinical and imaging criteria. This reference standard is modeled after the clinical setting of decision making in A-SAH patients, and will allow the translation of research results into clinical practice. However, we recognize that limitations and potential biases may occur at each level in the reference standard. The following is a brief discussion of the limitations that are most likely to occur in our study. At the primary level, using DSA may introduce bias by overestimating the CTP false-positive results. For example, A-SAH patients with vasospasm only involving the distal intraparenchymal vessels may not be detected on DSA and would be classified as a DSA-negative examination. According to our reference standard, this patient would be classified as a no vasospasm diagnosis. Given the working hypothesis that CTP is sensitive for detection of vasospasm, then a true-positive CTP in this patient would be considered as a false-positive result in the analysis. This limitation may have a minimal effect on the study results because DSA is a sensitive modality for detection of vasospasm and only few patients will be affected in this scenario. However, if this is a significant subgroup, then these potential cases will bias the study results of CTP toward a lower specificity. This represents a more conservative approach in determining the test characteristics of CTP.

At the secondary level, the clinical criteria defined for vasospasm does not include death related to vasospasm. A-SAH patients experience many complicating co-morbidities, including myocardial infarction, cardiopulmonary failure, and pulmonary embolus. Death is not sufficient evidence for assigning a vasospasm diagnosis because etiologies of death are difficult to isolate in this population. Most of these severely ill patients will undergo DSA with intra-arterial treatment and therefore be classified under the primary level of the reference standard. An additional safe-guard in the reference standard to account for this limitation is to include imaging criteria for evidence of delayed infarction at the secondary level and response to treatment at the tertiary level. Patients with true vasospasm related death, without a DSA examination, will likely develop delayed infarction or not demonstrate a response to treatment.

Last, at the tertiary level, patients who receive HHH therapy, and do not undergo a DSA exam or develop clinical or imaging sequela of vasospasm may demonstrate symptom improvement due to other etiologic factors rather than vasospasm. However, the standard of care is to assess all etiologic factors to explain symptom onset before starting medical therapy. This practice allows us to exclude other treatable etiologies, such as hydrocephalus, re-hemorrhage, postoperative infarction, and metabolic disturbances. The frequency of undetected etiologies resulting in symptom improvement without dedicated treatment is very low and unlikely to bias our results. On the other hand, patients who receive HHH therapy and do not demonstrate response to treatment may be given a diagnosis of no vasospasm. The natural progression of the disease in this subpopulation without clinical or imaging sequelae is to full recovery and the frequency of occurrence is very low. Again, as standard of care, patients who have symptoms of vasospasm and do not respond to HHH therapy, without other etiologic factors identified, will likely undergo DSA for further diagnostic testing and will be classified at the primary level.

In conclusion, this new reference standard is a simplified and systemic approach to determine the study outcome of a vasospasm diagnosis by incorporating both clinical and imaging criteria. To date, this multistage hierarchical design may be considered the best available method to determine the performance of CTP as a diagnostic tool in the evaluation of vasospasm. This new reference standard may provide a more comprehensive assessment of the entire A-SAH population for improved translation of research findings into clinical practice.