Robot-Assisted Transcranial Doppler Versus Transthoracic Echocardiography for Right to Left Shunt Detection

Abstract

BACKGROUND:

Right to left shunt (RLS), including patent foramen ovale, is a recognized risk factor for stroke. RLS/patent foramen ovale diagnosis is made by transthoracic echocardiography (TTE), which is insensitive, transesophageal echocardiography, which is invasive, and transcranial Doppler (TCD), which is noninvasive and accurate but scarce.

METHODS:

We conducted a prospective, single-arm device clinical trial of robot-assisted TCD (raTCD) versus TTE for RLS diagnosis at 6 clinical sites in patients who presented with an event suspicious for embolic cerebrovascular ischemia from October 6, 2020 to October 20, 2021. raTCD was performed with standard TCD bubble study technique. TTE bubble study was performed per local standards. The primary outcome was rate of RLS detection by raTCD versus TTE.

RESULTS:

A total of 154 patients were enrolled, 129 evaluable (intent to scan) and 121 subjects had complete data per protocol. In the intent to scan cohort, mean age was 60±15 years, 47% were women, and all qualifying events were diagnosed as ischemic stroke or transient ischemic attack. raTCD was positive for RLS in 82 subjects (64%) and TTE was positive in 26 (20%; absolute difference 43.4% [95% CI, 35.2%–52.0%]; P<0.001). On prespecified secondary analysis, large RLS was detected by raTCD in 35 subjects (27%) versus 13 (10%) by TTE (absolute difference 17.0% [95% CI, 11.5%–24.5%]; P<0.001). There were no serious adverse events.

CONCLUSIONS:

raTCD was safe and ≈3 times more likely to diagnose RLS than TTE. TTE completely missed or underdiagnosed two thirds of large shunts diagnosed by raTCD. The raTCD device, used by health professionals with no prior TCD training, may allow providers to achieve the known sensitivity of TCD for RLS and patent foramen ovale detection without the need for an experienced operator to perform the test. Pending confirmatory studies, TCD appears to be the superior screen for RLS compared with TTE (funded by NeuraSignal).

REGISTRATION:

URL: https://www.clinicaltrials.gov; Unique identifier: NCT04604015.

See related article, p 2851

Acute ischemic stroke patients should be considered for investigation of a source of cerebral embolism, and one element of the investigation is to screen for a right to left shunt (RLS), the most common source being a patent foramen ovale (PFO).¹ PFO is estimated to be present in ≈25% of the general population² but is overrepresented in the ischemic stroke population, especially those who are <60 years of age and without traditional vascular risk factors, where prevalence is estimated to be as high as 30% to 50%.^3,4 A RLS can be a conduit for a peripheral venous thrombus to embolize the cerebral arterial circulation (paradoxical embolization)³ or, as is the case with PFO, particularly with an atrial septal aneurysm,⁵ may itself be the thrombogenic source of acute cerebral ischemia.⁶ The presence or absence of RLS impacts the choice of stroke risk reduction therapies and prognosis.^7–12 Therefore, effective screening for RLS is a sine qua non of a thorough evaluation for embolic stroke with no other identified source to avoid exposing a patient to excess risk of stroke recurrence due to undertreatment.

Transthoracic echocardiography (TTE) with agitated saline contrast is the most common screening diagnostic for RLS but has been shown to have a low sensitivity of ≈45% for PFO^13,14 as compared with transesophageal echocardiography (TEE), making it a poor screening examination despite its widespread availability and noninvasive nature. TEE is the nonsurgical gold standard for PFO diagnosis^14,15 but is invasive, requires sedating medications that can limit Valsalva effort by the patient and negatively impact test sensitivity,¹⁵ and does not directly visualize extracardiac shunting. Transcranial Doppler (TCD) is very sensitive (96%) and specific (92%) for the diagnosis of PFO as compared with TEE,¹³ is noninvasive, can be performed at the point of care, allows for both calibrated Valsalva¹⁶ and body positioning¹⁷ to increase sensitivity, and has established shunt grading schema^16,18,19 that can assist in RLS evaluation and management, but is operator-dependent and limited by the availability of sonographers and physicians with expertise.²⁰

Recently, robot-assisted TCD (raTCD) technology,²¹ some with machine-learning-enhanced signal detection algorithms,²² has been introduced to clinical research and practice to help mitigate barriers to TCD performance. More specifically, raTCD can detect and maintain optimal cerebral blood flow velocity signals for embolic monitoring autonomously, with potential to expand the availability but maintain the high diagnostic accuracy of TCD for RLS diagnosis. However, the diagnostic accuracy of raTCD has never been prospectively tested against the most common RLS screening examination, TTE.

METHODS

Trial Oversight and Funding

This study comports with the Transparency and Openness Promotion Guidelines for authors publishing in the American Heart Association Journal, and the data sets can be made available by reasonable request to the corresponding author.

The study was a multicenter (conducted within 6 clinical sites), prospective, single-arm, nonsignificant risk, consecutively enrolled diagnostic yield device clinical trial. The trial was run between October 6, 2020 and October 21, 2021. Specific details about methodology, including prespecified outcomes, were published previously.²³ This trial was registered with ClinicalTrials.gov (https://www.clinicaltrials.gov; Unique identifier: NCT04604015). The trial and protocol were designed by an academic steering committee. The funders, NeuraSignal, Inc, had no influence on the final design or conduct of the trial, in the writing of the article, or in the decision to submit it for publication. The trial protocol (available in full in the Supplemental Material), and informed consent forms were reviewed and approved by central and institutional internal review boards as appropriate at each study site. The trial was performed in accordance with the principles of the Declaration of Helsinki. The trial was designed to align with STARD (Standards for Reporting of Diagnostic Accuracy Studies),²⁴ standards of device accuracy trials and the checklist available with Supplemental Material. The authors assume responsibility for the accuracy and completeness of the data and analyses, and for the fidelity of the trial and this report to the protocol.

Patients

The trial included adult (≥18 years) patients who experienced a clinical episode that, in the opinion of the treatment team, included an embolic acute neurovascular episode (eg, ischemic stroke or transient ischemic attack) on the differential diagnosis prompting patient referral for a TTE with agitated saline bubble contrast as part of routine clinical care. Specific subject inclusion and exclusion criteria are included below.

Subjects met all the following inclusion criteria to be enrolled in the study:

• 18 years of age or older

• Presentation with a clinical condition characterized by neurological signs and symptoms that, in the opinion of the investigator, include embolic stroke or transient ischemic attack in the differential diagnosis

• Scheduled for TTE study with agitated saline contrast (bubble study) within ±30 days of informed consent

• Ability to successfully perform a Valsalva maneuver.

• Signed informed consent

• Ability to comply with the protocol

Subjects were not enrolled in the study if any of the exclusion criteria were met:

• History of RLS/PFO closure

• Pregnancy or lactation at the time of study

• History of partial or full craniotomy/craniectomy within the past 6 months

• Presence of a physical limitation preventing TCD/Headmount placement

Trial Procedures

Enrolled patients underwent raTCD (NovaGuide Intelligent Ultrasound, NeuraSignal Inc, Los Angeles, CA) in addition to standard of care TTE with agitated saline contrast bubble study, both within 30 days of informed consent. The raTCD is a five-degree-of-freedom robotic unit paired with a signal optimization algorithm that supports traditional 2 MHz diagnostic TCD. Any other diagnostics for RLS testing (eg, TEE or TCD) were optional, performed only at the discretion of the treatment team. A clinical research coordinator performed the raTCD procedure. The research coordinators had no prior TCD experience before being trained to perform standard TCD bubble study technique¹⁹ and the study protocol for raTCD RLS testing. Injections of agitated saline contrast during raTCD were performed at rest and with calibrated Valsalva (mean flow velocity drop of at least 25% and obvious characteristic waveform changes), both in supine and elevated (45°) positions by the clinical research coordinator or bedside nurse. TTE raw data (still and video) were interpreted locally and reviewed in a cardiology core laboratory by independent, blinded experts. With the intent of having standard of care TTE as the control, TTE performance was not standardized but rather performed in accordance with the established clinical protocol of the local study site. The raTCD studies, including 60-second audio/video captures of the raw data, were interpreted in a TCD Core Lab by independent, blinded experts. RLS presence and size on raTCD were graded by Spencer Logarithmic Scale criteria¹⁶ and International Consensus Criteria.¹⁹ For the purpose of prespecified secondary outcome analysis,²³ large RLS was defined by >20 bubbles in the left heart on TTE²⁵ and Spencer Logarithmic Scale grade 3 or higher on raTCD.¹⁸ Further details on the trial procedures are available in the Supplemental Appendix.

Outcomes

The primary outcome was rate of RLS detection with TTE and raTCD in the intent to scan (ITS) cohort, which included site assessment of TTE (eg, local clinical interpretation). The primary safety outcome was any serious device-related adverse events. The key prespecified secondary outcome was rate of detection of large RLS on TTE and raTCD. The other prespecified secondary outcomes, including rate of absence of transtemporal windows and device performance parameters, are in the Supplemental Appendix.

Statistical Analysis

The study was powered based on the results of a meta-analysis¹³ reporting a pooled TCD sensitivity of 96.1% for PFO detection, while the pooled TTE sensitivity was estimated at 45.1% (absolute difference of 51%). For power calculations, we used a more moderate effect size of 40% increase in the sensitivity of raTCD TTE. A sample size of 100 subjects achieves 90% power to detect a difference of 40% between the 2 diagnostic tests whose sensitivities are 90% (TCD) and 50% (TTE). This procedure uses a 2-sided McNemar test with a significance level of 0.05. The mean prevalence of PFO in the population of patients with cryptogenic stroke was estimated to be at least 30%.²⁶ The proportion of discordant pairs has been set at 0.500. Given previous reports^27–29 indicating a prevalence of suboptimal transtemporal windows in 5% of Hispanic, 5% of White, 9% in African American, and 14% of Asian individuals, we increased our projected sample size by 20% (n=120). In addition, the final sample size was further increased to account for an anticipated dropout rate of at least 20%. Consequently, the study sample was set to at least 150 individuals. Data were analyzed on an ITS and per protocol basis. The ITS cohort was defined as subjects that met all inclusion/exclusion criteria and raTCD was attempted. The per protocol cohort was defined as subjects that met all inclusion/exclusion criteria, successfully completed the study with no protocol deviations and had complete data sets. Any data loss of the raTCD or TTE was treated as a dropout. We presented continuous parametric data using their mean values together with their corresponding SDs. We used median values for the presentation of nonparametric data and percentages for all dichotomous variables. Statistical comparisons between different subgroups were performed using the unpaired t test and Mann-Whitney U test as appropriate.

RESULTS

Characteristics of the Patients

From October 6, 2020 to October 20, 2021, a total of 154 patients were consented and enrolled, of whom 129 were evaluated on an ITS basis (Figure 1), and 121 subjects were evaluable as the per protocol cohort. Sixty one (47%) subjects were women and the mean age was 60±15 years in the ITS cohort. The qualifying clinical event was an acute ischemic stroke or transient ischemic attack in all subjects, with a majority (73; 57%) diagnosed with an embolic stroke of undetermined source. The no window rate in the ITS population was 7%. Baseline characteristics of the patients are presented in Table 1.

Figure 1.

BUBL patient recruitment diagram. SOC indicates standard of care; TCD, transcranial Doppler; and TTE, transthoracic echocardiography.

Table 1.

Demographics of the Study Population

Outcomes

For the primary outcome, in the ITS cohort, raTCD detected any RLS in 82 patients (64%) whereas TTE documented any RLS in 26 (20%) patients (absolute difference 43.4% [95% CI, 35.2%–52.0%]; P<0.001; Table 2). The per protocol analysis of this same end point is included in Table 2.

Table 2.

Outcomes

For the secondary prespecified outcome of large RLS detection, raTCD detected large RLS in 35 patients (27%) and TTE found large RLS in 13 (10%; absolute difference 17.0% [95% CI, 11.5%–24.5%]; P<0.001). TTE showed no RLS in 18 of 35 (51%) large RLS diagnosed by raTCD (Table 2).

There were few TCD and TEE data, which were optional diagnostics in this study. Overall, there were 14 cases with TEE (11%) and 6 of those also had evaluable TCD (5%) for cross-comparison (Table 3). Prespecified secondary analyses related to TEE and TCD included percent detection and are included in Table 4. There was a significant difference in percent detection of RLS between raTCD and both TCD (86% versus 57%; P=0.041) and TEE Core Lab analysis (86% versus 43%; P=0.041).

Table 3.

Outcomes in Subjects With TEE

Table 4.

RLS% Detection raTCD Versus TEE and TCD

Safety

There were no serious adverse events in this study related to the device or microbubble contrast injection. There were 2 nonserious adverse events (anxiety) in the ITS population.

DISCUSSION

This study is the first multicenter, prospective study of the diagnostic accuracy, feasibility, and safety of raTCD as compared with standard of care TTE for the diagnosis of RLS. In this study, raTCD was ≈3 times more likely to diagnose any RLS presence as compared with TTE. It was also safe and technically feasible to obtain quality raTCD results with an operator who had no prior TCD skills. Importantly, raTCD detected large¹⁸ RLS at ≈3 times the rate of TTE. Otherwise stated, TTE completely missed or underdiagnosed approximately two thirds of the large RLS diagnosed by raTCD (Figure 2). Considering TTE is the most common screening diagnostic for RLS, our results suggest RLS are frequently underdiagnosed. The fact that TTE showed no signs of any RLS in half of the large RLS diagnosed by raTCD should be a signal for change in practice to those caring for stroke patients. These data are only the most recent in a long line of observational studies over the last 3 decades noting a remarkably consistent outcome of TCD being more sensitive to detect RLS and PFO than TTE. While TTE is of use in the workup of embolic stroke because it provides diagnostic information other than RLS status, TCD—standard or robot-assisted—may be considered as the front-line screening examination for RLS rather than TTE.

Figure 2.

A transthoracic echocardiogram (TTE) and robot-assisted transcranial Doppler (raTCD) in one of the study subjects with a negative TTE and a strongly positive raTCD. The 4-chamber apical view of the TTE demonstrates opacification of the right atrium and ventricle (on the left of the figure) with agitated saline contrast and none of those bubbles crossing into the left atrium or ventricle, suggesting the absence of a right to left shunt (RLS). In stark contrast, one can see on the right part of the figure that many bubbles, generating the white streaks seen on the raTCD—enough to fill entire cardiac cycles—indicating a large RLS. In the setting of discrepancy, the positive raTCD study is to be trusted as TTE is subject to false negative and the pattern of the result of raTCD does not suggest false positive. Please see Video S2 for audio and visual detail.

Our results comport with prior studies. TCD has a long, globally published experience as a highly sensitive diagnostic for RLS detection,^18,30–37 and has been previously compared with TTE and TEE in smaller series^38–40 of varying study design, with consistently favorable (>90%) sensitivity as compared with the gold standard of TEE. For the specific diagnosis of PFO, 2 meta-analyses^13,14 have demonstrated an ≈40% difference in PFO rate of detection between TCD and TTE, with TCD demonstrating ≈95% sensitivity and TTE ≈45% sensitivity overall as compared with TEE. Our finding of an absolute difference of 43% in RLS detection between raTCD and TTE, collected prospectively in a multicenter device trial format, aligns with these previous estimations and lends credence to their veracity. Our results suggest that an autonomous, five-degree-of-freedom robot with machine-learning-supported signal location and optimization algorithms reinforcing standard TCD instrumentation can achieve this known sensitivity of TCD for the diagnosis of RLS without the need for expert practitioners, who are scarce, to perform the test. We also provide reassuring data regarding the safety of raTCD with agitated saline injection, consistent with previous reports of the excellent safety profile of agitated saline contrast TCD bubble studies for RLS detection.⁴¹

As previously mentioned, prior studies estimate RLS prevalence in a patient population similar to ours to be as high as 50%.^1–4 However, we found a higher percentage of 63.6%. These prior series were based on echocardiographic data. In light of our findings, we conclude that the higher than expected detection rate is not only from subject selection focused on patients with probable cerebral embolic events, which was a feature of these previous studies, but also the known ability of TCD to detect the presence of small and extracardiac shunts with greater sensitivity than echocardiography.

Our study has limitations. Our population was predominantly older and white and, considering transtemporal window adequacy varies with age, sex, and ethnicity, a more diverse study population may have yielded a different no windows rate, but this limitation is unlikely to have affected the primary outcome. We cannot comment definitively on accuracy parameters of raTCD versus TEE or standard TCD testing because very few patients had all optional diagnostic tests performed. That said, as outlined in Tables 3 and 4, there is a signal of accuracy that is in line with the aforementioned studies and routine clinical practice: raTCD was positive for RLS in all cases TEE was positive, there were no cases of TEE positive and raTCD negative for RLS (eg, “false negative”), and all of the “false positive” raTCD results were Spencer Logarithmic Scale grade 1, indicating a small shunt that could conceivably be beneath the resolution of TEE. There was, per se, a significant difference (P=0.041) between raTCD and both TEE (Core Lab analysis) and TCD for the percent detection of RLS favoring raTCD.

Another limitation of this study is that the technique for the standard of care TTE bubble study was not specified in the methods, whereas the TCD bubble study followed a study-defined protocol. We acknowledge that factors such as the quality of injection, patient positioning, Valsalva technique, and bubble contrast preparation may have introduced bias in favor of raTCD into these results, but the hope of this approach was to have true standard of care as the control of this study. In addition, participating sites included high-volume comprehensive stroke centers and the predominantly acute ischemic stroke/transient ischemic attack patient sample may not represent the exact clinical population seen at an outpatient neurology clinic or echocardiographic practice. However, our study provides data for a common clinical scenario where accurate RLS detection is paramount.

These findings have a mixed standing when contextualized within current guidelines on the use of TCD for RLS evaluation. A European multisociety-supported position article on the management of patients with PFO states “[contrast]-TCD has a higher sensitivity than [contrast]-TTE as a first-line investigation to detect a right to left shunt,” granting a level of evidence “A” and a “conditional” strength of recommendation. The diagram of recommended diagnostics for RLS detection aligns with our findings, namely that TCD is a reasonable front-line screen and, if TTE is negative, to proceed to TCD given the superior sensitivity and concordance with TEE.⁴² In the most recent guideline from the American Heart Association/American Stroke Association (AHA/ASA) for the prevention of stroke in patients with stroke and transient ischemic attack, TCD is given a level of evidence C-LD (limited data), class of recommendation 2b (weak) for use “[i]n patients with ischemic stroke or transient ischemic attack in whom PFO closure would be contemplated…” It is not mentioned in the subsection on PFO nor does it appear in any of the flowcharts of recommended diagnostic studies in spite of acknowledging “TCD compares favorably with TEE for detecting right-to-left shunting…”⁴³ Furthermore, according to the American Academy of Neurology Practice Advisory Update on PFO and secondary stroke prevention, “…TCD has been demonstrated to have similar sensitivity and specificity to TEE to detect right to left shunting…” and “in patients being considered for PFO closure, clinicians may use TCD with agitated saline contrast as a screening evaluation for right to left shunt.”⁴⁴ Our data support a revisitation of the AHA/ASA and American Academy of Neurology guidelines and recommended diagnostics for RLS detection and secondary stroke prevention given that the currently recommended workflow predicated on TTE as a screening examination and a focus on PFO closure alone rather than considering the multiple mechanisms by which RLS can present stroke risk, surely leads to significant underdiagnosis of RLS based on our findings, and thereby missed opportunities to effectively prevent stroke recurrence.

In conclusion, raTCD was ≈3 times more likely to diagnose any RLS, including large PFO, in this cohort of patients as compared with TTE. TTE failed to diagnose approximately two thirds of the large RLS diagnosed by raTCD. These findings, buttressed by the aforementioned global, decades-long experience with TCD for RLS testing, support TCD as the superior initial screening test for RLS as compared with TTE. The raTCD was safe and feasible for use by personnel without TCD expertise, suggesting that raTCD can achieve the known sensitivity of TCD for RLS without an experienced operator to perform the examination.

ARTICLE INFORMATION

Sources of Funding

This work was funded by NeuraSignal, Inc.

Disclosures

Drs Alexandrov, Rubin, Tsivgoulis, and Volpi have received payment as consultants to the study sponsor. C. Douville has received travel support from the study sponsor. The other authors report no conflicts.

Supplemental Material

Trial Protocol

Statistical Analysis Plan

Supplemental Appendix

Investigators

Supplemental Methods

Tables S1–S4

Figure S1

Video S1

STARD Checklist

CONSORT Checklist