Robotic tomographic ultrasound and artificial intelligence for management of haemodialysis arteriovenous fistulae

Abstract

Background:

Arteriovenous fistulae (AVF) and Arteriovenous Grafts (AVG) may present a problematic vascular access for renal replacement therapy (RRT), reliant on recurrent specialist nurse and medical evaluation. Dysfunctional accesses are frequently referred ‘out of the dialysis clinic’ for specialist sonographic examination, with associated delays potentiating loss of vascular access viability and/or need for emergency intervention. Definitive anatomical and functional diagnostics based in the dialysis unit may help to solve these delays and associated complications.

Objectives:

This publication reports a novel vascular access monitoring concept, Robotic Tomographic Ultrasound (RTU).

Research design:

Robotic Tomographic Ultrasound incorporates a semi-autonomous, robotic vascular ultrasound system and purpose designed analysis software that can be deployed at the point of care. Three-dimensional scan data, as well as conventional B-Mode and Doppler data are obtained by the system and transferred to a cloud based reporting and analysis software. Scans are remotely annotated and interpreted by a sonographer, with diagnostic data presented securely to clinicians on their preferred web based application/web connected device.

Results:

Software developed specifically for pre AVF mapping, maturation and monitoring protocols, analyse the data and then present interpreted results to all caring clinicians to assist with decision making. Vascular access planning can be determined with high confidence with data from the Map module. Maturation data can be presented in line with institutional requirements to the dialysis nurse, facilitating precocious needle access.

Conclusion:

Robotic Tomographic Ultrasound is a novel approach to vascular access management that may reduce the risk of loss of functional access by regular monitoring with the system; automated alerts guiding clinicians to the need for pre-emptive intervention, and the potential to increase longevity of the vascular access.

Introduction

End Stage Renal Disease (ESRD) is increasing in incidence and prevalence throughout the world ¹ and presents a significant burden to the health system at large. ² Renal replacement therapy can prevent many of the complications of ESRD, but is associated with complications itself, that place further burdens on the health system.

Haemodialysis is the most commonly chosen renal replacement therapy in the United States, ³ where Vascular Access (VA) is required for successful and sustainable therapy. The mature arteriovenous fistula (AVF) is associated with a lower rate of mortality and morbidity compared to other access types, such as arteriovenous graft (AVG) and tunnelled central venous catheter (CVC).^4,5 Moreover, it has a lower rate of thrombosis, ⁶ a longer expected lifespan, ⁷ maintains patency with fewer secondary interventions,^6,8 has lower rates of infection^5,9 and a lower rate of symptomatic central venous stenosis. ¹⁰ Despite multiple efforts over the past decade to increase fistula first access, ¹¹ most patients are initiated through a tunnelled CVC, ³ with less than a third having an AVF at initiation even when under nephrological care in the prior 12 months. ³

Achieving a functional AVF for patients with incident ESRD is difficult. This was demonstrated by Bylsma et al. ¹² who published a meta-analysis on AVF formation in 2017. They reviewed the results of 62,712 AVF access formations, demonstrating 1 year primary, primary assisted and secondary patency rates of 64%, 73% and 79%, respectively. However only 26% of fistulas created were mature at 6 months, indicating that a large number of patients requiring alternative access such as a central line for initial renal replacement therapy. 21% of AVFs were abandoned without use, representing a poor use of surgical resources. Moreover, in patients whose AVF reached maturation, the average time taken to reach maturity was 3.5 months. We have previously demonstrated that AVF formation with early, adjuvant endovascular treatment can bring about maturation rates exceeding 90% at 6 months, ¹³ reliant on early post formation ultrasound and clinical monitoring.

Currently, access site mapping and AVF monitoring require referral of patients out of the dialysis clinic. Ultrasound image data must be obtained by specialist sonographers, then interpreted by the surgeon/interventionist to create a patient access life-plan. ¹ Ambiguous ultrasound study results may require additional imaging such as Computed Tomography Angiography (CTA) and/or catheter digital subtraction angiography (DSA). Such investigation invariably delay a patient’s access to a functional access, and are reliant on patients attending multiple outpatient appointments and surgical/interventional procedures.

Poor patient compliance, referral out of the dialysis clinic for imaging and the requirement for outpatient specialist reviews to plan access interventions may in part explain why at 6 months post initiation, almost 50% of patients remain on access via tunnelled CVC^3,11 in the United States.

Point Of Care Ultrasound (POCUS), defined as ultrasound scans performed by a caring clinician at the patient beside, is used in many dialysis clinics to facilitate needle access and provide basic diagnostics of AVF function. Schoch et al. ¹⁴ identified various use cases for POCUS in VA care, including assessment of new AVF maturation, identifying landmarks and abnormalities and assessing for alternative cannulation sites amongst other benefits. The authors also identified significant impediments to wider uptake, including time required for assessment, staff reluctance to upskill (to undertake further skills training) in ultrasound and negative impacts on workflow.

Definitive POCUS in the dialysis clinic that does not rely on skilled staff would bring benefits for the patient. Venous and arterial ‘real estate’ could be determined in advance of the patient requiring haemodialysis, new AVFs could be scanned to determine suitability for cannulation, and mature AVFs/AVGs could be monitored and definitively assessed for abnormalities that may threaten patency or produce dialysis dysfunction.

This article presents a semi-autonomous ultrasound based solution for AVF interrogation that provides definitive image and flow data, without the requirement for trained staff to operate. Artificial Intelligence models integrated into the software enable actionable diagnoses to be produced at the point of care, thus democratising decision making and reducing the need for referral to a specialist imaging clinic.

Methods

The Vexev Ultrasound Imaging System, a robotic ultrasound imaging and software system that allows for semi-autonomous scanning of the upper limb. It is designed to be deployed at the point of care for example, dialysis clinic, physician’s rooms and deliver results directly to the clinicians attending the patient. Human Research Ethics Council approval was deemed to be unnecessary for the presentation of computer simulated hardware and software optimisation data derived from the authors/staffs limbs presented in this article.

Overview of system

The Vexev Ultrasound Imaging System consists of a custom developed automated robotic ultrasound imaging hardware, called the vxWave system (Table 1), and viewing and analysis software called vxView. Image data are obtained by the vxWave system with review and analysis by a sonographer or technologist on the vxView platform. A secondary software module, called vxAccess, has been specifically designed using Artificial Intelligence models for haemodialysis access management, and is the primary interface for non-sonographer caring clinicians to diagnose the state of the access.

Table 1.

Acronyms and expansions as well as proposed nomenclature for scanning methodology.

Acronym	Expansion	Definition
B-mode	Brightness Modality	A linear array of transducers simultaneously scans a plane through the body that can be viewed as a two-dimensional image on screen
CFD	Computational Fluids Dynamics	A field of science that, with the help of digital computers, produces quantitative predictions of fluid-flow phenomena based on the conservation laws governing fluid motion.
POCUS	Point of Care Ultrasound	A portable imaging system with scanning performed in a clinical area outside of a specialised vascular or radiology laboratory.
RTU	Robotic Tomographic Ultrasound	An imaging system incorporating an autonomous robotic arm that moves an transducer around a field of interest. Captured echo data is computationally processed by time as well as transducer location along x, y and z axis’s to reconstruct a 3-Dimensional data representation of the field of interest
TPU	Thermoplastic Polyurethane	An engineered plastic with material qualities specifically designed for transmission of high frequency sound waves, containment of fluid and sterilisation between uses.
USCT	Ultrasound Computed Tomography	An imaging modality in which image data are captured via an array of stationary transducers which send and receive soundwaves from known, fixed locations. Captured echo data is computationally processed by time as well as location of the transducer to reconstruct a 3-Dimensional digital tissue block of the field of interest
Vexev	Vexev	An Australian based Medtech startup company aiming to deliver robotic tomographic ultrasound capabilities for vascular and non-vascular applications
vxAccess	Vexev Access Software System	Computational software that has been specifically designed to meet the needs to the dialysis access clinician
vxWAVE	Vexev Wave Ultrasound System	The actual device containing the ultrasound imaging system into which a patient would place their limb
vxView	Vexev View Software System	Computational software that has been designed to allow for imaging specialists to interrogate and label captured ultrasound images, specifically, vascular structures and the presence of abnormalities.

Hardware design

A novel device that allows for ‘hands free’ scanning of a limb. The proposed nomenclature (Table 1) for this new imaging modality is Robotic Tomographic Ultrasound (RTU). RTU differs from the previously described ultrasound computer tomography ¹⁵ (USCT), an imaging modality in which image data are captured via an array of stationary transducers which send and receive soundwaves from known, fixed locations. In an RTU system, an ultrasound transducer is robotically moved around the limb, with echo data recorded by location, thence digitally reconstructed to produce tomographic slices of the limb.

The vxWave is a RTU system consisting of a wheeled chassis (Figure 1) which contains an Ultrasound beamformer and computational hardware, a Thermoplastic Polyurethane (TPU) scan bed filled with a temperature controlled liquid, a robotically articulated ultrasound transducer and a touch screen and keyboard for control of the device/review of output data. The robotic arm can move around a limb and along its length within the TPU scan bed. The total volume of the unit is very similar to a standard full sized Ultrasound system. A scan is performed by the patient placing their limb onto the height adjustable temperature controlled TPU scan bed (see the blue surface shown in Figure 1 below); the robotic arm then moves the transducer around the limb with complete acquisition of scan data obtained in less than 10 min in almost all cases. Image data are not displayed until all image post processing has been completed. Interaction with the image data is more analogous to that of X-ray or MRI, that is, the end user will not view the acquired data until all data is acquired and processed.

Figure 1.

vxWave device – Patient places their limb on the TPU scan bed allowing for scanning of forearm and upper arm AVF.

Image processing

The data acquired by an RTU scan encompasses all B-Mode and Doppler data normally acquired in a complete peripheral vascular ultrasound exam, with the addition of probe location data that allows integration of images from multiple sweeps into a ‘digital tissue block’. This allows computation of highly precise 3d structural and flow data that is similar in detail to data obtained by a CT or MRI scan. The lack of radiation and nephrotoxic contrast agents are amongst some of the benefits of RTU scanning over other modalities (Figure 2).

Figure 2.

Comparison of RTU with other imaging technologies.

As the RTU system acquires multiple ‘slices’ of ultrasound data from the same z axis points with acquisition angles often varying in all three planes, proprietary tomographic image reconstruction methods are used to ‘blend’ the slices and fill in the 3d volume space. Tomographic reconstruction of the ultrasound stack data and identification of vasculature using artificial intelligence models allows for the creation of a point cloud model of the vasculature (Figure 3). This data can then be used to determine inverse kinematics for acquisition of flow and longitudinal b-mode data.

Figure 3.

vxView clinician interface demonstrating 3D reconstruction of upper limb vasculature: (a) brachial, radial and ulnar arteries with charts demonstrating arterial diameters over length of specified vessel, (b) 3D reconstruction of forearm and upper arm veins with chart demonstrating diameter of cephalic vein over length in forearm, (c) arterial and venous segments allowing of determination of relationship between artery and perforating veins and (d) mapping scan clinical application allowing for inspection of B-mode images as marker is dragged along vessels, as well as application of rules to determine eligibility for AVF.

Flow data are obtained with near continuous sampling throughout the vasculature region of interest. The acquired flow data allows for all standard calculations (stenosis %, flow volume, resistive index, peak systolic velocity and acceleration time). Automated validity checks are performed on image and flow data prior to the system informing operators that the scan is complete and ready for review. Standard dimensional measurements such as diameter of vessel are able to be performed on the b-mode image data (Figure 3).

Due to the autonomous nature of the device, staff training to operate the device is minimal. The skill required for staff to complete an RTU scan is analogous to blood pressure measurement with an automatic sphygmomanometer.

Once processed, the scan data are transferred to the reporting and analysis software vxView. vxView is a web application hosted on a managed cloud service accessible from the device itself or on any internet connected device with a modern web browser. The current proposed workflow is for off-site skilled staff, that is, accredited vascular sonographers, to ‘label/annotate’ the 3d model of the vasculature (Figure 3), confirming the location/name of arterial structures, veins and presence of pathology for example, stenosis, aneurysm, thrombosis etc.

This labelled 3d interactive model serves as an intuitive user visualisation tool, allowing for users to understand the 3d geometry of the vasculature, with each vessel ‘split’ along their length into proximal, mid and distal segments. Stenosis and flow volumes are calculated automatically and added to a user-chosen report template which allows for the production of a standardised ultrasound report made bespoke to institutional and healthcare re-imbursement standards.

Results

Scan result data is presented to the caring physician on the vxAccess system. This system has been designed specifically for AVF management, with results pushed to patients/caring clinicians on their preferred web based browser or application.

Vascular access software module

vxAccess is a set of software modules designed to address the requirements of clinicians caring for patients with end stage kidney disease who likely require, or are currently receiving haemodialysis therapy via an AVF/AVG access site. The effective applications of ultrasound duplex imaging at the map, maturation and monitoring phases of the AVF lifecycle was described previously by Zamboli et al., ¹⁶ and is employed by vxAccess to provide actionable insights at each stage of the AVF lifecycle. vxAccess has hence been purpose designed in collaboration with dialysis clinics to allow for scanning to be able to be automatically performed during regular clinic visits, while not requiring skilled imaging staff on site to operate device.

To better integrate into the workflow of renal clinics or dialysis providers, and in contrast to normal medical image reporting systems, vxAccess displays patient level vascular access data (Figure 4), that is the patients current VA access, their vxAccess stage (map, mature, monitor) and a semantic colouring (green, orange, red) to indicate the need for review of that patient; Complete ultrasound duplex scan data are available for review, and it is expected the user will view these data when there are indications of access pathology or where there is an indication to review the geometry of the vasculature.

Figure 4.

vxAccess clinician interface utilising sematic colouring to draw the clinicians attention to potential AVF dysfunction/violation of pre-set rules of patency/function.

This novel data view allows clinicians to view the status of patients’ access at a glance without the need to open a specific scan report. The stage in patients’ journey from map to monitor is displayed in a linear visual component, with a ‘traffic light system’ of semantic colouring, based on a model of ultrasound criteria, to call attention to patients who may require VA review. Those criteria used to judge an AVF can be made bespoke to institutional requirements. Further to this, the system is able, via push notifications, to actively update users of the system to patient changes which may require attention.

Map

The vxAccess Map module exploits the novel 3d segmented data acquired by the vxWave to produce interactive 3D ‘maps’ of the vasculature. Distance between vessels, vessel size, flow data and vessel location with respect to skin are used as features for the assessment engine to output a report of the suitable vascular access options (Figure 3(d)). Where there is no suitable superficial vein for access creation, the clinician will be notified and can opt for scanning of the contralateral limb, or where no such vein is also present in that limb, plan for an AVG. Nephrologists can therefore be informed at a very early stage as to the presence of vascular access anastomosis sites, which may help inform the early discussions regarding renal replacement therapy planning.

Mature

The vxAccess Mature module uses a rules engine to notify clinicians, based on vessel spatial and flow data, when the AVF is likely to be able to support dialysis (Figure 5). Clinicians have the option to use ‘out of the box’ rules sets, such as the previously described and validated rule of 6’s¹ (brachial flow of greater than 600 ml/min, cannulation zone vein depth of less than 6 mm and outflow vein diameter of greater than 6 mm), or create and use their own rulesets.

Figure 5.

AVF maturation software demonstrating example post formation scan results: (a) immature AVF 2 weeks post formation due to outflow vein diameter and flow rate, (b) immature AVF at 4 weeks, again due to reduced outflow vein diameter and flow rate, (c) immature AVF at 6 weeks due to reduced outflow vein diameter and flow rate, now demonstrating likely aetiological juxta-anastomotic stenosis and (d) AVF post intervention to juxta-anastomotic stenosis, now meeting rules for cannulation.

Monitor

Due to the automated nature of scan data acquisition, it becomes feasible to acquire, with minimal disruption to the patient’s normal care episode, high fidelity ultrasound fistula monitoring data each time the patient attends for dialysis therapy.

These data along with complex flow data are able to be used as features for an assessment model to give AVF/AVG an overall health grading, either using a machine learning model or a simple rules based grading algorithm (Figure 6). In addition, the software can also be used to record dialysis performance parameters for example, Kt/V, URR, Arterial Pump Pressure, Venous Pump Pressures, Flow rate on Dialysis etc. It should be noted that due to the practicality of very frequent scanning, the derivatives of these data with respect to time are able to be included as features, presenting an opportunity for early detection of signals in the data; indeed this ability to collect detailed longitudinal data opens the opportunity for deep learning early prediction applications discussed later.

Figure 6.

Mature AVF analysed in Monitoring application: AVF does not violate institutional rules for maturation and is thus deemed a functioning AVF.

The software architecture means end users are able to be automatically alerted when velocity, vessel diameters or flow volumes exceed predetermined absolute or relative values. A simple example of this would be an alert sent out notifying a nephrologist that the outflow vein peak velocity of one of their patients has increased by more than 150 cm/s since their last scan 2 performed weeks prior.

Discussion

The Vexev Ultrasound imaging system is designed to improve the lives of ESRD patients through the streamlining of AVF/AVG care. It is a confluence of latest generation robotics, artificial intelligence and software engineering, designed such that all involved health care providers have visibility, and can be informed of the AVF status of patients as changes occur, allowing for prompt recognition and timely intervention.

ESRD presents an increasingly prevalent burden on the healthcare system, with many impediments to patients achieving a functional AVF. Despite well-publicised guidelines and strong evidence for HD to be initiated via an AVF, more than 80% ¹⁷ of patients in the United States initiate dialysis via a central venous catheter. This use of central venous catheter access persists to 6 months, remaining alarmingly high at more than 50%. ¹⁸ The reasons for this are invariably complex, but well described in the United States Renal Data System ³ reports. Improving information systems to provide a prompt diagnosis of available conduits for AVF creation, and detection of the presence of significant stenosis impairing AVF maturation may go a long way to enabling early treatment with increased timely maturation confirmation and subsequent AVF use. ¹³ Prompt detection of clinically significant AVF stenoses and indeed less apparent derangement of access function in mature AVF may also allow for preservation and hence longevity of this optimal access through timely corrective and preventative intervention.

Current information systems available to monitor and plan AVFs are fragmented. These systems can also vary between institutions, and ultimately rely on the patient and nurse care coordinator to ensure patients receive timely attention. AVF planning is reliant on duplex ultrasound scans and clinical assessment, both of which require experienced clinicians to perform and interpret findings. The determination of maturation can follow simple institutional rules such as the rule of 6s, are reliant on skilled individuals for example, the treating surgeon or nephrologist, to authorise access of a newly formed AVF. AVF monitoring systems such as Transonic HD03 Haemodialysis Monitor (Transonic Systems Inc, Ithaca NY, USA), SmartPatch monitoring system (Alio Medical, Broomfield CO, USA), may be combined with intra-access dialysis measurements of efficiency (e.g. arterial pressures, venous pressures, Kt/V, Urea Reduction Ratio etc) to determine the risk of fistula dysfunction, thrombosis and loss of access. However positive findings still need to be confirmed with duplex ultrasound scan ± fistulography to confirm the presence of a contributory stenosis. Such fragmentation of AVF management systems cause delay.

Integration of AVF mapping, maturation and monitoring outcomes to the one, widely available, clinician-user interface allows for prompt referral for definitive treatment, without the need for further investigations. The Vexev Ultrasound Imaging System therefore aims to address several pitfalls in the management of AVFs. The vxWave device aims to provide rapid vascular ultrasound results that are equivalent/superior to conventional ultrasound. The presence of the device at points of care such as dialysis units and nephrologists/surgeons’ rooms allows for diagnostic information to be acquired rapidly without additional outpatient referral for a conventional scan. The vxView system allows for human intelligence interrogation of the acquired data, to ensure the system meets institutional ultrasound standards, and is diagnostically accurate. The annotated scan data along with patient status data are then fed into vxAccess to enable the clinician interaction described above.

vxAccess enhances patient care over traditional systems through timely scans and efficient information management. The mapping module scan may be applied in the dialysis unit during the patients first dialysis session. Prompt referral for vascular access creation can then be made. Once created, frequent RTU scans during the first 6–8 weeks will allow for the detection of stenoses that may cause access thrombosis or failure to mature (Figure 5). Alternatively, scans in this early phase may demonstrate AVF that meet institutional criteria for early puncture. Once deemed matured and in the phase of active use, regular RTU scans may non-invasively detect stenoses that either cause AVF dysfunction or lead to loss of the access. These modules provide timely, accurate information, without any invasive procedure or need for external referral, thus presenting a significant improvement over traditional AVF management systems.

The potential applications of the Vexev Ultrasound imaging System are widespread. The acquisition of large AVF anatomical digitised data sets will provide the opportunity to build data models with the output of predicting AVF failure. Computational Fluid dynamics (CFD) have been studied in many parts of the vascular tree, but are rarely applied in routine clinical practice. Machine learning models may foreseeably be applied to CFD models of thousands of AVF sample data entries, allowing for prediction of AVF failure, perhaps from the first post AVF formation examination. This will allow for a better understanding of AVF patency, and anatomical drivers of stenosis and AVF failure.

The current iteration of the Vexev Ultrasound Imaging system is designed to perform ultrasound examinations of the upper limb vasculature. However, the acquisition of tomographic ultrasound data from the limbs will allow for examinations of musculoskeletal (joints, tendons, muscles and their compartments) and soft tissues (lipomas, cysts, lymph nodes). The addition of these imaging modalities will not require any significant redesign of the hardware component, rather the design of additional software modules to present the data.

At present the device and software are in mature prototype stage. Hardware and software development is ongoing to ensure accuracy of the acquired data. International clinical feasibility and comparison trials are planned in order to meet the requirements for FDA 510 K pathway approval, the results of which will be published in due course.

Conclusion

Robotic Tomographic Ultrasound with the Vexev Ultrasound Imaging System and management of data with vxView software module may provide a new tool in the management of upper limb vascular access, with the potential to increase the longevity of a patients AVF by providing actionable mapping information, facilitating timely access once the AVF has been shown to be mature, and reducing vascular access complications through timely detection of pathology.