The Use of Artificial Intelligence in Automation in the Fields of Gynaecology and Obstetrics – an Assessment of the State of Play

Abstract

The long-awaited progress in digitalisation is generating huge amounts of medical data every day, and manual analysis and targeted, patient-oriented evaluation of this data is becoming increasingly difficult or even infeasible. This state of affairs and the associated, increasingly complex requirements for individualised precision medicine underline the need for modern software solutions and algorithms across the entire healthcare system. The utilisation of state-of-the-art equipment and techniques in almost all areas of medicine over the past few years has now indeed enabled automation processes to enter – at least in part – into routine clinical practice. Such systems utilise a wide variety of artificial intelligence (AI) techniques, the majority of which have been developed to optimise medical image reconstruction, noise reduction, quality assurance, triage, segmentation, computer-aided detection and classification and, as an emerging field of research, radiogenomics. Tasks handled by AI are completed significantly faster and more precisely, clearly demonstrated by now in the annual findings of the ImageNet Large-Scale Visual Recognition Challenge (ILSVCR), first conducted in 2015, with error rates well below those of humans. This review article will discuss the potential capabilities and currently available applications of AI in gynaecological-obstetric diagnostics. The article will focus, in particular, on automated techniques in prenatal sonographic diagnostics.

Introduction

In 1997 the reigning world chess champion G. Kasparov was conquered by a computer (Deep Blue). Public awareness of artificial intelligence (AI), however, predates this success. In fact, the first successful AI applications were developed significantly beforehand and have now become an integral and accepted (often unrecognised) feature of our daily lives 4 . AI experts from large companies report that 79 percent of participants in surveys consider AI techniques as strategically highly significant or even vital for sustainable business success. Put another way, artificial intelligence has now become a mainstream technology worldwide in every industry. Competencies in core AI technologies (machine learning with deep learning, natural language processing and computer vision) are nowadays indispensable for larger companies 5 .

Although no single accepted definition of artificial intelligence exists, most exerts would agree that AI as a technology refers to any machine or system that can perform complex tasks that would normally involve human (or other biological) brain power 1 ,  2 ,  3 . Hence, the term artificial intelligence refers not just to a single technology but to a family of AI applications in a wide range of fields ( Fig. 1 ). Machine learning (ML) is a member of this family that focuses on teaching computers to perform tasks with a predetermined goal, without explicitly programming the rules for performing such tasks. It can be regarded as a statistical method that continuously improves upon itself through exposure to increasing volumes of data. This allows such systems to consecutively acquire the ability to correctly recognise objects from images, texts or acoustic data by searching for common properties and regularities and from which patterns can ultimately be extracted.

Fig. 1.

 Alan M. Turingʼs 1950 review paper on “machine” intelligence formed the conceptual basis for the introduction of the “Turing test” to ascertain whether a machine can be said to exhibit artificial intelligence. The development of artificial intelligence and its applications can be viewed in a temporal context: machine learning and deep learning are not merely related in name; deep learning is a modelling approach that enables, among other things, problems in modern fields such as image recognition, speech recognition and video interpretation to be solved significantly faster and with a lower error rate than might be feasible by humans alone 4 ,  47 ,  55 .

Deep learning, the flagship discipline of ML, unlike other machine learning methods, no longer requires direct intervention by humans. Such machine learning processes are made possible by artificial neural networks (ANN) or convolutional neural networks (CNN), which consist of several convolutional layers, followed by a pooling layer that aggregates the data of the filters and eliminates superfluous information ( Fig. 2 ). In this way, the abstraction level of a CNN increases with each of these filter levels. Developments in the field of computer-aided signal processing and the expansion of computing power with the latest high-speed graphics processors now allow an unlimited number of filter layers within a CNN to be created, which are thus referred to as “deep” (in contrast to conventional “shallow” neural networks, which usually consist of only one filter). Such learning is an adaptive process in which the weighting of all interconnected neurons changes so as to ultimately achieve the optimal response (output) to all input variables. The process allows either supervised or unsupervised approaches to neural network learning. In the former, ML algorithms employ a pre-coded data set to predict the desired outcome. In contrast, unsupervised approaches are supplied with only unlabelled (coded) input data in order to identify hidden patterns within them and, consequently, make novel predictions.

Fig. 2.

 Schematic design of a (feed-forward) convolutional network with two hidden layers. The source information is segmented and abstracted to achieve pattern recognition in these layers and ultimately passed on to the output layer. The capacity of such neural networks can be controlled by varying their depth (number of layers) and width (number of neurons/perceptrons per layer).

Since the early 2000s, deep learning networks have been successfully employed, for example, to recognise and segment objects and image content. AI-assisted voice control and speech recognition are based on similar principles, for instance, the Amazon Alexa, Google Home and Apple Siri voice assistants. Such technologies have a wide range of applications. Their developers, for example, claim that it is now possible to utilise audio capture (“coughing” apps) to identify vocal patterns characteristic of COVID-19 6 . Furthermore, automated evaluation of speech spectrograms can now be employed to identify vocal biomarkers for a variety of diseases, such as depression 6 ,  7 .

In terms of healthcare, computer-assisted analysis of image data is undoubtedly one of the most significant advantages of AI. In recent years, a new and rapidly developing field of research has emerged in this context, which, under the umbrella term “radiomics”, aims to employ AI to systematically analyse imaging data from patients, characterising over a range a large number of individual and distinct image features with regard to correlation and clinical differentiation. In contrast, the term “radiogenomics” refers to a specialised application in which radiomic or other imaging features are linked to genomic profiles 8 .

AI and Benefits for Gynaecological-Obstetric Imaging and Diagnostics

The initial hysteria that AI technologies could potentially replace clinical radiologists has now abated. In its place is an awareness that machine learning capabilities will enable personalised AI-based software algorithms with interactive visualisation and automated quantification to accelerate clinical decision-making and analysis time. The uptake of AI in other clinical fields, however, is still rather modest or hesitant 9 ,  10 ,  11 . Computer-assisted diagnosis (CAD) systems have actually been in use for more than 25 years, in particular in breast diagnostics 12 ,  13 . Novel deep-learning algorithms are employed to optimise diagnostic capabilities both in the area of mammography and in AI-supported reporting of mammasonography data sets, addressing issues limiting the use of conventional CAD systems (high development costs, general cost/workflow (in)efficiency, relatively high false positive rate, restriction to certain lesions/entities) 14 . This is illustrated by a recent US-British study in which a CNN was trained on the basis of 76 000 mammography scans, resulting in a significant reduction of false-positive and false-negative findings by 1.2 and 2.7% (UK) and 5.7 and 9.4% (USA), respectively, compared to initial expert findings 15 . In addition, consistent AI support can also sustainably reduce workload by automatically identifying normal screening results in advance that would otherwise have required traditional assessment 16 . OʼConnell et al. have published similarly promising data on AI-assisted evaluation of breast sonography findings. They demonstrated on the basis of 300 patients that, using a commercial diagnostic tool (S-Detect), automated detection of breast lesions using a set of BI-RADS descriptors was in agreement with the results obtained by ten radiologists with appropriate expertise (sensitivity, specificity > 0.8) 17 .

The advantage of deep learning algorithms has also been made explicitly evident in other application areas in our field. Cho et al., for example, have developed and validated deep learning models to automatically classify cervical neoplasms in colposcopic images. The authors optimised pre-trained CNNs in two scoring systems: the cervical intraepithelial neoplasia (CIN) system and the lower anogenital squamous terminology (LAST) system. The CNNs were capable of efficiently identifying biopsy-worthy findings (AUC 0.947) 18 . Shanthi et al. were able to correctly classify microscopic cervical cell smears as normal, mild, moderate, severe and carcinomatous with an accuracy of 94.1%, 92.1% and 85.1%, respectively, using various CNNs trained with augmented data sets (original colposcopy, contour-extracted and binary image data) 19 . In the view of Försch et al., one of the main challenges, generally, to increased integration of AI algorithms in the assessment of pathology and diagnosis of histomorphological specimens is that, at present, only a fraction of histopathological data is in fact available in digital form and thus accessible for automated evaluation 20 . This situation still applies to the vast majority of potential clinical AI applications 1 ,  21 .

Very comparable approaches have also been pursued over the last five years in reproductive medicine, in which successful attempts have been made, among other things, to utilise AI technologies for embryo selection. These have involved training CNNs to make qualitative statements based on image data and/or morphokinetic data, to predict the success of implantation 22 ,  23 . A study by Bori et al. involving more than 600 patients analysed not only the above-mentioned morphokinetic characteristics but also novel parameters such as the distance and speed of pronuclear migration, inner cell mass area, increased diameter of the blastocyst and length of the trophectoderm cell cycle. Of the four test algorithms, most efficient (AUC 0.77) was the one combining conventional morphokinetic and the above-mentioned morphodynamic features, with the latter two parameters significantly more likely to be associated with differences in implanted and non-implanted embryos 24 .

It is beyond dispute that, to date, only relatively few AI-based ultrasound applications have progressed fully from academic concept to clinical application and commercialisation. In addition to the importance of AI in prenatal diagnostics, as discussed in the following paragraphs, the advantages of AI-based automated algorithms have been very impressively demonstrated in reporting gynaecological abnormalities, a task that is bound to gain in importance, in particular, given the limited quality of existing ultrasound training 25 ,  26 ,  27 . Even though the first work in this field dates back more than 20 years 28 , significant pioneering work has been conducted in the last decade in particular, not least due to the extensive studies of the IOTA Working Group. Model analyses on risk quantification of sonographically detected adnexal lesions have been able to demonstrate the extent to which, on the one hand, a standardised procedure for qualified assessment and, on the other, a multi-class risk model (IOTA Adnex – a ssessment of d ifferent n eoplasias in the Adn ex a) validated on the basis of thousands of patient histories have made it possible to precisely and reproducibly assess the quality of sonographic findings of adnexal processes, thereby providing a significant boost to other study approaches (± AI) in this field 29 ,  30 ,  31 . The incorporation of the ADNEX model into a consensus guideline of the American College of Radiologists (ACR) clearly supports these findings. The decision to do so is remarkable, as the US professional medical associations are traditionally considered to be sceptical of ultrasound across all disciplines 32 . In a recent study on the validity of two AI models to determine the character (benign/malignant) of adnexal lesions (trained on grey-scale and power doppler images), Christiansen et al. demonstrated a sensitivity of 96% and 97.1%, respectively, and a specificity of 86.7% and 93.7%, respectively, with no significant differences to expert assessments 33 . The additional benefit of various ML classifiers, alone or in combination, has been investigated in several other approaches, which have likewise found that, in the future, AI approaches will be able to identify more ovarian neoplasms and be increasingly employed in their (early) detection 34 ,  35 ,  36 ,  37 ,  38 . In a recently published study, Al-Karawi et al. used ML algorithms (support vector machine classification) to investigate seven differing familiar image texture parameters in ultrasound still images, which, according to the authors, can provide information about altered cellular composition in the process of carcinogenesis. By combining the features with the best test results, the researchers achieved an accuracy of 86 – 90% 39 .

AI in Foetal Echocardiography

Naturally, any analysis of the value of AI raises the question of how automated approaches can benefit foetal cardiac scanning – both in terms of diagnostics and with regard to the practitioner – which is one of the most important but also more complex elements of prenatal sonographic examinations. Here, it is important to recognise that although detection rates of congenital heart defects (CHD) in national or regional screening programmes have improved demonstrably over the last decade, their sensitivity still ranges from 22.5 to 52.8% 40 . The reasons for this are complex – one of the main factors is without doubt the fact that the vast majority of CHDs actually occur in the low-risk population, with only approximately 10% occurring in pregnant women with known risk factors. Furthermore, a Dutch study suggests that, in addition to the lack of expertise in routine clinical practice, factors such as limited adaptive visual-motor skills in acquiring the correct cardiac planes and reduced vigilance seem to play a crucial role in identifying cardiac abnormalities 41 .

Experience from cardiology in adults has shown, among other things, that the use of automated systems (not by any means a novel conceptual approach) is demonstrably more efficient than a conventional (manual) approach and is likely to bridge the gap between experts and less experienced practitioners, while at the same time reducing inter- and intraobserver variance. Pilot studies on automated analysis of left ventricular (functional) parameters such as ventricular volume and ejection fraction on the basis of 2-D images and studies on AI-based tracing of endocardial contours in apical two- and four-chamber views using transthoracically acquired 3-D data sets have demonstrated accuracy comparable to manual evaluation 42 ,  43 . In this context, Kusunose defined four steps that are critical for developing relevant AI models in echocardiography (in addition to ensuring adequate image quality, these steps include level classification, measurement approaches and, finally, anomaly detection) 44 . Zhang et al. investigated the validity of a fully automated AI approach to echocardiographic diagnosis in a clinical context by training deep convolutional networks using > 14 000 complete echocardiograms, enabling them to identify 23 different viewpoints across five different common reference views. In up to 96% of cases, the system was able to accurately identify the individual cardiac diagnostic levels and, in addition, to quantify eleven different measurement parameters with comparable or even higher accuracy than manual approaches 45 . Does this mean that AI algorithms will replace echocardiographers or even cardiologists in the future? Should we be worried? Have we become members of the “useless class”, in the provocative words of Harari 46 ? The answers to these questions are unambiguous yet complex and equally applicable to foetal echocardiography. Although AI approaches will very soon be an integral component of routine cardiac diagnostics, examiners have a continuing or even increased responsibility to employ their clinical expertise to understand, monitor and assess automated procedures and, when errors occur, to take appropriate remedial action 47 . Arnaout et al. have successfully trained a model for creating diagnostic cross-sectional planes using 107 823 ultrasound images from > 1300 foetal echocardiograms 48 . In a separate modelling approach, they were then able to distinguish between structurally normal hearts and those with complex anomalies. The findings of the AI were comparable to those of experts. A slightly lower sensitivity/specificity (0.93 and 0.72, respectively, AUC 0.83) was documented by Le et al. in 2020 in their AI approach on nearly 4000 foetuses 49 . Dong et al. demonstrated how accurately a three-step CNN is able to detect different representations of four-chamber views based on 2-D image files and at the same time provide feedback on the completeness of the key cardiac structures imaged 50 .

In summary, it is beyond question that the essential prerequisite for efficient cardiac diagnostics, as discussed above, is the creation of exact cross-sectional images obtained during examinations. In the final analysis, this prerequisite should apply to all disciplines, and in particular that of functional imaging diagnostics. Hinton formulated this pertinently: “To recognize shapes, first learn to generate images” 51 . Noteworthy here is the recent approval by the US Food and Drug Administration (FDA) of an adaptive ultrasound system (Caption AI) to support and optimise sectional plane creation (and recording of video sequences) echocardiography in adults. In the view of the developers, this demonstrates how the enormous potential of artificial intelligence and machine learning technologies can be used specifically to improve access to safe and effective cardiac diagnostics 52 . Another commercially available high-sensitivity ultrasound simulator (Volutracer O. P. U. S.) has a comparable AI workflow. The simulator controls and adaptively corrects in real time the manual settings and transducer movements to achieve an exact target plane in any 2-D image sequence (irrespective of the anatomical structure) ( Fig. 3 ) 53 . A major advantage of these systems is undoubtedly their usefulness in particular in training and advanced training, since, among other things, the integrated self-learning mode can be used to automatically train, evaluate and certify operators without the need for experts to personally adjust settings 54 .

Fig. 3.

 Representation of the optical ultrasound simulator Volutracer O. P. U. S. Any volume data set (see also Fig. 5 ) can be uploaded and be adapted, for instance for teaching, by post-processing to acquire appropriate planes (so-called freestyle mode – without simulator instructions). In the upper right-hand corner of the screen, the system provides graphical feedback to assist movements to establish the correct target level. The simulation software also includes a variety of cloud-based training datasets that help teach users the correct settings using a GPS tracking system and audio simulator instructions with overlaid animations. Among other things, the system measures the position, angle of rotation and time until the required target plane is achieved and compares this with an expert comparison that can likewise be viewed.

Due to its comparatively small size, the foetal heart usually takes up a comparatively small area of the US image, and this, in turn, requires any algorithm to learn to ignore at least a portion of the available image data. Another difference to postnatal echocardiography is that the relative orientation and position of the heart in the image in relation to the position of the foetus in the uterus can vary considerably, further complicating image analysis 55 ,  56 . HeartAssist is an interesting approach to automated recognition, annotation and measurement of cardiac structures using deep-learning algorithms that is about to be launched on the market. HeartAssist is an intelligent software tool employed in foetal echocardiography that can identify and evaluate target structures (axial, sagittal) from 2-D static images (directly or as a single frame extracted from video sequences) of cardiac diagnostic sectional planes ( Fig. 4 ). Noteworthy is that, on the one hand, the tool can capture even partially obscured image information and integrate it into the analysis and, on the other hand, the image recognition is successful even with a limited sonographic window. This approach, like most algorithms to automate (foetal) diagnostics (e.g. BiometryAssist, Smart OB or SonoBiometry), is based on segmentation (abstraction) of foetal structures. It employs a wide variety of automated segmentation techniques (pixel, edge and region based models, as well as model and texture based models), which are usually combined to achieve better results 57 ,  58 .

Fig. 4.

 Four-chamber view of a foetal heart in week 23 of pregnancy. The foetusʼ spine is located at 3 oʼclock, the four-chamber view can be seen in a partially oblique orientation. In addition to abdominal and cardiac circumference, the inner outline of the atria and ventricles is automatically recognised, traced and quantified in the static image. Similarly, HeartAssist can annotate and measure all other cardiac diagnostic sectional planes (axial/longitudinal).

Particular significance of 3-D/4-D technology

The introduction of 3-D/4-D technology, now pre-installed on most US systems, has now opened up a range of diverse display options that are being increasingly utilised for automated image analysis and layer creation. With this technology some manufacturers are offering commercial software tools to facilitate a volume-based approach to foetal echocardiography and its standardised interpretation (Fetal Heart Navigator, SonoVCADheart, Smart Planes FH and 5D Heart). The latter algorithm facilitates a standardised workflow-based 3-D/4-D evaluation of the cardiac anatomy of the foetus through implementation of “foetal intelligent navigation echocardiography” (FINE) ( Fig. 5 ). This method analyses STIC (spatial temporal image correlation) volumes with the four-chamber view as the initial plane of volume acquisition. In the next step, predefined anatomical target structures are marked and the nine diagnostic planes needed for a complete foetal echocardiographic assessment are automatically reconstructed. Each plane can subsequently be evaluated independently of the others (e.g. quantitative analysis of the outflow tracts) and, if required, then be manually adjusted. Yeo et al. showed that 98% of cardiac abnormalities could be detected using this method 59 . This has been shown to be easy to learn and simplifies work-flows to evaluate the foetal heart independently of expert practitioners, a feature that is particularly important for capturing congenital anomalies in detail 60 ,  61 .

Fig. 5.

 5DHeart (foetal intelligent navigation echocardiography, FINE) program interface with automatically reconstructed diagnostic planes of an Ebsteinʼs anomaly of a foetus in the week 33 of pregnancy (STIC volume). The atrialised right ventricle is clearly visible as a lead structure in the laevorotated four-chamber view (cardiac axis > 63°). The foetusʼ back is positioned at 6 oʼclock by default after the automated software has been implemented (volume acquisition, on the other hand, was performed at 7 – 8 oʼclock, see Fig. 5 ). Analysis of the corresponding planes has also revealed a tubular aortic stenosis (visualised in three-vessel view, five-chamber view, LVOT and aortic arch planes).

Acquisition and quantification of objectifiable foetal cardiac functional parameters is similarly demanding and thus examiner-dependent, similar to manual level reconstruction. At this juncture, special mention should be made of speckle tracking echocardiography, which provides quantitative information on two-dimensional global and segmental myocardial wall movement and deformation parameters (strain/strain rate) on the basis of “speckles” caused by interference from random scatter echoes in the ultrasound image. The introduction of semi-automatic software (fetalHQ), which uses a 2-D video clip of the heart and manual selection of a heart cycle and corresponding marking of the annulus and apex, has now made it possible for less experienced practitioners to quantify the size, shape and contractility of 24 different segments of the foetal heart using AI-assisted analysis of these speckles 62 ,  63 ,  64 ( Fig. 6 ). Beyond this, AI methods to analyse Doppler-based cardiac function (modified myocardial performance index (Mod-MPI), previously termed the Tei index) have been developed in recent years and are now commercially available 65 ,  66 .

Fig. 6.

 Software tools for functional analysis of the foetal heart. Semi-automated approach to speckle tracking analysis using fetalHQ in the foetus examined in Figs. 3 and 5 with Ebsteinʼs anomaly ( a ). A selected cardiac cycle is analysed in the approach using automatic contouring of the endocardium for the left and/or right ventricle and subsequent quantification of functional variables such as contractility and deformation. Automated calculation of the (modified) myocardial performance index (MPI, Tei index) by spectral Doppler recording of blood flow across the tricuspid and pulmonary valves using MPI+ ( b ).

AI in standardised diagnostics of the foetal CNS

As mentioned above, the decisive advantage of automated techniques in prenatal diagnostics is clearly that they will allow less experienced practitioners to correctly identify highly complex anatomical structures such as the foetal heart or CNS in a standardised and examiner-independent manner. The basis for such tools is formed by transthalamic (TT) 3-D volume data sets (analogous to the sectional image setting needed to quantify biparietal diameter) acquired with AI-assisted post-processing and evaluation. These allow a primary examination of the foetal CNS with extraction of the transventricular (TV) or transcerebellar (TC) plane from the volume block (SonoCNS, Smart Planes CNS) or even a complete neurosonogram (5DCNS+) ( Fig. 7 ). After axial alignment of the corresponding B and C planes and marking of thalamic nuclei or the cave of the septum pellucidum, the latter algorithm also automatically reconstructs the coronal and sagittal sectional planes required for a complete neurosonogram ( Fig. 5 ). The latter working group documented successful visualisation rates of 97.7 – 99.4% for axial, 94.4 – 97.7% for sagittal and 92.2 – 97.2% for coronal planes in a prospective follow-up study using the 5DCNS+ modified algorithm 67 . A retrospective clinical validation study of more than 1100 pregnant women yielded similar results 68 . In contrast to the data of Pluym et al., the authors were able to show in their study, among other things, that this standardised approach could be used to collect biometric parameters that, in comparison, were similarly valid and reproducible to those obtained manually 69 . Ambroise-Grandjean et al. were similarly unequivocal when they showed in a feasibility study that the three primary planes including biometric measurements could be consistently reconstructed and quantitatively evaluated using AI (Smart Planes CNS) with low intra- and interobserver reproducibility [ICC > 0.98] 70 .

Fig. 7.

 (Semi-)automatic reconstruction after application of 5DCNS+ of an axially acquired 3-D volume of the foetal CNS (biparietal plane) in a foetus with a semilobar holoprosencephaly in week 23 of pregnancy. The complete neurosonogram reconstructed from the source volume comprises the 9 required diagnostic sectional planes (3 axial, 4 coronal and 2 sagittal planes). In the axial planes, automatic biometric measurements (not shown) are taken, which can be adjusted subsequently by hand at any time.

These algorithms are already in use in clinical practice, however, they usually require intermediate steps taken by the practitioner. Nevertheless, in the future specially trained CNNs will be able to fully automatically extract all sectional planes from raw volumes. Huang et al. demonstrated that “view-based projection networks” (CNN) from post-processed 3-D volumes (axial output volume and corresponding 90° sagittal/coronal rotations) could reliably detect and image five predefined anatomical CNS structures in parallel in three different 3-D projections, of which the best detection rates were achieved once again on the axial view 71 . The latter is due, among other things, to the gradual reduction in image quality inherent in the orthogonal B and C planes. The authors used the data sets of the INTERGROWTH-21 study group for their analysis. Precise B image quality and accuracy in sectional plane imaging is an indispensable prerequisite for 2-D-based AI approaches, especially for automated detection of abnormal CNS findings, as recently published by Xie et al. 72 . In this paper, CNN were trained using 2-D and 3-D datasets of approximately 15 000 normal and 15 000 abnormal standard axial planes and assessed for segmentation efficiency, binary classification into normal and abnormal planes, and CNS lesion localisation (sensitivity/specificity 96.9 and 95.9%, respectively and AUC 0.989). Before such AI approaches can be used in those areas where they would be of greatest benefit, namely in routine diagnostics, a number of “hurdles” still need to be cleared. These are primarily associated with the initial steps in imaging diagnostics (in keeping with Hintonʼs exhortation that quality is based on image generation), and this ultimately also applies to other foetal target structures in prenatal diagnostics 51 . It would be interesting here to determine, for example, to what extent such automated approaches enable standardised plane reconstruction in combination with DL algorithms to classify and thus accurately and reproducibly detect, annotate and quantify two- and three-dimensional measurement parameters, thereby enabling diagnostics that in the future are significantly less dependent on the presence of an expert practitioner. Of particular interest are, clearly, approaches in which, for example, specialised neural networks are used to optimise image acquisition protocols in obstetric ultrasound diagnostics, thereby shortening examination times and providing comprehensive anatomical information, even from, at times, obscured image areas. For example, Cerrolaza et al. demonstrated (analogous to deep reinforcement learning models for incomplete CT scans) that, even if only 60% of the foetal skull were captured in a volume dataset, AI reconstruction would nevertheless still be possible 73 ,  74 .

The potential of neural networks has also been demonstrated by recent papers by Cai et al. who developed a multi-task CNN that learns how to detect standard axial planes, such as foetal abdominal and head circumference (transventricular sectional plane), by detecting eye movements of the examiner when viewing video sequences 75 . Baumgartner et al. were able to show that a specially trained convolution network (SonoNet) could be used to detect thirteen different standard foetal planes in real time and correctly record target structures 76 . Yacub et al. took a similar approach, using a neural network to, on the one hand, ascertain the completeness of a sonographic abnormality diagnosis and, on the other, to perform quality control of the image data obtained (in accordance with international guidelines). No differences were demonstrated in this case compared to manual expert assessment 77 ,  78 . The same approaches to modelling now also form the (intelligent) basis for the worldʼs first fully integrated AI tool for automated biometric detection of foetal target structures and AI-supported quality control (SonoLyst) 5 . The potential of neural networks is also apparent in recent data from a British research group on AI-based 2-D video analysis of the workflow of experienced practitioners. This analysis allows systems to predict which transducer movements are most likely to result in the creation of precise target planes in abnormality diagnostics 79 . The same research group were able to demonstrate, on the one hand, that their initial AI models were able to automatically recognise video content (sectional planes) and add appropriate captions and, on the other hand, that specially trained CNNs were able to evaluate combined data from a motion sensor and an ultrasound probe, converting them into signals to augment correct transducer guidance 80 ,  81 .

AI and Other Clinical Applications in Obstetric Monitoring

Optimisation of biometric accuracy is another area where AI can be directly clinically relevant. Regardless of the assistance systems already mentioned (see above) and notwithstanding the significant improvement in ultrasound diagnostics over the past few years, such optimisation remains a challenge. The majority of foetal weight estimation models are based on parameters (head circumference, biparietal diameter, abdominal circumference, femoral diaphysis length) measured during conventional 2-D ultrasounds. Hitherto, the development of the soft tissue of the upper and lower extremities, although not directly biometrically quantifiable, was the established surrogate parameter for foetal nutritional status 82 . Three-dimensional measurement of the fractional limb volume (FLV) of the upper arm and/or thigh has been shown to improve the precision of foetal weight estimation, even in multiple pregnancies 83 . Automated techniques that allow much faster and, above all, examiner-independent processing of 3-D volumes (efficient recognition and tracing of soft tissue boundaries) have clearly demonstrated the clinical benefit of volumetric recording of FLV (5DLimbVol), which implements workflow-based, relevant, axially acquired 3-D data sets of the upper arm or thigh and incorporates them into conventional weight estimation ( Fig. 8 ) 84 ,  85 .

Fig. 8.

 Automated sectional plane reconstruction of a foetal thigh in week 35 of pregnancy to estimate foetal weight (soft tissue mantle of the thigh reconstructed by 5DLimb). After 3-D volume acquisition of the thigh aligned transversely, the soft tissue volume calculated in this way can be used to improve the accuracy of estimations of foetal weight.

AI has now also made it possible to automatically record sonographic parameters such as angle of progression (AoP) and head direction (HD), even as birth progresses. The first findings on this technique were published by Youssef et al. in 2017, who found that an automated approach is possible and can be used in a reproducible manner 86 . Just how far commercially available software solutions such as LaborAssist will improve clinical care remains to be seen, however.

A heated debate, illustrative of the occasional difficulties encountered in employing automated techniques in clinical practice, is under-way regarding the potential benefits of computer-assisted assessment of peripartal foetal heart rate (electronic foetal heart rate monitoring), which, due to the clear interobserver variability and subjectivity in assessing CTG abnormalities, could, at least theoretically, benefit from objective automated analysis. Prospective randomised data from the INFANT study group did not demonstrate any advantage over conventional visual assessment by medical staff present during delivery, neither in neonatal short-term outcomes nor in outcomes at two years 87 . The question of how far methodological weaknesses in the design of the study contributed to these non-significant differences between the study arms (Hawthorne effect) remains open 88 ,  89 , especially since other computer-based approaches delivered clearly promising data 90 .

To answer this, Fung et al. used data from two large population-based cohort studies (INTERGROWTH 21st and its phase II study INTERBIO 21st) to show machine learning can be employed to analyse biometric data from an ultrasound performed between weeks 20 and 30 of pregnancy along with a repeat measurement within the following ten weeks of pregnancy to determine gestational age to within three days and to predict the growth curve over the next six weeks in an individualised way for each foetus 91 . There is no doubt that AI will become ever more important in the future, potentially, for instance, in assessing and predicting foetomaternal risk constellations such as prematurity, gestational diabetes and hypertensive diseases of pregnancy 92 .

Summary

The authors of a recent web-based survey at eight university hospitals stated, among other things, that the majority of respondents tend to view AI in a positive light and, ultimately, believe that the future of clinical medicine will be shaped by a combination of human and artificial intelligence, and that sensible use of AI technologies will significantly improve patient care. The study participants considered the greatest potential lay in analysis of sensor-based, continuously collected data in electrocardiography/electroencephalography, in monitoring of patients in intensive care and in imaging procedures in targeted diagnostics and workflow support 93 . Specifically with regard to our field, it should be emphasised that the continuous development of ultrasound systems and the equipment associated with them, for instance high-resolution ultrasound probes/matrix probes for gynaecological and obstetric diagnostics, along with the inexorable introduction of efficient automated segmentation techniques for two and, in particular, three-dimensional image information, will increasingly influence and optimise the entire process chain in the future, from image data creation, analysis and processing through to its management.

A recent systematic review of more than 80 studies on automated image analysis found that AI delivered findings that were equivalent in precision to those from practitioners who were experts in their field. However, the authors also found that in many publications, no external validation of the various AI algorithms had been performed, or only inadequate validation. This situation, together with the collaboration between AI developers and clinicians, which is already well underway in many areas but still needs to be intensified, is currently making further implementation in relevant clinical processes even more difficult 94 . The current state of AI utilisation in healthcare is similar to the situation of owning a brand new car; making use of it will require both petrol and roads. In other words, the respective algorithms need to be “fuelled” with, for example, (annotated) image data, but they will only be able to fully realise their potential if the appropriate infrastructure – efficient and scalable processes with an AI-ready workflow – is in place 21 .

Outlook

AI systems continue to be developed and integrated into clinical processes, and with this comes tremendous expectations on how they will advance healthcare. What is certain is that integrating these tools is likely to fundamentally change work and training methods in the future. They will support all healthcare professionals by providing them with rapidly and reliably collected data and facts to interpret findings and consultations, which will, in the best case scenario, allow them to focus more on the uniquely human elements of their profession. Those tasks that cannot be performed by a machine because they demand emotional intelligence, such as targeted patient interaction to identify more nuanced symptoms and to build trust through human intuition, highlight just how unique and critical will be the human factor in deploying clinical AI applications of the future 95 . If nothing else, this reminds us that AI is a long way from truly replacing humans. Almost 100 years ago the visionary writings of Fritz Kahn (“The Physician of the Future”) were already foreshadowing current and future AI technologies in medicine: highly plastic constructivism in which technological civilisation and experimental science can synergistically transform the biology of the human body 96 ,  97 . One thing that emerges from these advances is that, notwithstanding all technical progress, humans have not yet, nor ever will, render themselves superfluous. Predictions that up to 47% of all jobs would be lost due to automation would seem to be unfounded; in healthcare in particular, more jobs are being created than are being lost 46 ,  98 ,  99 .

What is needed to optimally exploit the potential of AI algorithms is interdisciplinary communication and constant involvement of physicians as the primary users of these tools in the processes of developing and the modes of operating AI tools. In the absence of such involvement, the medicine of tomorrow will be shaped exclusively by the vision of engineers and will be less able to meet the actual requirements of personalised (precision) medicine 47 ,  100 . Table 1 summarises the most urgent research priorities for AI as formulated by the participants at the 2018 consensus workshop of radiological societies 101 ,  102 . From the perspective of gynaecology and obstetrics, mention should be made that regarding AI-assisted sonographic parameters, continued efforts are underway to optimise imaging (pre-/post-processing) in both conventional 2-D imaging and 3-D/4-D volume sonography, and that, similar to the established algorithms with an automated workflow, the need is for further AI technologies that provide intuitive user guidance, ease of use and general (cross-device) availability to efficiently analyse image and volume data. In addition, assisted systems for real-time plane adjustment and target structure quantification should be further pursued for routine diagnostics. Of particular note here, is the fact that it is now possible to incorporate pre-trained algorithms to analyse oneʼs own population-based data (transfer learning). This constitutes an attractive and, above all, reliable method, as training a new neural network with a large volume of data is computationally and time intensive 103 . The process adopts the existing, pre-trained layers of a CNN and adapts and re-trains only the output layer to recognise the appropriate object classes of the new network.

Table 1  Recommendations from the 2018 consensus workshop on translational research held in Bethesda, USA on advancing and integrating artificial intelligence applications in clinical processes (adapted from Allen et al. 2019, Langlotz et al. 2019 101 ,  102 ).

Research priorities for artificial intelligence in medical imaging
Structured AI use cases and clinical problems need to be created and defined that can be actually solved by AI algorithms.
Novel image reconstruction methods should be developed to efficiently generate images from source data.
Automated image labelling and annotation methods that efficiently provide training data to explore advanced ML models and enable their intensified clinical use need to be established.
Research is required on machine learning methods that can more specifically communicate and visualise AI-based decision aids to users.
Methods should be established to validate and objectively monitor the performance of AI algorithms to facilitate regulatory approval processes.
Standards and common data platforms need to be developed to enable AI tools to be easily integrated into existing clinical workflows.

It is highly likely that the greatest challenge facing targeted use of AI in healthcare in general, however, is not whether automated technologies are fully capable of meeting the demands placed on them but whether they can be incorporated into everyday clinical practice. To achieve this, among other things, appropriate approval procedures must be initiated, the appropriate (clinical) infrastructure must be established, standardisation ensured, and, above all, clinical staff must be adequately trained. It is clear that in the future these hurdles will be surmounted, but the technologies themselves may well take longer to mature. We should therefore expect to see still limited uptake of AI in clinical practice over the next five years (with more widespread uptake within ten years) 104 .