Review of Machine Learning and Artificial Intelligence (ML/AI) for the Pediatric Neurologist

Abstract

Artificial intelligence (AI) and a popular branch of AI known as machine learning (ML) are increasingly being utilized in medicine and to inform medical research. This review provides an overview of AI and ML (AI/ML), including definitions of common terms. We discuss the history of AI and provide instances of how AI/ML can be applied to pediatric neurology. Examples include imaging in neuro-oncology, autism diagnosis, diagnosis from charts, epilepsy, cerebral palsy, and neonatal neurology. Topics such as supervised learning, unsupervised learning, and reinforcement learning are discussed.

Section I: Introduction

The search to understand how humans and animals think is old and diverse: from Aristotle¹ to Broca and Wernicke’s studies^2,3 to localize specific brain functions. The field of Artificial Intelligence (AI) attempts to understand human intelligence and to create intelligent machines. AI and a branch of AI known as Machine Learning (ML) are gaining increased interest and utilization in precision medicine, clinical diagnosis, management, and research. Rather than an exhaustive treatise, we introduce AI and ML (AI/ML) with a brief history (Figure 1A and 1B) and review examples of how ML can be used in pediatric neurology.

Figure 1:

A Timeline of Artificial Intelligence/Machine Learning. A) Depicts events from 1940s to the present and B) from 1990s-present day, focusing the events that led up to one of the biggest achievements of AI: outperforming human champions in the game, Go. In Figure 1B, the state space is a measure of the possible configurations of a system (e.g., game). As shown by the dashed line, AI/ML has shown an exponential increase over recent years in it its ability to “solve” increasingly complex games (as measured against human expert performance). The bar graph for years 2010-2017 show a corresponding reduction in the error of deep learning systems in identifying objects in images within the ImageNet competition, with AlexNet developed in 2012 marking the dawn of Deep Learning and 2015 coincident with both reaching human-level accuracy (solid, horizontal line, “Human Level”) in the ImageNet competition and AlphaGo defeating Lee Sedol in Go.

Definitions

Definitions of AI/ML are established first, although some may interpret these terms differently. Al is the field that seeks to create intelligent machines, i.e. the “computational part of the ability to achieve goals in the world”.⁴ ML is a subset of AI that develops “computer algorithms that improve automatically through experience.”⁵ AI can be applied to any of the key facets of human cognition: perception, decision-making, and action.

ML uses data to infer a mapping of a set of inputs to a desired output, such as an Artificial Neural Network (ANN, Figure 2) that learns to predict whether a tumor is present on magnetic resonance imaging (MRI) given examples of MRI images that experts have annotated as a tumor is present or not – a process known as supervised learning (Figure 3A).⁶ The ANN is a crude, computational model of a biological neural network with simulated neurons, connection between those neurons, and groupings of neurons into “layers.” Information is passed from the input of the ANN (e.g., the intensity of each pixel of an MRI) to the first layer, from the first layer to the second, and so forth until the network outputs a desired signal (e.g., the probability of a tumor being present on MRI). These layers perform a series of multiplication, addition, and activations (e.g., the neuron can be turned “on” with an output of 1 if the input value exceeds a threshold), and the parameters governing these mathematical operations are trained on data to perform the desired operation (Figure 2). Different training mechanisms are discussed later in Section III. There is ongoing interest in developing ANNs that are more biologically plausible, called Spiky Neural Networks.⁷

Figure 2:

This figure depicts an artificial neural network with one input layer, one hidden layer, and one output layer. The input layer consists of M input features, (x₁, x₂, … , x_M). The single, hidden layer consist of D neurons, $\left({\eta }_1^{(1)},{\eta }_2^{(1)},\dots ,{\eta }_D^{(1)}\right)$. The superscript ⁽¹⁾ denotes the layer, where the input layer is layer 0 and the output layer, in this case, is layer 2. The hidden layer has nonlinear activation functions known as a Rectified Linear Unit (ReLU), which sets the output of the neuron to be the same as its input if the input is positive and 0 otherwise. The output layer has a sigmoid activation function in its single neuron, ${\eta }_1^{(2)}$, to convert the output into a single number between 0 and 1, e.g., for a supervised learning classification task for predicting whether the inputs correspond to a positive (y = 1) or negative (y = 0) instance and $\widehat{y}$ is the estimated likelihood of the input belonging to the positive class. The input to each layer is a weighted, linear combination of the outputs of the previous layer. A mathematical description of the neural network functioning is provided to the right for completeness. Note that neural networks may consist of any number of hidden layers and neurons per hidden layer, may vary in the types of activation functions, inputs, and outputs, may have various connection configurations, and more.

Figure 3:

Graphical representations of Supervised Learning (A), Unsupervised Learning (B), and Reinforcement Learning (C).

Al can be used in either a screening capacity (i.e., the machine makes a diagnoses or treatment recommendations before the physician sees the data) or as a second opinion. There are trade-offs. Humans are susceptible to the anchoring-and-adjustment heuristic.⁸ The consequence is that the physician might be inclined to overly rely on an AI if the AI is screening the patient, and the physician might under-rely on the AI if the AI is providing a second opinion. Moreover, AI can reason both about the likelihood of the test results given a potential disease and the prior probability of that disease occurring within the population. However, if the system is built using data from only individuals with rare diseases, then the system would not have the capacity to calibrate to the prior probability of the disease’s occurrence. As such, either the system should be trained on data from everyone (sick and healthy) or the system needs to be handed a prior probability for the occurrence of each disease in order to calibrate its predictions.

Al techniques that are not ML-based may tackle these same problems (i.e., perception, decision-making, and action) but would not use systems that learn from data like ML. One such example of a human-designed mapping would be expert systems, which is a computational representation of a set of IF-THEN rules solicited by domain experts.⁹

Al also includes subfields, such as Natural Language Processing (NLP), Computer Vision, or Robotics. NLP develops computational models for understanding human language for tasks such as translating English to Chinese (i.e., machine translation). Computer vision enables computers to recognize visual objects and can be used to identify abnormalities on images (discussed below, Section IV: Neuro-oncology). Computer vision can be used to identify a single type of abnormality, or an algorithm can be trained to identify multiple different abnormalities.¹⁰ Robotics develops intelligent machines that physically act within the world; thus, robotics encompasses includes the design of computational techniques and physical systems to interact with physical objects. For this review, however, we consider those “subfields” instead to be research communities focused on an application or extension of AI for a specific problem that may focus on a different set of algorithmic and modeling techniques (and hardware design, in the case of robotics). For a reference list of abbreviations and definitions of terms, please refer to Table 1.

Table 1:

A list of abbreviations and definitions used in this review.

Term	Abbreviation	Definition
Artificial Intelligence	AI	The field of creating machines that are intelligent.
Artificial Neural Network	ANN	A crude, computational model of a biological neural network with simulated neurons, connection between those neurons, and groupings of neurons into “layers.” See Figure 2.
Computer Vision	--	Computers recognize visual objects in the world
Convolutional Neural Network	CNN	A type of ANN that is most commonly used to analyze images, and uses layers of pattern-detecting filters to analyze images
Decision tree	--	A method to predict the output variable based upon different input variables
Deep Learning	--	Uses neural networks with three or more layers
Expert System	--	A computational representation of a set of IF-THEN rules solicited by domain experts
Explainable Artificial Intelligence	XAI	A subfield of AI that seeks to develop algorithmic mechanisms to afford users insight into the inner workings of these algorithms
Generative adversarial network	GAN	A class of machine learning frameworks in which two neural networks challenge each other -- a generator and a discriminator -- duel each other, with the generator attempting to create synthetic data (e.g., a fake MRI) that can fool the discriminator and a discriminator attempting to tease out fake from real data.
Logic Theorist	--	Finding a sequence of logical steps to prove a theorem
Machine Learning	ML	A subset of AI that develops and studies computer algorithms that improve automatically through experience
Natural Language Processing	NLP	Computational models for understanding human language for tasks
Random Forest Model	--	An ensemble learning method by using multiple decision trees. See Figure 5.
Reinforcement Learning	RL	A method of learning in which an agent needs to interact with or experience the world to learn the best possible sequence of actions to maximize its future rewards. See Figure 3C.
Robotics	--	The field that develops intelligent machines that physically act within the world
Supervised Learning	--	An ML algorithm that is trained upon labeled training data to predict the outcomes. See Figure 3A.
Support Vector Machine	SVM	A supervised learning method in ML to analyze data for classification and regression analyses. See Figure 4.
Transformer	--	A deep learning model that uses self-attention to differentially weigh each part of the input data as well as the outputs of each hidden layer of the neural network.
Unsupervised Learning	--	An ML algorithm discovers patterns in unlabeled training data; also referred to as knowledge discovery. See Figure 3B.

Section II: History of ML/AI

For brevity and simplicity in presentation, this history focuses on ML through the lens of supervised learning (Figure 1A and 1B). A few comments on other ML methods (i.e., unsupervised learning and reinforcement learning, Figures 3B and 3C) are discussed in Section III. An early work in AI is attributed to McCulloch and Pitts in 1943¹¹ where they encoded propositions in nets using binary neurons, which were later used for learning.¹² Donald Hebb¹³ later demonstrated learning by changing the connection strengths for neurons and Minsky and Edmonds in 1950 developed the first neural network (NN) computer to simulate a network of 40 neurons replicating functions of smaller systems of neurons.¹⁴

Meanwhile, Alan Turing in his “Computing Machinery and Intelligence” described ideas which would later become machine learning, reinforcement learning (Figure 3C), and genetic algorithms.¹⁵ He also described the “Turing test” as a valid test for intelligent behavior. In this test a human judge needs to distinguish the human participant from the unseen pair of a human and a computer participant using natural language questions and typed answers. The test still remains a high bar for computer algorithms today.¹⁵ If not for the work of Turing in developing AI to decrypt Nazi messages, the allies might have lost World War II.

The term, AI, was coined by John McCarthy at a 1955 conference.¹⁶ A popular work at this conference was Logic Theorist, which proved theorems by searching to find a sequence of logical steps to begin at the initial premise to arrive at the theorem’s conclusion.¹⁷ Logic theorist is the first non-numeric AI system and subsequent work resulted in a branch of AI studying cognitive architectures.^17-19 In 1958, Rosenblatt developed the “Perceptron” as the first ANN capable of learning a binary (yes/no) decision with a linear decision boundary¹⁹, but it was overhyped.¹⁴ Subsequent reports documented the failure of these systems,¹⁴ which led to the first “AI Winter” from 1974-1980. This winter came with a decreased funding and interest in AI.

The 1980s saw a renewal in AI research owing to the development of expert systems.⁹ While impressive, these hand-designed expert systems were brittle, required extensive domain-expertise, and lacked common sense. The initial hype followed by a lack of capability led to the second AI winter in the late 1980s.²⁰

A few standout methodologies were developed during this winter, including Hopfield networks²¹ (enabling ANNs to contain memories) and the backpropagation algorithm²², which allows ANNs to be trained more efficiently. Other progress in ML was made in defining the hardness of a learning problem using statistical learning theory. Vapnik formulated the empirical risk minimization framework along with a ML model called Support Vector Machines (SVMs, Figure 4), which provided guarantees on model performance.²³ In addition, probabilistic reasoning and Bayesian Networks were developed to encode uncertainty.²⁴

Figure 4:

Graphical representation of support vector machines, which can divide data into linear (A) or non-linear groups (B). This figure depicts two instances in which a support vector machine (SVM) is applied to classify positive (white circles, y = +1) and negative (black circles, y = −1) examples in data. To the left is the simpler case when the data are linearly separable (i.e., a SVM with a linear kernel can draw a line such that all examples from one class are on one side of the line and vice versa). To the right is the more complicated case where the data are not linearly separable, and a nonlinear kernel is required to separate the positive and negative examples. In the linearly separable case, we want to maximize the “margin,” defined as the inverse of the magnitude of the “support vector,” $\overrightarrow{\omega }$, such that the positive examples are above the separating line and the negative examples are below the line. This separation constraint is described by $y(\overrightarrow{\omega }{}^{T}x-b)\ge 1$, which describes a linear kernel. For the non-linearly separable case on the right, the data could be separated by a radial basis function kernel, a description for which is beyond the scope of this review.¹⁰¹

With the internet and the massive accumulation of electronic data led to an era in the 2000’s known as “Big Data” (Figure 1B).²⁵ Big Data enabled innovations, such as IBM Watson;²⁶ an ES operating on Big Data. The real success of Big Data was enabling the next era – Deep Learning.²⁷ The dawn of Deep Learning was marked by AlexNet,²⁸ a type of ANN architecture, which nearly cut in half the error rate in labeling an object within an image (e.g., labeling an image of a cat as “cat”). Deep Learning on Deep Neural Networks (DNNs), as opposed to “just” ANNs, is the confluence of five factors:

1. Improved ANN architectures (e.g., the convolutional neural network²⁷ (CNN) in AlexNet, which can be seen as a computational analogue for biological vision)²⁹,

2. Deeper ANN architectures (i.e., have more layers of artificial neurons) than the shallower networks of previous decades,

3. The same backpropagation algorithms developed in the 1980’s,

4. Access to Big Data, and

5. Access to more computational power with Moore’s Law³⁰ and graphics processing units, which enable fast operations over ANNs.²⁷

By 2015-2016, DNNs with Reinforcement Learning (RL) were used to achieve human-level performance in playing Atari Games³¹ and the game, Go, with Google DeepMind’s AlphaGo.³² More modern techniques have led to state-of-the-art performance in natural language understanding and generation,³³ protein folding predictions with AlphaFold,³⁴ and super-human performance in multi- and single-player games.³⁵ Another variation on neural networks include using generative adversarial network (GANs) -- a generator and a discriminator – duel each other, with the generator attempting to create synthetic data (e.g., a fake MRI) that can fool the discriminator and a discriminator attempting to tease out fake from real data.³⁶ GANs are entering the public arena with ANNs that can create art from user text prompts.³⁷ Transformers are another type of neural network model using a mechanism called “attention,” and have become the dominant approach for applications of supervised learning in NLP and CV.³⁸

One aspect of AI is the “black box” problem where the input(s) and output(s) are known but the inner workings are not. Explainable AI (XAI)^39,40 is a burgeoning subfield of AI that seeks to develop algorithmic mechanisms to afford users insight into the inner workings of these algorithms. A basic XAI mechanism is feature importance⁴¹ (i.e., conveying to the user which feature inputs were most consequential in the AI’s decision-making). Within XAI, researchers classify various approaches based upon transparency. For example, a decision tree would be highly transparent as it literally allows a human to simulate the operations of the machine. However, the computation of feature importance for a neural network model on MRI or EEG data might provide information about what subset of the data is important but not why. A helpful review on the importance of XAI is by Rudin.⁴²

Having discussed XAI generally, the question becomes: is XAI impactful and how? The answers are “yes” and “it depends.” In Dr. Matthew Gombolay’s prior work,⁴³ we have shown that XAI can increase a person’s trust in an AI system and increase the degree to which a person thinks the AI is human-like. However, that is potentially dangerous, as increasing trust and heightened anthropomorphization are risk factors for humans to incorrectly accept bad advice from a machine.⁴⁴ As such, one must be careful to balance the potential value of giving physicians insights into a machine’s decision-making without creating a situation in which the perceive intelligence of the explanation gives the physician a false sense of security that the machine is competent. How these different techniques can apply in pediatric neurology will be discussed later on in this review.

Section III: Types of ML/AI and Applications

Types of ML

ML can be divided into three main subtypes: supervised (predictive), unsupervised (descriptive), and reinforcement learning (RL, see Figure 3).

In supervised learning, the parameters of the ML model, such as an ANN, are trained (i.e., automatically tuned through optimization algorithms to maximize the model’s performance) to accurately predict the label (e.g., tumor or not) given the example (e.g., an MRI image) in cases where the input has not been seen before (e.g., an MRI for a new patient, Figure 3A). This focus on maximizing performance on test (i.e., novel) examples partly distinguishes ML from traditional statistics.

In contrast, unsupervised learning is a paradigm in which training data is unlabeled and the algorithm discovers patterns in the data, which can also be referred to as knowledge discovery.⁶ Unsupervised learning generally seeks to understand and extract the underlying processes within the data (Figure 3B). While the origins of unsupervised learning trace back to Grossberg in 1982,⁴⁵ unsupervised learning became prominent in the 2000’s.

Arguably, the most challenging class of learning is RL⁴⁶ in which an agent needs to interact with or experience the world to learn the best possible sequence of actions to maximize its future “rewards” (Figure 3C).⁴⁷ In general, RL approximates reward/punishment mechanisms used to train behaviors in animals. RL is critical in scenarios where the best action must be learned through trial and error and deducing which actions taken were responsible for a good or bad outcome. The analogy to animal training breaks down, though, as most RL systems start out knowing nothing, whereas animals generally have mastered fundamental motor and social skills to facilitate the trainer-trainee interaction. Nonetheless, the training mechanisms of some RL algorithms, such as temporal difference methods (TD),⁴⁸ have been identified in animal dopamine receptors.⁴⁹

General ML/AI Approach

In approaching an ML application, different methods can be used to develop a model to learn the data. Traditional statistical methods, such as linear or logistic regression, take the data (inputs) to generate a model (output) to explain the data. This process is similar to ML but with a different emphasis and associated set of techniques. In ML, the input is likewise data; however, the model is not the output. The model is the mapping from the input to the output, and the output (at least in supervised learning), is a prediction about the data.

The ML practitioner must make four decisions: 1) Selecting the type of ML (e.g., supervised, unsupervised, reinforcement learning, see Figure 3); 2) Designing the ML model (e.g., neural networks – Figure 2, support vector machines – Figure 4, random forest – Figure 5, and if a neural network is selected, how many layers will be used; examples of how these models are used will be discussed in Section IV); 3) Picking an objective function that scores how well the model is performing (e.g., the root mean squared error), and 4) Choosing a training procedure (e.g., whether to use a feature selection algorithm to search for and eliminate unhelpful components of the input data). While researchers in ML are more focused on novel deep learning architectures and while the proliferation of deep learning applications is profound, random forest models still often outperform neural networks in tabular data settings (e.g., excel spreadsheets with typical healthcare data).^50,51 As such, data scientists and ML practitioners alike commonly employ random forest models. Nonetheless, there is growing interest in enabling neural networks to handle such data more effectively.^52,53

Figure 5:

Graphical depiction of random forest, an ensemble of multiple decision trees. Each decision tree is created at random and combines different data features to classify the data. The random forest takes different random decision trees and uses the output from the multiple trees to make a decision.

While this review will not discuss many validation methods, K-fold cross validation is one way to assess the model accuracy. In this method, the data is used as both training and test data and is useful when insufficient data is available for separate sets. The dataset is divided into a k number of groups. Every group but one is used to train the model and then the other group is used as the test model, and this is performed k-1 times to develop the optimal model.⁶ Leave one out cross validation (LOOCV) is the extreme version in which one datapoint is left out and the model is evaluated based upon the remaining data.

Different combinations of ML techniques can be mixed, matched, and combined in a patchwork to create complex ML systems, which can become complicated and difficult to describe in a parsimonious fashion. A further complication is that no one ML approach is better than all others on all data⁵⁴ – one must tailor the ML approach to the problem at hand. Some applications of ML to healthcare will utilize a single method, whereas others will use different methods to find the optimal model for the data.

“Optimal” can be defined in different ways. For example, an “optimal model” for identifying tumors on imaging would be a model that is 100% sensitive and 100% specific, with identifying 100% of tumors 100% of the time and not mislabeling any tumors. The mislabeling of tumors would be the misclassification rate. However, one must be careful not to overly focus on achieving a 100% model performance on the training data as doing so often comes with poor test performance – a problem called “overfitting” in which the model learns spurious correlations in the data. For example, people know what a "tree" is by determining the right features (e.g., a tall, wooden plant with branches and leaves). Overfitting would be to look at the instance of trees in a single dataset, by counting all the leaves, such as three trees in a single dataset with 11,567 and 12,508, and 13,701 leaves each. An overfitting model would say that a tree has that exact number of leaves; otherwise, that object is not a tree. This would obtain 100% accuracy of the model on the training data but fails to generalize. Underfitting would be to say that a tree is anything with a leaf which would erroneously include other plants such as bushes, flowers, and vines. ML models are designed (e.g., regularization) to try to balance this over- vs. underfitting problem. ML researchers often employ the principle known as Occam's Razor, which prefers simpler models/explanations over more complicated ones, as simpler (fewer parameters) models tend to have a similar testing accuracy to their training accuracy, whereas more complicated models tend to have really good training accuracy but are wrong for testing. There are mechanisms to address overfitting and several other issues in ML, but such are outside of the scope of this review.

Section IV: Examples of ML in Pediatric Neurology

Here we will next highlight different studies in pediatric neurology when possible. ML/AI can be used in any aspect or any field of pediatric neurology, but this review is to highlight specific ways that ML/AI has been used for pediatric neurology.

Cerebral Palsy

The Gross Motor Function Measure-66 (GMFM-66) is an assessment tool used to measure gross motor function in patients with cerebral palsy (CP) that takes 45-60 minutes to administer. One study aimed to use ML to develop a shorter, more efficient version of the GMFM-66, or reduced version called rGMFM-66. Existing data was used from children with CP participating in a single center’s rehabilitation program. Random forest (Figure 5), SVM (Figure 4), and ANN (Figure 2) were self-learning models utilized to estimate a GMFM-66 score using fewest possible test items. Intraclass correlation coefficients (ICC) were used to evaluate agreements between tests. A total of 1217 GMFM-66 one-time assessments and 187 GMFM-66 assessments/reassessments after 1 year were evaluated. GMFM-66 scores were accurately predicted using SVM. The rGMFM-66 and GMFM66 had high ICCs for one-time assessments and evaluating change over 1 year. Thus, ML algorithms could administer the GMFM-66 more efficiently.⁵⁵

Neuro-oncology

Objective, standardized assessment of tumor burden and treatment response is essential in pediatric neuro-oncology. The Response Assessment in Pediatric Neuro-Oncology (RAPNO) working groups have developed objective criteria for tumor measurement and treatment response.⁵⁶ Manual tumor segmentation for volumetric measurements is a labor-intensive process, and methods to automate tumor segmentation using artificial intelligence have emerged. For example, a deep learning-based technique is used for two-dimensional and volumetric measurement of a mixed cohort of patients with high grade gliomas, medulloblastoma, or other leptomeningeal seeding tumors.⁵⁷

In the study, 704 preoperative and 1003 postoperative MRIs were split into training and testing sets. A convolutional neural network (CNN) was used for tumor segmentation. CNN is a type of ANN (Figure 2) that is most commonly used to analyze images, and uses layers of pattern-detecting filters to analyze images. Two-dimensional and volumetric measurements were made from the segmented T1 post-contrast and T2/FLAIR imaging sequences and these measurements were used to calculate RAPNO scores. Deep learning algorithm performance were compared to manual measurements and RAPNO scoring by ICC. Excellent agreement occurred between the automated and manually derived tumor volumes for both pre- and postoperative MRIs, (ICCs 0.896 to 0.960), and automated and manual RAPNO scores were also high (ICCs 0.851 to 0.909).⁵⁷ The study highlights the use of deep learning methods for pediatric brain tumors.

Epilepsy

ML can be applied to different data sources in epilepsy including analyzing ictal patterns on electroencephalography (EEG) to automate seizure detection and localization. This can be done using supervised learning employing spatiotemporal feature extraction from seizure labeled EEG segments or unsupervised ML methods that do not rely on manually labeled data.

One study used neural networks to classify video recordings of neonatal limb movements as different seizure types versus nonseizures. After training on 120 recordings, the authors achieved seizure detection with a sensitivity from 86-94% and specificity of 93 - 98%.⁵⁸ Additionally, SVM (Figure 4) was used in interictal intracranial EEG recordings and functional connectivity to predict Engel I seizure freedom outcomes with 100% sensitivity and 89% specificity in children.⁵⁹

A random forest classifier (Figure 5) was used to investigate clinical variables from demographics, insurance types, seizure rates to medication responses to demonstrate emergency department utilization in children with epilepsy. ⁶⁰ Additionally, ML has helped analyze genetic and clinical data to expand genotype phenotype correlates, automate neuroimaging analysis, aggregate neuropsychological profiles to predict outcomes of epilepsy surgery and responses to anti-seizure medications, highlighting its utility in clinical practice to advance the management of epilepsy.^61-63

Neuroimaging in Epilepsy

Over the last decades, several surface-based algorithms have been developed for fully automated detection of focal cortical dysplasia, the most prevalent epileptogenic cortical malformation ^64-72. While careful manual corrections of tissue segmentation and surface extraction are essential for accurate modeling of the main pathological characteristics of focal cortical dysplasia (FCD)⁷³, suboptimal processing of surface-based data leads to modest performance, even in cases with MRI visible lesions ⁷⁴. Consequently, current automated detection algorithms fail in up to 40% of patients ^67,68,71,75, particularly those with subtle FCD, and suffer from high false positive rates ⁷². Also, the limited set of features designed by human observes may not capture the full complexity of this brain malformation. Taking advantage of deep learning’s ability to learn independently abstract concepts from high-dimensional data, a recent study reached a sensitivity of 93% to detect FCD, while maintaining high specificity both in healthy and disease controls ⁷⁶. Importantly, the algorithm based on convolutional neural networks detected MRI-negative FCD with 85% sensitivity, thus offering a considerable gain over standard radiological assessment. This algorithm relied on T1-weighted and FLAIR images of patients with histologically confirmed FCD lesions collected across multiple epilepsy centres. Results were generalizable across age, hardware, and sequence parameters. A unique feature of the algorithm was Bayesian uncertainty estimation for risk stratification, which stratified predictions according to the confidence to be lesional. In other words, all lesional clusters were ranked based on confidence, thereby assisting the examiner to establish objectively the significance of findings.

Autism

Canvas Dx is an FDA market authorized⁷⁷ AI-based digital diagnosis aid, designed to support healthcare providers (HCPs) evaluating children for autism spectrum disorder (ASD) in primary care settings. Its use could shorten time to treatment initiation during the critical early years where interventions can produce greater benefits.⁷⁸ ML approaches were used in early product development to derive maximally predictive ASD behavioral features from the historical data of thousands of children with diverse conditions, presentations and comorbidities.^79-84 The two top performing classifiers were independently validated against novel medical record score data,^81,84 with subsequent prospective validation studies of the evolving device and algorithms.^82,85-88 Canvas Dx uses digital technologies to collect inputs from a caregiver, healthcare provider and trained video analysts, consistent with best practice recommendations for ASD evaluations.⁸⁹ Caregivers use a smartphone application to answer a brief questionnaire, and upload home videos. Trained analysts review the videos and the clinician completes their own questionnaires. The three inputs are combined into a mathematical vector for ML analysis and classification after which the device renders a decision of ASD positive or negative or abstains from a determination due to insufficient information. The device developers highlight the importance of abstention as a safety mechanism,^90,91 when developing AI for complex clinical tasks such as ASD diagnosis where degrees of uncertainty are anticipated, and additional assessment may be required.⁹² The Canvas Dx output is used by providers as an adjunct to their clinical judgment and criteria from the Diagnostic and Statistical Manual of Mental Disorders, fifth edition in diagnosis.

The Canvas Dx prospective pivotal validation study,⁹² was a double blinded comparator clinical trial conducted at 14 sites in 2019-2020 which included 425 study completers aged 18-72 months presenting with concern for developmental delay. The diagnostic output of the device was compared to the blinded diagnostic determinations of two or more experienced specialists. The device produced a determinate result (ASD positive or negative) for roughly one third of participants in the study, and abstained from diagnosing the remaining participants. For the participants who received a determinate (ASD positive or negative) result, device accuracy strongly aligned with clinicians (positive predictive value - 80.8% [95%CI, 70.3%-88.8%]; negative predictive value - 98.3% [90.6%-100%]; sensitivity 98.4% [91.6%-100%]; specificity 78.9% [67.6%-87.7%]). Promisingly, the study did not find evidence of performance inconsistencies across gender, race/ethnicity or socio-economic status. Follow-up studies would be needed, however, to confirm this finding.

Diagnosis using Natural Language Processing

Natural language processing (NLP) analyzes text and speech to infer meaning and can be used for the diagnosis, natural history and outcomes.⁹³ Unstructured data collected in the electronic health record (EHR) can be leveraged for structured data collection. The challenge is to define through domains the possible meaning of a given word, its synonyms, and its meaning in different situations. Despite these difficulties, we will detail two examples, the first addressing the challenge of seizure count and seizure cessation⁹⁴ and the second on the diagnosis of a rare epileptic and developmental encephalopathy, Dravet syndrome.⁹⁵

In the first example, a novel NLP pipeline automatically extracted seizure freedom and seizure frequency from clinical reports with validation against human annotations.⁹⁴ For individuals with epilepsy, the most important clinical markers used for clinical decisions are the presence or cessation of seizures and the frequency of seizures at the given time. Almost 79,000 reports of patients with epilepsy were used to identify the paragraphs of interest and to apply the algorithm through a number of questions. In parallel, a group of experts worked on manual annotations. The pipeline was created by using a combination of pretrained models with a relatively small number of additional steps to provide the requested information. This pipeline could be utilized to address other research questions in epilepsy, including frequency of seizures in a particular epilepsy syndrome or treatment response.

The second example is about the early diagnosis of a rare developmental and epileptic encephalopathy, Dravet syndrome.⁹⁵ The diagnosis of Dravet syndrome is often delayed as seizures at onset are febrile and patient can be misdiagnosed as febrile seizures (FS).⁹⁶ However, first seizures’ characteristics can be often identified for DS by an expert pediatric neurologist as seizures are long lasting, mainly hemiclonic (focal involving one hemisphere) and occurring after low grade fever or vaccine. This study aimed to explore blindly through NLP if these characteristics (concepts) are significantly more used in the EHR of individuals with DS before the age of 2 years compared to individuals with FS. These concepts, often reported by emergency room physicians for the first episodes, would allow an earlier diagnosis of DS and referral to expert centers that can confirm diagnosis and provide targeted care. In addition, those terms can be also automatically detected by software exploiting NLP helping non-expert physicians. This model can be applied to other rare epilepsies.

Neonatal Neurology

Leave-one-out cross validation (LOOCV) as discussed earlier, was used to evaluate a model using MRI features to predict language impairments in very-low-birth-weight neonates ≤ 1500 g and ≤ 32 weeks of gestational age.⁹⁷ MRI features included structural features (white matter abnormalities, and cerebellar asymmetry) and diffusion tensor imaging (DTI), and using the Bayley Scales, correlated with language outcomes at 18-22 months adjusted age. The number of participants was defined as N. The model was developed through supervised learning multivariate logistic regression models, with exhaustive search in the feature space. After selecting the best models, the models were evaluated using LOOCV with N-1 datapoints, and testing on the single participant that was left out. N-1 models were created with optimizing sensitivity and specificity. Cerebellar asymmetry was associated with lower receptive language subscores. Moreover, white matter microstructure on DTI could predict composite, expression, and receptive language outcomes.⁹⁷

Section V: Ethics of ML/AI

The ethics of ML/AI are important to address and a full discourse would need to be discussed in multiple manuscripts and discussions. For example, an incidental finding could be identified by the algorithm that was not the original intent. The AI/ML algorithm could be programmed to provide predictions about multiple potential diagnoses and not focus on a single diagnosis to reduce harm or misdiagnoses. Multiple ethical issues arise from using ML/AI in medical applications, and should be considered as AI/ML can improve diagnostic accuracy and efficiency, but should also not harm. Moreover, ethical issues can involve different levels including individual, interpersonal, group, institutional, and societal. For further discussion on ethical issues, please see this review.⁹⁸

Section VI: Conclusion

As the field of ML/AI advances with increased conversions to electronic medical record systems and increasing big data, additional applications of ML/AI will be used in medicine and research. Yet, there are many challenges still for AI (and ML). For example, ANN-based systems currently make decisions using neurons whose state and decision making is opaque and biased rather than human-interpretable.⁴² Moreover, generating meaningful language, images, music and behavior is still very hard. Further, most AI systems lack causal reasoning⁹⁹ (as opposed to reasoning based upon correlation) or common sense. ¹⁰⁰ Today, AI researchers generally think of “AI” as “specific” – a set of computational techniques applied for a specific problem. This connotation is in contrast to “general AI” or “Artificial General Intelligence” (AGI), which is the attempt to truly replicate a computational form of animal intelligence. As AI/ML advances, they will be increasingly utilized in research and in child neurology.