Artificial intelligence and machine learning in nephropathology

Abstract

Artificial intelligence (AI) for the purpose of this review is an umbrella term for technologies emulating a nephropathologist’s ability to extract information on diagnosis, prognosis, and therapy responsiveness from native or transplant kidney biopsies. Although AI can be used to analyze a wide variety of biopsy-related data, this review focuses on whole slide images traditionally used in nephropathology. AI applications in nephropathology have recently become available through several advancing technologies, including (i) widespread introduction of glass slide scanners, (ii) data servers in pathology departments worldwide, and (iii) through greatly improved computer hardware to enable AI training. In this review, we explain how AI can enhance the reproducibility of nephropathology results for certain parameters in the context of precision medicine using advanced architectures, such as convolutional neural networks, that are currently the state of the art in machine learning software for this task. Because AI applications in nephropathology are still in their infancy, we show the power and potential of AI applications mostly in the example of oncopathology. Moreover, we discuss the technological obstacles as well as the current stakeholder and regulatory concerns about developing AI applications in nephropathology from the perspective of nephropathologists and the wider nephrology community. We expect the gradual introduction of these technologies into routine diagnostics and research for selective tasks, suggesting that this technology will enhance the performance of nephropathologists rather than making them redundant.

Artificial intelligence (AI) has become an umbrella term for technologies emulating human “natural intelligence” or cognitive functions. In the context of nephropathology, we are interested in a machine’s ability to function as a nephropathologist by extracting clinically useful information from histological images of conventional and ancillary stains, transmission electron microscopy, and omics data. We can therefore quantify AI performance on the basis of its ability to inform about diagnosis, prognosis, and potential therapy responsiveness. This narrow definition excludes other AI-related research focusing on emotional or social intelligence, which, although certainly appreciated by a nephropathologist, is irrelevant to this review. We focus on AI for the primary tissue analysis. Although AI also plays a successful role in integrating information from downstream histopathological reports,¹ our focus is on the precursor analysis of raw histological images, more recently in the form of whole slide images (WSIs), that are the cornerstone of diagnostic and experimental nephropathology.

Computers have been used in histopathology for decades, usually for repetitive and cumbersome tasks and to enhance reproducibility. Compared to modern image analysis, these first applications were simple and offered pixel-based planimetry solutions for the quantification of fibrosis (e.g., Sirius red stains^2–4), capillary rarefaction,⁵ or immunohistological stains (e.g., C4d^6,7) by using public domain software packages such as Fiji,⁸ ImageJ (both National Institutes of Health, Bethesda, MD),⁹ or other commercially available software.

Since these early days, the transformation of the histopathology work environment from desk microscopes to the “virtual” slides of WSIs and dramatic improvements in hardware and software have enabled complex AI analyses of WSIs (~300–500 MB per WSI/slide for a kidney biopsy core scanned with a 40 objective), both for native and transplant biopsies, both online and off-line.¹⁰ The application of AI and machine learning, a subfield of AI, in nephropathology is within reach now.

The case for machine learning in nephropathology and precision medicine

Precision medicine shows promise for major improvements in native^11,12 and transplant nephrology.¹³ The foundation of precision medicine is the accumulation, integration, and analysis of massive, comprehensive, and precise data. As shown in Figure 1, a kidney biopsy WSI contains a wealth of information from conventional staining, single or multichannel immunohistological stains, and omics data. Over the past 100 years, nephropathology has become a powerful diagnostic tool integrating these data into a structured report, indispensable for the management of most native and transplant kidney diseases. However, even in a standardized report, many of these data are filtered and are not available for assessment in precision medicine. Successful examples of integrating nephropathology into big data–driven tools are the risk score for IgA glomerulonephritis¹⁴ using histopathological parameters from the Oxford Classification^15–17 and the iBox risk score for transplant kidney survival (http://www.paristransplantgroup.org/; accessed July 3, 2019) using Banff Lesion Scores.^18,19 Reproducibility of nephropathology parameters is crucial for both the development and clinical application of these tools. In fact, reproducibility of the relevant histopathological parameters is good among expert nephropathologists for the Oxford Classification¹⁶ (Table 1).^{16,18,20–30}

Figure 1 |. Nephropathology in the context of other disciplines contributing to big data in precision medicine.

Artificial intelligence can be used in all steps of tissue analysis and during the integration and analysis of data from other disciplines for the big data–based determination of diagnosis, prognosis, and treatment of an individual patient. IF, immunofluorescence; LM, light microscopy; Multi-IF, multichannel immunofluorescence; TEM, transmission electron microscopy.

Table 1 |.

Interobserver variation in nephropathological scoring in native and transplant kidneys

Disease	Parameter	Reproducibility	Reference
SLE	GN class	0.19–0.60	(20–25)
	SLE GN activity index	0.39–0.72	(21–25)
	Endocapillary hypercellularity	0.10–0.64	(20, 21, 23, 24)
	Leukocyte infiltration	0.28–0.32	(20, 21, 23)
	Subendothelial hyaline deposits	0.39–0.53	(21, 23)
	Fibrinoid necrosis/karyorrhexis	0.27–0.49	(21, 23, 24)
	SLE GN chronicity index	0.38–0.69	(21–24)
	Cellular crescents	0.51–0.55	(21, 23)
	Glomerulosclerosis	0.37–0.40	(21, 23)
	Fibrous crescents	0.23–0.59	(21, 23)
	Interstitial fibrosis	0.20–0.43	(21, 23, 24)
	Tubular atrophy	0.08–0.51	(21, 23, 24)
IgAN (Oxford Classification)	M for mesangioproliferation	Interexpert: ICC > 0.54 Expert vs. nonexpert: κ = 0.28	Interexpert (16); expert vs. nonexpert (26)
	E for endocapillary hypercellularity	Interexpert: ICC > 0.57 Expert vs. nonexpert: κ = 0.19
	S for segmental glomerulosclerosis	Interexpert: ICC > 0.46 Expert vs. nonexpert: κ = 0.51
	T for tubular atrophy and interstitial fibrosis	Interexpert: ICC > 0.78 Expert vs. nonexpert: κ = 0.53
	C for active crescents	Interexpert: ICC > 0.64 Expert vs. nonexpert: κ = 0.24
Transplant rejection (Banff Classification)	i for cortical tubulointerstitial infiltrates	κ = 0.320	(27)
	t for tubulitis	Since the most recent redefinition with the 2017 Banff update18 the reproducibility has never been re-examined.
	v for endarteritis	κ = 0.327	(27)
	g for glomerulitis	κ = 0.39	(28)
	ptc for peritubular capillaritis	κ = 0.38
	ci for interstitial fibrosis	κ = 0.306	(27)
	ct for tubular atrophy	κ = 0.314
	cv for arterial intimal fibrosis	κ = 0.360
	cg for glomerular basement membrane splitting	κ = 0.48	(28)

GN, glomerulonephritis; ICC, intraclass correlations; IgAN, IgA nephropathy; SLE, systemic lupus erythematosus.

Although nephropathology is and will remain the criterion standard for diagnosing most native and transplant kidney diseases, certain histopathological parameters required for prognostication and therapy planning have unsatisfactory reproducibility, particularly among nonexperts. These shortcomings could be improved by fully transparent artificial intelligence solutions in the hands of nephropathologists. By convention, ICC < 0.40 indicates poor interrater reliability, ICC =0.40–0.59 is moderate, ICC =0.60–0.79 is substantial, and ICC > 0.80 is outstanding.^29,30

This illustrates the value and potential of traditional light microscopy in the hands of experts who developed the scores.²⁶ However, nonexperts struggle to apply the Oxford Classification with the same accuracy.²⁶ Matters become even more complicated for lupus nephritis (LN), the leading cause of morbidity and mortality in patients with systemic lupus erythematosus.³¹ Nephropathology is still the criterion standard for diagnosing LN and the best predictor of end-stage kidney disease.^32–34 Nephropathologists are expected to assign a class from I to VI for lupus glomerulonephritis on the basis of light microscopy on a battery of stainings, immunohistology, and transmission electron microscopy results according to the recently revised International Society of Nephrology/Renal Pathology Society classification including activity and chronicity indices.³⁵ These 3 parameters, class, activity index, and chronicity index, inform prognosis and treatment decisions.^36–38 Because LN has a much wider histopathological spectrum than does IgA nephropathy, it is not surprising that reproducibility of these key parameters is suboptimal. The interpathologist reproducibility for class I to VI is between 0.19 and 0.60, with a median of 0.33 (Table 1), qualifying as poor to moderate.^29,30 Likewise, the interpathologist reproducibility for the activity and chronicity indices ranges from ~ 0.4 to 0.7.^{20–22,24,25} The situation is similar in transplant nephropathology. Here, the relevant nephropathological parameters are defined in the predominant³⁹ Banff Classification,^18,19 which are considered in the iBox prognostic tool.⁴⁰ Reproducibility of the Banff Lesion Scores g, ptc, i, t, cg, ci, and ct relevant for the iBox ranges from 0.306 to 0.48^27,28 (Table 1). Even the reproducibility of the prognostically most important diagnosis of antibody-mediated rejection in the class 2 diagnosis of the Banff Classification was considered unsatisfactory by highly trained nephropathologists from eminent centers.²⁸ These examples from native and transplant nephropathology illustrate the urgent need to improve reproducibility to supply quality histopathological data beyond the standardized report for these already existing and future tools that are expected to be extremely useful for diagnostic classifications, prognostication, and prediction of therapy responsiveness in precision medicine. Although educative efforts in nephropathology might be helpful in the interim, technological AI-based solutions could provide a long-term solution in the hand of nephropathologists. By definition, they deliver perfectly reproducible results, fit for precision medicine. In turn, nephropathology will continuously improve through analysis of increasingly comprehensive databases containing laboratory, serological, imaging, genetic, treatment, and follow-up data. These tools can be tuned, reflecting the progress of precision medicine and harnessing it for the advancement of nephropathology (Figure 1).

In addition to extracting information from routine WSIs, AI applications can be useful for other tasks. For example, AI and machine learning may be better suited for the precise and exhaustive extraction of the multidimensional information from multichannel immunofluorescence images or integration of omics (mRNA, microRNA, protein, and others) data from biopsy tissue.

Machine learning in image analysis and histopathology

Many nephropathological features or descriptors integrate subtle variations in local color and texture that are difficult to quantify directly. Attempts have been made to create robust features using texture filters,⁴¹ gray-level cooccurrence matrices,⁴² speeded-up robust features,⁴³ and many other approaches. However, these features are context independent and may be suboptimal for any given classification problem.

Machine learning as a subfield of AI gives computers the ability to learn without being explicitly programmed by adjusting themselves in response to the data they are exposed to. It is based on artificial neural networks (ANNs), which are inspired by, but not identical to, biological neural networks of the brain. ANNs learn from large data sets by automatically extracting features that they find useful for a particular task or classifier (Figure 2). An ANN will train itself for a classification task, identifying relevant features. Training can be supervised (e.g., replicating a manual annotation of podocyte nucleus vs. endothelial nucleus in a glomerular tuft), semi-supervised (e.g., WSI with rejection vs. WSI without rejection), or unsupervised. Interestingly, at the moment, these decisive features acquired through self-training by an ANN cannot be extracted and remain a “black box.” This current shortcoming explains the rare but sometimes blatant, seemingly obvious classification errors by ANNs. However, we are able to visualize the decisive features for a classification task in a given image (see Figure 3) with software such as Grad-CAM⁴⁴ or Lime,⁴⁵ providing the transparency necessary for building trust, which is essential particularly for clinical applications.

Figure 2 |. Development of machine learning algorithms and their architecture and functionality.

(a) The first neural networks used in image analysis were the simple perceptrons with a single input layer connected to a binary output. (b) Radial basis forward networks represent an intermediate stage with a hidden layer in between the input and output. (c) The current state of the art in the analysis of whole slide images is the convolutional neural network (CNN) that links the pixel input to the classification output via several interconnected hidden layers as shown in this fictional example for the classification of severity of T cell–mediated rejection (TCMR), antibody-mediated rejection (AMR), and scar in a rodent kidney allograft. Note that the hidden layers in such CNNs currently remain a black box. However, the trust-building mandatory transparency can be achieved with additional software, as shown in Figure 3. To optimize viewing of this image, please see the online version of this article at www.kidney-international.org.

Figure 3 |. Example of a preliminary convolutional neural network (CNN) solution for a classification task in transplant nephropathology.

The CNN (schematically depicted in the top right) was trained on 279 images from glomerular transections on periodic acid–Schiff (PAS)–stained slides from 6 transplant biopsies with antibody-mediated rejection (AMR) and 6 biopsies without AMR (no AMR). Data augmentation on this relatively small sample size was achieved with rotation. With this semi-supervised training on this augmented data set, the classification accuracy for individual glomeruli reached 91.3%. Interestingly, data augmentation through cropping resulted in lower accuracy. Grad-CAM was used for the superimposition of a heat map highlighting the decisive areas for classification. A trained human nephropathologist would look for the features of glomerulitis, glomerular basement membrane splitting, and perhaps also secondary focal and segmental glomerulosclerosis for this classification task. The misclassification example shows the advantage of the superimposed heat maps. Obviously, the decisive areas (superimposed as a polygon on the PAS image) are largely outside the glomerular tuft, alerting the pathologist of a possible misclassification. In contrast, the heat map in the example with the correct classification highlights tuft segments with glomerulitis (arrows) indicative of AMR. To optimize viewing of this image, please see the online version of this article at www.kidneyinternational.org.

State-of-the-art machine learning approaches focus on supervised methods, that is, methods that are dependent on training with particularly large data sets labeled by nephropathologists. When trained properly and with reliable data, these methods yield high accuracy for the respective classification. However, the annotation of large numbers of WSIs often requires significant time and effort by nephropathologists. A solution to this problem is offered by semi-supervised or unsupervised methods, with the latter not requiring any labeling at all. Although it is more challenging and less reliable to train networks without supervision, the reduced costs and time required for training might in some instances outweigh the reduced accuracy.

The output of a convolutional neural network (CNN) is typically a likelihood and thus a continuous parameter. Although nephropathology reports typically consist of discontinuous parameters such as diagnosis, grade, and stage, we do not think this discrepancy will be problematic. Nephropathologists are used to constantly filter visual input into output parameters such as nodes on the diagnostic decision tree and nominal or ordinal histopathological classifiers. This is done in full knowledge of the subjectivity of these decisions and the inevitable loss of information. Continuous AI output can still be simplified, alone or in combination, into nominal scales for, for example, diagnostic classes or ordinal scales for, for example, activity or chronicity. This categorization should be informed by meaningful big data analysis under consideration of the usual statistical performance parameters.

As a still recent and developing field, a number of limitations exist in digital pathology. Of particular concern is the lack of a common acquisition protocol for histological images, which results in wide inter- and intralaboratory variabilities in slide appearances, such as section thickness, stains from different manufacturers, and scan quality.

Despite these limitations, digital pathology and automating tasks on a large-scale promise huge benefits and will likely transform diagnostic and experimental nephropathology in the coming years. Pathomics, a recently introduced concept,⁴⁶ encompasses this idea of extracting subvisual patterns using feature analysis methods with the aim of characterizing morphological and behavioral patterns specific for different pathologies.

Machine learning architectures

A wide range of ANN architectures is available to automate classification tasks on the basis of a set of input features. These features can be quantitative measurements such as glomerular numbers and diameter, presence and density of infiltrates, or the strength of various immunohistochemical signals. They could also be segmentation tasks such as identifying glomeruli, arteries, cortex, and medulla. Commercial applications have been regularly used to perform such classifications on the basis of quantitative features.⁴⁷

The current state of the art in feature extraction is the CNN, a subtype of ANN.⁴⁸ Note that ANN is an umbrella term for these artificial networks with multiple layers whereas CNNs refers to a special architecture of the ANN defined by convolutional operations to increase the efficiency in image analysis. A CNN is a deep learning approach that integrates optimization of a set of feature extraction filters with the associated classification problem. Figure 2 shows the architecture of a CNN in comparison to much more simple predecessors. A CNN produces a bank of feature extraction filters that are ideally optimized to provide feature values tailored toward the desired nephropathological problem. CNN architectures are still an active area of research and have shown state-of-the-art performance for computer vision tasks.

CNNs are the most popular ANNs for the analysis of WSIs. This section discusses popular deep learning architectures that can reliably perform these tasks.

For the detection of important image structures, Faster R-CNN⁴⁹ is a popular architecture choice. This architecture consists of 2 networks: (i) a region proposal network for finding potential regions (each represented by a rectangular bounding box) containing the structures of interest and (ii) a refinement network to classify whether important structures are present in these proposed regions. Another popular choice is a YOLO (an acronym for “you only look once”) detector,⁵⁰ which splits the input images into a grid of multiple regions, passes them once through a CNN, and predicts class probabilities and bounding boxes for each region. In contrast to Faster R-CNN, which is a 2-stage detector, YOLO is a 1-stage detector.

Two-stage detectors commonly achieve higher detection accuracy, whereas 1-stage detectors are significantly more computationally efficient. For an overview, we refer the reader to a recent comprehensive review of recent object detection algorithms.⁵¹ Training of these detectors requires input data sets consisting of images (or WSIs) and rectangular bounding boxes around important structures within images (see Figure 3).

In addition to detecting structures of interest, recent networks also perform semantic segmentation to remove background from the detected structures. Specifically, the goal is to predict pixel-wise labels (background or foreground) using another network. Common choices of semantic segmentation networks include fully convolutional networks,⁵² U-Net,⁵³ and V-Net.⁵⁴ Training semantic segmentation networks requires a data set consisting of images and pixel-wise labels (also known as label masks). Recent architectures such as Mask R-CNN⁵⁵ and Mask Scoring R-CNN jointly train image detection and semantic (pixel-based labeling of, e.g., podocyte nucleus vs. cytoplasm) segmentation. This setting is commonly known as instance-based segmentation. The training of instance-based segmentation algorithms requires input images (typically WSIs), bounding boxes, and pixel-wise labels.

After detection and semantic segmentation, one can crop specific regions in images and categorize them into different disease types (e.g., IgA nephropathy vs. membranous nephropathy) or histopathological features (e.g., tubulitis and endarteritis). State-of-the-art architectures for this image classification task include ResNet,⁵⁶ DenseNet,⁵⁷ and Capsule Networks.^58,59 Recent work has shown that high-performing architectures can be discovered through automatic hyperparameter optimization techniques. Hyperparameters are used to steer the training algorithm. Examples of such automatically discovered effective networks include AmoebaNet,⁶⁰ which achieves top performance image classification, and MobileNet⁶¹ for fastest speed. Of course, one can also train the classification task with detection and semantic segmentation in parallel.

In our experience, classifying disease types using entire WSIs without first detecting important image structures is ineffective because of the high dimensionality of images and typically a small number of training examples (compared with hundreds of thousands or millions of images typically used for training deep networks).

Prerequisites for machine learning in pathology

The prerequisite for any machine learning approach in nephropathology is the availability of digital images or WSIs. The capability of generating WSIs in a high-throughput manner stimulated the emerging field of digital pathology with the fast development of CNN approaches we are currently experiencing.

Hardware requirements for the training of a CNN for image classification typically are a modern multicore computer with decent graphics processing units. Examples of such graphics processing units are Tesla V100, Titan V, Titan X, and GTX 1080 (NVIDIA, Santa Clara, CA). Graphics processing units significantly speed up the training through distributed computing, which could lead to several orders of magnitude faster training than with traditional central processing units. In a clinical or research setting, such equipment will render classifications for a WSI with acceptable computational delays. They could be run, for example, during the lag time between the finalization of the WSI and the review by a nephropathologist.

Machine learning in neoplastic cytopathology and histopathology

Most research effort in histopathology is devoted to neoplasia. Consequently, machine learning applications are most advanced in tumor pathology. In the next section, we give a short overview of the achievements in this field, illustrating the potential of AI in histopathology in general and its promise for nephropathology.

The first applications of machine learning were in cytopathology. The analysis of solitary cells or tissue fragments instead of whole slides was well suited for the less powerful computing methods of the past. Methods reducing the inherent subjectivity of cytopathological examination by introducing automated objective measurements have been developed since the 1950s and recently included ANNs (reviewed in Pouliakis et al.⁶²). Applications in histopathology on WSIs have been developed with CNNs for the detection of lymph node metastases. Under time constraints common in routine diagnostics, CNNs performed better than histopathologists, whereas without time constraints the performance was similar.⁶² A symbol-based machine learning approach performed well in classifying micrographs from follicular lymphomas.⁶³ Similarly, CNN-based approaches have shown promise in recognition of breast carcinoma,⁶⁴ adenocarcinoma infiltrates in prostate biopsies, breast cancer metastasis in sentinel lymph nodes,⁶⁵ and non–small cell lung carcinoma on WSIs.⁶⁶ Because of the recent success of immunotherapy in oncology, researchers have also developed deep learning models to label tumor-infiltrating mononuclear cells on hematoxylin and eosin (HE)–stained sections. For this, the training set consisted of virtual slides of testicular tumors in which a researcher marked mononuclear cells manually under the supervision of a pathologist. The algorithm was trained using 812 marked mononuclear cells and was able to count these mononuclear infiltrating cells with an accuracy of 0.89 compared with a pathologist assessment in the test set.⁶⁷ Similar deep learning approaches seem capable of assessing mononuclear infiltrates in diagnosing T cell–mediated rejection and microvascular inflammation relevant for antibody-mediated rejection.¹⁸ In native kidney biopsies, the scoring of glomerulocapillary leukocytes as in the Oxford Classification’s E parameter would benefit from such tools. This E parameter in particular, although prognostically important in the hands of reference pathologists,¹⁵ has shown poor reproducibility when scored by less experienced pathologists.²⁶ Immunohistochemical staining for the macrophage marker CD68 has been reported to be helpful, providing better reproducibility for surrogate markers for E.⁶⁸ Still, a deep learning algorithm developed on a training set with expert annotation providing E scores with perfect reproducibility and no need for additional immunostains would be highly desirable. The same argument could be made for the activity parameters endocapillary hypercellularity and interstitial inflammation in the revised National Institutes of Health activity grading of LN and, of course, also for the other parameters that are not based on leukocyte margination or infiltration.³⁵

Even underlying biological mechanisms can be unraveled with machine learning models. A CNN was able to classify tumors according to their underlying biology, such as microsatellite instability in gastrointestinal carcinomas, based on their appearance on HE-stained WSIs.⁶⁹ Usually, this would require ancillary staining or molecular pathology tests. Interestingly, albeit trained in 1 type of cancer, it was able to predict the molecular alterations in different types of cancer as well. In a similar approach, deep learning was used to predict virus-induced carcinogenesis with reasonable performance.⁷⁰ In another study, an attempt to visualize how the neural network perceives the typical feature of virus- versus non–virus-induced cancer showed clearly distinct forms, which human pathologists were not able to translate into reproducible features. Similarly, CNNs have shown good performance when classifying breast cancer into luminal A and B subtypes, which are related to distinct molecular subtypes.⁷¹ Moreover, estrogen receptor positivity, a critical marker for therapy response, could be predicted with 75% certainty on the basis of HE morphology.⁷² These striking examples illustrate 4 points: First, pathology and pathogenesis reveal themselves in histomorphology. Second, traditionally derived histopathological parameters can be reproducibly classified by machine learning models. Third, machine learning models can outperform histopathologists in some of these highly specialized tasks to detect these features. Fourth, in the hand of pathologists, these machine learning models can complement, simplify, and enhance the traditional histopathological assessment.

Machine learning in nephropathology

In comparison to oncopathology, the development of machine learning application in nontumor pathology is still in its infancy. Published data on machine learning analysis in non-neoplastic histopathology are relatively rare and devoted to find solutions to common and sometimes cumbersome tasks such as finding Helicobacter pylori in gastric biopsies.⁷³

Machine learning and ANNs have been used in nephrology for many years and have been used, for example, to personalize anemia management in end-stage kidney disease,⁷⁴ improve estimation of the glomerular filtration rate,⁷⁵ refine tacrolimus dosing in kidney transplantation,⁷⁶ and facilitate the early diagnosis of acute kidney injury⁷⁷ or chronic kidney disease.⁷⁸

Applications in nephropathology have been published for automated recognition (segmentation) of open or sclerosed glomerular transections in tissue sections in rodent and human kidneys, both after paraffin embedding and on frozen sections.^79–83 A recent publication has shown that ANNs are capable not only of glomerular segmentation but also of classifying diabetic nephropathy with sufficient precision.⁸⁴ Similar pipelines have been created that enable the rapid instruction of a CNN by nephropathologists for tasks such as the segmentation of glomeruli, glomerular cell types, and cortical tubular atrophy and interstitial fibrosis on mouse and human biopsy tissue.⁸⁵ Similarly, 2 groups have been able to train a CNN for the segmentation of all relevant tissue compartments of transplant kidney biopsies⁸⁶ and for the differentiation of glomerular components in diabetic nephropathy.⁸⁴ Recently, CNNs have shown better performance in fibrosis staging of native kidney biopsies than pathologists.⁸⁷ Figure 3 shows preliminary results from the training of a CNN to distinguish between photographs of glomeruli from a transplant biopsy with and without antibody-mediated rejection. Additional training data will clearly improve the already remarkable good accuracy of 91.3% (vs. 0.5 for a coin flip for this binary classification), considering that only a fraction of the glomeruli show lesions considered as indicative of antibody-mediated rejection by a human nephropathologist. It will be interesting for nephropathologists to review the decisive image hotspots from this semi-supervised training and compare them to traditional diagnostic features.

Obstacles: integration, standardization, data availability, and annotation

Diagnostic nephropathology has a unique position in anatomic pathology, because it is based on the integration of a battery of conventional paraffin stains, immunostains, and transmission electron microscopy. Considering the relatively high number of routine paraffin stainings required for nephropathology (at least HE, periodic acid–Schiff, Jones, and trichrome-elastica), staining variability poses another serious problem for machine learning algorithms in our field and is a well-known, sometimes irritating, problem even for the human nephropathologist.¹⁰ A solution to this problem is offered by color normalization techniques, whereby the color distribution of the target and the source images is matched in different ways. This commonly works only under conditions where the distribution of objects in both images is roughly the same, such as in consecutive level sections. Several technologies have successfully addressed this problem for the task of glomerular segmentation^88,89 without resorting to large databases with different staining methods. Another limitation is the current lack of standard data sets as well as data sets including all possible variations relevant for a certain classifier. Ultimately, the effectiveness of any CNN will depend largely on data available for training. Advances in transfer learning,⁹⁰ enabling knowledge transfer from, for example, more abundant animal tissue of a given disease to human biopsy tissue, could provide a solution to this problem. Transfer learning allows the transfer of features learned from images to more specific networks typically requiring dramatically lower numbers of biopsies for the transfer. Basic autoencoders trained to identify, for example, glomeruli on HE or periodic acid–Schiff-stained WSIs can be used as a starting point and re-trained with relatively little effort for other or more specific classifiers, such as the distinction between entities of glomerulonephritis. The requirement of these large annotated (e.g., mesangioproliferation, focal and segmental glomerulosclerosis, and tubular atrophy) data sets can be overcome with a different approach that has recently been described; this approach is not based on the trained detection of these features per se, but rather by flagging deviation from normal kidney histology,⁹¹ examples of which are usually abundantly present in urological nephrectomy specimens.

The major bottleneck for the creation of suitable training sets is the time-consuming manual annotation by a highly trained nephropathologist. For specific tasks, this bottleneck could be avoided through nonexpert annotation, such as the Amazon Mechanical Turk,⁹² provided that basic features can be explained effectively through an example. Of course, the usual caveat in research about the quality of input data determining the quality of the results applies and should inform the project strategy. In most cases, high-level annotations will likely require annotation by trained—or even panels of trained—nephropathologists. Platforms such as Labelbox (www.labelbox.com, last accessed May 2020) are useful to distribute such tasks among pathologists in a comfortable way.

The generation of extremely large data sets, matching a recent proposal of >40,000 histological cancer samples for oncopathology,⁹³ will require a joint international effort from the nephrology and nephropathology community, merging the current efforts from several institutions and consortia. For example, the Banff community has devoted a Working Group on Digital Pathology chaired by Kim Solez and Alton “Brad” Farris to the application of AI in transplant pathology.

Outlook and summary

Although we are clearly in the early stages of AI application in nephropathology, the potential demonstrated above in the field of oncopathology provides a promising outlook for the next decade. Nephropathology is arguably more complicated than neoplastic histopathology, because it requires grading across a continuous spectrum of inflammatory and degenerative diseases with multifactor (genetic, environmental, degenerative, autoimmune, etc.) causes. At minimum, a nephropathologist must integrate multimodal tissue diagnostics, which include conventional light microscopy with special stains, immunohistology for Ig and complement split products, other special immunostains (e.g., DNAJB9 for fibrillary glomerulonephritis),⁹⁴ transmission electron microscopy, and, among others, serological, immunogenetical, and genetic data.^95,96 Each of these modalities poses challenges regarding standardization and reproducibility within and between laboratories, introducing additional barriers to AI integration that must be controlled. Of course, one cannot expect AI to instantly offer some miraculous full diagnostic solution, emulating a skilled nephropathologist. We rather expect a gradual introduction for selected tasks and diagnostic scenarios into routine practice. For example, AI-aided flagging of leukocytes could help in diagnosing rejection in transplants whereas flagging of focal and segmental glomerulosclerosis could help in the diagnostic workup of steroidresistant nephrotic syndrome. Even some simple tools for, for example, automated counting and presentation of glomeruli might already be helpful in daily practice.

We do not expect AI to make nephropathologists redundant in the foreseeable future. AI will rather transform the nephropathologist’s work profile, for example, offering solutions for repetitive work that requires high levels of attention. The nephropathologist will have to acquire skills to manage the information derived from AI applications during the diagnostic workup, which will require a new, different set of skills. Therefore, the claim that medical professionals supported by machine learning could lose skills⁹⁷ reflects only the negative part of the constant advancement of medicine. Indeed, we urge more optimism and point out that so far the gains through new technologies have been far greater than the loss of human skills they replaced. Understanding of the functions, benefits, and limitations of AI and its classifiers will become crucial for nephropathologists. Moreover, procedures and rules will have to be devised and implemented to identify uncertainty and override questionable AI outputs when deemed necessary by the human nephropathologist who will ultimately be held accountable. To this end, AI developers must ensure that the decision-making process remains transparent. AI applications should complement traditional nephropathology in the hand of nephropathologists, eliminating human flaws and weaknesses. They should not be expected to deliver a substitute for nephropathology or skilled nephropathologists.

Moreover, the regulatory environment must be considered. For example, diagnostic applications using virtual slides have to meet the European Union directive (https://ec.europa.eu/growth/single-market/european-standards/harmonisedstandards/iv-diagnostic-medical-devices_en; accessed October 14, 2019) for Communauté Européenne (CE) certification. In the United States, conventional diagnostic applications using WSIs are provided only with recommendations issued by the College of American Pathologists⁹⁸ and the Food and Drug Administration (http://www.fda.gov/downloads/MedicalDevices/DeviceRegulationandGuidance/GuidanceDocuments/UCM435355.pdf; accessed October 14, 2019). At the moment, any clinical application using AI in nephropathology can be considered for “research use only.” The necessary full Communauté Européenne (CE) certification requirements will require close collaboration between researchers, commercial partners, legislators, and regulators. Establishing these novel technologies in an ethical fashion can only be achieved by involving all stakeholders including physicians, patients, and institutional review boards.⁹⁹ This is particularly important for the continuous update and improvement of clinical AI applications for which issues of consent and data ownership must be settled with all stakeholders, including patients.⁹⁹ For example, the European Union’s General Data Protection Regulation emphasizes that patients should have the right not to be subjected to decisions based on automated processing (https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A32016R0679; accessed May 20, 2020).

We are skeptical about the promise that AI can alleviate the shortage of trained nephropathologists in underresourced countries. For the foreseeable future, AI applications in nephropathology are expected to perform only in an environment where preanalytical procedures are tightly controlled and where AI integration and supervision by skilled nephropathologists is ensured. These requirements will also prevent nonspecialists or even patients to apply these tools to their own biopsies. AI for diagnostic purposes in nephropathology should remain in the hands of nephropathologists.

In summary, AI is a promising technology that can be expected to enhance the histopathological examination of native and transplant kidney. It will be exciting to see this new technology develop: Nephropathologists should not fear but embrace and use these new technologies in the global fight against kidney disease.

DISCLOSURE

All the authors declared no competing interests.