The potential of artificial intelligence-based applications in kidney pathology

Abstract

Purpose of review:

The field of pathology is currently undergoing a significant transformation from traditional glass slides to a digital format dependent on whole slide imaging. Transitioning from glass to digital has opened the field to development and application of image analysis technology, commonly deep learning methods (artificial intelligence) to assist pathologists with tissue examination. Nephropathology is poised to leverage this technology to improve precision, accuracy, and efficiency in clinical practice.

Recent findings:

Through a multidisciplinary approach, nephropathologists and computer scientists have made significant recent advances in developing artificial intelligence technology to identify histological structures within whole slide images (segmentation), quantification of histologic structures, prediction of clinical outcomes, and classifying disease. Virtual staining of tissue and automation of electron microscopy imaging are emerging applications with particular significance for nephropathology.

Summary:

Artificial intelligence applied to image analysis in nephropathology has potential to transform the field by improving diagnostic accuracy and reproducibility, efficiency, and prognostic power. Reimbursement, demonstration of clinical utility, and seamless workflow integration are essential to widespread adoption.

Introduction

Nephropathology is a specialized field of pathology relying on manual quantifications, e.g., number of glomeruli or the extent of lesions, which have diagnostic, prognostic and therapeutic consequences. Although significant standardization efforts have been made by the nephropathology community, assessing and quantifying individual lesions still remains prone to errors and inter-observer variability [1]. Additionally, certain scores have only limited prognostic power and some categorical diagnostic classes do not correspond to the clinical spectrum and stage of disease.

Computer-assisted image analysis has long been employed in nephropathology. However, the recent breakthroughs in digital pathology have opened new transformative possibilities for the field. One major technological advance is whole slide scanning, enabling high throughput digitization of glass slides to whole slide images (WSI). The progress in graphics processing units and storage systems facilitate the use of powerful image analysis techniques based on artificial intelligence (AI), in particular machine (ML) and deep learning (DL). Given the vast amount of phenotypic data contained in gigapixel WSI, these techniques open possibilities of novel, clinically relevant readouts derived from histology, potentially providing more exact information for treatment response or prognosis, or enabling more granular diagnoses [2,3].

ML and DL can be supervised, semi-supervised, or unsupervised to perform several different tasks, including classification (categorizing an image), semantic segmentation (labeling each pixel in an image, thus outlining object boundaries), quantification, clustering (grouping data points) or image regression (predicting a continuous variable - such as eGFR (estimated glomerular filtration rate) - from an image). DL is a machine learning method utilizing multi-layer artificial neural networks. Neural networks gradually transform input data through multiple processing layers to map input data (e.g. a histology image) to the desired output, e.g., a diagnosis [4]. Training a neural network entails updating the layer weights using iterative optimization methods. Currently, convolutional neural networks (CNN) are the most commonly used neural network architecture in nephropathology. Novel networks and methodology are being developed at an increasing pace [2,5,6], and expectedly will be implemented in nephropathology in future.

Promising applications of DL in pathology have been demonstrated, particularly in oncological pathology. Using only HE-stained histology slides, DL was able to infer clinically targetable mutations [7*] or virus presence [8]. By performing semantic segmentation and handcrafted feature extraction, a new prognostic feature, i.e. the area fraction of metastatic lymph nodes containing tumor tissue in gastric cancer, was identified [9]. This showed that precise large-scale quantification of histology holds the potential to uncover new clinically and biologically relevant features. Given the above examples, the fact that some DL methods require little manual interaction, and the increasing amounts of available data and computing power, we can expect major improvements and even a transformation of (nephro)pathology in the future.

Applications

Recent applications in disease classification, segmentation and quantification, outcome prediction, and other areas are described below. Table 1 enumerates specific studies’ type of dataset, species, ground truth used for training, and primary readouts.

Table 1:

Selection of recent studies using deep learning in kidney histopathology.

Application	Study ref.	Dataset	Species	Multicenter	Ground Truth	MainReadout
Classification	[10]	293 WSI	Human	yes	Concordance classification of 5 pathologists and 1 physician	CNN based classification of five glomerular phenotypes.
Classification	[13*]	5844 WSI	Human	yes	Pathologist-derived diagnostic class	Pre-Screening (triage) of kidney allograft biopsies based on histology alone using CNNs.
Classification	[28*]	95 WSI	Human	yes	Consensus of 5 pathologists	Grading of fibrosis in kidney biopsies.
Classification	[12]	68 WSI	Human	no	None (unsupervised)	Unsupervised clustering based classification of glomerular features in IgAN-nephropathy.
Detection & Quantification	[22**]	122 WSI	Human, Mouse, Rat	yes	IHC-, IF-guided	Open cloud based tool for podocyte detection and quantification in PAS-stained WSI.
Segmentation & Quantitation	[19*]	149 WSI	Human	yes	Serial annotation by 3 expert pathologists	WSI quantitation of glomeruli and %global glomerulosclerosis in frozen sections
Segmentation	[15**]	459 WSI	Human	yes	30,048 Expert-based annotations	WSI segmentation in different stainings. Only study to show feasibility of peritubular capillary segmentation.
Segmentation	[21*]	110 WSI	Human	no	Expert-based annotations and IF-images	Automated “podometrics” that identify diseased glomeruli in ANCA-GN.
Segmentation	[14**]	157 WSI	Human	yes	Expert-based annotations	First multiclass WSI segmentation CNN for kidney histopathology
Segmentation	[36]	51 WSI	Human	yes	Expert-based annotations	Segmentation of glomeruli and quantification of globally sclerotic glomeruli
Segmentation & Outcome prediction	[27**]	789 WSI	Human	yes	Expert-based annotations, clinical/pathological metadata	Features extracted by the CNNs can be used to predict graft loss in kidney transplants
Segmentation	[16*]	165 WSI	Human, Mouse, Rat, German Landrace Pig, Common Marmosets, Black Bears	yes	72,722 Expert-based annotations	Precise WSI segmentation and reproducible quantification of histology in several major animal models of renal disease
Virtual Staining	[30]	58	Human	no	Histochemical staining	Equivalence between virtual and histochemical stains as assessed by 3 expert pathologists

WSI = Whole Slide Image, CNN = Convolutional Neural Network, IgA = Immunglobulin A, ANCA-GN = Anti-neutrophil cytoplasmic antibody associated glomerulonephritis.

Disease Classification

Classification entails assigning a category to a datapoint, e.g., “segmental mesangioproliferation” as a label to the image of a glomerulus. However, nephropathological assessment remains partly subjective with interobserver variability [10], indicating the need for internationally accepted definitions of lesions [11]. In a study performing glomerular image classification, five experts classified glomerular images in 12 categories to train a classification model. However, for most lesions expert agreement was poor. Only five lesions were scored with a kappa above 0.4 and were used for training [10]. The best performance was achieved for the feature of capillary collapse. However, these models fall short of the clinical reality, in which one glomerulus commonly displays several histological features at the same time, e.g., a cellular crescent and fibrinoid necrosis.

The need for manual labelling and thus human bias can be circumvented using unsupervised learning. A study aiming at glomerular image classification in IgA nephropathy used unsupervised learning to find clusters in features extracted by a model [12]. The 10 identified clusters were then used as ground truth labels for a separate classification model. The outputs of the final layer of the network were aggregated per patient and used as a score to correlate with clinical findings. While this is clearly an interesting approach and some clusters classified by the model correlated significantly with clinical findings, e.g. systolic blood pressure, the correlation was rather poor (highest R2-value is 0.229) and the clinical impact thus might be rather low.

Weakly supervised DL, i.e. using patient-level labels, was explored for pre-classification of kidney allograft biopsies. This study used broad classes, i.e. normal, rejection or other diseases, in a large multicentre-cohort including real-world data with high variability [13*]. The investigated models showed good generalizability for the classes normal and rejection. The other diseases class performed less well, perhaps unsurprisingly given that this class included all pathological non-rejection diseases, including polyomavirus nephropathy and interstitial nephritis which can easily be confused with rejection without additional information and analyses. These models could potentially serve as a triage-tool to prioritize biopsy assessment by a pathologist. Additionally, visualizing the model predictions, e.g., using highly predictive tiles, might provide guidance for pathologists.

Segmentation and Quantification

Several studies have shown DL-algorithms for kidney histology segmentation, e.g. automated recognition and outlining of relevant histological substructures such as glomeruli, tubuli, vessels, etc. Although interesting from a methodological point of view, segmentation alone is of limited use to researchers and clinicians. However, segmentation is the basis for compartment-specific analyses and quantification of kidney histology using image analysis techniques, e.g. hand-crafted feature extraction.

The first kidney histology multiclass segmentation model was published mid 2019, using transplant biopsies and differentiating tubuli, i.e. proximal and distal, and glomeruli, i.e. globally sclerotic and all other. The algorithm showed strong similarity between model output and ground truth (weighted mean Dice coefficient 0.80–0.84) and correlated well with visual scores reported by kidney transplant pathologists [14**].

Another approach explored an optimal resolution for each segmented histological structure in kidney biopsies of patients with Minimal Change Disease [15**]. This study is the first that also showed the feasibility of peritubular capillary segmentation, which required a very large number of training examples, i.e. 19,720 expert-annotations. This would potentially allow quantification of microvascular inflammation in antibody-mediated rejection or microvascular dysfunction in chronic kidney diseases.

Preclinical studies in animal models of renal disease play an important role in investigating disease mechanisms and testing of potential novel treatments. One study developed DL-based multiclass segmentation for murine models of renal disease, but also in healthy kidneys from humans, rats, pigs, apes and bears [16*]. This allows a translational approach, using the same algorithm for preclinical and clinical research. Accuracy of segmentation was measured per instance, e.g., for each tubular cross section and not just the class “tubule” as a whole. Instance-based quantification enables capturing of distributions, e.g., the distribution of tubular diameters in a biopsy. As a proof of concept, a large-scale analysis of areas occupied by glomeruli, tubuli, vessels and interstitium, and also of tubular diameters allowing the analysis of tubular dilation was performed.

Another study showed that CNN-based quantification of interstitial fibrosis, tubular atrophy, and glomerulosclerosis in Periodic acid-Schiff (PAS) stained samples correlated with patient outcome variables at levels comparable to renal pathologists [17]. CNNs have also been applied to renal ultrasound images for automated grading of interstitial fibrosis and tubular atrophy, enabling noninvasive disease assessment [18].

Preoperative kidney transplant assessment requires rapid and accurate biopsy evaluation in time-constrained clinical environments, often with artifact-laden frozen slide preparations. One study developed a CNN which exhibited lower error rates for global glomerulosclerosis estimation from frozen H&E WSI than on-call pathologists; performance was further improved when evaluating pooling results from multiple levels of section [19*]. Enhancing evaluation robustness in this arena could potentially add vitally needed organs to the donor pool.

Cellular-level quantification is an additional emerging realm for DL techniques. Podocytes are highly specialized glomerular cells contributing to the glomerular filtration barrier. These cells are damaged in a wide variety of chronic and acute kidney diseases [20]. To quantify podocytes, WT1- and DACH1-stained immunofluorescence images were used as input for a U-Net based model that segments glomeruli and podocyte nuclear areas in ANCA-associated glomerulonephritis (ANCA-GN) and features such as podocyte distribution, density and glomerular dimensions are calculated [21*]. ANCA-GN patients could be identified in a cohort containing ACNA-GN patients and controls based on their derived podocyte-metrics. A similar approach applicable to rodent kidney tissue was recently published and made available as a free online tool [22**]. Also in this study, a model derived metric (podocyte-density) helped identify diseased glomeruli.

A combination of several types of CNN to identify podocytes and mesangial and epithelial cells in PAS-stained samples was used to determine intrinsic glomerular cell distribution, potentially leading to automated mesangial hypercellularity scoring [23]. Multispectral immunohistochemical staining for visualization of capillaries, macrophages, and lymphocytes was used to create artificial brightfield WSI for training a CNN to extract peritubular capillary extent, cell density, and cell distance from detected inflammatory cells [24].

Outcome Prediction

In an impressive multicenter study including data from 13,608 patients (89,328 patient years) a model for prediction of renal survival seven years after initial assessment in transplant recipients was developed based on histological, clinical, and immunological data and repeated measurements of eGFR and proteinuria [25*]. However, this study did not use histology directly, but pathologist derived scores and is thus not a direct tool for pathologists. Since the model prediction can be updated based on easily accessible data, i.e., eGFR and proteinuria, this model might be useful for patient surveillance. The model adequately reacted to treatments and thus could be used for monitoring therapeutic success. Since this model was validated in several large cohorts from different geographical regions it is likely to generalize to prospective real world settings.

A prediction tool for the development of end stage renal disease (ESRD) IgA-Nephropathy was reported. Clinical information such as blood pressure, histological information (MEST-C score) and therapeutic information (e.g., RAS-blockade or not) to predict ESRD at 5 or 25 years [26]. In a subset of patients that developed ESRD within 10 years of follow-up, a regression model was trained that predicted the time point of ESRD relatively well with a mean absolute error of 1.78 years.

A study segmented normal/diseased kidney tissue compartments, i.e. glomeruli, tubules and interstitium, and mononuclear leukocytes to extract slide-level features from two large cohorts [27**]. These slide level features were then correlated with Banff lesion scores and outcome parameters. Using these features, the authors were able to predict both short- and long term graft survival in baseline and 12-month kidney allograft biopsies significantly better than using Banff lesion scores alone.

One of the major predictors of renal survival is interstitial fibrosis and tubular atrophy (IFTA). Grading of fibrosis was explored in a DL framework operating on different scales, i.e. on WSI-level (resized to lower resolution) and high resolution image tiles cropped from the WSI [28*]. The framework predicted fibrotic grade, i.e. 0–10% IFTA, 11–25% IFTA, 26–50% IFTA and >50% IFTA and generalized well. Ground truth labels were acquired from five pathologists who scored the biopsies with good agreement (kappa = 0.622). However, semantic segmentation and quantification of standard stains specific for fibrosis could provide a more granular readout of interstitial fibrosis.

Other Applications

Pathologists routinely rely on a battery of special stains to evaluate renal biopsies, most commonly PAS, Jones methenamine silver (JMS), and Masson’s trichrome (trichrome). All of these stains require specialized reagents, are time-consuming, and add cost. Development of methods to transform tissue sections into these special stains would provide significant time and cost savings and could additionally minimize tissue sectioning, thereby preserving tissue samples for alternative molecular or other immunohistochemical studies. A virtual-staining method was described that uses a CNN to transform autofluorescence images of fixed, unstained tissue sections into histologically-stained images. Using a wide variety of tissues, including kidney, the investigators showed excellent correlation with traditional histochemical staining techniques [29]. Further work utilized a similar approach to transform H&E-stained WSI into PAS, JMS, and trichrome images that were equivalent to those generated using traditional histochemical staining techniques [30]. In addition to special stains, renal pathologists routinely utilize transmission electron microscopy to evaluate renal biopsy samples. Evaluating electron microscopic images is time-intensive, requiring expert review to generate high magnification views of diagnostic areas. DL techniques to TEM image restoration from fast TEM video streams demonstrate the feasibility of improving TEM images by reducing artifacts generated by motion and noise [31]. This work is an important advance towards automation of TEM image acquisition.

Discussion

The power of AI has the potential to transform the practice of pathology by providing pathologists with a wide array of tools to assist with examination of tissue samples. Implementation of computer-assisted diagnostic methods has the potential to improve pathologist efficiency, precision, and diagnostic accuracy. With the development of virtual staining techniques, there is significant potential for cost savings and tissue preservation for advanced molecular testing. Further improvements in accuracy will likely be achieved by combining clinical, molecular and histological data in multimodal modeling.

Integration of AI into the clinical workflow poses several unique challenges to the field. In order to leverage this technology, laboratories will need to develop the infrastructure to support large scale whole slide imaging. Although the cost of adopting digital pathology has been of concern, a recent study demonstrated a 5-year savings of $1.3 million following full scale digital pathology implementation [32]. Furthermore, with the recent relaxation of regulations for utilizing scanned WSI for primary diagnosis in the United States, a response to the emergent need to work remotely triggered by the SARS-CoV2 pandemic, adoption of whole slide imaging has become more widely accepted [33]. However, it is still far from common practice amongst the majority of pathology laboratories. It is anticipated that the further development of AI technologies applied to examination of WSI will serve to catalyze widespread adoption of digital pathology. As this technology advances, it will be critical to develop user-friendly software applications to deploy AI-methods into the clinical workflow. While the ultimate future state may be scanning of all renal biopsies and application of certain core algorithms prior to examination by the pathologist, a targeted and phased approach is more likely. Selecting slides after initial examination by the pathologist for scanning and quantitation of key parameters such as interstitial fibrosis and global glomerulosclerosis would be fairly straightforward to implement in a clinical environment, for instance. Evaluation of donor kidney biopsies, a process that relies on frozen section H&E tissue examination, could benefit from pre-analysis by AI-methods to quantify chronic damage [19*]. The pathologist would merely review the AI-output for accuracy prior to signing off on the case. Ultimately, deployment of AI-methods into the clinical environment will need to overcome the barriers of regulation and reimbursement.

The regulatory environment for AI-methods in pathology is rapidly evolving. Recently, technology for automated identification, grading, and quantifying prostate cancer on WSI of prostate core biopsies were granted Food and Drug Administration (FDA) approval [34]. This major development provides a framework for future algorithms to utilize as they advance from feasibility studies to clinically applicable techniques. Beyond FDA approval lies the issue of reimbursement for laboratories utilizing these advanced technologies.

The updated Current Procedural Terminology (CPT) codes allow for reimbursement when AI-based tools are used for specific applications. Computer-assisted morphometric analysis of in situ hybridization studies and computer-assisted quantitation of immunohistochemical markers are associated with billing codes 88365 to 88368 and 88361, respectively. FDA-approved platforms for computer-assisted immunohistochemical quantiation are currently available [35]. Although DL algorithms described for renal pathology specifically are not currently reimbursed, there appears to be a reimbursement pathway for the future.

Conclusion

In summary, the future for AI-based applications in renal pathology is promising. In order for these to translate into clinical practice, they need to demonstrate clear clinical utility, fit seamlessly into clinical workflows, and be cost-effective [35].

Key Points.

• Artificial intelligence methods, most commonly convolutional neural networks, are increasingly being implemented for renal pathology disease classification, histologic substructure quantitation, and outcome prediction.

• Computer assisted analysis and evaluation of kidney biopsy whole slide images may improve pathologist precision, accuracy, and reproducibility.

• The regulatory and reimbursement environment will dictate how rapidly these novel technologies will be implemented in clinical practice.