Deep Learning Resolves Myovascular Dynamics in the Failing Human Heart

Visual Abstract

Highlights

• •Using a U-Net–based pipeline, we developed a computational tool called CardioCount that allows for objective and consistent scoring of nuclei counts in microscopic images of cardiac tissue. CardioCount is available to run either through a Google Colab notebook or locally by downloading from GitHub.
• •We used CardioCount to determine whether reduced capillary density in the failing heart is associated with CEC cell loss or CM hypertrophy. We found that CM and CEC nuclear density proportionally decrease in the failing heart, suggesting that vascular rarefaction and CM hypertrophy are inter-related.
• •We used CardioCount to confirm prior work suggesting the cardiac cell cycling in the human heart is increased with mechanical unloading. However, we did not observe any change in cycling rates of CMs or CECs after LVAD placement.
• •When cycling was present in failing hearts, we found that individuals with higher levels of CM cycling also had higher levels of CEC cycling, suggestive of coupled growth of CMs and CECs in the adult human heart.

Summary

The adult mammalian heart harbors minute levels of cycling cardiomyocytes (CMs). Large numbers of images are needed to accurately quantify cycling events using microscopy-based methods. CardioCount is a new deep learning–based pipeline to rigorously score nuclei in microscopic images. When applied to a repository of 368,434 human microscopic images, we found evidence of coupled growth between CMs and cardiac endothelial cells in the adult human heart. Additionally, we found that vascular rarefaction and CM hypertrophy are interrelated in end-stage heart failure. CardioCount is available for use via GitHub and via Google Colab for users with minimal machine learning experience.

Although the innate regenerative capacity of the mammalian heart is limited, multiple groups have identified a low, but present, level of cardiomyocyte (CM) turnover in the adult human heart.1, 2, 3, 4 While such a low level of cycling has largely been considered insignificant, recent genetic ablation experiments in adult mice suggest that rare cycling CMs promote functional recovery after injury.⁵ However, the impact of CM cycling on human cardiovascular outcomes is unknown.

Elegant transgenic, fate-mapping methods have been developed to trace and quantify CM proliferation in adult mice and zebrafish.5, 6, 7, 8 However, such methods are not feasible for assaying CM cycling in the human heart. Traditional methods to quantify cycling cells from human tissue sections require manual curation of microscopic images. However, for low-frequency events, such as CM cycling, manual methods are often insufficient. Because of the sheer labor required to score thousands of images, manually curated assessments often lack the sensitivity to identify subtle phenotypes and could be subject to biases if readers are not sufficiently blinded. Not surprisingly, there can be wide variance in the frequency estimates of rare events such as CM cycling from lab to lab.⁹ A potential solution is to apply automated image segmentation routines to score microscopy images more quickly, reproducibly, and objectively. Once computationally prohibitive, better computing power has reinvigorated computer vision by enabling deep learning methods. Deep learning has subsequently been widely applied to various applications in biology, including image segmentation.¹⁰ However, these image analysis tools, although highly accurate, can be sensitive to imaging conditions and often require significant computational resources.

To address these issues, we developed CardioCount, a deep learning routine to score CM cycling in the human heart. CardioCount is a U-Net–based deep learning model with a ResNet50 backbone that was previously utilized to identify solar panels from drone imagery.¹¹ CardioCount identifies nuclei from antibody-labeled fluorescent images and can colocalize nuclear objects from multiple image channels. Inspired by the DeepLearning4Mic project, the software can run on Google Colab’s cloud resources and score thousands of images with few local computational tools.¹² We show that CardioCount is highly versatile and can be adapted to different image acquisition setups and to different species via transfer learning. To demonstrate the functionality of our deep learning model, we used CardioCount to assay 10,305 mm² of human cardiac tissue obtained from patients with and without end-stage heart failure (HF), including some who received left ventricular assist devices (LVADs). Using this tool, we confirm that loss of capillary density marks end-stage HF, but vascular rarefaction results from the proportional changes of cardiac endothelial cells (CECs) and CMs. Additionally, we demonstrate that as in zebrafish and mice, CM cycling is coupled with CEC cycling in the human heart.

Methods

Human cardiac samples

Human tissue samples were obtained from the Duke Human Heart Repository.¹³ Heart tissue was collected from unused donor hearts without HF (control hearts), from patients with end-stage HF who underwent LVAD implantation (pre-LVAD), and from LVAD recipients who underwent LVAD explantation (post-LVAD).

Histology and Immunostaining

Cryosections of fresh, frozen, human cardiac tissue were fixed with 4% paraformaldehyde, washed and permeabilized with 0.1% Tween (phosphate-buffered saline and Tween [PBST]), and subjected to heat-induced antigen retrieval with citrate buffer. After additional washing with PBST, the samples were blocked with 5% bovine serum albumin for an hour and incubated overnight with primary antibodies. Samples were then washed with PBST and incubated with secondary antibodies and DAPI (1:5,000) for 1 hour. Cryosections from neonatal mouse hearts were processed as previously described.¹⁴ Primary antibodies used for this work included: anti-PCM1 (Sigma-Aldrich, HPA023370, 1:100); anti-Erg (Abcam, ab92513, 1:25); anti-CD31 (BD Bioscience, 553370, 1:100), anti-Ki67 (Thermo Fisher Scientific, 4-5698-82, 1:100), anti-Tnnt (Developmental Studies Hybridoma Bank, CT3, 1:25), and anti-Cre (Cell Signaling Technologies, 15036, 1:50). Secondary antibodies were conjugated to Alexa-488 (Invitrogen, A21208, 1:1000), Alexa-594 (Invitrogen, A11037, 1:200), Alexa-633 (Invitrogen, A21052, 1:200), or Alexa-647 (Invitrogen, A21472, 1:200). To ensure quality control within each staining batch, a human colon tissue sample was included as a positive control for anti-Ki67 immunostaining.

Microscopy

Stained human tissue samples were imaged with a Zeiss CSU-X1 spinning disk confocal microscope or a Zeiss LSM 510 confocal microscope. Stained murine sections were imaged with a Zeiss CSU-X1 spinning disk confocal microscope, a Zeiss AxioImager M1 epifluorescent microscope, or a Zeiss LSM 510 confocal microscope.

Development of CardioCount

Images (N = 1,333) stratified across all patients were manually curated for ground truth by generating ground truth masks with Fiji/ImageJ (330 images for the Erg model, 330 images for the PCM1 model, and 667 images for the Ki67 model).¹⁵ Ten percent of the annotated data set was set aside for testing, 70% was used for training, and the last 20% was held for validation and hyperparameter selection. Raw RGB images (red: Erg or PCM1 for cell identity of CECs or CMs respectively, green: Ki67 for cell cycling, blue: DAPI for nuclei) were converted to 2 channel images (either RB to train the Erg and PCM1 models, or GB to train the Ki67 model). Two channel images and their corresponding ground truth masks were used to train a U-Net–based convolutional neural network (CNN), as described previously.^11,16

After training CNNs for each marker, the entire data set of 368,434 images was passed through each model. The probability map outputs from the CNN for these images were converted to object maps through a postprocessing pipeline by binarizing the images, grouping positive pixels into objects, and assigning a confidence score to each object. An object confidence threshold score was used to classify each object as a true positive (TP), false positive (FP), or false negative (FN) based on an intersection over union of 0.2 calculated as TP/(TP+FP+FN). Precision (TP/[TP+FP]), and recall [TP/(TP+FN)] were used to determine the F1 score (2TP/[2TP+FP+FN]), which was used to measure model accuracy and was calculated for each set of hyperparameters on the validation set. Hyperparameters that were tuned included: architecture type, learning rate, class weight ratio, loss weights, and the ratio between loss weights. Final models were selected by the highest F1 score on the validation set. To colocalize nuclear objects and identify double-positive cells, nuclei of Ki67⁺ cells were filtered by the presence of a centroid of an Erg or PCM1 nucleus within the borders of the Ki67⁺ nucleus. All code scripts are available on GitHub.

Murine cardiac samples

Cardiac tissue from neonatal mice was collected and processed as previously described.¹⁴ For AAV experiments, neonatal ICR mice were intraperitoneally injected with 10¹¹ vg of AAV.cc47 encoding Cre recombinase carrying pAAV.cTNT.iCre.^17,18 Neonatal hearts were collected 9 days after injection. pAAV.cTNT.iCre was a gift from William Pu (Addgene plasmid # 69916).

Adaptation of CardioCount to Murine Cardiac Images

CNNs for murine cardiac images were initially trained using optimized hyperparameters and model weights from the corresponding human data set. A total of 50 murine images were annotated and used as input. Five images were held out for the test set, while the training and validation sets were split in a 7:2 ratio. For transfer learning, a learning rate sweep was performed for the encoder (1E-2, 5E-3, 2.5E-3, 1E-3, 5E-4, 2.5E-4, 1E-4, and 5E-5). The learning rate for the decoder was set as 10 times that of the encoder. Following the learning rate sweep, a class weight ratio sweep was performed with various background-to-foreground ratios (1:500, 1:250, 1:100, 1:50, 1:25, 1:10, 1:5, 1:2, 1:1, and 2:1). Postprocessing was done as described for the human nuclear segmentation pipeline. The optimal model on the validation set was then run on the testing set for the F1 statistics.

Development of Area Segmentation Pipeline

To compute tissue area for nuclear density calculations, we developed an area calculation algorithm that calculates percent tissue coverage, defined as the number of pixels covered by tissue divided by the total number of pixels in the image. Segmentation and calculation of percent tissue coverage in images were calculated using an algorithm developed using the scikit-image, NumPy, and OpenCV libraries in Python.19, 20, 21 First, images were normalized by multiplying pixels by 255 and dividing by the 95th percentile of the pixel in the thresholded image, unless the image was considered a background image (more than 97% of pixels are near black). A series of dilation, area closing, binary erosion, area opening, binary dilation, binary erosion, area opening, and area closing was used in sequence to obtain the segmented image. Human verification of a set of random samples was conducted to ensure the areas calculated were appropriate.19, 20, 21

Statistical Analysis

Continuous data are presented as mean ± SD or median with 25th and 75th percentiles (Q1-Q3). Statistical analyses between unpaired data were conducted using Welch's t-test for normally distributed data or a Wilcoxon rank sum test otherwise. Shapiro-Wilk's test was used to evaluate data distribution. Categorical data are presented as count (%) and were analyzed using the Fisher exact test. Paired data were analyzed by fitting generalized additive models with random effects. Pearson's correlation coefficient (r) was used to evaluate the association between continuous variables using linear regression. If data were not log scalable due to zero-valued datapoints, a small positive constant was added to each datapoint to ensure normally distributed axes after log transformation. Statistical analyses and plot generation were performed in R software (R Project for Statistical Computing) using the dplyr, ggpubr, ggplot2, and mgcv packages.22, 23, 24, 25 An a priori P value <0.05 was used to determine statistical significance.

Study Approval

All human subjects research was approved by the Duke University Medical Center Institutional Review Board (Pro00005621). Informed consent was obtained for all subjects. All animal studies were approved by the Duke University Institutional Animal Care and Use Committee.

Results

A deep learning model to score cycling cells in the human heart

Based on the low rates of CM cycling previously reported, we reasoned that large amounts of cardiac tissue would need to be assayed to determine whether intrinsic rates of CM cycling vary between individuals. We leveraged the Duke Human Heart Repository, a bank of cardiac tissue specimens that contains tissues from individuals with various etiologies of HF as well as nonfailing cardiac tissue samples.¹³ Baseline characteristics for the population evaluated are provided in Supplemental Table 1. We assayed a total of 12 nonfailing control hearts and 32 hearts from patients undergoing LVAD implantation for end-stage HF (Supplemental Table 1, Supplemental Figure 1). Patients with HF were age matched to patients without HF, with an average age of 54.1 ± 9.7 years for the 32 HF patients and 53.6 ± 8.7 years for the patients without HF. In general, patients with HF had a lower left ventricular ejection fraction (17.5% [10.0%-20.0%] vs 55.0% [55.0%-62.5%]; P < 0.001, Wilcoxon rank sum test), had larger left ventricular diameters as measured by left ventricular internal end-diastolic diameter (6.60 [5.90-6.85] cm vs 4.50 [4.48-4.58] cm; P < 0.001, Wilcoxon rank sum test), and were more likely to have type II diabetes mellitus (56.3% vs 8.3%; P = 0.006, Fisher exact test). On average, patients with LVADs were diagnosed with HF 4.9 ± 5.8 years before LVAD implantation.

From each individual, we immunostained cryosections with PCM1 to mark CM nuclei, Erg to mark CEC nuclei, and Ki67 to mark cycling nuclei.^2,26,27 In total, we imaged 83.1 ± 36.9 mm² of cardiac tissue per individual, for a total of 10,305 mm² across 368,434 images. To score such a large number of images, we developed a customized U-Net–based deep learning algorithm based on prior work used to identify solar panels from satellite images (Figure 1).¹¹ Broadly, we established an approach to identify specific types of nuclei by colocalization of a cycling marker with a cell-type specific nuclear marker. We adapted this framework to develop specific models for resolving PCM1⁺ CM nuclei, Erg⁺ CEC nuclei, and Ki67⁺ cycling nuclei. As ground truth for model training and validation, we manually curated a library of 1,333 images (330 images for Erg, 330 images for PCM1, and 667 images for Ki67). Seventy percent of each data set was used for model training, with 20% set aside for validation and hyperparameter testing and 10% held out for final model F1 score statistics. Training data were passed through a U-Net CNN to determine model weights and to generate probability maps of nuclei locations. Raw probability maps were then converted into nuclei objects. Models were optimized based on precision-recall curves and tuned by sweeping across hyperparameters for learning rate, loss weights, loss-weight ratio, and class-weight ratio (Supplemental Figure 2A). Final models for identifying CM, CEC, and cycling nuclei achieved F1 scores of 0.85, 0.86, and 0.89, respectively (Figure 1, Supplemental Figure 2B).

Figure 1.

Generation and Validation of CardioCount

(A) Diagram of CardioCount’s machine learning pipeline for instance segmentation of RGB microscopy images. RGB images were converted to RB (red+blue channel) images as input to the Erg or PCM1 cell-identity models and GB (green+blue channel) images as input to the Ki67 cycling model. Probability map outputs were then converted to object maps using a postprocessing script. Lastly, cycling information was overlayed with cell-identity information by colocalizing nuclear objects. Ki67⁺ nuclei were filtered by the presence of a centroid of an Erg or PCM1 nucleus within the borders of the Ki67 nucleus. (B) Precision-recall (PR) curves for Erg, PCM1, and Ki67 models using test data (Erg, PCM1, Ki67). The red data point indicates the optimal machine learning model parameters, as determined by the best F1 score on the validation image set. (C) Representative application CardioCount to identify cycling cardiomyocytes from human cardiac sections immunostained for PCM1 and Ki67. Object instances of PCM1⁺ nuclei are outlined in the second subpanel; object instances of Ki67⁺ cells are outlined in the third subpanel, and a PCM1⁺Ki67⁺ double-positive object is outlined in the fourth subpanel. Scale bars are 50 μm. (D) Representative application CardioCount to identify cycling cardiac endothelial cells from human cardiac sections immunostained for Erg and Ki67. Object instances of Erg⁺ nuclei are outlined in the second subpanel; object instances of Ki67⁺ cells are outlined in the third subpanel, and an Erg⁺Ki67⁺ double-positive object is outlined in the fourth subpanel. Scale bars are 50 μm. CNN = convolutional neural network.

Myovascular ratio is largely maintained during disease

HF is a progressive disease, marked by CM hypertrophy and loss of CMs. Numerous studies have cited vascular rarefaction, or the loss of capillary density, as a marker of more advanced disease.28, 29, 30 However, whether this decrease in capillary density is associated with CEC loss or CM hypertrophy has yet to be resolved. To address this outstanding question, we used CardioCount to compare CEC and CM densities from pre-LVAD implantation HF samples to control samples. CEC and CM densities were computed by dividing nuclei counts by tissue areas calculated by a custom area segmentation pipeline. As expected, based on the CM hypertrophy known to occur in HF (Figures 2A and 2B), the density of CM nuclei was decreased by ∼33% in samples from patients with HF (135.1 [102.0-155.5] CM nuclei/mm² vs 201.5 [181.6-229.3] CM nuclei/mm²; P < 0.001, Wilcoxon rank sum test) (Figure 2C). When we evaluated CECs, we noted a similar decrease in nuclear density, with a ∼54.8% decrease in patients with HF compared with control patients (306.1 ± 60.9 CECs/mm² vs 138.2 ± 61.2 CECs/mm²; P < 0.001, 2-sided unpaired t-test) (Figure 2D). Based on the overall similarity in this decline, we compared the ratio of CEC/CM nuclei across patients. Patients without HF had largely the same ratio as those with HF (1.45 ± 0.46 CECs/CM vs 1.16 ± 0.53 CECs/CM; P = 0.123, 2-sided t-test) (Figure 2E), indicating that vascular rarefaction is either secondary to CM hypertrophy or due to a proportional loss of CECs and CMs. However, we noted a wider spread in the CEC:CM ratio among patients with HF compared with control patients, suggesting heterogeneity in the CEC:CM ratio within HF patients. When we evaluated which clinical factors might be related to CEC:CM ratio, only N-terminal pro–B-type natriuretic peptide (NT-proBNP) levels—a marker clinically used to mark ventricular stress and identify HF patients at high risk^31,32—were significantly correlated, with a decrease of 0.562 CECs per CM per doubling of NT-proBNP (r = −0.547; P = 0.007) (Figure 2F).

Figure 2.

Myovascular Dynamics With HF

(A and B) Representative images of human cardiac sections from a control patient and a patient with heart failure (HF). Sections in (A) are immunostained for PCM1 to mark cardiomyocyte (CM) nuclei (yellow), WGA to mark cell membranes (magenta), and nuclei (cyan). Sections in (B) are immunostained for Erg to mark cardiac endothelial cell (CEC) nuclei (yellow), WGA to mark cell membranes (purple) and nuclei (turquoise). Insets show a magnified image of a PCM1⁺ CM nucleus and an Erg⁺ CEC nucleus. Scale bars are 50 μm. Arrowheads indicate double-positive nuclei. (C) Violin plot of CM nuclear density in patients without heart failure (control patients) and with heart failure (HF/pre-LVAD). ∗∗∗P < 0.001, Wilcoxon rank sum test. (D) Violin plot of CEC nuclear density in patients without heart failure (control patients) and with heart failure (pre-LVAD). ∗∗∗P < 0.001 2-sided t-test. (E) Violin plot of the CEC:CM ratio in patients without heart failure (control patients) and with heart failure (pre-LVAD). P = 0.13, n.s. = not significant, 2-sided t-test. (F) Correlation of CEC:CM to NT-proBNP levels in patients with HF. Blue line is best-fit regression line. P = 0.007, Pearson correlation test. LVAD = left ventricular assist device.

Myovascular dynamics on LVAD support

LVADs are surgically implanted pumps that improve survival in patients with advanced HF. Blood is drawn into the LVAD from the left ventricle through an inflow cannula and propelled into the aorta, effectively decompressing the left ventricle. In select patients, LVADs have been associated with reverse remodeling and recovery of ventricular function.³³ Histologically, LVAD support leads to regression of CM hypertrophy and increased capillary density.³⁴ Because our samples from the LVAD cohort were collected at the time of LVAD insertion, we had a unique opportunity to use CardioCount to evaluate myovascular dynamics after LVAD support in individuals by also assaying LVAD explanation samples from the same individual. We obtained 20 matched pre-LVAD and post-LVAD cardiac tissue samples from the same patient along with an additional 5 unmatched pre-LVAD tissue samples and 7 unmatched post-LVAD tissue samples. On average, sex, age, race, and heart size measured by left ventricular internal end-diastolic diameter were consistent. However, post-LVAD patients compared with pre-LVAD patients had lower NT-proBNP levels (3,550 ± 4,040 pg/mL vs 11,300 ± 14,600 pg/mL) (Supplemental Table 2).

Consistent with prior work, we found that LVAD support resulted in an increase in the density of CM nuclei (126.6 ± 39.3 CMs/mm² vs 177.7 ± 50.9 CMs/mm²; P < 0.001, 2-sided paired t-test) and the density of CEC nuclei (138.2 ± 61.2 CECs/mm² vs 222.1 ± 116.8 CECs/mm²; P = 0.002, 2-sided paired t-test) (Figures 3A to 3D).³⁴ We did not observe any changes in CEC:CM ratio with LVAD support (1.16 ± 0.53 CECs/CM vs 1.27 ± 0.59 CECs/CM) (Figure 3E).

Figure 3.

Myovascular Changes After LVAD Support

(A and B) Representative images of human cardiac sections from a control patient and a patient with heart failure. (A) Sections are immunostained for PCM1 to mark CM nuclei (yellow), WGA to mark cell membranes (purple) and nuclei (turquoise). (B) Sections are immunostained for Erg to mark CEC nuclei (yellow), WGA to mark cell membranes (purple) and nuclei (turquoise). Insets show a magnified image of a PCM1⁺ CM nucleus and Erg⁺ CEC nuclei. Scale bars are 50 μm. Arrowheads indicate PCM1⁺ and Erg⁺ nuclei. (C) Violin plot of CM nuclear density in patients pre-LVAD and post-LVAD. ∗∗∗P < 0.001, generalized additive models with random effects with Wald P. (D) Violin plot of CEC nuclear density in patients pre-LVAD and post-LVAD. Lines connect paired samples from the same patient. ∗∗P = 0.002, generalized additive models with random effects with Wald P. (E) Violin plot of CEC:CM ratio in patients pre-LVAD and post-LVAD. P = 0.44, n.s.= not significant, generalized additive models with random effects with Wald P. For C to F, lines connect paired samples from the same patient. Abbreviations as in Figure 2.

Although CMs in the postnatal mammalian heart have largely exited the cell cycle, recent work has measured a low, but present, rate of CM turnover in the adult human heart,¹ including a recent report that the rate of CM cycling increases with LVAD support.⁴ Thus, we sought to characterize cellular cycling in our cohort by colocalizing Ki67⁺ nuclei with CECs and CM nuclei. Overall, we found evidence for rare cycling CMs and CECs either before or after LVAD support (Figures 4A to 4D). However, we noted considerable heterogeneity among patients with regards to CM and CEC cycling. We have recently described that coupled expansion of CMs and CECs is required for cardiac growth and regeneration in neonatal mice.¹⁴ To determine whether CEC and CM cycling might be coupled in the adult human heart, we regressed CEC and CM cycling rates for each patient. We found a modest correlation between CM cycling rate and CEC cycling rate (r = 0.59; P = 0.002) (Figure 4E), with a doubling in CEC cycling rate corresponding to a 1.47-fold increase in CM cycling rate, suggesting that coupled expansion of CECs and CMs also occurs in diseased human hearts.

Figure 4.

CM and CEC Cycling With LVAD Support

(A and B) Representative images of human cardiac sections from the same patient before and after LVAD support. (A) Sections are immunostained for PCM1 to mark CM nuclei (yellow), Ki67 to mark cycling nuclei (purple) and nuclei (turquoise). (B) Sections are immunostained for Erg to mark CEC nuclei (yellow), Ki67 to mark cycling nuclei (purple) and nuclei (turquoise). Insets show magnified images of Ki67⁺PCM1⁺ CM nuclei (A) and Ki67⁺Erg⁺ CEC nuclei (B). Scale bars are 50 μm. Arrowheads indicate Ki67⁺ nuclei. (C) Differences in % of Ki67⁺_, PCM1⁺ CM nuclei before and after LVAD support. P = 0.102, n.s. = not significant, generalized additive models with random effects with Wald P. (D) Differences in % of Ki67⁺_, Erg⁺ CEC nuclei before and after LVAD support. P = 0.46, n.s. = not significant, generalized additive models with random effects with Wald P. (E) Correlation of % of Ki67⁺_, Erg⁺ CEC nuclei with % of Ki67⁺_, PCM1⁺ CM nuclei. Note that a constant of 0.001 was added to log-scale zero-valued data. Blue line is best-fit regression line. P = 0.002, Pearson correlation test. Abbreviations as in Figure 2.

Transfer learning allows CardioCount to be generalized to other data sets

Because our models for CMs, CECs, and cycling cells have the same underlying architecture and perform similarly despite different staining morphologies, we reasoned that our models could be repurposed for additional applications using transfer learning. We first assayed the ability to score neonatal mouse hearts, which have smaller nuclear morphology and higher cell density than human myocardial tissue. Starting with the machine learning models trained on the human data set, we tuned the model hyperparameters with limited training/validation sets of neonatal mouse heart images (Figures 5A to 5C, Supplemental Figure 3). Combined training and validation set sizes of 0, 10, 15, 30, and 45 were used to train the model, whereas a testing set of 5 images was used to evaluate the transfer learning trained model. As expected, the model without transfer learning performed consistently worse than the other models; however, we did see a plateau in F1 score improvement after a transfer learning data set size of 15 images (Figure 5B). The neonatal mouse Erg data set consisted of images with 2,752 × 2,208 pixel dimensions instead of the standard 512 × 512 dimensions of the images used to train the human data set, suggesting that our modeling approach can be transferred to diverse imaging conditions.

Figure 5.

Application of CardioCount to Murine Hearts

(A) Schematic of transfer learning to assay murine data. Optimal model parameters for the human cardiac data sets were used as a starting point to train models for microscopy images of mouse hearts. (B) Final F1 scores for murine images as a function of different training and validation (Train/Val) set sizes. (C) Representative application of CardioCount to identify PCM1⁺ cardiomyocytes, Erg⁺ cardiac endothelial cells, and EdU⁺ cycling cells from a mouse cardiac section. The Erg⁺ cardiac endothelial cell image is a 512 × 512 crop of a full image. Identified objects are outlined in green for each marker. (D) Representative image from a neonatal mouse heart after injection with AAV-cTnT-Cre. Sections were stained for Cre (green), PCM1 (red), and DAPI (blue). The circles indicate Cre⁺ cells that were counted as PCM1⁺, and arrowheads point to Cre⁺ cells missed by CardioCount. Magnified images point to a correctly scored Cre⁺ nucleus and a missed Cre⁺ nucleus. (E) Comparison of scores from CardioCount to prior published work, when applied to the same set of images. Scale bars are 50 μm. CNN = convolutional neural network.

We next sought to benchmark our data against other data sets. We first tested the sensitivity of CardioCount for detecting CM nuclei based on PCM1 immunostaining. Neonatal mice were injected with AAV encoding Cre recombinase under the control of a CM-specific chicken troponin T promoter, followed by immunostaining of sections for Cre and PCM1 (Figure 5D).¹⁸ CardioCount was used to identify CM nuclei using the PCM1 and DAPI channels, and manually corroborated with Cre signals. Of 7,834 CM nuclei that were labeled with Cre, 7685 nuclei were correctly scored as CM nuclei, giving a sensitivity of 98.1%. Of note, the Cre⁺ cells that were missed tended to be out of plane or have atypical PCM1 immunostaining patterns. Finally, we compared CM cycling rates determined by CardioCount against our prior work that used a 2-step process to identify CM nuclei with a random forest model (ilastik),³⁵ followed by segmentation and colocalization with EdU signals in CellProfiler. Across 528 images scored for PCM1⁺ EdU⁺ cells, CardioCount produced a nearly identical output (r = 0.97; P < 0.001) (Figure 5E).

To enhance use, we developed an easy-to-use user interface for CardioCount in Google Colab, a free, cloud-based notebook environment that provides GPU access and directly integrates with Google Drive. The Google Colab implementation of CardioCount is aimed towards users with minimal machine learning experience or command line expertise. Users can upload their images to Google Drive and directly run the previously described human or mouse models on data sets through the Google Colab interface. Users can also perform transfer learning or train machine learning models from scratch without the need for any software setup on their local computers. One caveat to using the software is that it remains limited by Google’s resource partitioning, including memory and computational resources. The Google Colab CardioCount software works optimally for smaller image sets where fewer computational resources are needed to process the images. The full code scripts needed to run the machine learning pipelines on larger data sets are available on GitHub and can be implemented locally.

Discussion

Assays of cellular cycling and proliferation are fundamental for tissue regeneration research. Organs, such as the heart, with low basal proliferation rates are particularly problematic because an accurate assessment of rare events requires surveying large amounts of tissue to quantify enough events for accurate estimates, much less to be able to compare treatment effects. To address the need for automated and accurate scoring of cellular cycling, we developed CardioCount, a deep learning–powered pipeline to score cells from antibody-labeled immunofluorescent cardiac tissue. We adapted a prior machine learning platform used to identify solar panels from satellite images to identify labeled nuclei. We were able to use this framework to develop highly accurate models for identifying and scoring CMs, CEC, and Ki67⁺ cells in cardiac tissue sections with test set F1 scores of >0.85 on each of the 3 data sets. Our work differs from prior machine learning approaches that identify CM nuclei after immunostaining for sarcomeric proteins—specifically our models are based on nuclear markers and enable colocalization of different immunostains.³⁶ Our approach compares well with other prominent deep learning and cell segmentation techniques such as Cellpose, Mesmer, and CellProfiler, which regularly segment cells on data sets with F1 scores of 0.4 to 0.9 on general segmentation tasks with broad, diverse data sets.37, 38, 39, 40 Of note, our model assays CM cycling and cannot differentiate CM proliferation from endocycling, which typically requires lineage tracing approaches or cardiac dissociation to determine cellular ploidy.

To test the utility of our tool, we sought to address a long-standing question regarding the cellular basis for the vascular rarefaction that occurs in progressive HF. Consistent with prior work, we were able to identify decreases in both CM and CEC nuclear density in patients with HF. When we evaluated the overall CEC:CM ratio, we did not observe significant differences between end-state HF and control hearts, with approximately 1.2 to 1.3 CECs/CM, a figure that compares favorably with other approaches^1,41 (Figure 2). Our data suggest that vascular rarefaction may be a marker of CM hypertrophy or an indicator of proportional loss of CECs and CMs. However, we also noted that the CEC:CM ratio is highly related to NT-proBNP levels, a biomarker that clinically relates to higher filling pressure and worse prognosis (Figure 2F). Thus, our work is also consistent with prior studies suggesting that reduced coronary flow is related to worse HF and may further suggest that individuals with extremely elevated NT-proBNP levels might benefit from mechanical or molecular revascularization approaches. One potential confounder to our work is we assume that the proportion of mononuclear (∼74% in the normal heart) and multinucleated CMs stays constant during HF.⁴² Finally, we evaluated cellular dynamics of mechanical unloading with LVAD support. Unlike prior work, we did not identify a significant increase in CM cycling after LVAD support (Figure 4). However, different patient populations and different pump management strategies might partially explain why our results differ from prior work. Importantly, we did note that CEC and CM cycling are associated in the human heart, consistent with prior work in preclinical models and raising the possibility that revascularization strategies might be an important adjunct to tissue regeneration approaches (Figure 4E).¹⁴

Our goal was to establish a platform that could broadly be used to score tissue sections. Towards that goal, we have made our platform available using Google Colab to minimize the entry barrier in terms of computational resources and setup. We have also developed a transfer learning module to enable users to develop customized models for their own tissues of interest and imaging setups. As proof of principle, we applied our method to neonatal mouse section images under different microscopy conditions and show that our platform can be highly accurate even when trained with a small number of training images utilizing transfer learning. Overall, the use of accurate and reproducible algorithms for image quantification has the potential to improve experimental rigor and precision.

Although our models are highly promising, we acknowledge several limitations. CardioCount is a nuclei-based model, meaning that CM counts reported are only a proxy for the total number of CM cells obtained in the images. For this reason, the model works best on nuclear markers and may not be readily transferable to other types of markers. Additionally, though we expect the general approach of our model to work for other data sets, our models were specifically trained on human and mouse cardiac data and therefore may require larger amounts of training data if transfer learning were to be applied to noncardiac contexts. Finally, our Google Colab implementation of the machine learning training remains limited by the hardware Google provides on its free tier to Google Colab users. The Colab notebook works optimally only on smaller image sets, and for larger image sets, the scripts likely need to be run locally.

Conclusions

We developed CardioCount, a deep learning–based computational pipeline that can be used to segment nuclei and count cells from diverse microscopy setups. Using this tool on a large data set of human myocardium, we were able to make unique observations regarding the composition of human cardiac tissue, including coupled growth between CECs and CMs, the lack of any association between LVAD support and CM cycling, and findings suggestive of vascular rarefaction as a marker of CM hypertrophy. CardioCount is publicly accessible via GitHub or Google Colab, and we hope this platform can be used broadly for similar applications of tissue segmentation.

Perspectives.

COMPETENCY IN MEDICAL KNOWLEDGE: Cardiovascular disease is a leading cause of death worldwide, and understanding cardiac tissue composition and cycling is essential for developing effective therapies. Current methods to quantitatively assay cardiac tissue composition by histology rely on manual scoring, often limiting the amount of tissue that can be assayed. We built a deep learning tool to score large amounts of cardiac tissue. When applied to images of failing hearts, we find that end-state heart failure is associated with a decrease in capillary density, but that this is proportional to changes in myocyte nuclear density. We also show that there is coupled cycling of cardiac endothelial cells with cardiomyocytes in the human heart, like our previous observations in zebrafish and mice. Taken together, we find that the cellular makeup of the failing heart is dynamic and provides a resource for largescale histologic assessment of the heart.

TRANSLATIONAL OUTLOOK: We developed an objective and consistent machine learning model that can segment and count nuclei in thousands of microscopic images. Tools like this will allow for rigorous assessment of cardiac cell composition and will allow for robust, and unbiased, estimates of rare events such as cardiomyocyte cycling. This resource can enhance the rigor of translational studies that rely on quantitative assessments of cardiac tissue. Due to the generalizability of the CardioCount framework, CardioCount may also be applied more broadly for nuclear segmentation of tissue sections in other contexts.

Funding Support and Author Disclosures

This work was funded by a Stead Society Grant (Dr Karra), a Duke University Strong Start Physician Scientist Award (Dr Karra), and NHLBI grant R01 HL15777 (Dr Karra). The authors have reported that they have no relationships relevant to the contents of this paper to disclose.