Deep Image Segmentation for Cardiomyocyte Proliferation

Corresponding Author

The adult mammalian heart has limited regenerative capacity, which derives from the long-lived nature of adult cardiomyocytes (CMs). Indeed, CM renewal occurs at a minimal, albeit measurable, level in the adult human heart.¹ Investigating whether CM renewal is associated with other measures of improved cardiovascular outcomes is challenged by the technical difficulty in accurately quantifying CM cell division. Current methods for counting CMs, although rigorous in the right hands, are technically challenging and error prone. In mice, the technical challenges are helped by advanced transgenic fate-mapping techniques, which enable more precise tracking and quantification of CM division but are obviously unavailable to human samples.

Manual curation can compromise sensitivity; introduce potential bias; and, unsurprisingly, lead to substantial interlaboratory discrepancies in CM cycling estimates. One solution to these challenges lies at the intersection of cardiovascular biology and machine learning. The exponential growth in computational power and advancements in deep learning–based image analysis have catalyzed the adoption of automated image segmentation techniques, propelling the field toward more rapid, reproducible, and unbiased analyses. In this issue of JACC: Basic to Translational Science, Karpurapu et al² have demonstrated some significant advances in this direction.

Karpurapu et al² focused on the dynamics of myovascular cells, specifically CMs and cardiac endothelial cells (CECs), in human heart failure and after mechanical unloading by a ventricular assist device.² The authors developed CardioCount, a deep learning–based pipeline for identifying nuclei from antibody-labeled fluorescent images and colocalizing nuclear objects from multiple image channels. CardioCount leverages the U-Net architecture³ with a ResNet50 backbone.⁴ Briefly, the U-Net is an “encoder-decoder” architecture. First, an input image is gradually down-sampled (reducing the spatial resolution) while maintaining necessary contextual information. This part of the model is called an encoder. Next, from reduced images, a decoder restores the spatial resolution and combines features newly discovered by the encoder to produce a “segmentation mask.” U-Net is a classical and widely adopted convolutional neural network architecture for biomedical image segmentation. Residual networks have also been shown to work well for U-Nets. Thus, CardioCount’s success is intuitive. As training data, the authors manually curated the segmentation masks in about 1,000 images. These images used specific markers to identify CM and CEC nuclei and Ki67 to mark nuclei in the cell cycle. They applied the trained model on ∼368,000 images obtained from the Duke Human Heart Repository.

Karpurapu et al² found an interesting coupling between CMs and CECs in the adult human heart, suggesting that the dynamics of these cell types are inter-related in the heart failure samples. They found that CM and CEC nuclear density in the failing heart decrease proportionately, suggesting a biologic link between vascular rarefaction and CM hypertrophy. Furthermore, in failing hearts exhibiting cell cycling, individuals with higher levels of CM cycling also had higher levels of CEC cycling, supporting a coordinated growth dynamic between CMs and CECs in the adult human heart.

Overall, CardioCount is an impressive feat and an important addition to the list of tools deploying deep learning methods in cardiovascular research. Further advancements will require addressing a few aspects of the methodological approach and selecting best practices. First, as the authors noted, manual curation is a source of bias, yet they started with manually curated “ground truth” images. When multiple human annotators perform curation, which is preferred to a single annotator, unavoidable variations are introduced into the model. Uncertainty-aware Bayesian segmentation approaches could help build a more rigorous model in these cases. Another approach to minimize the variability is “scribble segmentation.”⁵ Rather than drawing precise boundaries, human annotators draw small scribbles within the objects of interest, and the model identifies boundaries. However, we acknowledge that addressing these issues is difficult, and an optimal approach probably does not currently exist.

We think the field essentially needs “human-in-the-loop artificial intelligence” in which human experts remain and iteratively improve models like CardioCount. An initial model is trained using the best available options and resources, but groups of human experts continue providing feedback to the model, and the model is retrained to incorporate that feedback. Building a supportive framework for the CardioCount user community will be crucial for the tool’s success.

Another methodological concern is the limited training data size, but in the future, researchers following up on CardioCount can use generative adversarial networks-based approaches.⁶ Thirdly, in the current work, the authors could have offered a more careful inspection of the cases in which CardioCount “makes mistakes.” The model’s precision, recall, and F1 score are satisfactory, but it is a missed opportunity not to ask how the accuracy metrics could be improved. Inspecting a model’s mistakes is often a rewarding experience. It reveals the model’s limitation and indicates if there is any fundamentally challenging aspect of the data that requires a different type of model. Finally, although there is a strong rationale for using U-Net, we think it is another missed opportunity not to try alternative methods such as Mask R-CNN.⁷ Benchmarking multiple methods can reveal the relative strengths and weaknesses of the primarily chosen method and lead to a better solution.

In conclusion, the community welcomes tools like CardioCount. These tools can open new avenues for therapeutic intervention and offer unprecedented insights into the development and regeneration of the human heart. Future studies will incorporate improved statistical and machine learning methodologies and validate the findings of these tools.

Funding Support and Author Disclosures

Dr Martin is a cofounder and owns shares in YAP Therapeutics. Dr Samee has reported that he has no relationships relevant to the contents of this paper to disclose.