Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2021 Jan 1.
Published in final edited form as: Ann Biomed Eng. 2019 Sep 23;48(1):477–489. doi: 10.1007/s10439-019-02366-2

High frequency spectral ultrasound imaging to detect metastasis in implanted biomaterial scaffolds

Grace G Bushnell 1,*, Xiaowei Hong 1,*, Rachel M Hartfield 1, Yining Zhang 2, Robert S Oakes 1, Shreyas S Rao 3, Jacqueline S Jeruss 1,4, Jan P Stegemann 1, Cheri X Deng 1,5,+, Lonnie D Shea 1,2,+
PMCID: PMC6930322  NIHMSID: NIHMS1540508  PMID: 31549327

Abstract

For most cancers, metastasis is the point at which disease is no longer curable. Earlier detection of metastasis, when it is undetectable by current clinical methods, may enable better outcomes. We have developed a biomaterial implant that recruits metastatic cancer cells in mouse models of breast cancer. Here, we investigate spectral ultrasound imaging (SUSI) as a non-invasive strategy for detecting metastasis to the implanted biomaterial scaffolds. Our results show that SUSI, which detects parameters related to tissue composition and structure, identified changes at an early time point when tumor cells were recruited to scaffolds in orthotopic breast cancer mouse models. These changes were not associated with acellular components in the scaffolds but were reflected in the cellular composition in the scaffold microenvironment, including an increase in CD31+CD45− endothelial cell number in tumor bearing mice. In addition, we built a classification model based on changes in SUSI parameters from scaffold measurements to stratify tumor free and tumor bearing status. Combination of a linear discriminant analysis and bagged decision trees model resulted in an area under the curve of 0.92 for receiver operating characteristics analysis. With the potential for early non-invasive detection, SUSI could facilitate clinical translation of the scaffolds for monitoring metastatic disease.

Keywords: Metastasis Detection, Spectral ultrasound imaging, Pre-metastatic niche, Cancer, Cancer diagnostics

Introduction

Metastasis is responsible for 90% of cancer related deaths11. The transition to stage IV disease has a devastating prognosis for many reasons. First, metastasis is often not detected until whole organ systems have been compromised. Currently, a definitive diagnosis of metastasis relies on the use of positron emission tomography (PET), computed tomography (CT), or magnetic resonance imaging (MRI) to locate and visualize a metastatic lesion, typically >7–8mm in diameter13. Second, widespread metastatic disease is often more resistant to therapy, more aggressive, and overall much harder to treat. Early detection of metastatic disease or risk of metastasis has significant potential to reduce cancer mortality by allowing interventions when the burden of disease is low.

Early detection strategies have primarily focused on the use of blood as a liquid biopsy. Liquid biopsies including circulating tumor cell (CTC), circulating tumor DNA (ctDNA), and exosome detection in the blood are emerging as a technology to stage patients beyond the presence, size, and molecular characteristics of the primary tumor6. However, the connection between the presence and relative number of each of these markers and metastasis is still unclear. For example, while a high number of circulating tumor cells (5 CTCs in 7.5 mL blood) correlates with poor prognosis32, studies in animal models have shown that 99.99% of CTCs do not have the ability to metastasize16 and thus understanding when to intervene in a patient that has a high number of CTCs is still elusive. Biomaterial scaffolds that capture metastatic tumor cells1 extend beyond liquid biopsy to capture immune cells associated with the metastatic niche and metastatic tumor cells themselves24, 7, 30. These technologies have been successful in mouse models of breast24, 7, 30, ovarian12, prostate8, melanoma22 and hematologic cancers24 and have captured tumor cells (as measured by flow cytometry, bioluminescence imaging, immunofluorescence, and histology), reduced metastatic burden7, 30, and improved survival30. Studies from various groups have independently confirmed the crucial role of the foreign body response to the biomaterial implant to recruiting tumor cells1, 5, 7, 8, 22, 30. The altered immune system in a tumor-bearing subject is reflected in the immune cells available to participate in the foreign body response, resulting in formation of an environment permissive to recruitment of tumor cells7,30.

A crucial component of this detection platform will be non-invasive monitoring of the biomaterial implant for metastasis. Inverse spectroscopic optical coherence tomography (ISOCT) has been implemented in this system to detect the presence of tumor cells at the earliest stages of metastatic cell dissemination, prior to detection of tumor cells in vital organs per our previous investigations7,30. However, optical imaging techniques are highly specialized and not currently available in the clinic for non-ocular applications. Additionally, while the resolution of optical techniques is high, the penetration depth is low 34, 35 such that it is challenging to translate this technology for non-invasive imaging through human skin.

Ultrasound imaging is the most commonly used imaging modality in radiology due to its noninvasive nature and optimal balance of resolution and penetration depth suitable for many clinical applications. However, conventional ultrasound imaging mainly focuses on morphology and limited in its ability to reliably and quantitatively determine tissue status in an objective fashion. Spectral ultrasound imaging (SUSI), a quantitative imaging technique extended from the conventional grayscale B-mode ultrasound imaging, utilizes the raw radiofrequency (RF) backscattered signals in ultrasound imaging to extract quantitative information associated with tissue properties. Power spectra of the RF signals are computed and calibrated to remove system effects. Linear regression of the calibrated spectrum over the frequency bandwidth of imaging is performed to obtain spectral parameters including slope, mid-band fit (MBF), average acoustic scatterer diameter (ASD), and average acoustic concentration (ACC). These spectral characteristics have been previously used to characterize atherosclerotic plaque composition26, detect malignancy in prostate15, pancreas, and lymph node23 in vivo, as well as to non-invasively monitor the development of tissue-engineered constructs in vitro17, 18, 20, 33.

In this study, we demonstrate the use of SUSI to detect changes in biomaterial scaffolds that correspond with spontaneous metastatic tumor cell arrival in vivo. We investigated the changes in SUSI parameters that occur within biomaterial scaffolds from healthy to tumor bearing mice using models of orthotopic human and mouse breast cancer metastasis. We also developed a classification algorithm to classify healthy and tumor bearing mice according to SUSI parameters measured in scaffolds. In addition, we evaluated the biological components of the scaffold in order to identify those that were responsible for of the detected changes in SUSI parameters between tumor bearing and healthy samples. Our results show that the use of SUSI in combination with biomaterial scaffolds that capture metastatic tumor cells may have the potential to create an advantageous, non-invasive platform for identifying metastatic disease at its earliest stages in order to reduce cancer mortality in metastatic disease.

Materials and Methods

Scaffold fabrication and implantation

Microsphere preparation

PCL microspheres were prepared as previously described30. Microspheres were generated via emulsification of a 6% (w/w) PCL solution (Lactel Absorbable Polymers, Birmingham, AL; Inherent viscosity 0.65–0.85 dL/g) in dichloromethane with a 10% (w/v) poly(vinyl alcohol) solution with 1 min of homogenization at 10,000 rpm. Dichloromethane solvent was evaporated via stirring for 3 hours. Following evaporation, microspheres were collected via 2000 × g centrifugation for 10 minutes. Microspheres were washed > 5 times in deionized water to remove excess PVA. Microspheres were ready for use after lyophilization for >48h.

Scaffold fabrication

Microporous scaffolds were prepared by combining prepared PCL microspheres and sodium chloride (crystals 250–425 μm in diameter as isolated via mesh sieves) at a ratio of 1:30 (w/w). The mixture was then pressed in a steel die (2 mm in height and 5 mm in diameter) at 1500 PSI for 45 seconds using a hydraulic press. The pressed disks were then heated on a hotplate at 60°C for 5 minutes per side to melt PCL microparticles around salt crystals in order to form a continuous structure. The salt porogen was subsequently removed by rocking in water for 1.5 hours at room temperature. Prior to implantation in mice, scaffolds were sanitized using 70% ethanol, rinsed with sterile water, and dried on a sterile surface. Leached and sanitized scaffolds were kept at −80°C until use.

Scaffold implantation

Animal studies were performed in accordance with institutional guidelines and protocols approved by the University of Michigan Institutional Animal Care and Use Committee (IACUC, PR000007801). Scaffolds were implanted into the subcutaneous space of 8-week-old female NOD/SCID-IL2Rγ−/− (NSG) or Balb/c mice purchased from Jackson Laboratory (Bar Harbor, ME) as previously described30. A minimum of 4 scaffolds/condition were used for all studies. Sample size for each experiment is described in the figure captions. For the implantation procedure, animals were anesthetized via inhalation of isoflurane (2%), administered Carprofen analgesia (5 mg/kg, s.c. injection), the upper back was shaved and disinfected using a Betadine swab followed by an ethanol swab, and this procedure was repeated 3 times. A fenestrated sterile field was draped over the surgical area and a 1 cm incision was made in the upper back. Following incision, subcutaneous pockets were created perpendicular to the incision to both the left and right, into which sanitized scaffolds were inserted (2 scaffolds/mouse). The skin was then closed using sterile 7 mm wound clips (Reflex, Roboz Surgical Instrument Co, Gaithersburg, MD).

Tumor Inoculation

Tumor inoculations were performed by injection of human 2 × 106 MDA-MB-231BR-tdtomato-luc2 (Northwestern University Developmental Therapeutics Core, Evanston, IL) or mouse 4T1-tdtomato-luc2 (Perkin Elmer, Waltham, MI) cells in 50 μL PBS (Life Technologies, Carlsbad, CA) into the fourth right mammary fat pads of 10-week-old female NSG mice or Balb/c mice respectively (Jackson Laboratory, Bar Harbor, ME). Two cell lines were used to increase the translatability of this work. Human cell lines replicate the biology of human tumor cells but require immunocompromised mice. Mouse cell lines can be used in immunocompetent mice and thus better model the interactions of tumor cells with the immune system. Cell lines were confirmed to be pathogen free and authenticated by short tandem repeat DNA analysis and compared to the ATCC STR profile database (DDC Medical, Fisher Scientific, Hampton, NH).

Ultrasound Imaging

The Vevo 770 system (Visualsonics, Toronto, Canada) with RMV-708 scan head (a 55 MHz center frequency single element transducer, 40 MHz bandwidth, 4.5 mm focal distance, 1.5 mm depth of focus [−6 dB], 30 μm axial resolution, 70 μm lateral resolution) was used to image scaffolds. Scaffolds were imaged with Vevo 770’s B-mode and 3D scan mode for visualization at 100% power, 10 frames per second, 12 mm × 12 mm field of view, with cardiac mode on. For better volume reconstruction, a step size of 32 μm over a total scan distance of 8.99 mm was applied for 3D imaging. Radiofrequency (RF) mode was used to acquire raw RF data for SUSI analysis. RF signals were acquired in 3D mode, with a sweep distance of 8.99 mm and 200 μm step size. Power was reduced to 50% to avoid saturation in the raw signal and 250 lines were captured for each frame at a sampling rate of 419.78 MHz.

Spectral Ultrasound Imaging (SUSI) Analysis

Grayscale (GS) value

For verification, the grayscale values from the RF signals were also generated to form B-mode images which were confirmed to the B-mode images obtained with the regular mode of the scanner. Hilbert transformation was applied to the raw backscattered RF data along each scan line (z) at a given location y to obtain the complex analytical signal p(y,z). Grayscale values were determined as the mean absolute value of the complex analytical signal as:

GS(y,z)=log10|p(y,z)|. (1)

Spectral parameters

For each scan line within an image, segments of RF signals were generated by gating the signals using a sliding hamming window (0.2 μs duration, rounded to 84 samples given a sampling rate of 419.78 MHz, and a 0.1 μs offset or overlap). Power spectra of the RF segments were calculated by taking Fast Fourier Transform (FFT) of the segmented signals, with an FFT length of 256. For calibration and removal of system-dependent factors, the power spectrum of each segment was divided by the power spectrum of the signal from a perfect reflector (oil-water interface) obtained at the same imaging setting. Slope (m) and mid-band fit (MBF) were determined using linear regression to the calibrated power spectrum within a −9 dB bandwidth of the imaging frequency, as described previously17,18.

Acoustic scatter diameter (a), which is the effective size of acoustic scatters in a tissue, was determined from the slope (m), the geometry index (n = 4)18, 25, the center frequency of the calibrated (fc = 37.7 MHz), and the bandwidth of the imaging frequency (b= 39.37 MHz) using:

a=2*0.25n[b(1b24)]b3fc2m105.5fc. (2)

Acoustic concentration (CQ2) was calculated from MBF, scatter diameter (a), and a shape dependent factor (E = 53.25)18, 25:

CQ2=e(0.23(MBFg1ng2(a2)2))Ea2(n1), (3)

where

g1(fc,b)=4.34[ln(fc(1b24)0.5(2+b2b)1b)1], (4)
g2(fc,b)=76.9fc2(3+b24). (5)

Average scatterer diameter (ASD) and average acoustic concentration (AAC) were calculated as the average value of the scatter diameter (a) and acoustic concentration (CQ2) respectively within the chosen region of interest (ROI) in an image. AAC values are given in decibel scale.

Parametric images for spectral parameters

Parametric images for each spectral parameter including m, MDB, ASD, and ACC were generated by color encoding and overlaying the spectral parameter values on the underlying/corresponding grayscale ultrasound images. For analysis, a region of interest in the image was selected, and the average value for each SUSI parameter within the ROI was calculated by averaging all the pixel values within the ROI. In addition, the 95% confidence intervals were calculated for an ROI for each sample and the number and percentage of pixels above or below the 95% confidence interval were determined for each sample. The 95% confidence interval allowed a measurement of the spread of data or heterogeneity of a sample, instead of purely looking at the parameter value alone as previously described31.

RT-qPCR Analysis

Scaffolds were explanted from tumor-free and tumor bearing mice, flash frozen in isopentane, and stored until use. Total RNA was isolated from explanted scaffolds via homogenization in TRIzol® (Thermo Fisher, Waltham, MA). Samples were centrifuged at 10,000 × g to remove non-soluble particles. Total RNA was isolated using Direct-zol™ RNA Miniprep Plus kit (Zymo Research Corp, Irvine, CA). RNA concentration and purity were assessed by light absorbance via NanoDrop 2000c (Thermo Fisher). cDNA was generated from RNA via reverse transcription using SuperScript™ VILOTM cDNA synthesis kit (Thermo Fisher). Taqman® probes were purchased from Thermo Fisher including Mm00495386_m1 (Lox), Mm00439498_m1(Mmp2), Mm00801666_g1 (Col1a1), Mm01210125_m1 (Col4a1), Mm01256744_m1 (Fn1) and reference genes Gapdh, Tbp, Ywhaz, Hmbs, Ubc. RT-qPCR was performed on CFX ConnectTM Real-Time PCR Detection System (Bio-Rad Inc, Hercules, CA) with CFX Manager Software.

Flow Cytometry

Cells were prepared for analysis via flow cytometry as previously described30. Scaffolds were removed from mice, minced using a sterile scalpel blade, digested using Liberase TL (Roche, Basel Switzerland) and strained through a 70 μm filter (Falcon, Corning, NY) to produce a single cell suspension of all cells that infiltrated the biomaterial scaffold from the mouse including fibroblasts, immune cells, and endothelial cells as previously reported. Cells were collected via centrifugation at 500 × g for 5 min. Following isolation of a single cell suspension, cells were blocked using anti-CD16/32 (Biolegend, San Diego, CA) and stained with anti-mouse CD45 AF700 (Biolegend) and anti-mouse CD31 BV421 (Biolegend). Samples were run on MoFlo Astrios Flow Cytometer (Beckman Coulter) using the appropriate lasers and filter sets and data was processed using FlowJo (TreeStar Inc., Ashland, OR).

Decellularization of scaffolds

Scaffolds were decellularized using the method described by Aguado et al.2 In brief, scaffolds were explanted from tumor-free and tumor bearing mice and washed in dPBS, then treated with sequential 30 min washes in 1%, 2% and 3% Triton X-100 (Sigma Aldrich, St Louis, MO), followed by an overnight incubation in 0.1% sodium dodecyl sulfate (SDS) with shaking at 4°C. This treatment was repeated the following day until scaffolds were completely decellularized.

Preparation of collagen gels with isolated cells

The single cell suspensions of scaffold-infiltrating cells (heterogeneous population of immune cells, endothelial cells, fibroblasts) were prepared as described above for flow cytometry. Cells were added to 4 mg/mL collagen I, 5x DMEM solution, Fetal Bovine Serum, and 0.1N NaOH at a ratio of 1:5:2:1:1 resulting in a collagen gel with a final concentration of ~500,000 cells/mL, 1x DMEM, 2 mg/mL collagen I, 10% FBS, and 0.01N NaOH. Gels were plated in a 24 well plate and allowed to crosslink for 30 min at 37°C prior to imaging.

Experimental timeline

A schematic of experimental timeline is shown in Figure S1. For early metastasis detection, SUSI imaging was performed on day 33 after scaffold implantation or day 5 after tumor inoculation implantation. For late stage detection, SUSI was performed on day 43 after scaffold implantation or day 15 after tumor inoculation.

Classification of tumor free and tumor bearing using SUSI parameters

Linear discriminant model and bagged decision tree model, built based on the number of pixels outside of the 95% confidence interval for SUSI parameters, were used for classification of tumor bearing and control mice. In brief, the number of pixels outside of the 95% confidence interval for each SUSI parameter across all experiments was placed into a matrix, which was split into a training cohort (6 tumor free mice and 8 tumor bearing mice) and a testing cohort (14 tumor free and 18 tumor bearing mice). First, unsupervised hierarchical clustering on the dataset was performed using MATLAB function clustergram. Next, a class-based linear discriminant model for the training cohort was generated using fitcdiscr. Similarly, a bagged decision trees model for the training cohort was built using MATLAB function TreeBagger. Each of the models was used to independently predict the testing cohort and the classification results were compared to the known class of each mouse (i.e. tumor free or tumor bearing). The scores for both models were plotted and based on these results the sum of both scores (with a cutoff of 1) was used to determine the receiver operating characteristics (ROC) for each model individually and the combined model using the MATLAB function perfcurve.

Computational and Statistical Analysis

Regions from at least 10 frames/images from each scaffold were identified and quantified by GS value and spectral parameters. All results are presented as mean ± standard error of mean (SEM). Statistical comparisons of parameters between groups were made using Student’s t-test for unpaired samples. A p value of <0.05 was considered statistically significant and indicated by an asterisk (*) on all plots. Further details for statistical tests between groups are included in each figure caption.

Results

SUSI detects changes in mouse-tumor bearing mice with micrometastatic disease relative to tumor free

We first assessed the ability of SUSI to distinguish between tumor bearing mice with micrometastasis and healthy controls. To this end, we used scaffolds from healthy mice as controls and 4T1 breast tumor bearing mice as our experimental condition. Scaffolds were explanted at day 14 following inoculation and imaged ex vivo. Day 14 post tumor inoculation represents a time point at which tumor cells could be present both in the scaffold and in the lung and liver as micrometastases30. Grayscale images with 3D reconstruction and 2D grayscale cross section images of scaffolds for tumor free (control) and tumor bearing conditions (Figure S2A) clearly show the morphology of the explanted scaffolds, which demonstrated no apparent changes in implant structure at the macroscopic scale. On the other hand, parametric images of mid-band fit, slope, ASD, and AAC demonstrate subtle alterations in the parameter values and distribution (Figure 1A). Quantification of the parameters did not show changes in average grayscale and midband fit values in the whole samples, but demonstrated clear distinctions between scaffolds from tumor free and tumor bearing mice at the micrometastatic stage (Figure 1B) in other SUSI parameters, with significantly lower slope for tumor bearing scaffolds compared to control (0.103 ± 0.076 for control and 0.0583 ± 0.078 dB/MHz for tumor bearing, n = 6, p=0.001). Average acoustic scatter diameter (ASD: 20.82 ± 1.88 μm for control and 21.90 ± 1.83 μm for tumor bearing, p=0.001) and average acoustic concentration (AAC: 42.77 ± 3.02 dB[mm−3] for control and 44.41 ± 3.17 dB[mm−3] for tumor bearing, p=0.002) were both higher for tumor bearing relative to tumor free condition.

Figure 1: SUSI detects changes in scaffolds from mouse-tumor bearing mice with micrometastasis relative to tumor-free.

Figure 1:

(A) Grayscale images of scaffolds from tumor free and tumor bearing mice overlaid with parameter values for mid-band fit, slope, ASD, and AAC. (B) SUSI parameters for control and tumor bearing scaffolds including grayscale, mid-band fit, slope, ASD, and AAC. (C) The number of pixels lower than the lower bound of the 95% confidence interval of the sample for each SUSI parameter. N = 6 scaffolds/condition. Error bars s.e.m. *p<0.05 via two-sided t-test. Scale bar indicates 1 mm.

In addition to these changes of the average tissue SUSI parameter 9, we assessed changes in sample homogeneity using the number of pixels outside of the sample 95% confidence interval for each SUSI parameter obtained from the whole scaffolds explanted from control and tumor bearing mice (Figure 1C). Taking into account the spatial homogeneity of the parameters as observed in the images, this approach could offer a unique metric for the evaluation of scaffolds explanted from the mice because of their intrinsic multicomponent nature as cells entered and colonized the biomaterial scaffolds after implantation. Interestingly, this approach resulted in more consistent trend in the differences between tumor bearing and control scaffolds. For all parameters, the number of pixels outside of the 95% confidence interval was lower in tumor bearing mice, including grayscale (23017 ± 8772 pixels for control and 20210 ± 6859 pixels for tumor bearing scaffolds, p=0.04) and mid-band fit (426 ± 197 pixels for control and 369 ± 142 pixels for tumor bearing scaffolds, p=0.05) which were not significantly different by the average parameter value itself (Figure 1B). These results indicate a more homogeneous property for the scaffolds in tumor bearing mice compared to those in control mice, suggesting changes within the scaffolds implanted in tumor bearing mice for 14 days relative to control.

SUSI detects changes in mouse- and human-tumor bearing mice relative to tumor free at early stages of metastatic disease

We next investigated the ability of SUSI to identify changes in scaffolds at the early stages of metastasis. We chose day 5 following inoculation of breast adenocarcinoma cells, as this time point is associated with occasional tumor cells in the scaffold yet not metastasis to vital organs7, 30. We also tested both mouse 4T1 and human MDA-MB-231-BR tumor cells in order to investigate the ability of SUSI to identify changes in both immunocompetent mice with mouse breast cancer cells, which better models the contributions of the immune system to metastasis, and an immunocompromised mouse with human cancer cells, which better model behavior of human breast cancer cells. Similar to the tumor bearing mice with micrometastasis at day 14, we found no visible macroscopic alterations in implant architecture in 2D images of scaffolds in mice bearing either human (231-BR) or mouse (4T1) tumors compared to their tumor free controls (Figure S2B,C), but minor alterations were observed in SUSI parametric images (Figure 2A,C). Here, we focused on assessing the homogeneity of the samples using the number of pixels outside of each sample’s 95% confidence interval for each SUSI parameter. Again, similar to tumor bearing mice with micrometastasis at day 14, we found the number of pixels outside of the sample 95% confidence interval to be significantly lower in scaffolds from tumor bearing mice relative to control for both human and mouse breast tumors (Figure 2B, D). For all SUSI parameters, the number of pixels outside the 95% confidence interval was significantly decreased in scaffolds from mice bearing day 5 human 231-BR tumors relative to control. For mice bearing mouse 4T1 tumors, the number of pixels for all SUSI parameters except for AAC (624 ± 234 pixels for control and 561 ± 207 pixels for tumor bearing) were significantly decreased (see figure caption for p-values) for tumor bearing scaffolds compared to control. Collectively, these results demonstrate the capacity of SUSI to identify differences between scaffolds from tumor bearing and tumor free mice at early stages of disease.

Figure 2: SUSI detects changes in mouse- and human-tumor bearing mice relative to tumor-free at early stages of metastasis.

Figure 2:

(A) Grayscale images of scaffolds taken from control and 231-BR tumor bearing mice at day 5 post-inoculation overlaid with parameter values for mid-band fit, slope, ASD and AAC. (B) The number of pixels lower than the 95% confidence interval for each sample and each SUSI parameter (grayscale p=0.001, mid-band fit p=0.002, slope p=0.001, ASD p=0.001, and AAC p=0.003) for control and day 5 231-BR tumor bearing scaffolds. (C) Grayscale images of scaffolds taken from control and 4T1 tumor bearing mice at day 5 post-inoculation overlaid with parameter values for mid-band fit, slope, ASD and AAC. (D) The number of pixels lower than the 95% confidence interval for each sample and each SUSI parameter (grayscale p=0.03, mid-band fit p=0.03, slope p=0.02, ASD p=0.04, and AAC p=0.06) for control and day 5 4T1 tumor bearing scaffolds. N = 4 scaffolds for control and N = 6 scaffolds for tumor bearing for NSG mice bearing 231-BR tumors. N=6 scaffolds/condition for balb/c mice bearing 4T1 tumors. Error bars s.e.m. *p<0.05 via two-sided t-test. Scale bar indicates 1 mm.

SUSI detects changes in cellular composition within scaffolds with tumor progression and metastasis

We next sought to identify the biological changes that might be responsible for the changes detected by SUSI in the scaffolds. We first investigated changes in extracellular matrix (ECM) associated genes as these are likely to be observed by SUSI 17, 18, 20, 33 and may also be altered in the natural pre-metastatic niche1, 2, 21, 29. The expression of Col1a1 (p=0.86), Col4a1 (p=0.73), Fn1 (p=0.38), Lox (p=0.74), and Mmp2 (p=0.78) in the whole scaffold samples exhibited no significant differences between control and 4T1 tumor bearing scaffolds at day 7 post inoculation (Figure 3A), suggesting the changes detected by SUSI at the early stage of day 5 were not related to ECM within the scaffolds.

Figure 3: SUSI detects changes in cellular composition with tumor progression and metastasis.

Figure 3:

(A) qRT-PCR data showing normalized gene expression for ECM associated genes including Col1a1, Col4a1, Fn1, Lox, and Mmp2 for scaffolds from control and day 15 4T1 tumor bearing mice (N = 3 scaffolds/condition). (B) Flow cytometric analysis of CD31+CD45− endothelial cells in scaffolds from control and day 15 4T1 tumor bearing mice (N = 4 scaffolds/condition). (C) SUSI analysis of decellularized scaffolds taken from control and day 5 4T1 tumor bearing mice showing fold change from control of the number of pixels under 95% confidence interval for each sample for each SUSI parameter including grayscale, mid-band fit (MBF), ASD, and AAC (N ≥ 4 scaffolds/condition). (D) SUSI analysis of scaffold-derived cells in a collagen gel taken from control and day 5 4T1 tumor bearing mice showing fold change from control of the number of pixels under 95% confidence interval for each sample and each SUSI parameter including grayscale, MBF, ASD, and AAC (N ≥ 4 scaffolds/condition). (E) SUSI analysis of spleen-derived cells in a collagen gel taken from control and day 5 4T1 tumor bearing mice showing fold change from control of the number of pixels under 95% confidence interval for each sample and each SUSI parameter including grayscale, MBF, ASD, and AAC (N ≥ 4 scaffolds/condition). For all plots error bars s.e.m. and *p<0.05 via two-sided t-test.

We next investigated changes in cell populations within the scaffolds. Previous work on scaffolds in tumor bearing mice has identified major changes in immune cell populations as a result of tumor development and metastatic progression 14, 7, 30, however other stromal populations had not been investigated. Therefore, we performed flow cytometry to determine the numbers of CD31+ endothelial cells present in each scaffold (Figure 3B). We found that a significantly increased number of CD31+CD45− endothelial cells were present in the scaffolds from 4T1 tumor bearing mice at day 5 relative to control scaffolds (1237 ± 252 cells for control and 1909 ± 385 cells for tumor bearing scaffolds, p=0.0267), suggesting that increase in the number of CD31+CD45− endothelial cells in scaffolds may be responsible for the increased homogeneity detected by SUSI. Previous work has identified alterations in immune cell populations due to tumor inoculation in both implanted scaffolds and spleens7, 30, and our results are consistent with that finding.

We next sought to determine if the changes detected by SUSI would be recapitulated in decellularized scaffolds (Figure 3C) or collagen gels loaded with cells isolated from scaffolds (Figure 3D) or spleen harvested from mice (Figure 3E). In the decellularized scaffolds (Figure 3C), no statistically significant differences between control and day 5 4T1 tumor bearing scaffolds were observed, suggesting minimal role of the acellular biomaterial scaffold itself in the changes detected by SUSI (grayscale p=0.659, MBF p=0.674, slope p=0.162, ASD p=0.259, AAC p=0.219).

For collagen gels loaded with scaffold-derived cells, the number of pixels outside of the 95% confidence interval decreased for all parameters in tumor bearing relative to control (Figure 3D), consistent with the whole scaffolds. However, differences between control and collagen gels with tumor bearing scaffold-derived cells fell short of statistical significance (grayscale p=0.333, MBF p=0.289, slope p=0.291, ASD p=0.183, AAC p=0.289), suggesting that deposition of cells isolated from the scaffolds in a collagen gel construct may not completely recapitulate the complexity of the whole explanted biomaterial scaffolds.

Previous work has identified that alterations of immune cell populations due to tumor inoculation are similar in scaffolds and spleens, with more changes in spleens7,30. Thus, we also compared scaffold-derived cells in a collagen gel to spleen-derived cells in a collagen gel in this study. Interestingly, a consistent trend of changes was observed in collage gels with spleen-derived cells where a statistically significant lower number of pixels outside of the 95% confidence interval for all SUSI parameters was detected (Figure 3E), suggesting that tissue of the spleen became more homogeneous and that indeed SUSI was able to detect the larger changes that occurred in the spleen due to tumor inoculation (grayscale p=0.001, MBF p=0.002, slope p=0.001, ASD p=0.001, AAC p=0.002). In addition, the changes in the spleen may be reflective of the effect observed in the scaffold as we have previously reported30. Representative parametric images for scaffold cells in collagen gel, decellularized scaffolds, and spleen cells in collagen gel can be found in Figure S3.

SUSI parameters classify tumor free and tumor bearing mice with good sensitivity and specificity

We next built classification models to discriminate between tumor bearing and control mice based on a consistent decrease in the number of pixels outside of the 95% confidence interval for SUSI parameters observed in tumor bearing scaffolds compared to control. A schematic of the method used to develop classification models is shown in Figure 4A. A heatmap with unsupervised hierarchical clustering across samples showed the number of pixels outside of the 95% confidence interval for each parameter and each sample in the test cohort (Figure 4B). Using unsupervised hierarchical clustering (lines above heatmap), most tumor free control samples (TF) cluster on the left and most tumor bearing samples (TB) cluster on the right of the heatmap. However, the number of trees generated in the clustering indicates that the samples are heterogeneous in each classification and that a simple cutoff value for each parameter cannot suffice to predict tumor status. Thus, a linear discriminant model and bagged decision trees model were created to classify the test cohort based on the training cohort. Each model gave each sample a score from 0 to 1 based on the likelihood of being tumor free (0) or tumor bearing (1) and these scores are plotted in Figure 4C, demonstrating good separation in scores of tumor free and tumor bearing scaffolds. A receiver operating characteristics (ROC) curve was created for each model individually as well as the linear addition of both models, with any score greater than 1 classified as tumor bearing (Figure 4D). The area under the curve (AUC) for the combined model was found to outperform each model alone (Combined AUC: 0.92, Linear Discriminant Analysis AUC: 0.88, and Bagged Decision Trees AUC: 0.87).

Figure 4: SUSI parameters are able to classify tumor free and tumor bearing mice with good sensitivity and specificity.

Figure 4:

(A) Schematic of method used to classify samples as tumor free (TF) or tumor bearing (TB). A training cohort of n=6 TF and n=8 TB mice were used to build a linear discriminant model and a bagged decision trees model. The test cohort (n=14 TF and n=18 TB) were tested using the models for classification. (B) Heatmap with unsupervised hierarchical clustering of test cohort data normalized across each parameter. (C) Classification of test cohort data and score given by each model indicating prediction of either TF status (score of 0) or TB status (score of 1). (D) Receiver Operating Characteristic (ROC) curve for tumor status classification showing the classification accuracy for the combined score including both bagged decision trees and linear discriminant analysis models and each model alone

Discussion and conclusion

Metastasis is responsible for 90% of cancer-related deaths, in part because no clinical strategies currently detect metastasis prior to the compromise of organ function. In this study, we investigated whether SUSI may be used to detect early metastasis in a biomaterial scaffolds using breast cancer mice models. Our results show that the number of pixels outside of the sample 95% confidence interval for all SUSI parameters was lower in tumor bearing scaffolds, suggesting increased sample heterogeneity due to metastasis. This result was consistent with the changes observed previously in tumor cell arrival and immune/stroma alteration with the development of the pre-metastatic niche and transition to metastasis in implanted scaffolds7, 30. We have also successfully detected changes associated with metastasis in both models using ISOCT7, 30. Interestingly, the changes detected by SUSI appear to exhibit progression from early metastasis to micrometastatic disease. For 4T1-inoculated mice, all parameters are significantly altered between tumor free and tumor bearing at day 15 whereas at day 5 only 4/5 parameters were significantly altered. Such progression with metastasis over time has not been observed before.

We found cell population alterations within the scaffolds from tumor-bearing mice compared to control to be associated with the observed alterations in SUSI parameters. While changes in ECM are associated with the pre-metastatic niche 14, 28, 29 and metastasis to implanted biomaterial scaffolds2, we did not find significant alterations in ECM associated genes by qRT-PCR, possibly due to large animal to animal variability in mRNA content of ECM associated genes. In addition, SUSI results of decellularized scaffolds showed no changes from tumor free to tumor bearing scaffolds. Instead, we found a significant increase in the number of endothelial cells in the scaffolds, which are also associated with metastasis and the pre-metastatic niche19, 21. We have previously discovered a number of other significant changes in immune cells at this time point7, 30, which may also be part of the cellular changes SUSI detects between tumor free and tumor bearing scaffolds. Based on the results reported here, we conclude that SUSI detects the altered foreign body response in scaffolds from tumor-bearing mice compared to tumor-free mice. This result is potentially advantageous because we are not detecting the presence of tumor cells themselves, but the changes to the microenvironment that allow tumor cells to arrive at the biomaterial. While these alterations could be measured via flow cytometry for surface markers, we envision that SUSI monitoring of scaffolds would allow for a non-invasive indication that the scaffold microenvironment is changing, followed by biopsy for confirmation in a clinical setting.

Finally, classification models based on the SUSI data classified tumor bearing and tumor free mice with good sensitivity and specificity, suggesting the utility of SUSI of implanted biomaterial scaffold as a promise strategy for non-invasive early detection of metastasis. In practice, the scaffold may be non-invasively monitored using SUSI to identify changes indicative of metastasis risk and then the scaffold explanted and analyzed via histology, RT-qPCR, or other techniques to validate the arrival and presence of immune cells (myeloid derived suppressor cells etc. 7, 30) and gene signatures (unpublished data, Robert S. Oakes) associated with metastasis as well as directly quantifying the arrival of tumor cells. We envision this scaffold platform and imaging technology being used in patients who have received a diagnosis of invasive breast cancer, completed chemotherapy and surgery, and are deemed cancer-free but are at risk for metastatic recurrence. Implantation of the scaffold under the arm, near the axilla and subsequent non-invasive monitoring of a scaffold implant is complementary and potentially advantageous over liquid biopsy approaches as the scaffold identifies the risk of metastasis based on presence of metastatic niche forming cells and tumor cells that have metastasized to a tissue and are similar to the metastatic lung10.

To our knowledge, this is the first time SUSI has been used to monitor metastasis development in an engineered metastatic site. While serial imaging was not performed at more time points, we performed analysis at two important individual time points including one early in metastasis and one when micrometastasis has occurred in the lung. Importantly, demonstration of SUSI to detect metastasis at the earliest stages may have significant potential.

In conclusion, we demonstrate the utility of SUSI to detect changes associated with metastasis to biomaterial scaffolds in both syngeneic mouse and xenogeneic human models of spontaneous metastasis before tumor cells are detectable in vital organs. SUSI imaging of biomaterial scaffolds may provide an advantageous platform in the early and non-invasive detection of metastasis, with the potential to lead to early intervention and reduced mortality from cancer metastasis.

Supplementary Material

10439_2019_2366_MOESM1_ESM

Acknowledgements

Thanks to the NIH for their support through 5R01CA173745-06 and R01DE026630. G.G.B. is a recipient of the NSF Graduate Research Fellowship and NRSA F31 CA224982-01. G.G.B., X.H., S.S.R., J.S.J., C.X.D., and L.D.S. conceived the presented hypotheses and experimental designs. G.G.B and X.H. carried out experiments with support from S.S.R, R.M.H, Y.Z., and R.S.O, and completed computational analysis with support from R.S.O. G.G.B., X.H., and L.D.S. wrote the manuscript with support from J.S.J. and C.X.D. L.D.S. supervised the project with support from J.S.J. and C.X.D. All authors discussed the results and reviewed the final manuscript.

Grant Support: The authors acknowledge support from the National Institutes of Health NIH-Director’s Transformative Research Award-R01CA173745 and R01CA214384 (to L.D. Shea and J.S. Jeruss) and NIH R01DE026630 (to C. X. Deng and J. P. Stegemann). G.G. Bushnell is a recipient of the NSF Graduate Research Fellowship and NRSA F31 CA224982-01.

Competing Interests: The scaffold as a platform for metastasis detection is described in a current patent application US20170281798A1 by assignee Northwestern University with inventors Lonnie D. Shea, Samira M. Azarin, Robert M. Gower, Jacqueline S. Jeruss and US2017012556 by assignee Regents of the University of Michigan with inventors Lonnie D. Shea, Shreyas S. Rao, Samira M. Azarin, Jacqueline S. Jeruss, and Grace G. Bushnell.

Footnotes

Data Availability Statement:

Data associated with this manuscript is available through a private repository with password “Bushnell-Hong” at the following linkhttps://1drv.ms/u/s!AmVnzTmi4yrLhI1lf215PCMrJzK7WA?e=S4xYlg. After publication data will be made publicly available.

Publisher's Disclaimer: This Author Accepted Manuscript is a PDF file of an unedited peer-reviewed manuscript that has been accepted for publication but has not been copyedited or corrected. The official version of record that is published in the journal is kept up to date and so may therefore differ from this version.

References

  • 1.Aguado BA, Bushnell GG, Rao SS, Jeruss JS and Shea LD. Engineering the pre-metastatic niche. Nature biomedical engineering 1: 0077, 2017. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Aguado BA, Caffe JR, Nanavati D, Rao SS, Bushnell GG, Azarin SM and Shea LD. Extracellular matrix mediators of metastatic cell colonization characterized using scaffold mimics of the pre-metastatic niche. Acta Biomaterialia 33: 13–24, 2016. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Aguado BA, Hartfield RM, Bushnell GG, Decker JT, Azarin SM, Nanavati D, Schipma MJ, Rao SS, Oakes RS and Zhang Y. A synthetic pre-metastatic niche mimic alters the primary tumor and tumor microenvironment. Advanced Healthcare Materials 1700903, 2018. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Aguado BA, Wu JJ, Azarin SM, Nanavati D, Rao SS, Bushnell GG, Medicherla CB and Shea LD. Secretome identification of immune cell factors mediating metastatic cell homing. Scientific reports 5: 17566, 2015. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Aguado BA, Wu JJ, Azarin SM, Nanavati D, Rao SS, Bushnell GG, Medicherla CB and Shea LD. Secretome identification of immune cell factors mediating metastatic cell homing. Scientific reports 5: 2015. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Alix-Panabieres C and Pantel K. Clinical applications of circulating tumor cells and circulating tumor DNA as liquid biopsy. Cancer discovery 2016. [DOI] [PubMed] [Google Scholar]
  • 7.Azarin SM, Yi J, Gower RM, Aguado BA, Sullivan ME, Goodman AG, Jiang EJ, Rao SS, Ren Y and Tucker SL. In vivo capture and label-free detection of early metastatic cells. Nat Commun 6: 8094, 2015. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Bersani F, Lee J, Yu M, Morris R, Desai R, Ramaswamy S, Toner M, Haber DA and Parekkadan B. Bioengineered implantable scaffolds as a tool to study stromal-derived factors in metastatic cancer models. Cancer Research 2014. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Bianchi MT, Phillips AJK, Wang W and Klerman EB. Statistics for Sleep and Biological Rhythms Research:From Distributions and Displays to Correlation and Causation. Journal of Biological Rhythms 32: 7–17, 2017. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Bushnell GG, Hardas TP, Hartfield RM, Zhang Y, Oakes RS, Ronquist S, Chen H, Rajapakse I, Wicha MS, Jeruss JS and Shea LD. Biomaterial scaffolds recruit an aggressive population of metastatic tumor cells in vivo. Cancer Research canres.25022018, 2019. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Chaffer CL and Weinberg RA. A perspective on cancer cell metastasis. Science 331: 1559–1564, 2011. [DOI] [PubMed] [Google Scholar]
  • 12.de la Fuente A, Alonso-Alconada L, Costa C, Cueva J, Garcia-Caballero T, Lopez-Lopez R and Abal M. M-trap: exosome-based capture of tumor cells as a new technology in peritoneal metastasis. JNCI: Journal of the National Cancer Institute 107: 2015. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Erdi YE Limits of tumor detectability in nuclear medicine and PET. Molecular imaging and radionuclide therapy 21: 23, 2012. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Erler JT, Bennewith KL, Cox TR, Lang G, Bird D, Koong A, Le Q-T and Giaccia AJ. Hypoxia-induced lysyl oxidase is a critical mediator of bone marrow cell recruitment to form the premetastatic niche. Cancer cell 15: 35–44, 2009. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Feleppa EJ, Kalisz A, Sokil-Melgar JB, Lizzi FL, Liu T, Rosado AL, Shao MC, Fair WR, Wang Y and Cookson MS. Typing of prostate tissue by ultrasonic spectrum analysis. IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control 43: 609–619, 1996. [Google Scholar]
  • 16.Fidler IJ Metastasis: quantitative analysis of distribution and fate of tumor emboli labeled with 125I-5-iodo-2-deoxyuridine. Journal of the National Cancer Institute 45: 773–782, 1970. [PubMed] [Google Scholar]
  • 17.Gudur M, Rao RR, Hsiao Y-S, Peterson AW, Deng CX and Stegemann JP. Noninvasive, quantitative, spatiotemporal characterization of mineralization in three-dimensional collagen hydrogels using high-resolution spectral ultrasound imaging. Tissue Engineering Part C: Methods 18: 935–946, 2012. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Gudur MSR, Rao RR, Peterson AW, Caldwell DJ, Stegemann JP and Deng CX. Noninvasive quantification of in vitro osteoblastic differentiation in 3D engineered tissue constructs using spectral ultrasound imaging. Plos One 9: e85749, 2014. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Hiratsuka S, Watanabe A, Sakurai Y, Akashi-Takamura S, Ishibashi S, Miyake K, Shibuya M, Akira S, Aburatani H and Maru Y. The S100A8-serum amyloid A3-TLR4 paracrine cascade establishes a pre-metastatic phase. Nature cell biology 10: 1349, 2008. [DOI] [PubMed] [Google Scholar]
  • 20.Hong X, Stegemann JP and Deng CX. Microscale characterization of the viscoelastic properties of hydrogel biomaterials using dual-mode ultrasound elastography. Biomaterials 88: 12–24, 2016. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Kaplan RN, Riba RD, Zacharoulis S, Bramley AH, Vincent L, Costa C, MacDonald DD, Jin DK, Shido K and Kerns SA. VEGFR1-positive haematopoietic bone marrow progenitors initiate the pre-metastatic niche. Nature 438: 820–827, 2005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Ko C-Y, Wu L, Nair AM, Tsai Y-T, Lin VK and Tang L. The use of chemokine-releasing tissue engineering scaffolds in a model of inflammatory response-mediated melanoma cancer metastasis. Biomaterials 33: 876–885, 2012. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Kumon RE, Pollack MJ, Faulx AL, Olowe K, Farooq FT, Chen VK, Zhou Y, Wong RC, Isenberg GA and Sivak MV. In vivo characterization of pancreatic and lymph node tissue by using EUS spectrum analysis: a validation study. Gastrointestinal endoscopy 71: 53–63, 2010. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Lee J, Li M, Milwid J, Dunham J, Vinegoni C, Gorbatov R, Iwamoto Y, Wang F, Shen K and Hatfield K. Implantable microenvironments to attract hematopoietic stem/cancer cells. Proceedings of the National Academy of Sciences 109: 19638–19643, 2012. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Lizzi FL, Ostromogilsky M, Feleppa EJ, Rorke MC and Yaremko MM. Relationship of ultrasonic spectral parameters to features of tissue microstructure. IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control 34: 319–329, 1987. [DOI] [PubMed] [Google Scholar]
  • 26.Nair A, Kuban BD, Obuchowski N and Vince DG. Assessing spectral algorithms to predict atherosclerotic plaque composition with normalized and raw intravascular ultrasound data. Ultrasound in medicine & biology 27: 1319–1331, 2001. [DOI] [PubMed] [Google Scholar]
  • 27.Nakagawa S and Cuthill IC. Effect size, confidence interval and statistical significance: a practical guide for biologists. Biological Reviews 2007. [DOI] [PubMed] [Google Scholar]
  • 28.Peinado H, Lavotshkin S and Lyden D. The secreted factors responsible for pre-metastatic niche formation: old sayings and new thoughts In: Semin Cancer BiolElsevier, 2011, p. 139–146. [DOI] [PubMed] [Google Scholar]
  • 29.Peinado H, Zhang H, Matei IR, Costa-Silva B, Hoshino A, Rodrigues G, Psaila B, Kaplan RN, Bromberg JF and Kang Y. Pre-metastatic niches: organ-specific homes for metastases. Nature Reviews Cancer 17: 302, 2017. [DOI] [PubMed] [Google Scholar]
  • 30.Rao SS, Bushnell GG, Azarin SM, Spicer G, Aguado BA, Stoehr JR, Jiang EJ, Backman V, Shea LD and Jeruss JS. Enhanced survival with implantable scaffolds that capture metastatic breast cancer cells in vivo. Cancer Research 76: 5209–5218, 2016. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Røraas T, Petersen PH and Sandberg S. Confidence intervals and power calculations for within-person biological variation: effect of analytical imprecision, number of replicates, number of samples, and number of individuals. Clinical chemistry 58: 1306–1313, 2012. [DOI] [PubMed] [Google Scholar]
  • 32.Stathopoulou A, Vlachonikolis I, Mavroudis D, Perraki M, Kouroussis C, Apostolaki S, Malamos N, Kakolyris S, Kotsakis A and Xenidis N. Molecular detection of cytokeratin-19–positive cells in the peripheral blood of patients with operable breast cancer: Evaluation of their prognostic significance. Journal of clinical oncology 20: 3404–3412, 2002. [DOI] [PubMed] [Google Scholar]
  • 33.Stukel JM, Goss M, Zhou H, Zhou W, Willits RK and Exner AA. Development of a High-Throughput Ultrasound Technique for the Analysis of Tissue Engineering Constructs. Annals of biomedical engineering 44: 793–802, 2016. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34.Yi J and Backman V. Imaging a full set of optical scattering properties of biological tissue by inverse spectroscopic optical coherence tomography. Opt Lett 37: 4443–4445, 2012. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.Yi J, Radosevich AJ, Rogers JD, Norris SC, Çapoğlu i. R., Taflove A and Backman V. Can OCT be sensitive to nanoscale structural alterations in biological tissue? Optics express 21: 9043–9059, 2013. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

10439_2019_2366_MOESM1_ESM

RESOURCES