Microfluidics-based patient-derived disease detection tool for deep learning-assisted precision medicine

Abstract

Cancer spatial and temporal heterogeneity fuels resistance to therapies. To realize the routine assessment of cancer prognosis and treatment, we demonstrate the development of an Intelligent Disease Detection Tool (IDDT), a microfluidic-based tumor model integrated with deep learning-assisted algorithmic analysis. IDDT was clinically validated with liquid blood biopsy samples (n = 71) from patients with various types of cancers (e.g., breast, gastric, and lung cancer) and healthy donors, requiring low sample volume (∼200 μl) and a high-throughput 3D tumor culturing system (∼300 tumor clusters). To support automated algorithmic analysis, intelligent decision-making, and precise segmentation, we designed and developed an integrative deep neural network, which includes Mask Region-Based Convolutional Neural Network (Mask R-CNN), vision transformer, and Segment Anything Model (SAM). Our approach significantly reduces the manual labeling time by up to 90% with a high mean Intersection Over Union (mIoU) of 0.902 and immediate results (<2 s per image) for clinical cohort classification. The IDDT can accurately stratify healthy donors (n = 12) and cancer patients (n = 55) within their respective treatment cycle and cancer stage, resulting in high precision (∼99.3%) and high sensitivity (∼98%). We envision that our patient-centric IDDT provides an intelligent, label-free, and cost-effective approach to help clinicians make precise medical decisions and tailor treatment strategies for each patient.

INTRODUCTION

Cancer is a complex and diverse disease, and patients within a specific subtype can respond differently to treatments and drugs due to various factors, including tumor heterogeneity.¹ This heterogeneity is characterized by spatial, temporal, genomic, and proteomic profile differences, challenging cancer eradication and resulting in lower survival rates and increased treatment costs. The conventional diagnostic approach of solid tumor biopsy is now obsolete due to its drawbacks, including invasiveness, patient discomfort, and the inability to provide real-time information on the tumor.

Liquid biopsy, which evaluates circulating biomarkers in peripheral biofluids for tumor diagnosis, prognosis, and monitoring, has the potential to revolutionize clinical oncology by offering a less invasive alternative.² Circulating tumor cells (CTCs), circulating tumor DNA, and exosomes are among the circulating biomarkers widely studied for liquid biopsy-based clinical oncology.³ However, next-generation liquid biopsy, which relies on the deep and broad sequencing of circulating biomarkers, requires data science-based analysis for comprehensive profiling. Due to the heterogeneity in clinical presentation, analysis based on several multiplex parameters may not fully capture the diverse tumor information from clinical samples, and variations in the sample and imaging can affect precision.

Artificial intelligence (AI) has emerged as a promising tool for data-driven analytics due to the successful development of high-performance computing capabilities, such as central and graphics processing units.⁴ Artificial intelligence-related technologies such as machine learning, artificial neural networks, and deep learning have recently flourished and penetrated many fields, including disease and healthcare management. AI has succeeded in cancer,^5,6 cardiovascular disease,^7,8 and chronic disease management⁹ by utilizing multi-dimensional data from clinicians and biologists.^10,11

Here, we developed an Intelligent Diseases Detection Tool (IDDT). This automated and precise tool can detect patient-derived cell clusters in liquid biopsy samples, specifically for gastric and breast cancer. By incorporating patient-derived circulating tumor cells (CTCs) and tumor-associated immune cells into 3D cell clusters, the IDDT provides a more accurate representation of the complex in vivo tumor microenvironment. Due to the inherent heterogeneity of clinical samples, characterizing the cultured cell clusters with multiple specific parameters posed a significant analytical challenge.

To enhance the presentation of clinical outcomes, we employ a novel deep learning-assisted methodology for the precise and automated analysis of a cancer prediction score. This score reflects patient prognosis, more accurately delineating specific treatment cycles and cancer stages.

IDDT uses a novel deep learning-assisted approach to precisely and automatically analyze a cancer prediction score, which reflects patient prognosis levels with specific treatment cycles and cancer stages.^2,12 The IDDT can distinguish healthy individuals accurately (n = 12) from cancer patients (n = 55) with high sensitivity (99%) and specificity (97.75 ± 1.30%) using a vision transformer. The training and validation of the IDDT were performed using liquid biopsy samples collected from cancer patients (n = 55), which were divided into training, validation, and testing datasets (6:2:2). By using small volumes of clinical liquid blood biopsy (∼200 μl per channel) and with a rapid (∼2 weeks; 1 treatment cycle) turnaround time, the IDDT enables the formation of patient-derived cell clusters that correlate with patient prognosis.

Using a vision transformer, the IDDT has demonstrated high accuracy, sensitivity, and specificity in distinguishing healthy individuals from cancer patients. This technology can reduce healthcare workloads and automatically generate precise decisions for individual patients. We envision the microsystem-based deep learning IDDT as potentially invaluable in developing patient-derived cell clusters, accurately stratifying patient cohorts, and paving the way for the next generation of patient-centric healthcare.

RESULTS

Development of an integrated approach for Intelligent Disease Detection Tool

To meet the growing demand for a highly precise assessment tool for disease detection, we aimed to develop an integrated and efficient approach termed the Intelligent Disease Detection Tool (IDDT). The IDDT combines a tumor model with a deep learning algorithm to detect patient-derived cell clusters from liquid biopsies. Due to the label-free process based on the proliferative capabilities of cancer cells, the IDDT demonstrates broad applicability in various cancer types while maintaining valuable information on tumor heterogeneity, disease progression, and response to therapy.

We used a microfluidic biochip of two layers to establish a patient-derived tumor model. The top layer was a barrier to retaining fluidics and solutions while preventing channel contamination. The bottom layer consisted of a tapered microwell array, allowing comprehensive 3D interaction between immune and cancer cells [Figs. 1(a)–1(c)]. Each microwell had a diameter of 250 × 150 × 150 μm³ (length × width × depth) and was fabricated using a detailed process described in a previous report.¹³ We employed phase-contrast microscopy to visualize the morphology of cell clusters obtained from patients. A specialized deep learning algorithm, Mask R-CNN, was tailored to our needs. This algorithm is capable of pinpointing regions of interest (RoIs), classifying the phenotype of these patient-derived cell clusters, and making accurate assessments about patient conditions, including their treatment cycles and cancer stages. Following extended cultivation in a hypoxic environment, the circulating tumor cells (CTCs) within the microwells showed persistent proliferation, ultimately resulting in cell aggregation and adhesion.

FIG. 1.

A microfluidics-based tumor model combined with deep learning for developing an Intelligent Disease Detection Tool (IDDT). (a) Schematic illustration of the IDDT workflow. (1) Whole blood samples were collected from patients’ veins. (2) Red blood cells were removed by lysing, and the remaining cell deposits were seeded into the IDDT for culturing. (3) After culturing for ∼2 weeks, the patient-derived tumor model was established with ∼300 cell clusters for each patient. (4) The label-free phase-contrast microscopy images of the patient-derived cell clusters were captured and stored for analysis. (5) Customized deep neural networks were developed to identify the regions of interest and stratification of the healthy and patient cohorts. (6) Regions of interest (i.e., microwells) were identified successfully. (7) Healthy donors and cancer patients were distinguished, and patients were stratified with a specific treatment cycle and cancer stage with high accuracy. (b) Details of the microfluidics-based patient-derived tumor model. Schematic diagrams of patient-derived tumor model with eight culturing channels comprised of two PDMS layers: (i) the top barrier layer and (ii) the bottom tapered microwell layer. The representative microscopy image of the tapered microwell array is shown. Scale bar, 100 μm. Each microwell’s length, width, and depth are 250, 150, and 150 μm, respectively. (c) The photo of the patient-derived tumor model-assisted IDDT. Scale bar, 1 cm. (d) Representative phase-contrast microscopy images of the patent-derived tumor model. Scale bar, 50 μm.

Meanwhile, other nucleated cell constituents in the blood gradually diminish. After the 14-day culture period, cell clusters derived from patients manifested heightened compactness and adhesion, reflecting the patient response. In contrast, samples from healthy individuals predominantly exhibited cellular debris in the resultant cell culture [Fig. 1(d)].

The IDDT approach combines a patient-derived 3D tumor model system based on a microfluidic platform with deep learning-assisted algorithmic analysis to enable the cultivation of patient-derived tumors, in situ imaging, detection, analysis, and decision-making. The IDDT process comprises seven steps [Fig. 1(a)]: (1) blood collection, (2) red blood cell (RBC) lysis, (3) tumor model establishment, (4) microscopy image capture, (5) deep neural network establishment, (6) identification of regions of interest, and (7) clinical decision-making. These steps allow for the rapid and accurate identification of patient-derived cell clusters and the correlation of these clusters with patients’ conditions, including treatment cycles and cancer stages.

The process for utilizing the IDDT involves several steps. First, a dataset must be prepared, followed by the training of the model. Two training sets were created for each magnification level to ensure accurate predictions of microscopy images at different magnifications. Image augmentation techniques were employed to improve the accuracy of the results. The next step involves establishing a deep neural network (step 5), which integrates Mask R-CNN (Mask Region-Based Convolutional Neural Network) and vision transformer. Mask R-CNN is trained separately for each microscopy magnification, while the vision transformer is used for classification. The following step (step 6) involves the identification of regions of interest, wherein Mask R-CNN is utilized to detect the microwell’s position, and the program segments each into a single image. Finally, in step 7, the single images are sent to the vision transformer for further classification, leading to precise decisions regarding the patient’s condition.

Optimization of analysis time point and cluster viability in a patient-derived tumor model

After RBC lysis, nucleated cells containing CTCs and tumor-associated immune cells from patient blood with a concentration of 4–10 × 10⁶ cells/ml were homogeneously seeded into the channels of the microfluidic biochip. To determine the optimal analysis timepoint after cluster cultivation, we established patient-derived cell clusters from gastric and breast cancer patients at the IV stage (Table S1 in the supplementary material). The cluster size, thickness, and roughness of cell clusters at distinct timepoints were evaluated in a preliminary cohort of cancer patients (n = 4). The phenotypes of the clusters were evaluated under a 6-day and 14-day period by measuring the cluster size, thickness, and roughness using in situ cluster imaging quantitatively [Figs. 2(a) and 2(b); Figs. S1–S3 in the supplementary material]. The results demonstrated that patient-derived clusters significantly increased (p < 0.001; breast cancer: 1.48-fold, gastric cancer: 1.30-fold) in the cluster size and decreased (p < 0.001; gastric cancer: 1.16-fold) in grayscale values from the sixth day to fourteenth day [Figs. 2(a) and 2(b)]. Therefore, clinical samples were analyzed for in situ cluster imaging and analysis after 14-day cultivation.

FIG. 2.

Optimization of the patient-derived tumor model establishment. (a) Boxplot of cluster size of patient-derived cell clusters from gastric (n = 2; day 6: 4919.18 ± 1552.7 μm², day 14: 7273.98 ± 1857.85 μm²) and breast cancer patients (n = 2; day 6: 5246.13 ± 1857.85 μm², day 14: 6803.48 ± 1716.80 μm²) vs culturing days (i.e., 6 and 14 days). (b) Boxplot of grayscale values of patient-derived cell clusters from gastric (n = 2; day 6: 136.93 ± 24.14, day 14: 118.49 ± 17.86) and breast cancer patients (n = 2; day 6: 125.92 ± 21.32, day 14: 126.41 ± 20.21) vs culturing days (i.e., 6 and 14 days). *** represents p ≤ 0.001. (c) Representative images of clusters in microwells under different perfusion flow rates. (d) Cell viability of clusters from two gastric cancer samples cultured for 14 days. (e) Representative fluorescence images were obtained under the in situ cell viability assay. Scale bar, 100 μm.

The stability of the cell clusters is a vital consideration for the functionality of the IDDT approach. We evaluated the stability of cluster growth under medium culture perfusion at different flow rates. We demonstrated that the morphology of the cell clusters remained stable at flow rates ranging from 0 to 500 μl/min, indicating that the perfusion of the culture medium did not impact the growth of the cell clusters in the microwells [Fig. 2(c)]. Under actual cell culture conditions, we have identified an optimized flow rate of 100 μl/min. This selection ensures thorough mixing of cells with essential nutrients within the microchannel, thereby facilitating the periodic refreshment of the culture medium.

The viability assessment of patient-derived cell clusters was conducted by applying the in situ viable cell fluorescent indicator, calcein-AM (green), in conjunction with the nuclear dye Hoechst (blue) for total cell enumeration. Stained cluster images were captured using a Nikon Eclipse Ci-L fluorescent microscope at 10× magnification. Image processing and viable cell quantification were performed using ImageJ software. Hoechst-labeled cells (30–60 cells per well) were identified as the total cell count within a single microwell. The ratio of the living cell count to total cell count was then calculated to ascertain cell viability [Fig. 2(d)].

The results indicated that the viability of seeding cells was maintained at over 95%. Furthermore, after 14 days of incubation, the cluster cells retained a relatively high viability of 88.64 ± 3.79% [Fig. 2(e)]. Therefore, the microfluidic tumor model and IDDT approach demonstrated the ability to maintain cell viability during the experimental period, ensuring the reliability of the patient-derived cell clusters for further analysis and assessment.

Integration of deep learning-assisted algorithmic analysis by Mask R-CNN and vision transformer for classification

To enhance the precision and efficiency of the disease detection process, we aimed to use deep learning techniques and a microfluidic disease model to accurately classify cell clusters and distinguish between cells from healthy individuals and patients. Our two-stage approach combined the Mask R-CNN and vision transformer algorithms, leveraging both strengths to achieve optimal results. Using Mask R-CNN to segment individual microfluidic wells enabled us to isolate each patient-derived cell cluster as a unique training sample, reducing the time and computational resources required and allowing for the independent optimization of both algorithms.

The Intelligent Disease Detection Tool (IDDT) utilized two distinct algorithms with the microfluidic disease model to classify cell clusters accurately. In the first part of the IDDT process, Mask R-CNN [Fig. 3(a)] was used to segment the regions of interest (ROIs), which were tapered microwells in the input datasets. In the second part, the vision transformer algorithm was employed to classify the cell clusters within a single microwell as originating from healthy donors or cancer patients. After obtaining the ROI with Mask R-CNN, additional algorithms can be integrated for further analysis.

FIG. 3.

Overview of the IDDT assisted by deep learning. (a) Overview of prediction model preparation and training. The deep learning model can be split into four parts: (i) Preparation of the segmentation dataset; (ii) preparation of the classification dataset for training the vision transformer; (iii) training the model and output of the prediction; (iv) post-processing. (b) Overview of prediction part. (c) Preparation of the segmentation dataset by the SAM.

We created two independent training sets and utilized image augmentation techniques to improve the accuracy of predicting microscopy images at different magnifications (e.g., 100× and 200×). Mask R-CNN was trained separately for each magnification using the training and validation datasets. The IDDT program then integrated Mask R-CNN and the vision transformer in the prediction process. The algorithm employed the vision transformer (ViT) algorithm for classifying cell clusters. Unlike traditional convolutional neural networks (CNNs), ViT is better at capturing complex spatial relationships within images, a key factor for accurate classification in our task. Despite being more resource-intensive, the performance improvement justified its use.

During the testing phase, the images were input into the IDDT, and Mask R-CNN detected the microwell’s position with the assistance of the microwell size. Once the microwell position was obtained, the program segmented each into a single image and sent it to the vision transformer for further classification. The results showed that using the vision transformer, the IDDT could accurately identify each microwell using the prediction results from Mask R-CNN and predict the patient cohort (e.g., healthy or pre-treatment, gastric, breast cancer).

The IDDT achieved a high % accuracy rate of 98% in classifying cell clusters, indicating the effectiveness of combining deep learning techniques with patient-derived data for accurately identifying and characterizing cell clusters. This approach holds great potential for detecting and predicting various diseases, including cancer, which can lead to more accurate diagnoses and personalized treatment plans.

Validation of IDDT for high stability and robustness

We aimed to validate the IDDT as a state-of-the-art deep learning tool for cancer prognosis and treatment to ensure its stability and reliability. The validation process involved training on 4350 images, validation on 1450 images, and testing on 1450 images from various cancer types, including gastric and breast cancer [Figs. 4(a)–4(c)]. While the adequacy of a sample size depends on multiple factors, previous studies have shown that deep learning models can achieve satisfactory performance with similar or even smaller sample sizes (<2000 images).¹⁴

FIG. 4.

System loss and accuracy of the vision transformer. (a) Vision transformer’s loss and accuracy curve on the training and validation sets on the full dataset. (b) The loss and accuracy curve of the vision transformer on the training set and validation set on the healthy and gastric cancer dataset. (c) The loss and accuracy curve of the vision transformer on the training set and validation set on the healthy and breast cancer datasets.

The system’s accuracy increased as the number of training iterations increased to 50 epochs, indicating the successful establishment of the deep networks. The validation process showed similar accuracy to the validation datasets, and the test datasets validated the robustness of the IDDT. The convergence of the loss and accuracy curves during training and validation further supports the successful establishment of the deep networks [Figs. 4(a)–4(c)]. Therefore, based on the information provided, the sample size used for the IDDT validation process is adequate to confirm its stability and reliability. The accuracy remained consistent across the validation and test sets, indicating that the IDDT is a reliable tool for cancer diagnosis and treatment in translational oncology.

Dataset training and augmentation using Unet and Mask R-CNN

The IDDT utilizes Mask R-CNN [Fig. 3(b)] to segment the regions of interest (ROIs) in the tested images and determine the location and size of each microwell [Fig. 5(a)]. Here, we tested five different neural networks, including Unet,¹⁵ Unet++,¹⁶ ResUnet,¹⁷ ResUnet++,¹⁸ and Mask R-CNN [Fig. 5(b), Table I], to identify ROIs from the tested images. Among these, we found that Mask R-CNN was the most effective in identifying the ROIs. While Unet is a neural network that provides precise segmentation, ResUnet and ResUnet++ improve accuracy using various techniques. Based on the comparison of the testing results in Table I, Mask R-CNN emerged as the most effective neural network for our purposes. The score presented in Table II, which shows a 98.3% accuracy for all types of patients and healthy donors, reflects the patient response based on the cluster formation rate of patient-derived cell clusters obtained from liquid (blood) biopsies, as previously reported.

FIG. 5.

The design of the ROI segmentation for Mask R-CNN. (a) The architecture of Mask R-CNN. (b) The prediction results of Mask R-CNN of different microscopy magnifications.

TABLE I.

The result of the segmentation algorithm.

Model	Type	mIoU
100× Microscope magnification	Unet	0.8695
	Unet++	0.8792
	ResUnet	0.8242
	ResUnet++	0.8750
	Mask R-CNN	0.9020
200× Microscope magnification	Unet	0.6837
	Unet++	0.6752
	ResUnet	0.6144
	ResUnet++	0.5240
	Mask R-CNN	0.8198

TABLE II.

Classification results of the vision transformer.

Model	Dataset	Accuracy	Type	Precision	Sensitivity	Specificity
All patients and healthy	Validation	0.9890	All cancers	0.989	0.996	0.957
			Healthy	0.983	0.957	0.996
	Testing	0.9876	All cancers	0.990	0.997	0.960
	Validation	0.9841	Healthy	0.986	0.960	0.997
Pretreatment and healthy			Pretreatment	0.985	0.964	0.993
			Healthy	0.984	0.993	0.964
	Testing	0.9886	Pretreatment	0.993	0.971	0.997
			Healthy	0.987	0.997	0.971
Gastric cancer and healthy	Validation	0.9855	Gastric cancer	0.990	0.988	0.98
	Testing	0.9788	Healthy	0.977	0.980	0.988
			Gastric cancer	0.988	0.980	0.977
			Healthy	0.961	0.977	0.980
Breast cancer and healthy	Validation	0.9921	Breast cancer	0.996	0.991	0.993
			Healthy	0.987	0.993	0.991
	Testing	0.9829	Breast cancer	0.989	0.982	0.983
			Healthy	0.974	0.983	0.982

We employed the Segment Anything Model (SAM) to produce the dataset,¹⁹ which reduced production time by 90%. SAM is a foundational model for image segmentation that demonstrates strong generalization across a wide range of datasets [Fig. 3(c)]. Integrating the Segment Anything Model (SAM) significantly bolstered our training efficiency. It is worth noting that we refrained from conducting any secondary training or modifications to SAM's pre-trained model. The algorithm was employed with an additional filtering process to ensure that the output dimensions align with the size specifications of the microwells in our study. The IDDT used Unet and Mask R-CNN to train and augment the dataset. For images under 100× microscopy magnification, 244 were used for training (80% of the dataset), and 64 were used for validation (20%). For images under 200× microscopy magnification, 125 were used for training (80% of the dataset), and 30 were used for validation (20%). The dataset contained 463 annotated images. Before feeding into the Mask R-CNN model, the images were normalized and resized to 512 × 512 pixels. During training, data augmentation techniques such as horizontal flip, vertical flip, random rotation, random center crop, warp, and HSV (Hue, Saturation, Value) variation were employed to expand the dataset and enhance its features. Incorporating two training sets for every magnification level is crucial to alleviate overfitting, yielding a more dependable and resilient model performance than relying on a single dataset approach.

The results indicate that training the IDDT separately for 100× and 200× microscopy magnification produced more accurate predictions than training them together. Specifically, the results showed that the mean Intersection over Union (mIoU) between the predicted results and the ground truth was 0.9020 and 0.8198 for 100× and 200× microscopy magnifications, respectively (Table I).

Precise classification of healthy and patient cohorts using a vision transformer

The vision transformer (ViT) employed in this study treats images as a sequence of patches, facilitating extensive interactions among global features. These images are partitioned into grids, with their flattened patches linearly embedded and positionally encoded. Subsequently, transformer layers process these embeddings, and a linear layer employs the transformed tokens for final classification.

In contrast to convolutional neural networks (CNNs), the ViT utilizes self-attention to capture global dependencies, preceding the exclusive focus on local regions. This capability enables the ViT to acquire more intricate patterns across the entirety of the image. ViT’s reduced reliance on handcrafted features and predefined architectures renders it highly adaptable and efficient in accommodating various image characteristics. This, in turn, augments our model’s robustness and generalization capacities.

To validate the suitability of the IDDT for screening cancer patients, we carried out a clinical study approved by an institutional review board with Certificate No. XHEC-NSFC-2020-078. Whole blood samples (n = 71; approximately 5 ml) were collected from 55 cancer patients [including gastric cancer patients at varying treatment stages (n = 26), breast cancer patients at varying treatment stages (n = 20), and nine individuals with other cancer types, including colon cancer, lung cancer, and pancreas cancer], as well as healthy donors (n = 12) (Tables S1–S4 in the supplementary material).

The IDDT utilized a vision transformer to classify cell clusters derived from patient samples. The model was trained and augmented using the dataset, which was first processed to segment individual microwell images. The resulting labeled images were saved, resulting in 7250 images. We used the pre-trained vision transformer (ViT) model, accelerating model convergence. Positive controls were derived from untreated patients, while negative controls were obtained from healthy samples. The labeled images were then divided into training, validation, and testing datasets in a 6:2:2 ratio following annotation. The dataset was further divided based on the classification groups. For the All Patients & Healthy Group, we used a total of 7250 images, split as follows: training: 4350 images; validation: 1450 images; and testing: 1450 images. For the Breast Cancer & Healthy Group, we used a total of 3829 images, divided into the following: training: 2303 images; validation: 761 images; and testing: 765 images. For the Gastric Cancer & Healthy Group, we used a total of 4477 images, allocated as follows: training: 2689 images; validation: 892 images; and testing: 896 images. For the Pre-treatment & Healthy Group, we used a total of 2864 images, distributed as follows: training: 1541 images; validation: 660 images; and testing: 663 images. The classification result is shown in Table II.

After integrating Mask R-CNN and the vision transformer, the IDDT could identify each microwell’s position and predict the cell clusters within each ROI. The results demonstrated that the vision transformer achieved a high level of accuracy, with an accuracy rate of approximately 98% on the test dataset. Additionally, the confusion matrix revealed that the IDDT exhibited high sensitivity (97%) and high specificity (98%).

The IDDT system demonstrated excellent performance in accurately classifying cell clusters from various clinical samples, as shown in Table II. The system achieved an overall accuracy of 98.9% based on the clinical cohorts (n = 55; Tables S2–S5 in the supplementary material) and control cohort (n = 12; Table S5 in the supplementary material) [Fig. 6(a)]. The accuracy was 98.4% for the pre-treatment (n = 7) and healthy donor samples [Fig. 6(b)], 98.6% for gastric cancer patients and healthy donors [Fig. 6(c)], and 99.2% for breast cancer patients and healthy donors [Fig. 6(d)]. These results indicate that the IDDT system can accurately stratify outcomes from various cancer types, demonstrating broad applicability in cancer diagnosis and treatment. Cell clusters provide valuable information about the characteristics and behavior of cancer cells, and the algorithm calculates samples to derive potential sample characteristics, which can inform the development of personalized and targeted treatment strategies. The IDDT system can provide clinicians with valuable insights to improve patient outcomes by accurately identifying and characterizing cell clusters.

FIG. 6.

The classification results of vision transformers for classification. (a) The confusion matrix of all cancer patients and healthy donors. (b) The confusion matrix of pre-treatment and healthy donors. (c) The confusion matrix of gastric cancer and healthy donors. (d) The confusion matrix of breast cancer and healthy donors.

Stratification of patients with a specific treatment cycle and cancer stage

To further evaluate the IDDT’s applicability in clinical settings, we used the neural network to stratify cancer types based on the clinical cohort (n = 55). A confusion matrix was generated to analyze inter-class variability in classification accuracy among patients with different treatment cycles and intra-class variability in distinguishing between healthy donors and cancer patients [Fig. 7(a)].

FIG. 7.

The stratification results of vision transformers for the different treatment cycles and the different cancer stages. (a) The confusion matrix of the different treatment cycles. (b) The confusion matrix of different cancer stages. (c) The vision transformer’s loss and accuracy curve on the training and validation set on the healthy and different treatment cycle datasets. (d) The vision transformer’s loss and accuracy curve on the training and validation sets on the healthy and different cancer stage datasets.

We demonstrated that the IDDT showed high overall accuracy (∼86.76%) in stratifying the subtypes of cancer patients, including healthy donors (n = 12), pre-treatment (n = 8), treatment cycle 1 (n = 8), cycle 2 (n = 3), cycle 3 (n = 5), cycle 4 (n = 4), cycle 5 (n = 8), cycle 6 (n = 10), and cycle more than 7 (n = 9). The results showed that healthy donors and patients in treatment cycle 3 (90.20% accuracy) were accurately stratified with excellent performance (97% accuracy).

To further evaluate the clinical stratification capability of the IDDT system, we applied it to distinguish between healthy donors and cancer patients with different cancer stages. The dataset included healthy donors (n = 12) and patients with stage I (n = 5), stage II (n = 10), stage III (n = 13), and stage IV cancer (n = 26). The confusion matrix showed an overall high accuracy of 93.10% [Fig. 7(b)]. These results demonstrated the effectiveness of the IDDT system in accurately stratifying cancer patients based on their cancer stages.

The IDDT's training progress was monitored by plotting the loss and accuracy curves during training and validation for subtyping (i.e., treatment cycles) stratification [Fig. 7(c)] and cancer staging [Fig. 7(d)]. The system loss of both training and validation decreased to low levels (0.04) after a few iterations (50 epochs). In contrast, training and validation system accuracy increased to high levels (∼99%).

Overall, the IDDT demonstrated robust utility for samples obtained routinely at different treatment cycles and cancer stage stratifications, suggesting its potential as a clinical tool for clinicians to make informed decisions regarding patient treatment.

MATERIALS AND METHODS

Fabrication of the microfluidics-based tumor model

The integrated microfluidics-based tumor model was fabricated with standard photolithography and soft lithography techniques to meet the following requirements: (i) creation of a 3D microenvironment for cells to interact with one another, (ii) mimicking tumor microenvironment conditions in vivo, and (iii) providing ease of establishment for the patient-derived cell cluster. The microfluidic-based tumor model comprised two polydimethylsiloxane (PDMS) layers, i.e., (i) top barrier layer and (ii) bottom tapered microwell layer, assembled with plasma treatment (Fig. S1 in the supplementary material).

The top barrier layer was fabricated to hold the culture medium within 5 mm thickness and could hold up to 500 μl culture medium. The VeroClear barrier mold was fabricated using a 3D printer (Objet260 Connex3, Stratasys, USA). The mold was baked in the oven overnight to remove the residual printing reagent. To create a microwell layer capable of generating densely packed clusters with minimal susceptibility to shear flow during the fluid exchange, we employed photolithography techniques to design and fabricate a silicone mold featuring an array of tapered microwells.^13,14 This mold encompasses eight arrays, each comprising one thousand tapered microwells with dimensions of 250 μm in length, 150 μm in width, and 150 μm in depth.

To fabricate PDMS (Sylgard 184 Silicone Elastomer Kit, Dow Corning, USA) with the mixture ratio of 10:1 (elastomer vs curing agent) was prepared and then poured into the molds for casting patterns. Before putting them into the oven, the molds with PDMS were placed into a vacuum pump to remove bubbles. After baking for 2.5h at 70 °C, the first PDMS layer with trapped microwells and the second barrier layer was peeled off from the molds. The first and second PDMS layers were assembled with plasma treatment for 5 min with 700 mm. Eventually, the assembled chip with two layers was placed into an oven for baking for 2h at 70 °C.

Clinical samples’ preparation

Blood samples were collected from clinical patients at various treatment cycles (n = 59, pre-treatment, and over eight treatment cycles) and healthy donors (n = 12). The institutional review board approved this study under ethical approval (Certificate No. XHEC-NSFC-2020-078). All patients consented to be included in the study. The blood samples were collected in EDTA-coated vacutainer tubes (Becton-Dickinson) and mixed with red blood cell lysis buffer (Life Technologies) for 5 min at room temperature (15–25 °C) and then centrifuged at 1000 g for 5 min to remove the supernatant. The lysis reaction was washed with sterile phosphate-buffered saline (PBS) three times.

Cell cluster culture

After performing the RBC lysis procedure, the cell pellets were suspended with Roswell Park Memorial Institute (RPMI) 1640 culture medium [supplemented with 10% fetal bovine serum (FBS) and 1% penicillin–streptomycin] and mixed gently.¹³ Following the dilution of nucleated cells to a concentration of 1 × 10⁶/ml, a 200 μl aliquot of the patient's blood cell suspension was uniformly introduced into the microchannel, employing either a hand pipette or a syringe pump.

After cell seeding, the microfluidic-based tumor model was placed in a dish (150 mm) and incubated under humidified conditions with 5% CO₂ at 37 °C for 14 days.¹³ The medium was refreshed every 2 days, i.e., removing 200 μl of the old culture medium with a hand pipette and replacing it with a fresh medium. Similarly, a syringe pump could add and remove culture media at a 100 μl/min flow rate to maintain the cell cultures.

Label-free monitoring of tumor models

To monitor and capture the cluster morphology derived from clinical samples using the microfluidic-based tumor model, a phase-contrast microscope (Nikon, Eclipse Ci-L, Japan) was utilized. The parameters (e.g., exposure time, ISO sensitivity, and white balance) of the CCD camera on the microscope were fixed for the same illumination conditions while capturing the culture resultants.

Mask R-CNN establishment

Mask R-CNN²⁰ follows the idea of Faster R-CNN. The feature extraction adopts the ResNet-FPN architecture, adding an additional mask prediction branch. The ResNet-FPN architecture combines the advantages of ResNet’s deep residual learning and FPN’s multiscale feature representation, enabling efficient detection of objects at various scales and improving the model’s overall accuracy. The additional mask prediction branch in Mask R-CNN generates a binary mask for each detected object, providing pixel-level segmentation.

Mask R-CNN used SGD (stochastic gradient descent) as the optimizer, trained for 20 epochs with a learning rate of 0.004 and a weight decay of 1 e⁻⁴, and the official pre-train model as the pre-training weight. Vision transformer used SGD as the optimizer, trained for 50 epochs, the learning rate was 0.001, the learning rate dropped to 0.01, the loss calculation used cross-entropy loss, the official pre-train model was used as the pre-training weight, the model type selected the base size, and the patch is 16.

Vision transformer establishment

Vision transformer²¹ is a neural network based on self-attention, and the algorithm divides the image into 16 × 16 tokens. Then, the algorithm converts the token into token embedding and sends it into the encoder part for classification. After classification, the IDDT could predict whether each sample is positive or negative.

Statistical analysis

Precision value is defined by TP/(TP + FP), where TP (true positive) represents the cases where the model predicted positive, and the actual label is also positive; and FP (false positive) represents the cases where the model predicted positive, but the actual label is negative. The sensitivity value is defined by TP/(TP + FN), where FN (false negative) represents the cases where the model predicted negative, but the actual label is positive. Specificity is defined by TN/(TN + FP), where TNs (true negatives) represent the cases where the model predicted negative, and the actual label is also negative.

DISCUSSION

Conventional approaches to treating diseases are established based on clonal growth. In recent years, personalized medicine has emerged as an alternative approach focusing on understanding and treating patients on an individual level.^22–24 This involves using technology to gather and analyze personal patient data for disease diagnosis, treatment, and monitoring, allowing physicians and clinicians to develop tailored interventions for each patient. However, such personalized medicine approaches generate a large amount of data, which can be overwhelming to manage. Engineering technologies such as artificial intelligence (AI), genome-guided therapy, nanomedicine, and microfluidics are being used to develop more precise and personalized clinical strategies to address this challenge. These advancements are changing the landscape of clinical trials and decision-making, allowing for more effective and targeted treatments for patients.²⁵

Patient cohorts exhibit heterogeneous characteristics, including various symptoms across individuals, making detecting, classifying, and predicting disease prognosis extremely challenging. Disease models have been developed to mimic the disease microenvironment and pathological processes within the human body to understand better how diseases develop and test new treatment approaches.

Microfluidics has emerged as a cost-effective and efficient tool to mimic disease environments on a single chip. It has found applications in various fields, such as disease evaluation,^26,27 point-of-care testing,^28–31 organs-on-a-chip,³² and microfabrication.³³ Microfluidics provides an effective method to culture cell composites in vitro, with the potential to develop patient-derived models for various applications.^34,35 The convergence of microfluidics with other technologies, such as AI, nanomedicine, and genome-guided therapy, offers new opportunities for developing precise and personalized clinical strategies to support clinical trials and decision-making.

Data-driven algorithms such as artificial neural networks with deep hidden layers have been developed to analyze patient-derived cell cultures effectively and extract valuable information. These algorithms can solve non-deterministic polynomial-time hardness problems, making them ideal for analyzing heterogeneous diseases in clinical settings.^36,37 We demonstrated an innovative approach that combines microfluidic tumor models with an Intelligent Disease Detection Tool (IDDT) that utilizes a deep learning algorithm to evaluate patient prognosis quickly. The IDDT algorithm has two main components: a deep learning algorithm core module and a traditional algorithm module for detecting scale-space extrema and localizing key points.

IDDT can potentially be a patient-centered, cost-effective, and highly accurate method for improving cancer care outcomes. In comparison with other techniques employing algorithmic analysis (Table III), the IDDT exhibits distinctiveness in four key facets: (i) It necessitates only a minimal sample volume (∼200 μl) for the establishment of a 3D patient-derived tumor model sourced from a patient’s blood biopsy. This process yields high-throughput cluster formation, generating up to 1000 patient-derived cell clusters. (ii) After deep learning model training, test image classification is immediate, with a brief analysis time of 0.5–2 s per image. (iii) The IDDT employs Mask R-CNN and a vision transformer, resulting in exceptionally accurate microwell identification, achieving an accuracy of approximately 98%. (iv) It achieves a precise prediction score of approximately 99.3% using datasets from both healthy donors (n = 12) and cancer patients (n = 55). This allows for a clear and effective stratification of subtypes using the vision transformer. Compared to traditional algorithms such as ellipse recognition and edge detection, the IDDT employs Mask R-CNN and a vision transformer, resulting in highly accurate microwell identification (approximately 98%). The tool requires only a small sample volume of around 200 μl to establish a 3D patient-derived tumor model, with high-throughput cluster formation generating up to 1000 patient-derived cell clusters. Obtaining raw phase-contrast images is also rapid, reducing clinical decision-making timelines. Once the model is trained, tested images can be classified immediately, with a short analysis time of 0.5–2 s per image. Future work can involve a large amount of data that can be used to establish a deep learning model for monitoring patients’ recovery status. This could further enhance the IDDT’s effectiveness and precision, ultimately contributing to more efficient and accurate cancer detection, diagnosis, and monitoring, thereby improving patient outcomes and overall healthcare experiences.

TABLE III.

Comparison of the IDDT with other existing techniques. N/D, not determined; IDDT, Intelligent Disease Detection Tool; SMART, spheroid monitoring and AI-based recognition technique; DHM, digital holography microscopy.

Technique	Sample source	Cancer type	Platform	Data type	Prediction algorithm	Accuracy	Algorithm type	Samples (training validation and testing dataset)	Reference
IDDT	Liquid biopsy	Multiple (label-free)	Microfluidic-based tumor model	Images	Mask R-CNN ViT	98%	Machine learning	4350 (training) 1450 (test) 1450 (test)	Our work
SMART	Cancer cell line	Breast, colon, and lung cancer cells	96-well non-adhesive U-bottom plates and flow-containing micro-physiological system	Images	Excess perimeter index and multiscale entropy index	N/D	Physical parameters	N/D	38
Microfluidic biochip with conventional neural network	Cancer cell line	Breast cancer cells	Microfluidic biochip	Images	Conventional neural network	>93.2%	Machine learning	6144 (training; 80% of fourfold of 1920 images) and 1536 (testing; 20% of fourfold of 1920 images)	23
Inline-DHM	Cancer cell line	Breast cancer cells	Microfluidic device	Hologram	Classification and regression tree	∼99%	Machine learning	100 000 (training)	39
Combined microfluidic deep learning approach	Tissue-derived cancer cells	Lung cancer cells	Microfluidic device	Images	ResNet18	>98.37%	Machine learning	70 664 240 (training), 1413 (validation), and 1413 (testing)	40
Machine-based genetic profiling	Liquid biopsy	Lung tumors	N/D	Gene profile	Random Forest	83%	Machine learning	32 healthy donors and 60 lung cancer patients	41
Deep learning	Solid tumor biopsy	Colorectal polyps	H&E stained whole-slide	Images	ResNet	93.0%	Machine learning	458 (training) 239 (internal test)	42
Deep learning	Solid tumor biopsy	Colorectal adenoma	Slides	Images	DeepLab v2 with ResNet-34	90.4%	Machine learning	177 (training) 194 (internal test)	43
Image classification	Solid tumor MRI	Glioblastoma multiforme	N/D	Images	ResNet34	80.72%	Machine learning	66 methylated and 89 unmethylated tumors	44

Tumor-related diagnosis methods such as DNA and genetic cell analysis are robust in their niche applications but cannot be traced back to the source of the genetic material.^45–47 Here, the IDDT uses the frequency of cluster formation to reflect patient prognosis, allowing the assay to serve as a patient-derived tumor model that can be used for downstream screening and analysis. Cluster formation percentage in the assay correlated inversely (p < 0.001) with patient treatment cycles (pre-treatment: 88.6%; <3 weeks post-treatment: 84.4%; 3–5 weeks post-treatment: 66.7%; >5 weeks post-treatment: 42.6%), as previously demonstrated.¹²

Machine learning algorithms have a certain tolerance for prediction results represented by false positives and negatives. Here, patient-derived cell clusters were classified by our customized deep learning algorithm based on physical properties such as size, shape, and compactness. Our algorithm demonstrated an accuracy of 98% in distinguishing between healthy donors (n = 12) and cancer patients (n = 55). Moreover, the algorithm could identify false-positive samples and improve its accuracy.

T-SNE (t-Distributed Stochastic Neighbor Embedding) visualization for the transformer offers a clearer understanding of data organization and relationships, enabling better comprehension of classification results for informed decision-making (Fig. 8). This enhanced visualization technique enables researchers and clinicians to comprehend the classification results better and make more informed decisions. Our results demonstrate that healthy individuals and cancer patients are distinctly segregated, while the separation between treatment cycles and cancer stages is less pronounced. This observation suggests that while the integrated methodology excels in distinguishing healthy individuals from cancer patients, additional refinement might be necessary to enhance its capacity for discerning variations in treatment cycles and cancer stages. The lack of distinct separation may also be attributed to the differences in individual patients and the diverse treatment regimens employed.

FIG. 8.

The visualization of the vision transformer and T-SNE: (a) T-SNE visualization for the transformer for healthy individuals and cancer patients, and the different treatment cycles and cancer stages. (b) Microwell images of healthy individuals and cancer patients. The x and y axes represent abstract dimensions capturing the variation within the high-dimensional data. (c) Microwell images of healthy individuals and cancer patients, along with images of microwells on day 6 and day 12. (d) Microwell images of healthy individuals and cancer patients with different treatment cycles. (e) Microwell images of healthy individuals and cancer patients at various cancer stages.

The combination of the transformer and T-SNE capitalizes on the strengths of both techniques, thereby offering a robust framework for scrutinizing intricate data. As deep learning models, transformers excel in capturing complex relationships within datasets. At the same time, T-SNE, as a dimensionality reduction method, adeptly projects high-dimensional data onto a lower-dimensional space for effective visualization. By amalgamating these two methodologies, we present a potent tool for examining and illustrating the heterogeneity inherent in cancer-related data. As a result, this may contribute to developing tailored therapies and more precise prognostic assessments, ultimately facilitating better clinical outcomes. T-SNE visualization offers an exciting opportunity for advancing cancer research and personalized treatment strategies.

Integrating deep learning and traditional algorithms in the IDDT aims to enhance accuracy and robustness. The IDDT’s neural network offers a fast and straightforward approach to obtaining results, potentially improving the next generation of disease diagnosis, prognosis, and treatment in patient-centric healthcare. The IDDT algorithm was used to classify patient data based on different treatment cycles and was effective in accurately categorizing patients despite variations in the treatment processes and internal samples. The algorithm’s strong performance extended to different cancer stages, highlighting its versatility and potential utility in many patient populations.

Future efforts can be directed towards creating a standardized database to improve the accuracy of the predictions. Our results demonstrated the feasibility of using a neural network in classifying patient cohorts and emphasized the importance of having a standardized database to achieve more accurate results in the future. We envision that these advanced intelligent tools will aid clinicians in developing precise treatment strategies for patients, leading to better clinical outcomes and ultimately contributing to the goal of patient-centric healthcare.

CONCLUSIONS

We developed a deep learning-assisted IDDT using a microfluidics tumor model to rapidly establish patient-derived cell clusters for routine monitoring. We used several deep neural networks to segment ROIs within the datasets and classify patient-derived cell clusters. Our approach has the potential to benefit clinicians in making precise decisions for individual patients, decentralize healthcare, improve cancer diagnosis, and promote in-house prognostic support. The IDDT offers a rapid and precise way to monitor patient-derived cell clusters and stratify clinical cohorts with high sensitivity and specificity. We envision our approach as a valuable contribution to next-generation disease diagnosis, prognosis, and treatment, ultimately improving clinical outcomes for cancer patients. The IDDT can serve as a complementary routine screening evaluation tool for clinicians, ultimately achieving patient-centric healthcare.

SUPPLEMENTARY MATERIAL

See the supplementary material for the additional data that support our findings. It comprises five tables (Tables S1–S5) detailing the clinical demographics of cancer patients and healthy donors involved in our study. They provide information such as cancer type, TNM stage, cancer stage, treatment cycle, age, and gender. Figures S1–S3 present a detailed correlation analysis of cell clusters derived from gastric and breast cancer patients, comparing their size, grayscale values, and standard deviation of grayscale values (SDGVs) over different culture durations (6 and 14 days). Significant differences are highlighted.

NOMENCLATURE

ASPP

atrous spatial pyramidal pooling

CTC

circulating tumor cell

FBS

10% fetal bovine serum

IDDT

Intelligent Disease Detection Tool

mIoU

mean intersection over union

PDMS

polydimethylsiloxanes

PLA

polylactic acid

PBS

phosphate-buffered saline

RPMI

Roswell Park Memorial Institute medium

ViT

vision transformer

AUTHOR DECLARATIONS

Conflict of Interest

One or more authors have a patent related to this work.

Author Contributions

H.H. and Y.Z. contributed equally to this work.

Haojun Hua: Conceptualization (equal); Methodology (equal); Writing – original draft (equal); Writing – review & editing (equal). Yunlan Zhou: Investigation (equal); Methodology (equal). Wei Li: Conceptualization (equal); Methodology (equal); Writing – original draft (equal); Writing – review & editing (equal). Jing Zhang: Methodology (equal). Yanlin Deng: Investigation (equal); Methodology (equal); Writing – original draft (equal); Writing – review & editing (equal). Bee Luan Khoo: Conceptualization (equal); Funding acquisition (equal); Supervision (equal); Writing – review & editing (equal).

DATA AVAILABILITY

The data that support the findings of this study are available within the supplementary material.