Real-time cancer detection with an integrated lensless fluorescence contact imager

Abstract

Microscopic tumor cell foci left in a patient after surgery significantly increase the chance of cancer recurrence. However, fluorescence microscopes used for intraoperative navigation lack the necessary sensitivity for imaging microscopic disease and are too bulky to maneuver within the resection cavity. We have developed a scalable chip-scale fluorescence contact imager for detecting microscopic cancer in vivo and in real-time. The imager has been characterized under simulated in vivo conditions using ex vivo samples, providing strong evidence that our device can be used in vivo. Angle-selective gratings enhance the resolution of the imager without impacting its physical size. We demonstrate detection of cancer cell clusters containing as few as 25 HCC1569 breast cancer cells and 400 LNCaP prostate cancer cells with integration times of only 50 ms and 70 ms, respectively. A cell cluster recognition algorithm is used to achieve both a sensitivity and specificity of 92 % for HCC1569 cell samples, indicating the reliability of the imager. The signal-to-noise ratio (SNR) degradation with increased separation is only 1.5 dB at 250 μm. Blood scattering and absorption reduce the SNR by less than 2 dB for typical concentrations. Moreover, HER2+ breast cancer tissue taken from a patient is distinguished from normal breast tissue with an integration time of only 75 ms.

1. Introduction

Removal of all disease, both gross and microscopic, is necessary for successful cancer surgery. Tumor cells left in the patient, known as microscopic residual disease (MRD) or a positive margin, significantly increase the chance of cancer recurrence across cancer types. However, identification of MRD intraoperatively remains a pervasive challenge. The lack of accurate spatial information on the presence, location, and quantity of MRD forces physicians to either empirically treat wide areas of normal-appearing tissue with surgery or radiation in an effort to eradicate all cancer cells or risk leaving tumor cells behind. In breast cancer, for example, an estimated 65% [1] of the 230,000 women [2] diagnosed annually in the United States undergo lumpectomy and MRD is found in 25–40 % of these women [3–5]. Left untreated, MRD doubles the local recurrence rate, from 15 % to 30 % over 15 years [6], and decreases survival [7]. Consequently, re-excision is recommended in many cases [8, 9]. Even empirical resection of additional tissue, also called shave margins, results in a 20 % rate of MRD [5]. Similarly, rates of MRD after radical prostatectomy for high risk prostate cancer range between 20 % and 50 % [10, 11]. However, the tight physical confines of the prostate bed prohibit re-excision. Patients are instead treated with eight weeks of postoperative radiation therapy to the tumor bed, often with months to years of hormone therapy, resulting in both short and long term side effects. Despite these intensive therapies, only 45 % of these patients remain disease free four years later [12]. Therefore, to achieve excellent oncologic outcomes across cancer types, while reducing the need for additional therapy, real-time intraoperative assessment of MRD with precise localization is critically needed.

Pathologic assessment is the gold standard for postoperative MRD evaluation, but it remains elusive in the operating room. In the laboratory setting, microscopic disease is readily identified on the excised tumor specimen surface using a combination of specific molecular labeling and high-magnification microscopic evaluation of a few hundred cells at a time. If cancer is found on the excised tumor surface, it is assumed by proxy that residual cancer is present on the adjacent tissue remaining in the tumor bed. However, tissue processing and microscopy times restrict evaluation to the postoperative setting, necessitating an additional surgical procedure if residual tumor is determined to be present. Inaccurate co-registration of the pathologic specimen to the tumor bed and sampling error on the excised specimen occur frequently and lead to potentially missing MRD.

Successful intraoperative MRD evaluation requires (1) the ability to image the entire surface area, eliminating sampling error, (2) real-time operation, allowing for seamless surgical integration and instant re-excision of residual tumor, and (3) high sensitivity imaging devices. Pure visual or tactile methods for detecting cancer are limited to detection of millimeter-scale or larger areas of tumor [13, 14], corresponding to 100s of millions of cells. In breast cancer, 200 cells is considered to be a clinically significant number [15]. Such small clusters are effectively invisible to conventional methods that do not make use of molecular markers to distinguish tumor cells from their morphologically similar normal tissue counterparts.

In order to perform intraoperative imaging, cancer tissue in the patient needs to be labeled preoperatively. The labeling will not require any extra time during the operation. This can be accomplished using an injection of an antibody-fluorophore compound [16–20]. The antibody molecularly-targets cancer cells and the fluorophore allows visualization of the cancer. These compounds require as little as several hours to bind and the cancer will remain fluorescent for several days. Thus the injection can occur on the same day as the operation or in one of the preceding days as part of the preoperative outpatient workflow. There have been numerous successful clinical trials using this strategy for a variety of cancers including ovarian [21], head and neck [22], breast [23], and esophageal (NCT02129933). There are also several clinical trials ongoing (NCT02422979 for head and neck cancer and NCT03052127 for ocular melanoma). High tumor-to-background ratios have been reported, including in the clinical trials (5–10 in [22] and up to 9 in [23]), indicating that non-specific binding of the molecular agents to healthy tissue remains low enough to allow MRD imaging. The fluorophores used for in vivo imaging emit in the far-red and near-infrared where there is negligible tissue autofluorescence. At least several dyes are proved to be non-toxic and are FDA approved, including indocyanine green [24]. Additionally, commercial entities have developed dyes, such as IR700DX and IR800CW, that have been tested successfully in non-human primates [25] and humans [22, 23] and are being used in ongoing clinical trials (NCT02422979, NCT03052127).

Despite recent advances in targeted molecular imaging agents to label cancer in vivo, imaging devices themselves remain the limiting factor. Current intraoperative fluorescence imagers [21, 26] that place a large microscope above the tumor bed require the use of rigid and bulky optics to guide, focus, and filter excitation and emission light. This makes them inadequate for MRD detection for two key reasons: (1) they are restricted to line-of-sight vision, missing the majority of the complex tumor cavity surface, and (2) they image far from the tumor bed, with reduced resolution and sensitivity as light diverges.

These large-scale fluorescence microscopes are incompatible with the small, 1–3 cm, diameter tumor cavities common in modern minimally invasive surgeries such as lumpectomy and robotic assisted prostatectomy. The entire 3–30 cm² surface area of these cavities need to be rapidly imaged (<1 min) by a small, highly maneuverable imager with the ability to detect about 100 to 1000 cancer cells, necessitating imaging speeds of up to 0.5 cm²/s. Miniaturized optical elements are difficult to fabricate and they often suffer from increased aberrations and other issues such as ghost images [27–29]. Goggle-based approaches [30, 31] are easier to use, but suffer from low sensitivity due to the distance from the tumor bed and cannot image the complex tumor cavity sidewalls. A variety of handheld devices for residual cancer detection have been designed that use optical coherence tomography (OCT) [32], spectroscopy [33–35], birefringence [36], magnetic tracers [37], radio tracers [38], or fluorescence [39]. These are either too large to fit inside the resection cavity itself [32] or only image single-points [33–37]. Techniques using fiber optic bundles [38, 39] suffer from a fundamental trade-off of flexibility versus imaging area, prohibiting rapid imaging of the entire tumor bed surface in a small, complex-shaped, resection cavity.

Contact image sensors eliminate these challenges by largely dispensing with optics in lieu of proximity to the tissue surface to achieve high sensitivity and the required spatial resolution. Integrated circuit (IC) technology is ideally suited for contact imaging as it is inherently planar, with thickness ranging from 500 μm down to less than 10 μm [40], and is scalable from single millimeter dimensions [41] up to several centimeters [42], due to the parallel, arrayed nature of the pixels. Scalability and direct integration with surgical tools is therefore possible with minimal form-factor disruption. Consequently, these imagers can be designed for a range of tumor cavity shapes and sizes and easily mounted on highly maneuverable probes, since only electrical signals, rather than light, need to be transmitted outside the tumor cavity. Furthermore, CMOS-based imagers can be disposable due to their low-cost batch fabrication.

There has been recent interest in using the metal interconnect in IC processes to make optical structures [43, 44]. In this paper, we demonstrate a lens-free fluorescence contact imaging platform using angle-selective gratings (ASGs) for cellular and tissue, in vitro and in vivo, imaging of fluorescently-labeled prostate and breast cancer cells as a model to evaluate the sensitivity and resolution of our system. By replicating in vivo conditions, we assess the effect of blood and fluid overlaying the tissue surface on the sensitivity and resolution. We also demonstrate applicability to human tissue by imaging excised human HER2+ breast cancer tissue slices. Cancerous tissue taken from a breast cancer patient is readily distinguished from both healthy tissue and background signal with integration times of only 75 ms.

2. Results

2.1. Development of angle-selective contact imaging system

Traditional fluorescence microscopy (Fig. 1(a)) requires bulky and rigid optics to maintain dual optical pathways and a high performance optical filter with 4–6 orders of magnitude of rejection to separate the bright excitation light from the relatively weak emission light, differing in wavelength by only a small Stokes shift of about 50 nm [45–47]. Contact imagers dispense with optics and instead rely on proximity to the sample to obtain high spatial resolution and sensitivity by gathering light before it significantly diverges. For in vitro systems, the sensor can be placed directly against the sample and excitation light can be applied using illumination through the sample (Fig. 1(b)). However, for in vivo intraoperative devices, the presence of the human tissue in the traditional en face illumination pathway requires oblique illumination from the side of the sensor, precluding direct placement against the tissue (Fig. 1(c)). This excitation light can be provided from either small laser diodes, LEDs, or a flexible 1D fiber optic array injecting light along the imager-tissue interface. The required spacing between the tissue and the sensor must allow for excitation at or below the total internal reflection critical angle, approximately 75° for a tissue-glass interface [48]. For a 2 mm wide sensor this requires an imager-tissue separation of 0.5 mm, increasing as the sensor dimension is increased.

Fig. 1.

(a) Conventional fluorescence microscope. (b) In vitro contact imaging. (c) In vivo contact imaging using an image sensor with ASGs to recover the lost resolution due to light divergence. (d) Image sensor with ASGs bonded onto a PCB. (e) Optical filter epoxied using clear optical epoxy on top of image sensor with ASGs. Dark epoxy is used for electrical isolation.

However, the resolution of contact imagers degrades with increasing imager-tissue separation due to the divergence of unfocused light. While surgical “resolution” (i.e. the minimum amount of resectable tissue) is on the millimeter to centimeter scale, the need for higher spatial resolution is driven by the requirement to distinguish a small fluorescent MRD signal from the background signal due to non-specific antibody binding to healthy tissue. Given that this background can vary over the tissue surface due to variations in antibody distribution and tissue heterogeneity, high spatial resolution on the order of the cell cluster size to be detected is required for accurate background subtraction and edge detection. The resolution of the contact imager is determined by the angle of view (AOV) of individual pixels and not by the pixel pitch that can be on the micron scale in modern IC processes. Taking the full-width half-maximum (FWHM) of the angular response to be the AOV of standard CMOS photodiodes, its value can be in the range 100–120° and often it will not be the same in both the horizontal and vertical directions [49, 50], leading to uneven blurring in the image. This corresponds to a minimum field of view (FOV) for each pixel of approximately 0.62 mm with a 0.5 mm spacing between the image sensor and the imaged sample. This wide FOV per pixel limits the spatial resolution and cannot be mitigated by smaller pixel pitch.

In order to improve spatial resolution while minimizing imager-tissue separation, we directly fabricate ASGs on each photodiode in our image sensor (Fig. 1(c)). The ASGs do not focus light, but rather reject light that is not incident perpendicular to the sensor surface. This results in a reduction of the AOV of each pixel to 36° [51, 52], corresponding to a FOV of 0.22mm, spanning approximately 120 cells, for a 0.5 mm tissue-to-imager spacing. The pixel pitch is 55 μm, smaller than the FOV such that the resolution is not impacted. The image sensor has an active imaging area of 1.98 mm by 4.4 mm and a total area of 2.25 mm by 4.7 mm. The architecture is readily scalable to 1 cm² or larger. It is bonded to a printed circuit board (PCB) for testing purposes (Fig. 1(d)). A thin-film interference filter fabricated on a 3 mm by 5 mm, 0.5 mm thickness, fused silica substrate is epoxied on to the surface of the angle-selective image sensor using a clear optical epoxy (Fig. 1(e)). The fused silica substrate also serves to provide the spacing between the tissue and the image sensor for applying the excitation light. An optically opaque (black) epoxy, also visible in Fig. 1(e), is filled around the image sensor for electrical isolation. The sub-centimeter form factor of the sensor and filter combination is small enough for maneuverability inside a small resection cavity, allowing imaging of all side walls, while the large surface area ensures rapid imaging of the tumor cavity.

2.2. In Vitro characterization using a 3D cell culture model

In order to evaluate the sensitivity of our image sensor system, we imaged 3D cancer cell culture models in vitro. HER2-overexpressing HCC1569 breast cancer cells and PSMA-overexpressing LNCaP prostate cancer cells cultures were grown in a Matrigel layer approximately 40 μm (three to four cell layers) thick covered by a 150 μm thickness coverslip. These cells were fluorescently stained with anti-HER2 and anti-PSMA, respectively, followed by secondary antibody conjugated to a quantum dot, Qdot 705, for fluorescence visualization. Corresponding images were taken by both the custom imager and a fluorescence microscope. The microscope images were taken with a long 1 s integration time for clarity (Fig. 2(a,b)) and the corresponding custom sensor images were taken with only a 50 ms integration time (Fig. 2(c,d)). All the cancer cell clusters visible in the microscope image are identified in the custom sensor image. Clusters that are close to each other, with spacing close to or below the approximate 0.22 mm resolution of our image sensor, appear blurred together in the custom sensor image. The improved resolution relative to that of a conventional image sensor with wider angular response allows small, scattered clusters of cancer cells to be distinguished from background, which would not be possible if the emission light was spread widely over the surface area of the sensor.

Fig. 2.

Images of two locations of slides containing HCC1569 breast cancer 3D cell cultures. (a,b) Long, 1 s, integration time fluorescence microscope images. (c,d) Corresponding 50 ms images taken with the custom contact image sensor. (e) Histogram of HCC1569 and LNCaP cell clusters for SNR and size. The colorbar indicated how many cell clusters fall into each bin. HCC1569 clusters appear in the upper-left and LNCaP clusters appear in the lower-right. Integration times of 50 ms and 70 ms were used for HCC1569 and LNCaP samples, respectively. (f) Histogram showing SNR per decade calculated for each HCC1569 and LNCaP cluster imaged.

While signal-to-background ratio has been proposed as a metric for intraoperative imagers [53], this is not a sufficient or complete metric for characterization of MRD imagers. In contrast, the signal-to-noise ratio (SNR) includes the effect of measurement variability, including any background variability, and is a more appropriate metric, as noise presents the true limit to measurement accuracy. A uniform background level can be subtracted out using information from neighboring tissue. Therefore, we characterize sensitivity using the SNR as defined in section 3.4. Fig. 2(e) shows a histogram of cell clusters for SNR and cluster size in number of cells. The colorbar indicates how many clusters fall into each bin of the histogram. The SNR is calculated from the pixel with the maximum signal within a cluster in the custom sensor image and the cluster size is estimated from the corresponding microscope image. Results are shown for 109 HCC1569 cell clusters (upper-left) and 18 LNCaP cell clusters (lower-right). HCC1569 cancer cell clusters consisting of as few as about 25 cells are readily detectable with greater than 10 dB SNR, with an integration time of 50 ms. Consistent with the lower receptor (PSMA) expression on prostate cancer relative to HER2 expression on breast cancer cells [54, 55], the signal and the SNR are lower for LNCaP cell clusters and clusters consisting of greater than approximately 400 cancer cells are detectable with an integration time of 70 ms. Fig. 2(f) depicts a histogram of cell clusters versus the SNR per decade defined by

$$\frac{\mathit{\text{SNR}}}{{\text{log}}_{10}(N)},$$ (1)

where SNR is the signal-to-noise ratio for each cluster and N is the corresponding cluster size in cells. The SNR per decade is 9.1 ± 1.5 dB and 4.1 ± 0.6 dB for HCC1569 and LNCaP clusters, respectively.

2.3. Sensitivity and specificity

Analysis was performed on captured images to represent more accurately the requirements of intraoperative tumor identification. Sensitivity and specificity were determined from the component true positive, false positive, true negative, and false negative values. In order to evaluate the sensitivity and specificity of this measurement, we use an algorithm for mapping cell clusters identified in the fluorescence microscope images to corresponding distinct regions in the custom sensor images. The two images are overlaid and cell clusters that fall within a defined capture radius around the center of a cluster identified on the custom sensor image are considered to be true positives. The capture radius consists of the sum of a constant value, the offset radius, and an intensity-based value. Details and examples of this process are explained in section 3.5. Two modes of the algorithm are used. In the fully-automated mode, cell clusters are automatically detected in the custom sensor images and no user input is required. In the semi-automated mode of operation, additional cell clusters not automatically detected can be manually indicated by the user. The sensitivity and specificity for varying offset radius are plotted in Fig. 3 for the HCC1569 cancer cell clusters. The results for the fully-automated algorithm appear in Fig. 3(a). The results for the semi-automated algorithm appear in Fig. 3(b), where additional cancer cell clusters were identified by an oncologist. If the offset radius is set too small the algorithm does not adequately account for blurring in the image and many areas of cancer cells go undetected, leading to reduced sensitivity. If the offset is set too large, areas that are not cancerous can be misidentified, reducing the specificity. For an offset radius of 340 μm, the sensitivity and specificity are both 85 % using the fully-automated mode. Using the semi-automated mode of operation, the sensitivity and specificity can be improved to 92 % for an offset radius of 280 μm. This indicates that cancer can be reliably detected using the custom imaging system.

Fig. 3.

(a) Sensitivity and specificity when using the fully-automatic image recognition algorithm. For 340 μm offset radius, the sensitivity and specificity are both 85 %. (b) Sensitivity and specificity when using the semi-automatic recognition algorithm. Missed cancer cell clusters are manually identified in the custom sensor image. The sensitivity and specificity improve when using the semi-automated algorithm and for 280 μm offset radius, both sensitivity and specificity are 92 %.

2.4. Impact of increasing imager-tissue separation

During surgery, variation in the imager-tissue separation is introduced by blood and other fluid buildup on tissue. Given that a 250 μm layer of blood is opaque, representing a reasonable upper limit of intraoperative thickness, the image sensor needs to be robust to variations of this order. We examine the effect of sample-to-imager distance variation in this section, and the effect of scattering and absorption through blood in the next section.

We varied the separation in vitro by increasing the distance between the imager surface and the HCC1569 breast cancer cell cultures as shown in Fig. 4(a). A varying number of spacers, 150 μm thickness glass coverslips, coupled together using immersion oil of approximately 100 μm thickness are used to increase the distance from the nominal 0.65 mm to 1.8 mm. One hundred 50 ms integration time images of the same area were taken with the custom image sensor and averaged together to increase SNR and accurately evaluate the degradation in spatial resolution. The custom sensor images with increasing distance are shown in Fig. 4(b1–g1). To more easily visualize the change in spatial resolution, the images are normalized in Fig. 4(b2–g2), with each image scaled relative to its own maximum pixel. The reference 1s microscope image is shown in Fig. 4(h).

Fig. 4.

(a) Test setup for measuring distance variation. (b1–g1) One hundred averaged 50 ms custom sensor images taken at increasing distance from 0.65 mm to 1.8 mm in 0.25 mm intervals. The maximum SNR in each image is noted. (b2–g2) Normalized versions of the images to more clearly visualize the degradation in imaging resolution with increasing distance. (h) 1 s integration time microscope image for reference.

The two primary effects of increased imager-tissue separation are (1) a reduction in spatial resolution and (2) a decrease in SNR. The reduced resolution is primarily visible in Fig. 4(b2–g2) where two distinct cancer cell spots gradually blur together to form one larger spot as the distance is increased. Sub-millimeter resolution is achieved even at a large 1.8 mm separation and the SNR is only degraded by a gradual 6 dB mm⁻¹ as imaging distance is increased. With an expected intraoperative distance variation of 250 μm, the expected SNR variation is less than 1.5 dB.

2.5. Impact of blood scattering and absorption

While tumor cavities are irrigated continuously during surgical procedures, a residual thin layer of diluted blood is common. Without this irrigation, blood in the tumor cavity would obscure the surgeons’ view, hampering their ability to resect tissue. In order to evaluate the effect of a layer of blood with varying dilutions, we placed an adhesive chamber on top of a coverslip and filled the chamber with different dilutions of blood in phosphate-buffered saline (PBS). The liquid layer in the chamber is 250 μm thickness. The chamber was then placed in between our custom sensor and coupled to the slide holding HCC1569 breast cancer cell cultures using immersion oil (Fig. 5(a)). The total cancer cell to image sensor distance is approximately 1.25 mm. One hundred 50 ms integration time images were taken and averaged for each blood dilution in order to evaluate the effect of each dilution on the resolution of the image. The images are shown in Fig. 5(b1–g1) when filling the chamber with pure PBS, 100X, 30X, 10X, and 3X dilutions of blood in PBS, and undiluted blood, respectively. To more easily visualize the images, they are normalized in Fig. 5(b2–g2), with each image scaled relative to its own maximum pixel. The 1 s integration time fluorescence microscope image is shown in Fig. 5(h) for reference.

Fig. 5.

(a) Test setup for measuring the effect of different dilutions of blood. (b1–g1) One hundred averaged 50 ms custom sensor images taken for a pure PBS (clear) solution, 100X, 30X, 10X, and 3X dilutions of blood in PBS, as well as undiluted blood. The maximum SNR in each image is noted. (b2–g2) The images are normalized in order to more clearly visualize the increased noise for solutions containing more blood. (h) 1 s integration time microscope image for reference.

There is no noticeable reduction in spatial resolution as the blood concentration is increased, but there is a reduction in the SNR. The blood scatters the emission light isotropically, essentially trading signal for background (reducing SNR). Due to the presence of the ASGs, the majority of the scattered light is rejected by the image sensor and spatial resolution remains unaffected. While the SNR reduction is significant for high blood concentrations, up to 10 dB for a 250 μm layer of undiluted blood, with proper tumor cavity irrigation, dilutions on the order of 30–100X are common, and the SNR reduction will be limited to less than 2 dB.

2.6. Human HER2+ breast cancer tissue imaging

We demonstrate that tumor borders are clearly detected and tumor distinguished from normal tissue by imaging human HER2+ cancer tissue from a breast cancer patient. Slices from frozen tissue are mounted on a glass slide and imaged simultaneously using a fluorescence microscope and our custom sensor. The 75 ms integration time microscope and custom sensor images are shown in Fig. 6(a) and Fig. 6(b), respectively. In order to reduce noise, both microscope and custom sensor images are filtered with a 2D Gaussian filter with standard deviation equal to two pixels on the custom sensor, or 110 μm. The filtered versions are shown in in Fig. 6(c) and Fig. 6(d), respectively. A long integration time 5 s integration time microscope image is shown in Fig. 6(e) for reference. The brightest areas correspond to cancer tissue, the darkest areas to background from the slide, and the intermediate brightness areas to healthy tissue. The tumor margin is confirmed with immunohistochemistry and hematoxylin and eosin (H&E) stain images as shown in Fig. 6(f1) and Fig. 6(g1) of the areas indicated in the two labeled boxes in Fig. 6(e). Additional higher magnification fluorescence images (red) overlaid on DAPI images (blue) are shown in Fig. 6(f2) and Fig. 6(g2) for reference.

Fig. 6.

Images of human HER2+ breast cancer tissue slices. (a) Raw 75 ms fluorescence microscope image. (b) Raw 75 ms custom sensor image. (c) Fluorescence microscope image filtered with 2D Gaussian filter with standard deviation 110 μm. (d) Custom image filtered with 2D Gaussian filter with standard deviation 110 μm. (e) 5 s integration time fluorescence microscope image for reference. (f1,g1) Two regions, labeled f and g in (e), were examined using hematoxylin and eosin (H&E) as shown in order to confirm tumor margin. (f2,g2) Higher magnification fluorescence images (red) are shown overlaid on DAPI stain images (blue) of the same regions.

The tumor margin is clearly visible in the custom sensor images, with improved SNR after filtering. The unfiltered microscope image has significant noise, but after filtering has similar resolution and noise to the filtered image of the custom sensor. While the long integration time microscope image shows the cancer and healthy tissue in higher resolution, this is not possible with short integration times due to excessive noise. For the same, 75 ms integration time, our custom sensor is used to find the tumor margin with similar performance, but with a significantly smaller form factor than the fluorescence microscope, allowing it to be used intraoperatively.

3. Materials and methods

3.1. 3D cell culture models

For characterization of the imager, three-dimensional (3D) cell culture models of breast and prostate cancer were used to generate cell clusters of varying size. Breast cell cultures consisted of HCC1569 (HER2-overexpressing) (ATCC) cultured in RPMI with 10 % FBS. 3D culture models were generated in Matrigel as described in prior work [56, 57]. Cells were stained with HER2/ErbB2 (29D8) rabbit monoclonal antibody (#2165, Cell Signaling, Danvers MA) followed by anti-rabbit quantum dot (Qdot) 705 (Q11461MP, Thermofisher Waltham, MA). Prostate cell cultures consisted of LnCAP cells, stained with anti-PSMA rabbit antibody (#D4S1F, Cell Signaling, Danvers MA) followed by anti-rabbit Qdot 705. Each staining step takes approximately one hour at room temperature, for a total of two hours.

3.2. Ex vivo tissue margin

Ex vivo tissue margins were generated from patient samples of HER2+ and triple-negative breast cancers and compared with normal tissue counterparts from the same patient. Anonymized patient samples were obtained from Bioserve (Betsfiel, MD). Tissue was frozen in order to preserve enzyme and antigen functionality and 6 μm sections (slices) were cut for imaging. These slices of HER2+ tumor were stained with anti-HER2 antibody and secondarily with Qdot 705. Each staining step takes approximately one hour at room temperature, for a total of two hours. Adjacent slices were H&E stained for comparison. The slices were allowed to return to room temperature before imaging, preserving their optical properties.

3.3. Test setup

The custom imager with optical filter is mounted on a PCB. A field-programmable gate array, also mounted on the PCB, is used to communicate between the image sensor and a PC. A slide containing 3D cancer cell cultures is placed on an XY-stage (MLS203) above an inverted fluorescence microscope (Leica DM IRB) with a 2.5X objective. The custom imager is suspended above the microscope and positioned in contact with the cell sample slide using a Thorlabs XYZ stage (RB13M). Light from an Excelitas X-Cite 120LED attached to the microscope is passed through an optical filter to illuminate the 3D cultures at 450 nm. Images are collected at 700 nm by the custom image sensor and an Hamamatsu ORCA-Flash4.0 V2 microscope camera.

3.4. SNR calculation

The signal-to-noise ratio (SNR) was calculated by taking 100 consecutive images at a set integration time, 50 ms for HCC1569, 70 ms for PC3 cells. The SNR in dB is then calculated as

$$\mathit{\text{SNR}}=20\cdot {\mathit{\text{log}}}_{10}(\frac{\overline{V_{\mathit{\text{pixel}}}}}{\mathit{\text{std}}(\overline{V_{\mathit{\text{pixel}}}})}),$$ (2)

where $\overline{V_{\mathit{\text{pixel}}}}$ and std(V_pixel) are the mean and standard deviation of the voltage of each pixel over the 100 images, respectively. The SNR for each cell cluster is defined as the SNR of the pixel with maximum intensity within that cluster.

3.5. Cell cluster recognition for sensitivity and specificity

In order to evaluate sensitivity and specificity, clusters of cancer cells were detected using both a fully-automated and semi-automated process in the custom sensor image and a fully-automated process in the microscope image. The positions of the clusters were compared to determine correctly and incorrectly detected cancer cell clusters. The detection process is illustrated in Fig. 7. The raw custom sensor image Fig. 7(a) is non-linearly scaled to make the equalized image Fig. 7(b). This image is thresholded and clusters are automatically identified in red when using the fully-automated mode of operation. If the semi-automated mode is used, additional clusters can be manually by the user. The manually added clusters are shown in orange in Fig. 7(c). An overlay of the identified clusters and the equalized image is shown in Fig. 7(d). A similar process is used to identify clusters in the microscope image as shown in Fig. 7(f–i), but all clusters are automatically detected. An overlay of the identified clusters in both images on top of the equalized custom sensor image is shown in Fig. 7(e). Areas of the custom sensor image assigned to a cluster are marked within a circle with radius equal to the capture radius as described below. Microscope clusters within the capture radius of a cluster are classified as true positives and marked with green squares. Clusters that fall outside of this radius are classified as false negatives and marked with light blue circles. A false positive corresponds to a cluster identified in the custom sensor image that has no corresponding cluster in the microscope image. A value for true negative is calculated by dividing the area outside all capture radii by the area of a circle with radius equal to the offset radius. An overlay of the identified clusters on the equalized microscope image is shown for reference in Fig. 7(j).

Fig. 7.

(a) Raw custom sensor image. (b) Non-linearly scaled equalized custom sensor image. (c) Regions are identified with red and orange markers. (d) Regions are overlaid on the equalized image. (e) The clusters identified in the custom sensor image appear overlaid with the clusters that are identified in the microscope image using a similar procedure to the one described in (a–d) and illustrated in (f–i). (j) An overlay of the identified clusters on the microscope image for reference.

The capture radius for each cluster identified in the custom sensor image consists of the sum of a constant factor and an intensity-dependent quantity:

$$\mathit{\text{CR}}=\mathit{\text{OR}}+\mathit{\text{IR}},$$ (3)

where CR is the capture radius, OR is the offset radius, and IR is the intensity radius, all with units μm. The offset radius is a constant factor and accounts for blurring in the custom sensor image. The intensity radius is given as:

$$\mathit{\text{IR}}=\mathit{\text{IF}}\cdot I,$$ (4)

where IF is the intensity factor with units μm V⁻¹, and I is the mean signal level of all the pixels identified to be within a cluster. Fig. 8 shows an example custom sensor image with the offset radius indicated as a brown line and the intensity radius indicated with a red dashed line. True positives (green squares) and false negatives (light blue circles) are also shown.

Fig. 8.

The cell clusters identified in the custom sensor image (red cross) and microscope image are compared. Clusters from the microscope image are classified as true positives (green square) if they fall within the capture radius of any cluster identified in the custom sensor image. Otherwise they are considered false negatives (light blue circle).

4. Discussion and conclusion

Although the presence of MRD after a resection is known to significantly increase the chance of cancer recurrence across cancer subtypes, thorough, rapid visualization of MRD with 200 cells resolution within the small, complex, tumor bed remains a persistent challenge. Our scalable contact fluorescence image sensor can both fit and be maneuvered within a tumor cavity while imaging rapidly. It is enabled by a combination of on-chip ASGs integrated in a high-sensitivity custom CMOS image sensor and an integrated optical filter.

Using 3D breast and prostate cancer cell cultures as model systems, we characterized our imager on a variety of in vitro samples, establishing sensitivity, response to imager-cell separation, and scattering due to blood. We demonstrated detection of small HCC1569 clusters containing as few as 25 cells with high SNR for only a 50 ms integration time. A 70 ms integration time was used to detect LNCaP clusters containing as few as 400 cells. The 0.22 mm sensor resolution is high enough for edge detection and accurate MRD cell cluster recognition.

Using a relatively simple fully-automated cell cluster recognition and detection algorithm, we can achieve a high sensitivity and specificity of 85 %. The sensitivity and specificity can be improved to 92 % by allowing the user to manually identify additional clusters in the custom sensor image. We take into account only the center location of a cluster in the custom sensor image and its intensity level for automatic recognition, and not its precise morphological structure. The algorithm and the sensitivity and specificity could be further improved by recognizing the cluster boundary and morphology. This would be especially beneficial for clusters that are oblong or irregular, rather than spherical. However, despite the use of a simple and straightforward to implement algorithm, the sensitivity and specificity are high. This indicates that the imager is a reliable platform that allows easy identification of residual cancer. Additionally, machine learning techniques can also be applied to generate an accurate classification algorithm [38, 58].

Importantly, we have also demonstrated our imager’s robustness to imager-tissue variation due to the ASGs which block diverging light. Imager distance variation can be caused by blood or other liquid between the sensor and the tissue being imaged or due to the hand motions of the surgeon. In response to distance variation, there is only a 6 dB mm⁻¹ decline in SNR. For practical distance variations of less than 250 μm, the SNR degradation is less than 1.5 dB. Sub-millimeter resolution images are obtained even at 1.8 mm imaging distance.

During surgery, irrigation is used to keep the tumor cavity clear of blood in order to visualize the tissue surface and any small bleeding blood vessels. Electrocautery devices are used to stop this bleeding. The vast majority of the tumor bed is not cauterized, but only irrigated to ensure hemostasis and easy visualization. However, this process leaves a liquid residual diluted blood layer. We have demonstrated imaging through this layer. With 250 μm layers of 30X–100X diluted concentrations of blood there is less than a 2 dB reduction in SNR measured with our imager. The blood layer does not degrade resolution, but only decreases SNR as blood concentration is increased, indicating that the scattered light is largely blocked by the ASGs.

We have also demonstrated imaging and detection of tumor margin in thin, 6 μm, slices of frozen breast cancer tissue taken from a breast cancer patient. Tumor tissue and healthy tissue are easily distinguishable with integration times of only 75 ms. By imaging a tissue slice, we have demonstrated imaging for a worst-case scenario that represents MRD.

Currently our imager is mounted on a rigid PCB for electrical connections, however it can be mounted on a flexible PCB similar to those used for other medical probes and imaging devices [59, 60]. These flexible PCBs do not have the bending radius restrictions that fiber optic bundles do, allowing them to be used intraoperatively. Our complete imaging system will both have the small-scale and maneuverability necessary for in vivo use.

In summary, using HER2+ breast cancer and PSMA-overexpressing prostate cancer as model systems, we have demonstrated highly sensitive cell cluster imaging in a scalable, lens-free platform. This platform can be used to image any type of cancer with a targeted antibody and the sensor is small enough to be used inside tumor cavities typically found in modern minimally invasive surgical procedures. We have shown that our sensor is robust to both imaging distance variation and the presence of a layer of blood in the imaging path, indicating that it maintains a high performance and is able to detect cancer rapidly under real-time operative conditions.

Funding

Department of Defense (DoD) (PC141609); the Mary Kay Ash Charitable Foundation (P0506430); NIH/NCRR UCSF-CTSI grant (UL1 TR000004); the American Society of Clinical Oncology (ASCO) (4300-134078-44).

Disclosures

The authors declare that there are no conflicts of interest related to this article.