Surface plasmons-phonons for mid-infrared hyperspectral imaging

Abstract

Surface plasmons have proven their ability to boost the sensitivity of mid-infrared hyperspectral imaging by enhancing light-matter interactions. Surface phonons, a counterpart technology to plasmons, present unclear contributions to hyperspectral imaging. Here, we investigate this by developing a plasmon-phonon hyperspectral imaging system that uses asymmetric cross-shaped nanoantennas composed of stacked plasmon-phonon materials. The phonon modes within this system, controlled by light polarization, capture molecular refractive index intensity and lineshape features, distinct from those observed with plasmons, enabling more precise and sensitive molecule identification. In a deep learning–assisted imaging demonstration of severe acute respiratory syndrome coronavirus (SARS-CoV), phonons exhibit enhanced identification capabilities (230,400 spectra/s), facilitating the de-overlapping and observation of the spatial distribution of two mixed SARS-CoV spike proteins. In addition, the plasmon-phonon system demonstrates increased identification accuracy (93%), heightened sensitivity, and enhanced detection limits (down to molecule monolayers). These findings extend phonon polaritonics to hyperspectral imaging, promising applications in imaging-guided molecule screening and pharmaceutical analysis.

Surface phonons enhance the molecular identification capabilities in hyperspectral imaging.

INTRODUCTION

Hyperspectral imaging represents a mapping technique combining imaging and spectroscopy to characterize the spatial distribution of biochemical constituents (1). Encompassing the mid-infrared (IR), near-IR, and visible regions, it finds extensive application in diverse domains, such as continuous monitoring of multiple biomolecules (2). In particular, mid-IR hyperspectral imaging (MIR-HSI) uniquely offers visualized location information and conformational fingerprint features for molecular screening (3), supplementing existing modalities such as polymerase chain reaction (PCR) (4), quantum biosensors (5), and optical dynamic monitoring of live biosamples (6). Initially constrained by low intrinsic molecular absorption cross sections, MIR-HSI necessitated large sample volumes for observable changes. Advancements in nanophotonics have broadened its capacities to encompass advanced functionalities. For instance, nanoantennas are vital nanophotonic components constructed from metals (7–10), dielectrics (11), or even DNA (12–14). Nanoantenna-enabled trace detection in hyperspectral imaging has been demonstrated through enhanced light-molecule interactions (3), known as surface-enhanced IR absorption spectroscopy (SEIRAS) (15–22). In addition, compact, miniaturized hyperspectral imaging is attained by leveraging the subwavelength scale and extraordinary light manipulation capabilities of nanoantennas. Real-time dynamic hyperspectral imaging is achieved by encoding and reconstructing multidimensional light properties (amplitude, phase, and wavelength) using nanoantennas. Machine learning–augmented nanophotonics is advancing hyperspectral imaging toward intelligent clinical diagnosis. Collectively, nanophotonics is propelling rapid progress in hyperspectral imaging. Exploring integration pathways between nanophotonics and hyperspectral imaging represents a promising technological frontier.

Surface phonon polaritons (referred to as “phonons”) are collective oscillation modes of atomic displacements in polar dielectrics (23), occurring within the frequency range bounded by the longitudinal optic (LO) and transverse optic (TO) phonon frequencies, denoted as the “Reststrahlen” band. It provides remarkable optical characteristics, such as strong electromagnetic field confinement, long lifetime, and low loss (24). Recent advancements have used phonons in spectroscopy (25), exemplified by the successful utilization of highly confined phonons within Ge-SiC resonators for achieving ultrasensitive IR spectroscopy (26). In addition, phonon manipulation via van der Waals nanostructures has shown promise for miniaturized IR spectroscopy applications (27). Furthermore, the coupling of phonons with surface plasmons has been demonstrated to attain vibrational ultrastrong coupling (28), underscoring its potential for enhancing spectroscopic capabilities. These demonstrations provoke consideration regarding the potential integration of phonons with hyperspectral imaging to fully leverage their distinctive optical characteristics. Such inquiry could offer avenues for conceptual and technological advancements in nanophotonic hyperspectral imaging.

To date, surface phonon polariton–enhanced hyperspectral bioimaging remains undocumented (table S1). The first inquiry arises regarding the phononic excitation and interaction mechanisms with molecules in hyperspectral imaging. Phonon excitation entails overcoming the momentum mismatch between incident photons and phonon modes (29). In addition, phonon resonance is confined to a narrow Reststrahlen band, thereby incapable of spanning the entire molecular vibrational fingerprint range (30). The presence of strong nonlinearity, characterized by the “slowing down” of spectral shift and “pinning” of phononic resonance (31), adds complexity to the feasibility of phononic resonance directly interacting with molecular vibrations for phononic bioimaging. The second inquiry is the autonomous manipulation of phonons and plasmons. Illustrated by a bioimaging system integrating phononic and plasmonic elements, both use a shared light source and photodetector. The simultaneous presence of phonon and plasmon signals in hyperspectral images may give rise to undesired mutual interference and signal masking. Therefore, ensuring independent control over phonons and plasmons is crucial for attaining seamless system integration.

Here, we investigate the phonons for hyperspectral imaging by developing a plasmon-phonon–integrated hyperspectral bioimaging system, using asymmetric cross-shaped nanoantennas composed of stacked plasmonic and phononic materials (Fig. 1). Initially, the phonon mode within the system is excited through plasmon-phonon coupling, overcoming momentum mismatch, instead of the direct excitation of narrow phonon resonances by light. This approach effectively addresses the limitation of the narrow excitation range associated with phonon resonance. Then, the activated phonons interact with molecules to detect molecular refractive index instead of molecular vibrations. This enables phonons to capture signals distinct from molecular vibrations compared to plasmons. The polarization of incident photons can serve as an effective means to control plasmon and phonon excitations within the system. In a deep learning–assisted imaging demonstration targeting the two spike proteins of the severe acute respiratory syndrome coronavirus (SARS-CoV) (32), we demonstrate that phonons exhibit enhanced identification capabilities, achieving the de-overlapping and observation of the spatial distribution of two mixed SARS-CoV spike proteins. Furthermore, we illustrated the versatility of our system across various scenarios, including species identification, concentration prediction, spectral reconstruction, and hyperspectral imaging of protein monolayers using both laboratory and clinical SARS-CoV samples.

Fig. 1. Workflow of the plasmon-phonon hyperspectral imaging system.

(A) Schematic view of dual plasmon-phonon (DP) nanoantennas featuring immobilized SARS-CoV spike proteins (S-protein-1 and S-protein-2) on their surface. The proteins are anchored as a mixed monolayer employing the self-assembled monolayer (SAM) technique. The absorption spectra of the two proteins exhibit overlap at their amide groups (circular inserts). The excitation of phonons and plasmons is controlled by the polarization of the incident light. (B) Acquisition of hyperspectral datacubes for DP nanoantennas using a tunable QCL-based mid-IR spectral imaging system. Polarization along the short arm of antennas for plasmon excitation is termed plasmon polarization, whereas polarization along the long arm for phonon excitation is referred to as phonon polarization. Sequential acquisition of hyperspectral datacubes under the plasmon and phonon polarizations is performed for DP nanoantennas. (C) Multimodal deep neural network (MM-DNN) model for hyperspectral image processing. (D) Output prediction of de-overlapped distribution for mixed monolayers of S-protein-1 and S-protein-2 on DP antennas. Scale bars (insets), 300 μm. Serial numbers 1 to 5 depict key steps: (1) Sample immobilization, (2 and 3) polarization-controlled acquisition of plasmon and phonon hyperspectral datacubes, (4) image processing, and (5) prediction. The 3D model of S-protein-1 and S-protein-2 was imported from RCSB Protein Data Bank, DOI: 10.2210/pdb8h16/pdb (S-protein-1) and 10.2210/pdb7vhh/pdb (S-protein-2). The S-protein-1 data were deposited by Zhang et al. (56), in 2022, under the title “Structure of SARS-CoV-1 Spike Protein (S/native) at pH 5.5, Open Conformation”. The S-protein-2 data were deposited by Liu (57), in 2021, under the title “Delta variant of SARS-CoV-2 Spike protein”. All rights reserved.

RESULTS

Methodology

The nanophotonic device consists of asymmetric cross-shaped Au nanoantennas and SiO₂ nanoantennas, both stacked and situated on a CaF₂ substrate, termed dual plasmon-phonon (DP) nanoantennas (Fig. 1A). Plasmons are excited in the Au nanoantennas, while phonons are excited in the SiO₂ nanoantennas, which are controlled by the polarization of the incident light. Figure 1 illustrates the systemic workflow for in vitro hyperspectral bioimaging of trace samples using DP nanoantennas. The procedure comprises five steps. Step 1 involves immobilizing a monolayer of two SARS-CoV spike proteins (S-protein-1 and S-protein-2) onto the surface of DP nanoantennas using self-assembled monolayer (SAM) technology (Fig. 1A). The absorption spectra of the two proteins exhibit overlap at their amide groups, presenting a challenge for molecular identification. Our method uses the plasmons and phonons of DP nanoantennas to identify these proteins with spectral overlap. Then, in steps 2 and 3 (Fig. 1B), hyperspectral imaging datacubes are collected under polarization control, where one datacube contains plasmon signals and the other is phonon signals. It shows that phonons capture different spectral features from those observed with plasmons, enabling more accurate identification of molecules (top of Fig. 1C). Then, in step 4, the plasmon and phonon datacubes acquired in steps 2 and 3 are input into a multimodal deep neural network (MM-DNN) to predict the de-overlapped images of samples in step 5 (Fig. 1D). The spatial information for individual components within mixed samples is observed in the de-overlapped images. Three pivotal scientific and technical aspects in the workflow warrant emphasis.

The first aspect involves the excitation of phonon modes, wherein our approach incorporates patterned phonon nanoantennas positioned beneath plasmonic nanoantennas. Incident light passing through the plasmonic nanoantenna is localized into an enhanced near field, thereby coupling with phonons to activate phonon modes (fig. S1). The coupling mechanism involves the interaction between plasmon-associated near fields and phonon-associated lattice vibrations (28), facilitating their energy and momentum transfer (33). Our phonon signal encompasses the entire Reststrahlen band (fig. S8), in contrast to the narrower coverage of one-third observed in phonon modes through direct interaction with light (34). The broadening mechanism operates as follows: Direct excitation leverages the metallic properties of phononic materials within the Reststrahlen band (35). Phononic nanoantenna resonances exhibit high-quality factors and narrow linewidths due to the low optical loss of phononic materials, localized within the Reststrahlen band. However, our approach involves coupling plasmonic modes with phononic materials to excite phonon modes, entailing the interaction between plasmon near fields and phonon lattice vibrations. With lattice vibrations spanning the entire Reststrahlen band, the phonon signal extends beyond this band rather than being confined within it. Besides, the stacked design of DP nanoantennas exhibits heightened sensitivity compared to conventional single-layer antennas. The sensitivity is related to the spatial overlap between E-field and target molecules. Increasing the proportion of the electric field exposed to free space, where target molecules reside, can amplify this spatial overlap. To describe it, we define the ratio of E-field exposed to free space (R_E) as the proportion of the electric field exposed to free space in the entire region S, where S denotes the region adjacent to the antenna’s end face (see fig. S1, text S1, and Materials and Methods for details). The simulation reveals that the R_E for the single-layer nanoantennas is 52.7%, while the DP nanoantennas achieve 72.3%. The increased R_E of DP nanoantennas is attributed to the coupling of phonons and plasmons, altering the electric field distribution of nanoantennas. This coupling attracts the electric field closer to the coupling interface, situated beyond the substrate. In contrast, a notable portion of the electric field of conventional antennas resides within the substrate. This observation is further supported by the electric field distribution depicted in fig. S2.

The second is polarization control for plasmonic and phononic imaging. The asymmetric design of DP nanoantennas features unequal arm lengths (a short arm and a long arm). The asymmetric crosses exhibit two distinct resonances along their arms under polarization control (figs. S9 and S10) (36, 37). We assign plasmon modes to the short arms and phonon modes to the long arms. Specifically, the plasmonic resonance frequency of short arms is strategically positioned away from the phonon frequency, while aligning with the vibrational frequency of protein samples for SEIRAS-based vibrational imaging (fig. S8). In contrast, the plasmonic resonance frequency of long arms deliberately aligns with the phononic Reststrahlen band to excite phonon modes for imaging. Hence, manipulation of the polarization direction allows for the selective excitation of plasmonic modes in the short arms or phonon modes in the long arms, denoted as plasmon polarization and phonon polarization, respectively.

The third is the different interactions of plasmonic and phononic modes with molecules. Molecular optical properties related to light-matter interactions include refractive index (n) and extinction coefficient (k). In the mid-IR region, k manifests as the vibrational absorption of molecules, detectable through the near-field enhancement of plasmon modes (38–40). Nonetheless, the overlapping of vibrational absorption in mixed proteins presents a challenge for molecular identification (Fig. 2A). We observed notable differences in the lineshape and intensity of the n within the overlapping region. These differences can be captured by phonon modes through phonon-molecule coupling (top of Fig. 2B), thus offering enhancements for molecular identification. This detection mechanism, termed n-sensitive phonon variation, will be investigated in the later section. It is different from the prevailing n-detection mechanism based on conventional nanoantennas (41), wherein detection relies on resonant frequency shift, yielding an output value indicating the extent of the shift. In phonon modes, the lineshape and intensity of phonon signals are linked to the frequency-varying feature of n (top of Fig. 2C), establishing a one-to-one correlation (fig. S11). It is advantageous for molecular screening and identification. Notably, the parameters n and vibration-related k can be mutually converted through Kramers-Kronig relations (42). However, the n-k conversion based on the nanoantenna’s measured spectrum is inaccurate due to signal distortion attributed to the Fano effect (43) within the antenna-molecule coupling system (fig. S12). Furthermore, the conversion is time-consuming for hundreds of thousands of spectra in hyperspectral image datacube, diminishing the de-overlap speed. In addition, the error accumulation in the conversion could reduce the accuracy of molecular identification.

Fig. 2. Response mechanism of DP nanoantenna’s phonon and plasmon signals to samples.

(A) Calculated refractive index (n) and vibration-related extinction coefficient (k) of S-protein-1 and S-protein-2. (B) Schematic of plasmon and phonon modes in DP nanoantennas, with mode excitation controlled by incident light polarization. Phonon mode (top) detects protein samples through phonon-protein coupling, while plasmon mode (bottom) interacts with protein samples via enhanced near fields, namely, the SEIRAS effect. (C) Simulated phonon and plasmon signals of DP nanoantennas in protein detection. The phonon signals (red curves, top) are sensitive to the n of proteins, while plasmon reflection spectra (lower panel) reveal enhanced detection for the k of proteins.

Refractive index-sensitive phonon variation

Next, we theoretically and experimentally investigate the response mechanism of phonon signals to samples, namely, n-sensitive phonon variation. We derive coupled equations and subsequently solve them to determine the spectral dispersion of reflection as (text S2)

$$R_0={\mid r\mid }^2={\mid -\frac{{\mathrm{\xi }\mathrm{\sigma }}_{\text{se}}}{2+{\mathrm{\xi }\mathrm{\sigma }}_{\text{se}}}\mid }^2$$ (1)

where ξ is the vacuum impedance of the external waves, and σ_se is surface conductivity. This foundational model is inadequate in describing the phonon signal accurately. In the model, the phonon signal assumes a Lorentz shape (fig. S3), contrasting with the authentic phonon signal characterized by two modes, representing out-of-phase atomic lattice vibrations, with distinct TO and LO phonon frequencies. This issue has been mentioned in the previous work (28). To solve it, we formulated a dual-damped oscillator model (text S3). The model yields the spectral dispersion of reflection as follows

$$R_{\text{phonon}}={\mid r\mid }^2={\mid -\frac{{\mathrm{\xi }\mathrm{\sigma }}_{\text{se}\_\text{phonon}}}{2+{\mathrm{\xi }\mathrm{\sigma }}_{\text{se}\_\text{phonon}}}\mid }^2$$ (2)

$${\mathrm{\sigma }}_{\text{se}\_\text{phonon}}=-{\mathit{\text{in}}}_{\mathrm{s}}\mathrm{\omega }\frac{{\mathrm{\Gamma }}_{\mathrm{m}1}{\mathrm{\Gamma }}_{\mathrm{m}2}}{{\mathrm{\Gamma }}_{\mathrm{p}}{\mathrm{\Gamma }}_{\mathrm{m}1}{\mathrm{\Gamma }}_{\mathrm{m}2}-{c_1}^2{{\mathrm{\kappa }}_1}^2{\mathrm{\Gamma }}_{\mathrm{m}2}-{c_2}^2{{\mathrm{\kappa }}_2}^2{\mathrm{\Gamma }}_{\mathrm{m}1}}$$ (3)

where Г_m1,m2,p are frequency-related terms of dissipative oscillator and external force, κ is coupling strength, and c is a constant. Figure 3A is the calculated reflection spectra of the plasmon-phonon coupling system using the revised model. It is closer to the experimental measurements in contrast to the results of the basic model (fig. S4). Figure 3B illustrates phonon signals characterized by intensity and shape information. As the molecular n in the coupling system varies, the phonon signal (I) undergoes concurrent variations in both intensity and shape (Fig. 3C). It demonstrates the sensitivity of the phonon signal to the n. The intrinsic physical mechanism is postulated as follows: Upon loading molecules, the molecule refractive index change (Δn) causes a redshift in the plasmon frequency (44). However, the phonon frequency in the SiO₂ antenna is fixed and not affected by Δn (28). This will cause a change in the matching between plasmon resonances and phonon polarons, referred to as detuning (δ). The detuning change (Δδ) further causes the phonon signal variation (ΔI) because the plasmon-phonon coupling intensity is influenced by δ (43). To substantiate this hypothesis, we varied the plasmon frequency in the model while maintaining a fixed phonon frequency (fig. S5), thereby simulating the Δδ. We observe that the Δδ induces the ΔI, affirming that Δδ is the determinant factor rendering n-sensitive phonon variation.

Fig. 3. Theoretical analysis and optimization of n-sensitive phonon variation.

(A) Calculated reflection spectra of plasmon antennas with/without phonon coupling. (B) Extracted phonon signal curve. It is characterized by both intensity (Int. I) and shape information. (C) The n of molecules changed from 1.2 to 1.26. There is a one-to-one correlation between the n and the phonon signal curve in (B). (D) Calculated 2D mapping of phonon signal intensity as a function of coupling strength (κ) and normalized detuning (δ). (E) Calculated phonon intensity and sensitivity as a function of δ. The three curves of phonon signal intensity in the top correspond to the three specific configurations in (D). By derivation, the phonon signal intensity is calculated as the phonon sensitivity in the bottom. (F) Calculated 2D mapping of phonon sensitivity as a function of κ and δ. (F) is the derivative of (D). (G) Calculated 2D mapping of phonon sensitivity when the damping γ_p changes from 18 to 1 THz.

Next, we optimize the sensitivity of the phonon signal. According to Eq. 2, when detuning (δ) and coupling intensity (κ) are considered as variations, we can obtain a two-dimensional (2D) mapping of phonon signals, illustrating their dependency on δ and κ, as depicted in Fig. 3D. In terms of κ, the phonon intensity exhibits an initial increase followed by saturation as κ rises. The saturation is reached upon entry into the strong coupling regime (κ > γ) (45), denoted as point b. Regarding δ, the phonon intensity exhibits a progression characterized by an initial increase followed by a subsequent decrease with increasing δ. Figure 3E illustrates the computed phonon sensitivity by deriving the derivative of phonon intensity concerning the δ direction. As observed, sensitivity is at its minimum when phonon intensity reaches its maximum (point b with δ = 0). Two configurations with maximal sensitivity exist at points a and c, where δ ≠ 0. This principle should serve as a universal guide for the meticulous and highly sensitive design of the DP nanoantennas. This conclusion remains valid in the comprehensive mapping diagram encompassing all κ (Fig. 3F). Notably, sensitivity saturation occurs earlier with the increase of κ, specifically at point d (κ = 25 THz), compared to the saturation of phonon intensity at point b (κ = 28 THz). Hence, to attain heightened sensitivity, the κ of the DP nanoantennas should be positioned in proximity to point d through adjustment of the phonon antenna thickness. In conjunction with δ and κ, the phonon sensitivity is influenced by the damping (γ) of the coupled system, intricately linked to the energy dissipation rate. There are two kinds of damping in the harmonic oscillator model: bright-mode damping (γ_p) and dark-mode damping (γ_m). A decrease in γ_p results in a concurrent increase in both maximum phonon intensity and sensitivity (Fig. 3G and fig. S6). Regarding γ_m, its influence on phonon intensity and sensitivity is comparatively minimal in comparison to γ_p (fig. S7). Hence, the damping of the DP nanoantennas should be meticulously designed with a reduced value.

Next, the DP nanoantennas were fabricated to empirically validate the aforementioned theoretical analysis. Figure 4 (A to C) depicts the scanning electron microscopy (SEM) and atomic force microscopy (AFM) images of the DP nanoantennas, illustrating a well-defined pattern outline and affirming the efficacy of the fabrication process. We examine detuning (δ) by changing the long-arm length of nanoantennas to adjust the plasmon resonance. This exploration establishes the association of δ with n-sensitive phonon variation. Figure 4D is a 2D mapping of experimentally measured reflection as a function of antenna length and wave number when the thickness of the SiO₂ nanoantenna is 30 nm. The phonon signal is observed within the Reststrahlen band, spanning from ω_LO to ω_TO. Zero detuning is defined as the coupling state where the uncoupled plasmon resonance (R₀) aligns with the longitudinal vibration of the phonon at ω_LO. Then, the correlation between detuning and both phonon signal strength and sensitivity can be derived from the 2D mapping plot (Fig. 4G). The observed trend in Fig. 4G aligns consistently with that depicted in Fig. 3E. In addition, the phonon intensity varies with δ, reaching an optimum at δ = 0, whereas the optimal sensitivity diverges, reaching its peak when δ ≠ 0. These experimental findings substantiate earlier theoretical conclusions regarding δ.

Fig. 4. Experimental demonstration of detuning-induced phonon variation.

(A to C) SEM and AFM images of the DP nanoantennas. (D to F) 2D mapping of experimentally measured reflection as a function of antenna length and wave number when the thickness of the phonon nanoantenna t_phonon is 30 (D), 70 (E), and 110 nm (F). The polaritonic resonances of the upper (R_upper) and lower (R_lower) branches are analytically fitted using eq. S19. R₀ is the resonance of plasmonic nanoantennas without coupled with the phonon. Zero detuning is defined as the coupling state at the intersection between the uncoupled resonance (R₀) and the phonon longitudinal vibration (ω_LO), where the normal mode splitting (Ω) is measured. (G to I) Experimentally measured phonon signal intensity (Int. I) and sensitivity as a function of δ. (G) to (I) are calculated based on the data in (D) to (F).

Regarding coupling intensity (κ), its examination involves altering the thickness of the phonon antenna. Figure 4 (D to F) depicts the 2D mapping of experimentally measured reflection for phonon antenna thicknesses of 30, 70, and 110 nm, respectively. Common to all is the splitting of plasmon resonance (R₀) into upper and lower polaritonic resonances (R_upper and R_lower) when R₀ bifurcates near ω_LO. The splitting (Ω) is linked to the coupling strength, which, in turn, is correlated with the thickness of the phonon antenna (t_phonon). Mathematically, ultrastrong coupling is defined as a normalized coupling strength (η) exceeding 0.1. η is calculated as g/ω_m, where ω_m represents the phonon frequency and coupling strength (g) is derived from Ω. Calculation results (table S2) reveal that η rises with increasing t_phonon, and it reaches 0.116 (>0.1) at t_phonon = 110 nm, indicating entry into the ultrastrong coupling regime. The derived phonon intensity and sensitivity (Fig. 4, H and I), calculated from Fig. 4 (E and F), validate the findings regarding κ presented in Fig. 3. For instance, as κ varies, both the maximum phonon intensity (Int. I_max) and sensitivity (S_max) exhibit proportional increases. Specifically, as t_phonon varies from 30 to 110 nm, S_max rises from 6 to 75, and Int. I_max increases from 3.5 to 70.

Species classification

In the subsequent sections, we present three demonstrations to investigate the plasmon-phonon imaging system. First is the label-free classification of mixed SARS-CoV spike protein samples characterized by severe spectral overlap. Initially, the DP nanoantennas present polarization-dependent characteristics with a notable transition at a specific polarization angle, Φ = 39° (Fig. 5A). Then, we examined the spectral response upon separate loading of S-protein-1 and S-protein-2 onto DP nanoantennas. Under plasmon excitation (Fig. 5B), IR vibrational signals of S-protein-1 and S-protein-2 were observed. Subsequent determination of secondary structure information (Fig. 5C) was achieved through the Levenberg-Marquardt algorithm (fig. S13). Specifically, antenna-enhanced mid-IR spectroscopy captured six protein conformations, including amide I coil, nucleic acid, and α-helix and β-sheet of amides I and II (fig. S14). Upon phonon excitation (Fig. 5D), discernible phonon signals attributable to S-protein-1 and S-protein-2 were detected, exhibiting distinct intensity and line shapes corresponding to each protein (Fig. 5E). These variations are identifiable through deep learning algorithms. Next, mixed S-protein-1 and S-protein-2 with varying concentrations (table S3) were loaded onto DP nanoantennas, followed by the collection of plasmonic vibrational signals and phonon signals (Fig. 5F). The dimensional reduction of these two signals was performed using a principal components analysis (PCA) algorithm (Fig. 5G). Most data points exhibited clear separation. Certain points that appear to overlap in Fig. 5G are discernibly separated when viewed from another perspective (fig. S15). In contrast, when individually analyzing the phonon signal (fig. S16) or plasmon signal (fig. S17) using the PCA algorithm, separating these overlapping spectra poses a challenge. This suggests that the molecular features contained in the plasmon and phonon signals are different, and their combination enhances the decoupling of overlapping spectra by the algorithm. Then, the PCA outcomes were input into a support vector machine (SVM) algorithm for species identification. The confusion map of the SVM outcome indicated a classification accuracy of 93.4% for S-protein-1 and S-protein-2 (Fig. 5H). To investigate the algorithmic influence, we used four primary classification algorithms (decision tree, k-nearest neighbor, SVM, and discriminant analysis) to scrutinize the measurement data (fig. S18). The results indicated accuracy fluctuations within 10%, with SVM demonstrating the highest accuracy. Notably, there was a reduction in accuracy observed solely for the S-protein-2 in comparison to S-protein-1. Further investigation suggested that this decrease was attributed to the narrower concentration intervals of S-protein-2; as the concentration interval decreased, accuracy diminished (fig. S19). For comparison, conventional plasmonic nanoantennas (fig. S20) were used to gather signals from mixtures of S-protein-1 and S-protein-2. Subsequently, these signals were classified using PCA and SVM algorithms. The results indicated that the PCA data points exhibited obvious overlap (Fig. 5I), and the classification accuracy of SVM was 22.8% (Fig. 5J). Substantial enhancement in accuracy through alternative common algorithms was not evident, as accuracy remained within 20% (fig. S21). In the parallel coordinates plot depicting the classification process, as the process progressed, the overlapping data points gradually segregated for DP nanoantennas (fig. S22), offering visual confirmation of the handling of overlapping spectra.

Fig. 5. Species classification demonstration.

(A) 2D mapping of normalized experimentally measured reflection as a function of polarization angle (Φ) and wave number. It shows the polarization-controlled transition of plasmonic and phononic modes in DP nanoantennas. Long-arm polarization: Φ = 0. Short-arm polarization: Φ = 90. (B) Measured spectra of S-protein samples under plasmon mode. (C) Extracted vibration signal of S-protein samples. (D) Measured spectra of S-protein samples under phonon mode. (E) Extracted phonon signals corresponding to S-protein samples. (F) Plasmon and phonon signals of S-protein samples at different concentration configurations (denoted as C1 to C8, see table S3 for details) using DP nanoantennas. (G) Results of dimensionality reduction of data in (F) using principal components analysis (PCA). (H) The confusion map for support vector machine (SVM) outcome showing a classification accuracy of 93.4%. (I) PCA results of conventional plasmonic nanoantennas for comparison with DP nanoantennas. (J) Corresponding SVM results indicating a classification accuracy of 22.8%.

Concentration prediction and spectral reconstruction

We construct a MM-DNN–based regression model, comprising (i) feature concatenation and (ii) the DNN architectures (Fig. 6A). The DNN model includes an input layer, a hidden layer, and an output layer. The input layer comprises 3426 nodes, corresponding to the spectral wave number points, while the output layer consists of two nodes, aligning with the number of species in the sample. For MM-DNN model training, we conducted mixture experiments, as well as single-analyte experiments that measured individual analytes within the concentration range of 0 to 400 ng/μl. The training dataset comprised spectral libraries encompassing diverse protein configurations, along with corresponding concentration labels (Fig. 6B). The comprehensive training and validation datasets comprised 1,713,000 spectral data points. The MM-DNN model accuracy reached 100% (fig. S23). Then, the newly obtained measured spectral data were input as a test set (Fig. 6C) to the trained MM-DNN model (Fig. 6A) and output concentration predictions for all components within the mixture. The predicted outcome closely aligned with the actual concentration, exhibiting an average error of 8.08% (top of Fig. 6D). The test set, excluded from model training, demonstrates the model’s capability to predict S-protein-1 and S-protein-2 mixtures across concentrations ranging from 0 to 400 ng/μl. Potential expansion of the training dataset’s concentration range could further extend this predictive capacity. Moreover, spectral reconstruction can be attained by leveraging the predicted concentration ratio of the mixture, the spectrum of the mixture, and the predefined spectra of S-protein-1 and S-protein-2. Taking sample 20 as an example, the concentration ratio of S-protein-1 and S-protein-2 was 6:1 (bottom of Fig. 6D), and the individual spectra at varying concentrations are presented in Fig. 6B. Via spectral matching, the total overlapping spectrum of the heterogeneous sample (top of Fig. 6E) was de-overlapped and reconstructed into the individual spectra of S-protein-1 and S-protein-2 (bottom of Fig. 6E).

Fig. 6. Concentration prediction and spectral reconstruction demonstration.

(A) The MM-DNN-based regression model for concentration prediction and spectral reconstruction. Step 1: Training and validation data are used to train the model. Step 2: The experiment data are put into the trained model. Step 3: Output the concentration prediction. Step 4: Spectral reconstruction. (B) Plasmonic and phononic spectra of S-protein-1 and S-protein-2 samples with various concentrations, which are used as training and validation sets. Top: Plasmonic IR vibrational signals and phonon signals. Bottom: concentration labels. (C) Newly measured plasmonic and phononic spectral sets of S-protein samples (denoted as samples 1 to 20), which are used as the test set. (D) Concentration prediction results of the data in (C). Top: The average error of prediction. Bottom: True and predicted concentration value of S-protein-1 and S-protein-2 samples. (E) Spectral reconstruction results for the sample 20. Top: Measured spectra of the mixed S-protein sample. Bottom: Reconstructed spectra of S-protein-1 and S-protein-2.

Hyperspectral imaging of laboratory and clinical SARS-CoV protein monolayers

These demonstrations are based on (i) S-protein samples and (ii) clinical saliva samples. SAM technology was used to immobilize samples on the device (fig. S24). This well-established method is widely recognized for its efficacy in the covalent immobilization of proteins (46). Both S-protein-1 and S-protein-2 were immobilized at identical concentrations. First, we examined the hyperspectral imaging of nonmixed monolayers by immobilizing S-protein-1 and S-protein-2 on distinct regions of DP nanoantennas [Fig. 7A (a)]. Phononic and plasmonic hyperspectral image datacubes were then acquired [Fig. 7A (b)]. The manifestation of the coffee ring effect was noted at the sample periphery, attributed to drying at the edges, preventing effective rinsing. Following this, the MM-DNN model was trained and used to predict the distribution of S-protein-1 and S-protein-2, with 100% training accuracy and 93% validation accuracy. The resulting protein monolayer distribution aligned with the observed immobilization of the samples [Fig. 7A (c to e)]. Second, we investigated the hyperspectral imaging of heterogeneous monolayers by immobilizing S-protein-1 and S-protein-2 monolayers in the same regions [Fig. 7B (a)]. We conducted hyperspectral imaging with phononic and plasmon polarizations (figs. S25 and S26), and coffee rings were not observed [Fig. 7B (b)]. Followed by MM-DNN model prediction, the successful observation of the distribution of heterogeneous protein monolayers ensued [Fig. 7B (c to e)]. Evidently, the immobilization of protein molecules exhibited nonuniformity, with unsuccessful protein immobilization observed in certain regions. For comparative analysis, we investigated the detection of heterogeneous protein monolayers under three configurations: imaging without nanoantenna, imaging with conventional nanoantennas, and imaging with our DP nanoantennas (fig. S27 and movie S1). As a result, the absence of nanoantennas yielded no observable signal, while samples enhanced with conventional nanoantennas exhibited spectral features. However, directly feeding these spectra into the DNN model demonstrated a low accuracy of 49% due to the spectral overlapping (fig. S28). The identification and distribution observation of S-protein-1 and S-protein-2 in the sample remained challenging. In contrast, our method disentangled the overlapping spectra of hyperspectral imaging with an accuracy of 93% and a processing speed of 230,400 spectra/s (fig. S29), facilitating the observation of protein monolayer distributions. In addition, no distinguishable differences in bonding preference between the two proteins on the device surface were observed when immobilized at identical concentrations (fig. S30).

Fig. 7. Label-free protein monolayer hyperspectral imaging.

(A) Hyperspectral imaging of SARS-CoV protein monolayers separately located in different areas. (a) Schematic diagram of sample distribution on the DP nanoantennas. (b) Plasmonic and phononic hyperspectral image datacubes. (c) The image predicted by the MM-DNN model showing the distribution of S-protein-1 and S-protein-2. Individual display of (d) S-protein-1 and (e) S-protein-2. (B) Hyperspectral imaging of monolayers mixed located on the device. (a) Sample distribution. (b) Plasmonic and phononic image datacubes. (c) The predicted image. Individual display of (d) S-protein-1 and (e) S-protein-2. (C) Imaging results of clinical saliva samples collected from a COVID-19-positive individual on various days post-initial symptom onset. (a) Days 1, (b) 4, and (c) 10. (d) The calculated total intensity of the positive signals in the hyperspectral image. (D) Imaging results of clinical saliva samples at different times after saliva ex vivo. (a) 12, (b) 24, and (c) 48 hours. (d) The corresponding total intensity. Scale bars (insets), 300 μm. The error bars in (C) and (D) represent the mean ± SD of S-protein-2 in saliva samples, with n = 3.

Next, we examined clinical saliva samples containing SARS-CoV-2 spike proteins. The comprehensive methodology for saliva sample collection and handling is described in Materials and Methods. Despite the presence of other proteins and molecules in saliva samples, our method holds promise for the identification of spike proteins. Figure 7C shows the predicted imaging results of saliva samples on various days post-initial symptom onset, which reveals clear observation of the spike protein on days 1 and 4, disappearing by day 10. The total intensity [Fig. 7C (d)] indicated a higher viral load during the early stages of symptom onset, reflecting active viral replication. As the immune response and other factors come into play, the viral load tends to decrease. Figure 7D illustrates the results of saliva samples at various time points post-saliva ex vivo. The spike protein exhibited a slight increase followed by a decline over time, potentially attributed to the presence of a limited number of upper respiratory tract cells in the saliva (47). Viral invasion induces host cell rupture, releasing newly produced viral particles (48). The observed decrease may result from protein denaturation after prolonged ex vivo exposure. Various SARS-COV-2 detection methods are compared in table S4. Lateral flow immunochromatographic strip (LFICS) and PCR are established methods. LFICS is known for its rapidity and cost-effectiveness but exhibits slightly lower accuracy compared to other methods. PCRs are highly specific to the target sequence, minimizing false positives or errors, but it is relatively time-consuming. Our method is not outstanding for routine negative/positive diagnostics of SARS-CoV-2. It features imaging-guided molecular screening by providing molecular spatial distribution information in a noninvasive and sensitive manner.

DISCUSSION

We have investigated the phonons for hyperspectral bioimaging by developing a plasmon-phonon system. We demonstrate that phonons enhance the molecular identification capabilities in hyperspectral imaging. Our contributions are threefold. The first is the expansion of phonon polaritons into mid-IR hyperspectral imaging. We address three inquiries regarding phonon-enhanced hyperspectral imaging: the excitation of phonons, the interactions of phonons with molecules, and the control of phonon and plasmon excitation. First, phonon excitation through plasmon-phonon coupling broadens phonon signals compared to direct excitation of phonon modes by light. Then, phonons interact with molecules to detect molecular refractive index instead of molecular vibrations. Moreover, we demonstrate the polarization-controlled phonons and plasmons in the system.

The second is hyperspectral imaging with enhanced spectral de-overlapping. We note that artificial intelligence (AI) algorithms have been applied to resolve overlapping spectra in complex dynamic systems (15). However, the AI de-overlapping via direct input of overlapping spectra into an AI model demonstrates a low identification accuracy of 49% (fig. S28) in hyperspectral imaging, where each image datacube contains hundreds of thousands of overlapping spectra. This reduced accuracy could be attributed to the accumulation of de-overlap errors in the context of extensive spectral data. Mathematically, this stems from insufficient features within the database. We demonstrate that phonons capture the intensity and shape features of molecular refractive index, distinct from those observed with plasmons. This allows enhanced molecular identification capabilities in hyperspectral imaging. The hyperspectral imaging demonstration of two mixed SARS-CoV spike proteins achieves a de-overlapping speed of 230,400 spectra/s and an identification accuracy of 93%.

The third is a toolkit for imaging-guided biodetection. Our SARS-CoV imaging experiments demonstrate the potential of our approach in imaging-guided biodetection, in a noninvasive and label-free manner. Compared with our earlier study on plasmons and phonons (25), this work shows advancements in polarization-controlled phonon excitation, signal analysis, imaging applications, and detection performance (table S5). Notably, the plasmon-phonon method can be extended to the far-IR and terahertz regions (35); however, implementing it in the visible and near-IR regions using current material systems is challenging due to the nature of phonon polaritons. Moving forward, the commercialization of our technology is paramount. Nanoimprint lithography stands out as a highly competitive, cost-effective large-area nanopatterning technique from a fabrication standpoint (49, 50). Its scalability, cost-effectiveness, and capability to achieve high-resolution patterning render it an appealing option for mass production and market integration of optical technologies (51, 52). Leveraging this technology, we anticipate a substantial reduction in the cost of our devices.

MATERIALS AND METHODS

FDTD simulations

A proprietary commercial software package, FDTD Solutions, developed by Lumerical Inc., was used for conducting both spectral and near-field simulations. The incident radiation source used in these simulations consisted of a plane wave light source characterized by polarization aligned along the nanoantennas’ arm direction. To accurately represent the periodic nature of the nanoantennas along the x and y axes, we implemented periodic boundary conditions, while a perfectly matched layer was implemented along the z axis to ensure radiation absorption. It is important to note that the thin titanium (Ti) adhesion layer was intentionally excluded from the simulation process. The spatial distribution of the electric field (E-field) intensity was computed using a 3D frequency-domain power monitor. For the material properties, the refractive index of CaF₂ was set to 1.4, while the refractive indices of gold (Au) and silicon dioxide (SiO₂) were obtained from the dataset in (53). To maintain simulation accuracy, the minimum mesh size was defined as 20 nm along the X and Y directions and 10 nm along the Z direction. This mesh size was chosen to be 10 times smaller than the smallest dimension of the nanoantennas, ensuring the fidelity of the simulations. The ratio of E-field exposed to free space (R_E) is calculated by $R_{\mathrm{E}}=({\int }_0^{z_0}\langle {\mid E/E_0\mid }^2\rangle \mathit{dz})/({\int }_{-z_0}^{z_0}\langle {\mid E/E_0\mid }^2\rangle \mathit{dz})$, where |E/E₀|² represents the average E-field intensity within the region adjacent to the antenna end face (fig. S1 and text S1), which fluctuates along the z axis, computed from the simulated E-field distribution. The z₀ is set to 400 nm.

TCMT modeling

The temporal coupled-mode theory (TCMT) was used to model the coupling behavior between nanoantennas and molecules. On the basis of TCMT, a classical radiating oscillator model is a set of two coupled harmonic oscillators that are subjected to the external force. In this work, we improved the classical radiating oscillator model to be suitable to model the coupling of plasmon and phonon. Specifically, we developed a radiating oscillator model with dual-damped vibrational modes whose frequencies correspond to the TO and LO phonon frequencies. The comparative analysis of experimental and computational outcomes demonstrated the accuracy of our model (fig. S4). More detailed derivation is provided in the supplementary information. The dispersion curve and the 2D mapping in Fig. 3 were calculated using the dual-damped modes model.

Device nanofabrication

The nanofabrication procedure for the DP platform was initiated by subjecting the CaF₂ wafer to a meticulous cleaning process involving immersion in acetone and subsequent ultrasonic treatment for a duration of 10 min. Subsequently, the wafer was subjected to rinsing with isopropanol and dried using nitrogen gas. A crucial step in the process entailed subjecting the wafer to oxygen plasma treatment for a duration of 5 min. Following this, a layer of PMMA e-beam lithography resist (950 PMMA A5), measuring 400 nm in thickness, was evenly spin-coated onto the wafer surface at 3000 rpm. Subsequent thermal bakes were performed, and a commercially available electron-conducting polymer (Espacer 300Z from Showa Denko Singapore) was spin-coated at a speed of 2000 rpm to mitigate charge accumulation during e-beam exposure. The subsequent step involved exposing the nanoantenna pattern through e-beam lithography. Upon completion of the exposure, the sample underwent a development process involving the use of deionized water, a mixture of methyl isobutyl ketone/isopropyl alcohol (IPA; in a 1:3 ratio), and isopropanol. Following this, the deposition of SiO₂/Ti/Au films with prescribed thicknesses was carried out sequentially using e-beam evaporation. This was followed by an immersion period of 24 hours in acetone to remove the unexposed resist, ultimately yielding the final nanoantenna pattern. It is worth noting that the fabrication process for the conventional nanoantenna closely resembles that of the DP platform, with the exception that, after exposure, only the materials Ti and Au need to be deposited through e-beam evaporation.

Ethics clearance, patient selection, sample collection, and handling

All COVID-related samples were collected with approval from the National Healthcare Group Domain-Specific Board of Singapore (reference number 2012/00917) and the institutional review board of the National University of Singapore (reference code H-20-006). Saliva samples were obtained from a patient diagnosed with COVID-19 infection (confirmed by PCR testing). Specifically, the saliva samples were collected using Salivette tubes as per the manufacturer’s guidelines. Before saliva collection, the patient confirmed that he had fasted, refrained from taking oral medications, and abstained from brushing his teeth for a minimum of 30 min. These tubes comprise a cotton swab that the patient is directed to chew for a specified duration. Afterward, the swab is placed into an inner tube, which is then inserted into an outer tube to capture liquid saliva during centrifugation at 1000g for 3 min. After centrifugation, saliva samples underwent viral inactivation via treatment with Triton X-100. Inactivated samples were immediately frozen and stored at −20°C until testing. SARS-CoV-1(-2) protein samples were acquired from live SARS-CoV cultures within the Duke-NUS Medical School laboratory. The SARS-CoV-2 used in this study is the Delta variant (Delta AY.4.2), whereas SARS-CoV-1 corresponds to the SARS coronavirus Tor2. The spike proteins were purified using conventional methods (54) and stored at −20°C until testing. Before testing, protein samples underwent dilution to specific concentrations using phosphate-buffered saline (PBS) buffer. Notably, this technology is in the developmental stage and has not yet undergone clinical trials. The influence of individual differences of patients such as age, gender, and health status on the detection results was not investigated in this study. The study involved a single patient for saliva sample collection due to the focus on method validation rather than patient diversity. To ensure the reliability of the method, we collected 6 ml of saliva per sampling and conducted three replicate experiments (using 1.5 ml for each detection).

Immobilization of SARS-CoV protein monolayer

Initially, the chips were subjected to immersion in an acetone solution for 30 min at room temperature to eliminate impurities. This was followed by a meticulous triple rinse procedure using deionized water and IPA, concluding with a gentle drying process utilizing nitrogen gas. Subsequently, the chips were subjected to an overnight incubation period in a solution comprising 0.5 mM 3-mercaptopropionic acid (3-MPA; procured from Sigma-Aldrich) dissolved in ethanol. This step facilitated the formation of a SAM characterized by carboxyl (COOH) functionality on the alkane thiol (55). The COOH groups on the SAM were then activated through incubation of the chips in a cross-linking solution consisting of 75 mM 1-ethyl-3-(3-dimethylaminopropyl) carbodiimide (EDC) and 15 mM N-hydroxysuccinimide (NHS) for a duration of 30 min. EDC, functioning as a zero-length cross-linker, served to activate the carboxyl groups originating from 3-MPA, while NHS participated in the formation of stable amide bonds with the EDC-activated carboxyl groups. Notably, the pH of the reaction milieu played a pivotal role in modulating the cross-linking process, influencing both the extent of activation of carboxyl groups and the reactivity of NHS molecules toward the activated carboxyl groups. pH adjustment was precisely carried out to attain a pH value of 6.1 using MES buffer (initially at pH 5.5). Subsequent to this critical step, the treated chips were immersed in a coupling buffer containing the SARS-CoV protein in a PBS solution, allowing for the immobilization of the SARS-CoV protein onto the SAM. During this phase, the pH was meticulously maintained at 7.2 using a PBS buffer (pH 7.4). Ultimately, the functionalized chips underwent thorough washing with a PBS buffer solution, and culminated with a drying process at a temperature of 4°C.

FTIR measurement

The IR spectrum of our experimental platform was acquired by employing a Fourier transform IR (FTIR) spectrometer, in conjunction with an IR microscope fitted with a liquid nitrogen–cooled mercury cadmium telluride detector. The dimensions of the measurement region were precisely regulated using knife-edge apertures, establishing a defined area of 100 μm by 100 μm. A spectral resolution of 4 cm⁻¹ was used, with data acquisition comprising 20 scans per measurement in standard conditions. In reflection mode, background data were collected using a gold mirror. To safeguard the optical pathway against environmental interference, a continuous supply of nitrogen gas was introduced into the microscope through specially designed accessories.

Quantum-cascade laser–based mid-IR microscopy

MIR-HSI measurements were conducted using a mid-IR microscope featuring a quantum-cascade laser (QCL) known as Spero-QT, developed by Daylight Solutions Inc. This instrument was equipped with four distinct QCL modules, each providing spectral coverage spanning from 950 to 1800 cm⁻¹. Data acquisition in the reflection mode was facilitated by using a high-quality 4× IR objective, characterized by a pixel size of 4.25 μm and a numerical aperture (NA) of 0.3. In addition, a microbolometer focal plane array with 480 × 480 pixels, which operated without active cooling, was used for efficient data collection. The spectral resolution of 2 cm⁻¹ was chosen to ensure precision in the gathered data. The field of view at wide-field IR imaging mode is 2 × 2 mm² (0.3 NA). To ensure the integrity of the measurements, a continuous stream of dry nitrogen was used to purge the microscope system.

Data processing

The differential absorbance spectra were derived through the subtraction of the measured reflectance spectra from a baseline that was obtained using asymmetric least squares algorithms. Subsequently, the integrated phonon signal was determined by performing an integration of the extracted phonon in Reststrahlen bands. The normalized detuning δ was computed as (ϖ − ϖ₀) / ω₀, with ϖ and ϖ₀ representing the shifted and initial frequencies, respectively. Furthermore, we used the PeakFit algorithm based on the Levenberg-Marquardt algorithm to identify absorption bands associated with protein secondary structures within a spectroscopic dataset.

Multimodal deep neural network

In terms of classification demonstration, we first used PCA for the dimensionality reduction of the measured spectral dataset. To facilitate visualization of the feature space, PCA was performed in MATLAB_R2020a. In this case, a covariance matrix was computed using a factorization of singular value decomposition for the normalized set of features from which the eigenvectors and eigenvalues were extracted. Each PCA was constructed as a linear combination of the initial features. The first three PCAs were then used to display 3D scatter plots of the features. The subsequent step in the analysis involves the utilization of PCA-processed data as an input for the SVM model, which serves as a versatile tool for constructing hyper-planes within high or even infinite-dimensional spaces. It is noteworthy that the efficacy of the SVM model in achieving a satisfactory separation between classes hinges upon the identification of a hyper-plane characterized by the maximum distance to the nearest training data points belonging to any class. Furthermore, the decision tree, k-nearest neighbor, SVM, and discriminant analysis techniques were implemented using the Classification Learner App of MATLAB.

As for the concentration prediction and spectral reconstruction, we developed a MM-DNN model based on Python 3.6 using the sci-kit-learn package. The MM-DNN model consists of feature concatenation and the DNN architectures. Early fusion is used to concatenate the features from plasmonic and phononic datacubes into a single, joint representation. Subsequently, feature extraction and data dimensionality reduction are applied to the fused dataset. The architecture of the DNN model consists of fully connected layers, also called dense layers (Fig. 6A). These layers use the rectified linear unit activation function, a widely favored choice in deep learning due to its computational efficiency and facilitating faster convergence during training. The MM-DNN model encompasses input, hidden, and output layers. The input layer comprises 3426 nodes, corresponding to the wave number points within the spectrum, while the output layer consists of two nodes, aligning with the component numbers. In addition, the model incorporates two hidden layers, each comprising 64 nodes, strategically designed to extract intricate features from the input data. These hidden layers empower the model to discern intricate relationships and facilitate precise predictions. The loss function used in the model is the mean square error, quantifying the average squared disparity between predicted and actual values. The optimizer adopted in the final model is Adam, a potent optimization algorithm that dynamically adjusts the learning rate based on the gradient of the loss function, enhancing the model’s training efficiency. Throughout training, the model iteratively fine-tunes the weights and biases within the layers to minimize the loss function, consequently enhancing prediction accuracy.

In MM-DNN–aided hyperspectral imaging, the architectural configuration of this model differs from that used for concentration prediction demonstrations (fig. S29). First, plasmonic and phononic datacubes are fused. Next, feature extraction and dimensionality reduction using PCA are executed, reducing the feature space to 100 dimensions. Subsequently, the input layer is composed of 100 nodes, each corresponding to the PCA-transformed data points, while the output layer comprises three nodes, aligning with the distinct category labels. Moreover, our model incorporates four hidden layers, each consisting of 64 nodes. These hidden layers are equipped with the softmax activation function, a widely used activation function in the realm of machine learning, in particular in the context of multiclass classification problems. To quantify the model’s performance, we use the “categorical_crossentropy” loss function, which gauges the dissimilarity or error between the predicted probability distribution and the true probability distribution of class labels for each input data point.

Materials and apparatus

3-MPA, EDC, NHS, MES, and PBS were purchased from Sigma-Aldrich (Singapore). The clinical saliva samples were obtained from Duke-NUS Medical School. SEM analyses were conducted using the Hitachi Regulus 8230 field-emission SEM. IR spectral measurements were obtained with an FTIR spectrometer (Cary 660, Agilent Technologies) and an IR microscope (Cary 610, Agilent Technologies). The fabrication of nanoantenna patterns involved electron beam lithography with Raith GmbH equipment. The metal deposition process used ultrahigh vacuum electron-beam evaporation with the ATC-T Series from AJA Int.

Supplementary Materials

This PDF file includes:

Texts S1 to S3

Figs. S1 to S30

Tables S1 to S5

Legend for movie S1

References

Other Supplementary Material for this manuscript includes the following:

Movie S1