Correlated Materials Characterization via Multimodal Chemical and Functional Imaging

Abstract

Multimodal chemical imaging simultaneously offers high-resolution chemical and physical information with nanoscale and, in select cases, atomic resolution. By coupling modalities that collect physical and chemical information, we can address scientific problems in biological systems, battery and fuel cell research, catalysis, pharmaceuticals, photovoltaics, medicine, and many others. The combined systems enable the local correlation of material properties with chemical makeup, making fundamental questions of how chemistry and structure drive functionality approachable. In this Review, we present recent progress and offer a perspective for chemical imaging used to characterize a variety of samples by a number of platforms. Specifically, we present cases of infrared and Raman spectroscopies combined with scanning probe microscopy; optical microscopy and mass spectrometry; nonlinear optical microscopy; and, finally, ion, electron, and probe microscopies with mass spectrometry. We also discuss the challenges associated with the use of data originated by the combinatorial hardware, analysis, and machine learning as well as processing tools necessary for the interpretation of multidimensional data acquired from multimodal studies.

Graphical Abstract

The functionality of materials and biological systems from batteries, fuel cells, catalysts, and photovoltaics to biological tissues and cells is traditionally studied with macroscopic characterization techniques. However, the underlying functionality is defined by the chemical organization, with characteristic length scales on the order of microns to nanometers. The push toward studying materials and systems at the nanoscale addresses challenges like developing lighter, more energy efficient, economical structural energy materials, as well as understanding biological complexity in a wide range of applications.^1–5 Current techniques capable of spatially resolving nano- to mesoscale features are limited by the amount of chemical information they offer. For instance, techniques such as atomic force microscopy (AFM) with spatial imaging resolution as low as 1 nm provide almost no chemical information.^6,7 In contrast, chemical imaging approaches can supply chemical information on the molecular level but are not capable of imaging physical properties with high resolution. Therefore, by coupling modalities that collect physical and chemical information, one can begin to address scientific problems using methods tailored to studying systems that require both nanoscale spatial resolution as well as high chemical specificity. The combined multimodal platforms enable local correlation of material properties with chemical makeup, allowing one to answer fundamental questions in how the chemistry and structure of the material drive functionality and physical properties observed on the macroscale.

Here, we review recent progress and offer a perspective for multimodal chemical imaging in characterization of a rich sample spectrum in a correlated manner. We discuss the use of data combinatorial hardware platforms and analysis as well as machine learning and processing tools that are becoming necessary for interpretation of the multidimensional data acquired from multimodal studies.

WHY MULTIMODAL IMAGING?

Understanding the complex functionality of inorganics and ceramics, soft and polymeric materials, and especially biological systems necessitates multiple sources of information, placed in the context of sample history (preparation conditions, medical history, etc.), that are related to properties and likely future behaviors. Correspondingly, published and exploratory studies include multiple characterization and imaging modalities. For example, in solid-state chemistry and materials science, the classical characterization approach will include X-ray, optical, and electron microscopy, techniques often performed on the same or similar samples.^8–14 The question then arises as to what constitutes multimodal imaging and what the challenges and benefits of such an approach would be. Here, we offer a classification of the levels at which multimodal studies can be performed.

As a first class of problems, we consider the case when macroscopic measurements are available along with the imaging data, i.e., data sets of the type R(w) and A(x, y), where w is a parameter (e.g., wavelength or temperature), and A is the image data. A typical example of these data sets can be X-ray scattering data and optical microscopy. Here, it may be possible to establish the relationship between the data (for example, quantify the fraction and number of constituent phases from microscopy) and then use this information to determine identity from scattering. This analysis is generally simple when R(w) is a linear combination of the component spectra and classical linear unmixing methods with known (multiple regression) or unknown end-members are applicable. For these cases, the statistical methods can establish the uncertainties in the recovered signal and, hence, material characteristics. In cases in which the signal R(w) has a complex dependence on microstructure (e.g., conductivity or dielectric properties), more-complex microstructural methods are required, and in general, this approach is rarely used. Note that the former problems can be considered as a special class of multimodal imaging for the case in which one of the imaging modes has zero resolution.

In the second class of problems, we consider the case in which the two or more imaging data sets corresponding to dissimilar imaging modalities, A(x, y) and B(x′, y′) are available, and the spatial grids (x, y) and (x′, y′) are unrelated. In other words, the imaging is performed at different regions. Where the properties of the sample can be assumed (or are known) to be spatially uniform, the information from A(x, y) and B(x′, y′) can be compared. For example, observations of the atomic species and point defects at the atomic level can be performed using scanning transmission electron microscopy (STEM) and directly compared to the information on the atomic structure and valence states via electron energy loss spectroscopy (EELS). Similar studies can be performed via scanning tunneling (STM) and non-contact atomic force microscopy (nc-AFM). However, for bulk materials this approach is more difficult because STEM visualizes atomic columns (i.e., projection of atomic structure on the image plane), whereas STM and nc-AFM are surface-sensitive techniques. Moreover, even for 2D materials such as graphene or layered chalcogenides, simultaneous (or even sequential) imaging of the same region is a very complex problem due to the difficulties in finding the same area, possible changes of the surface, and the material during sample handling. However, if a sufficiently large body of imaging data on the atomic configurations in both modalities are available, equivalence between the two can be established based on a statistical distribution. Additionally, high-veracity identifiers (e.g., defect size) that do not define structure unambiguously but significantly reduce the number of possible variants can be used. Similarly, for mesoscopic imaging, the average comparison can be performed assuming that readily identifiable characteristics can be used for validation and classification of the structural elements.

The third class of the problems correspond to the case in which the data sets A(x, y) and B(x′, y′) are obtained from the same spatial region, and the grids (x, y) and (x′, y′) overlap. In this case, the functional properties A and B are explored from (roughly) the same location. In general, the primary initial task of the image analysis workflow becomes the co-registration between the spatial grids, potentially augmented by interpolation or pan-sharpening to extrapolate the data to a single spatial grid yielding a compound object [A, B](x″, y″). Once such data are available, fundamentally different opportunities to explore and to derive knowledge from the material data open. This can be considered a full quantifier of the dissimilar properties A and B of the material, allowing the establishment of a correlative relationship between A and B within the material class. For example, the optical properties can be directly correlated with mass-spectrometric traces, providing the information that can be used to decipher optical measurements.

Finally, the fourth class of the multimodal imaging problems corresponds to a case in which the two measurements A and B are spatially incongruent. For example, measurements A are taken on the surface, A(x, y), whereas the measurements B are taken on volume, B(x, y, z). In this case, the natural question is whether the information can be combined. Specifically, can A offer a boundary condition that allows the reconstruction of material properties within the volume B? This class of problems are becoming very common due to the broad propagation of 3D structural mapping tomography tools and the fact that many physical phenomena including mechanical and ferroelectric are extremely long-range and are affected by the boundary conditions as a consequence of generative physics models.

INFRARED AND RAMAN SPECTROSCOPIES WITH SCANNING PROBE MICROSCOPY

Infrared (IR) spectroscopy is commonly used to characterize bulk chemical composition based on the infrared light interaction with matter via absorption or emission. Raman spectroscopy, which is based on inelastic scattering of monochromatic light in the visible, near-infrared, or near-ultraviolet ranges is another powerful optical technique used for the chemical mapping of materials. Raman and IR spectroscopies offer information on chemical bonds and local chemical environments based on the spectroscopic signatures recorded. While powerful, these conventional instruments are averaged over an ensemble of molecular species, thus providing an average description of local chemistry. Adaptation of these approaches to microscopic platforms provides an avenue by which to map chemical signatures in a spatially resolved manner but are intrinsically limited by the diffraction limit to modest spatial resolutions of ~2.5–75 μm and ~0.25–1 μm for IR and Raman microscopes, respectively.^15,16

However, collecting infrared spectra with high spatial resolution and visualizing the spatial distribution of chemical properties, or functional groups would provide useful insights hitherto inaccessible with a classical setup. This type of a study can be based on the absorption of light by matter and the local excitation of molecular vibrations. The vibrational frequencies of these types of transitions are defined by the chemical surroundings, allowing us to match the experimentally observed peaks with certain chemical regions. Large volumes of the peak-group assignments have been tabulated, making infrared spectroscopy a reliable technique for chemical analysis in a variety of areas such as organic chemistry,¹⁷ inorganic chemistry,¹⁸ industrial process control,¹⁹ and sensors.²⁰ Moreover, the peak shape can indicate changes in the molecular conformation and orientation²¹ to reveal thermodynamic properties²² as well as the defect formation.²³ Thus, tracking specific peaks, or collecting spectra on a spatially dense grid can image chemical properties in areas of interest and surroundings. One of the most promising ways to overcome the diffraction limit in spatial mapping and obtain high-resolution local chemical maps is to combine atomic force microscopy (AFM) with optical spectroscopy.²⁴ Adhesion and elastic modulus can be overlaid with chemical maps highlighting interplay between crystallinity, composition, and intermolecular interaction between and within single domains. Combined infrared vibrational scattering scanning near-field optical microscopy (IR s-SNOM) with force–distance spectroscopy can also be used for the simultaneous characterization of both nanoscale optical and nanomechanical molecular properties²⁵ (Figure 1).

Figure 1.

Multidimensional data set showing maps of PS and PMMA microdomains. Adopted with permission from ref 25 under the Creative Commons Attribution License. Copyright Beilstein Journal of Nanotechnology 2018.

Additionally, the AFM probe can be used as a detector and capture light interactions with matter down to ~10 nm. Using near-field effects, scanning near-field optical microscopes (SNOM) have become popular, with aperture- and apertureless versions commercially available. Raman²⁶ and nanoscale IR^27,28 with signal amplified by the scanning probe have demonstrated the potential of nanoscale chemical signal acquisition. In tip-enhanced Raman scattering (TERS),²⁹ the intrinsically low intensity of Raman scattering is successfully overcome³⁰ with near-field amplification by plasmonic and chemical enhancements. As a result, TERS finds³¹ its applications in imaging of graphene,³² polymers,^33,34 silicon-based structures,³⁵ and semiconductors^36,37 as well as biological systems.³⁸ Subnanometer resolution chemical images have been demonstrated recently³⁹ (Figure 2). However, experimental parameters may alter the Raman spectra qualitatively and quantitatively by, e.g., the pressure applied to the tip, and distance to the species being probed, which complicates the analysis of the data.⁴⁰

Figure 2.

TERS imaging of molecular domains adsorbed at silver terraces. (a) Schematic of STM-controlled TERS. The molecular structures of the porphyrin molecules under study (ZnTPP and H2TBPP) are shown in the upper-right corner. (b) STM image of two adjacent porphyrin molecular domains (−1 V, 5 pA). (c) STM image simultaneously acquired during TERS imaging of the area denoted by the dashed square in b (−0.1 V, 1 nA, 7 nm × 7 nm, 32 pixels × 32 pixels, 1 s per pixel). The boundary between the molecular domains is highlighted by a yellow line. (d) TERS spectra, averaged over the blue and green squares (3 pixels × 3 pixels) shown in panel c, extracted from the datacube for ZnTPP and H2TBPP molecules, respectively. (e, f) TERS images reconstructed based on single-peak analysis for the Raman peaks at ~701 cm⁻¹ (panel e, integrated over 687–736 cm⁻¹) and ~918 cm⁻¹ (panel f, integrated over 890–959 cm⁻¹). Adopted with permission from ref 39 under a Creative Commons Attribution Non-Commercial Share-Alike 4.0 International License. Copyright Nature 2018.

By tuning an IR laser across a spectral region and measuring the deflection of the cantilever, one can generate a vibrational spectrum that matches those measured by conventional attenuated total reflection (ATR) Fourier transform infrared spectroscopy (FTIR) instruments but is spatially resolved. This approach can also be used more generally with light outside of the IR region, which is referred to as photothermal induced resonance (PTIR). Traditionally, the IR evanescent field in a total internal reflection geometry has been used to induce thermal expansion of the sample. In practice this is limited by the availability of the ATR crystals as substrates. To resolve this issue, in more-modern set-ups, infrared light is introduced from the side, which allows any thermally conductive substrate to be used as a sample. The next major improvement was in the resonant signal enhancement.⁴¹ Here, the excitation rate of the quantum cascade laser is tuned to match the contact resonance of the scanning probe. Deposits as thin as 4 nm can be analyzed using this approach. With AFM cantilever resonance enhancement, it is possible to conduct IR measurement in liquid environment as contact resonances of higher cantilever modes are less dampened.⁴² In addition, tapping-mode AFM-IR has been developed by Anasys for the characterization of soft matter, which suffers from the physical contact between the sample and scanning probe. Another intriguing approach involving the substitution of the conventional AFM tip with microscale optical transducers has been recently demonstrated,⁴³ offering drastic improvement of the signal-to-noise ratio. Additional improvements also include light polarization control.⁴⁴

The analysis of polymer blends⁴⁶ and nanocomposites⁴⁷ is a well-suited application for AFM-IR. For example, it is possible to investigate the internal structure of polymer blends; while the phase image of two high-impact polypropylene samples show similar internal structure of the multilayer rubber particles, the AFM-IR reveals that the chemical distribution is very different (Figure 3). Quantitative analysis of nano-domains in polymer-based materials that relates chemical composition and localized thermal properties can be carried out using this approach.^45,48 AFM-IR has been used to confirm the uniformity of the polymerization reaction of PEDOT⁴⁹ and P3HT.⁵⁰ CNT-reinforced thermoset composites have been investigated revealing the distribution of chemical interfaces.⁵¹

Figure 3.

A pair of samples of high-impact polypropylene (HIPP): (a, d) AFM phase images of AFM-IR absorption maps at 974 cm⁻¹; (b, e) warmer colors indicate stronger absorption and higher polypropylene content. (a, b) Left column, HIPP-1; (d, e) right column, HIPP-2. In HIPP-1, the rigid cores of the particles are rich in polypropylene, which is highly crystalline, whereas for HIPP-2, the major component of the rigid cores is polyethylene with a high degree of crystallinity. The formation of these very different structures and compositions in the core and rubber domains may be attributed to different chain structures and compositions of the copolymers in the alloys produced by different catalysts. Reprinted with permission from ref 45. Copyright 2018 Elsevier.

The sensitivity levels allow the characterization of additive migration in the industrial samples.^41,52 Finally, it is possible to perform AFM-IR measurements on biological samples such as cells^53–55 and tissue⁵⁶ without staining. The development of cataract in human lenses has been recently characterized with AFM-IR,⁵⁷ revealing the differences in protein secondary structure between clear and opaque lens samples. Because the infrared spectra is sensitive to molecular vibrations, the distribution of the organic species in the hybrid materials such as methylammonium lead halides can be clearly observed.⁵⁸ The direct registration of the local chemical changes generates an additional channel of information that aids the electrical characterization of samples and opens perspectives on the electromigration.

OPTICAL MICROSCOPY AND MASS SPECTROMETRY

Multimodal chemical imaging techniques coupled with mass spectrometry have a particularly bright future in the biomedical and biological sciences. Although single-mode imaging is well-established in many scientific subfields, the combination of various complementary imaging modes offering correlated information is still only emerging. These studies typically involve an optical imaging technique combined with a mass spectrometry technique or a combination of various mass spectrometric modalities that are intrinsically complementary. As the complexity of the system under study increases, as is the case with biological systems, the value added by each imaging mode becomes increasingly more evident. The capability to detect highly diverse molecules found in biological specimens benefits from the various types of imaging modes, such that one mode can detect species undetectable by another.⁵⁹ A combination of nanoscale infrared spectroscopy and mass spectrometry allows us to circumvent some limitations imposed by either techniques used alone (Figure 4). For example, while AFM-IR images have higher resolution, the sampling depth is a complex function of tip temperature, scanning parameters (e.g., scan speed), and the heat transfer of the surface. By including mass spectrometry data into the analysis, it is possible to highlight the chemical composition of the surface.

Figure 4.

(a) 2D IR and (b) 2D MS chemical images for PMMA and P2VP as well as (c) the related 2D topography image. Zoomed-in regions of (d, e) IR and (f, g) MS chemical images shown in panels a and b corresponding to areas I and II, respectively, indicated by the white squares in the full-size images. Arrows indicate about 1.6 μm wide gaps between PMMA features that are visible in the images in panels c–g. Adapted with permission from ref 60. Copyright 2018 American Chemical Society.

Significant challenges in terms of imaging data analytics are still pervasive. These challenges scale with the complexity of the system and become more significant as instrumentation is refined, and both spatial and spectral resolution increase. With this trend, data density for each imaging mode becomes increasingly difficult to manage. For example, latest generation of matrix-assisted laser desorption/ionization time-of-flight mass spectrometry imaging (MALDI-ToF-MSI) instruments are now capable of acquiring spectra at a rate of 50 pixels s⁻¹ by combining high repetition-rate lasers, synchronized with fast-moving sample stages, feeding into hardware that can sustain high data write speed.⁶¹ These experiments can generate data files on the order of TBs per sample, resolved at 50 μm², containing ~10 million individual spectra.

Another significant challenge is in situ real-time molecular identification of biochemical species, essential for mapping molecules in biochemical pathways and understanding their larger biological impact. In the case of proteins, original approaches for spatially targeted liquid microextractions,⁶² spatial extractions coupled to label-free liquid chromatography tandem mass spectrometry experiments,⁶³ in situ tryptic digestion,⁶⁴ and in-source decay experiments⁶⁵ among others are being developed. In the case of small molecules (metabolites and lipids), molecular identification relies heavily on high-resolution mass analyzers combined with tandem MS experiments.⁶⁶ The lack of retention-time information, as with chromatographic experiments, and the fragmentation of metabolomics databases are some of the remaining challenges that prevent real-time identification of metabolites and lipids during mass spectrometry imaging (MSI) experiments.^67,68

Various MS approaches have been utilized in a multimodal environment combined with orthogonal analytical methodologies for biomedical applications. Sweedler et al. were some of the first teams to combine secondary ion mass spectrometry (SIMS) and matrix-assisted laser desorption and ionization mass spectrometry (MALDI-MS) for the purposes of imaging rat spinal cord tissue sections. This multimodal approach enhanced the chemical coverage of the MSI experiment, revealing molecular properties of the chemically diverse but anatomically discrete motor and sensory cell networks.⁶⁹ Phospholipids, proteins, and neuropeptide distributions were obtained from single 20 μm² sections. Analyte identities were preliminary assigned by mass-to-charge ratio matches, followed by liquid chromatography–tandem mass spectrometry (LC–MS/MS) experiments. Later work also combined SIMS and MALDI, but in a hybrid, single ion source design,⁷⁰ enabling ion images of individual invertebrate neurons, mammalian spinal cord, and cultured neural networks (Figure 5). Confocal Raman microscopy coupled to SIMS and registered via an array of chemical microdroplets was later used for imaging quinolone signaling molecules in Pseudomonas aeruginosa biofilms, important in human diseases such as cystic fibrosis and in bacterial ecology studies.^71,72

Figure 5.

C₆₀ SIMS images of an Adelpha californica neural network obtained with a hybrid SIMS–MALDI ion source coupled to a commercial quadrupole-time-of-flight mass spectrometer, allowing both high-resolution and tandem MS measurements directly from biological specimens. PC: phosphocholine. The scale bar corresponds to 200 μm. Reproduced by permission from ref 70. Copyright 2014 Springer Nature.

Pioneering work by Heeren et al. demonstrated the combined application of magnetic resonance spectroscopic imaging (MRSI), metal-assisted SIMS, matrix-enhanced SIMS, and MALDI-MS for in vivo and ex vivo measurements on metastatic and nonmetastatic breast cancer xenograft models.⁷³ Principal component analysis of the multimodal imaging data revealed distinct tumor microenvironments characterized by their characteristic molecular signatures, revealing the altered choline metabolism and transport characteristic of cancer cells. Multimodal imaging can also be performed in a two-step fashion, with the sampling event separated in space and time from the detection event. The combination of laser capture microdissection with continuous online atmospheric pressure chemical ionization mass spectrometry enabled the sampling event to take place at higher resolution than typical for most MS imaging approaches (a few micron to submicron), whereas the detection was provided by downstream thermal vaporization of the generated tissue aerosols and reaction with an atmospheric pressure plasma leading to proton transfer and mass spectrometric detection.⁷⁴

Multimodal imaging MS has also been successfully used in several other biological applications. Ewing et al. employed multimodal imaging via time-of-flight secondary ionization mass spectrometry (ToF-SIMS) and MALDI-MS to investigate spatial distributions of the environmental toxin β-N-methylamino-l-alanine (BMAA) in hippocampus sections of a rat. BMAA has been causatively linked to neurodegenerative disease pathology and, in a rat model, learning and memory impairments.⁷⁴ More recent work has demonstrated the simultaneous imaging of both N-glycans and proteins in the same tissue section via MALDI-MS.⁷⁵ Using a single technique but two sequential on-tissue digestion procedures with PNGaseF and trypsin, complementary images on leiomyosarcomas, myxoid liposarcomas, and colorectal carcinoma tissues were produced. Fixed adrenal cells prepared for and imaged by transmission electron microscopy were also imaged by both ToF-SIMS and nanoSIMS.⁷⁶ Ewing et al. have recently demonstrated the multimodal use of nanoparticle-assisted laser desorption ionization mass spectrometry (NP-LDI MS), MALDI-MS, and gas cluster ion beam (GCIB) SIMS to investigate intact lipids in mouse brain tissues.⁷⁷ GCIB SIMS acts as a semisoft ionization method that closes the gap between conventional SIMS and MALDI techniques in terms of internal energy deposition and ion fragmentation. More recently, a “trimodal” MALDI-MS approach imaging positive and negative lipids and proteins at a resolution of 10 μm was reported, revealing spatially correlated lipid and peptide distributions involved in Aβ plaque pathology in Alzheimer’s disease.⁷⁸

The aforementioned body of work and its success in utilizing multimodal imaging techniques in mass spectrometry are just a sliver of a growing community reliant on these and similar tools to advance research across a wide biomedical front. It is expected that the number of tools and researchers utilizing these approaches will continue to grow with techniques increasing in complexity and in produced data. Some of the complexity alone stems from higher-resolution mass spectrometry techniques operating at nanoscale, and coupled with scanning probe microscopy techniques, described in the next section.

ION, ELECTRON, AND PROBE MICROSCOPY WITH MASS SPECTROMETRY

In the last few decades a whole class of MSI techniques was developed to map distribution of the chemical composition with spatial resolution ranged from nanometers to tens of microns.^79–83 Here, the surface and the subsurface chemistry of the sample is analyzed via a focused physical probe (ionic, optical, thermal, etc.), which releases the analyte species from a small area on the sample. The specific probes define the pros and the cons of a given technique. For instance, ionic probes used in SIMS allow for chemical imaging with submicron spatial resolution^84,85 but lead to significant fragmentation of large molecules,⁸⁶ complicating data interpretation in biological and polymeric systems. However, optical probes as used in MALDI, discussed at some length in the previous section,⁸⁷ allow the direct identification of large molecules (e.g., lipids, peptides, and proteins)^88–91 but require an appropriate matrix and suffer from low spatial resolution, limited by the size of laser beam down to few micrometers. Released secondary species in MSI are further analyzed using different types of the mass detectors (time-of-flight, magnetic sector, orbitrap, etc.) to acquire information on the local chemical composition at a certain spatial point on the surface. Rastering the beam and probe thus produces chemical maps containing information on the distribution of chemical species. When the electron microscopy signal is combined with secondary ion spectrometry, high-resolution imaging is supplemented by the chemical sensitivity intrinsic to MS (Figure 6).

Figure 6.

(a) PIES image: Overlay of BF-TEM image (contrast inverted) and SIMS images of ⁶Li⁺ and ⁷Li⁺. (b–d) High-magnification TEM images (contrast as acquired) corresponding to the boxed hotspots. (e–g) SIMS images of ⁶Li⁺ (green) overlaid on ⁷Li⁺(red) corresponding to panels b–d, respectively. The arrows in the TEM images indicate nanoparticles rich in ⁶Li⁺ (green) and ⁷Li⁺ (red). The color scales indicate secondary ion counts in linear scale. Reproduced with permission from ref 92 under a Creative Commons Attribution 4.0 International License. Copyright 2018 Nature Publishing Group.

Time-of-flight secondary ion mass spectrometry (ToF-SIMS) is one of the MSI techniques that uses a focused ion beam to release secondary ions from the surface and direct them into the time-of-flight analyzer to detect their mass-to-charge ratio. This tool requires high- or ultrahigh-vacuum conditions. The chemical spatial resolution in imaging mode for ToF-SIMS is defined by the spot size of the focused primary beam. Liquid metal ion guns (LMIG), commonly used in static and dynamic SIMS, routinely provide ~100 nm spot sizes. The surface analysis can be complemented by additional sputtering sources, allowing the removal of significant amounts of the material. This extends chemical investigation into the bulk, down to few micrometers in depth. The time-of-flight mass analyzer used in ToF-SIMS offers mass resolution m/Δm in the ~10³ – 10⁴ range, along with wide mass range (up to 10⁵ Da) and high transmission efficiency (above 50%).⁹³ The time-of-flight approach also enables parallel acquisition of the chemical information on all chemical species seen by the detector.^94–97 This produces high-dimensionality data sets in which an entire mass spectrum can be recorded at each voxel of a 3-dimensional sample map; revealing the surface and bulk chemical composition of a material with sub-micrometer spatial resolution. ToF-SIMS has found its application in life sciences, allowing us to probe the distribution of relevant species within the biological sample (Figure 7).

Figure 7.

Correlative back-scattered electron and nanoSIMS analysis of multilamellar lysosomes (MLL) within amiodarone-treated macrophages; (A) BSE image zoomed on an area containing profound multilammelar lysosomes for the full image) and (B) the corresponding NanoSIMS ¹²⁷I⁻ secondary ion map, which shows the locations of amiodarone accumulation. (C) ¹²⁷I⁻-to-³¹P⁻ ratio image of the selected area (the ratio color scale: 0–0.08, blue–pink, respectively). (D) Plot of the ¹²⁷I⁻ and ³¹P⁻ secondary ion intensities measured in lysosomes, showing a linear relationship between the amount of the drug and phospholipids accumulated in MLLs. Reproduced with permission from ref 98. Copyright 2017 The Royal Society of Chemistry.

While there are obvious strengths to chemical sensitivity in the ToF-SIMS, its capabilities can be extended beyond chemical mapping by combining the tool with atomic force microscopy (AFM) in the same vacuum chamber. This multimodal imaging AFM/ToF-SIMS platform enables nano-scale characterization of chemical and physical properties of the sample along with the surface morphology. The chemical information acquired by ToF-SIMS, at chemical resolution of 50–100 nm can be supplemented with functional sample response measured by AFM at resolution down to 1 nm. This combination has been recently used by Sostarecz et al. to characterize chemistry and phase behavior of lipid films⁹⁹ and by Belianinov et al. to study ion induced changes in layered copper indium thiophosphate ferroelectrics.^100–102 Furthermore, a combined AFM/ToF-SIMS platform opens pathways for chemical characterization of local physical behavior at the nanoscale. In these types of experiments, local chemical phenomena induced by the physical field of the AFM cantilever tip (including but not limited to mechanical, electric, thermal, magnetic, etc. excitation) can be studied by ToF-SIMS. Recently, this approach enabled a study focused on the growth of metal–organic frameworks with AFM tip grafting,¹⁰³ explore chemical phenomena associated with local polarization switching in ferroelectrics¹⁰⁴ (Figure 8), and investigate electrochemical response of lithium-ion cathodes¹⁰⁵ and chemical interaction between AFM tip and the sample.¹⁰⁶

Figure 8.

Combined AFM–ToF-SIMS investigations of the electro-resistive switching by AFM tip in BiFeO₃ thin film. (a) Topographical changes associated with electroresistive switching. (b) ToF-SIMS XY map of Bi⁺ on the surface. (c–g) XZ depth profiles across a modified region, base elements of the BiFeO₃ film, SrRuO₃ buffer layer, and LaAlO₃ substrate: (c) Fe⁺, (d) Sr⁺, (e) Al⁺, (f) La⁺, and (g) Cs₂O⁺. (h) Overlay of Fe⁺ (red), Sr⁺ (blue), and Al⁺ (green). Reprinted with permission from ref 104. Copyright 2018 American Chemical Society.

AFM topography offers quantitative roughness and depth measurements, which can be used before and after ion beam sputtering in 3D profiling ToF-SIMS measurements. In this type of work, the depth of the sputtered crater as measured by AFM, allows sputter-rate calibration and accurate volume reconstruction, assuming sample homogeneity.^107–110 The authors point out that the chemical resolution is still dictated by the spot size of the beam. However, the geometric correction offered by the AFM topography can offer invaluable information at the data reconstruction step, in which changes in morphology can significantly impact interpretation.^111–115

Thus, a combined AFM/ToF-SIMS platform offers a perfectly complementary set of tools for correlated chemical and functional investigations of wide range of materials. Achieving lateral chemical spatial resolution below 50 nm with primary-source LMIGs is still challenging. Gas field ion sources (GFIS) using helium or neon ion beams as a primary source, combined with mass spectrometers, were recently suggested by Wirtz et al., based on the commercially available helium ion microscope (HIM) Zeiss ORION NanoFab.¹¹⁶ While the source design for this tool is significantly more complex than for LMIG, the spatial resolution (~0.5 nm with He and ~1.9 nm with Ne) is a significant breakthrough for the SIMS community.

Helium ion microscopy (HIM) is a relatively young technique that can produce higher-resolution images than scanning electron microscopy (SEM).^117,118 GFIS deliver high-brightness ion beams and the interaction volume for helium ions is smaller than for electrons in SEM.¹¹⁹ The overall result is enhanced resolution and a greater depth of field for HIM compared to SEM.¹²⁰ In addition to the imaging of emitted secondary electrons, the helium ion beam can be used to obtain information on ions released by sputtering. High-brightness and high-current (5–10 pA) ion beams and a low-energy spread below 1 eV can be theoretically focused to area smaller than 1 nm,¹²¹ making the GFIS interesting as a primary ion source for SIMS.¹²² No general restrictions exist that prohibit the use of gases other than helium in GFIS, although practical reasons limit the use to species with sufficiently high ionization energies, i.e., to helium and neon.¹²³

While the small focus size of the GFIS beam directly translates into a higher spatial resolution, for SEM the limitations for the analysis of secondary ions are set by the lateral dimensions of the collision cascades induced in the sample. For helium, the collision cascade is situated quite deep below the sample surface but because only a small fraction of the impact energy is deposited close to the surface, sputter yields (the fraction of sputtered particles to incident ions) are low (<1 for He).¹¹ For neon, the surface area that secondary particles are sputtered is significantly larger due to shallower collision cascades.^124,125 Sputter yields are higher (30× in comparison to He) and comparable to O and Cs ion beams routinely utilized in SIMS.

The initial work of Wirtz et al.¹¹⁶ led to the development of a HIM-SIMS tool based on the Zeiss ORION NanoFab microscope utilizing a custom set of retractable extraction optics to feed into a double focusing magnetic sector mass spectrometer with four detectors in a Mattauch-Herzog design. This system showed chemical imaging with 13 nm lateral resolution on a standard sample. Another implementation of HIM-SIMS platform was demonstrated by Klingner et al. who extended a Zeiss ORION NanoFab by time-of-flight based mass analyzer, using a pulsed primary ion beam and a multichannel plate (MCP)-based ion detector.¹²⁶ Altogether HIM-SIMS is considered as a perspective direction of development for MSI, allowing chemical imaging with ultimate spatial resolution.

Although various SIMS techniques provide a great set of the tools for chemical imaging with submicrometer and even nanometer spatial resolutions, several fundamental problems hamper its application for identification of large (biological) molecules. Primary ion beams used in SIMS for ionization lead to molecular fragmentation. Furthermore, SIMS measurements require vacuum, which is not compatible with many materials. This significantly complicates chemical characterization of biological and some other systems in SIMS. A hybrid solution, utilizing sharp AFM tips for material desorption from the surface of the sample, has been recently suggested.^127–129

An AFM tip can be used to mechanically uptake sample material from the surface. The work by Lee et al.¹²⁸ suggested using field ionization to release the scratched material into a mass spectrometer for elemental analysis. The entire SPM–MS system was held under vacuum along with a special probe design that enabled tip transfer (together with the collected sample) from a surface sampling position to a field desorption emitter position. In other work by Hoffmann et al.,¹³⁰ the AFM tip was used to mechanically sample material from the surface, with a subsequent temperature ramping step for the controlled release of the material by thermal desorption. Initial work on coupling direct thermal desorption from the sample surface with heated AFM cantilever probes with mass spectrometry has been developed by Price et al.¹³¹ This method used an offline approach to capture material that was thermally desorbed under either volatilization or pyrolysis on a sorbent in close vicinity to the desorption site. This enabled the analysis of the desorbed material by gas chromatography coupled with mass spectrometry. Other researchers also used similar approaches to draw material from the desorption site through a capillary into a separate collection device for the deposition onto graphite, which allowed for subsequent laser desorption and two-photon ionization.¹³²

Several combined AFM/MS platforms using thermal desorption have been developed. Their general applicability has been illustrated for small organic molecules including dyes, pharmaceuticals, explosives and pesticides.^133–135 The application to polymers also has been shown (Figure 9) based on the detection of small characteristic fragments, such as for poly(methyl methacrylate) (PMMA), polystyrene (PS), and poly(2-vinylpyridine) (P2VP).^136–138 The overall sensitivity of the combined AFM–MS platform can be benchmarked by the minimal detectable amount of sample material (e.g., the smallest desorption crater), and future developments are focused on improving the precision with which the material is removed.

Figure 9.

Combined SIMS–SPM 3D mapping of a PS/PMMA blend. Field of view: 10.6 μm × 9.8 μm. Reproduced with permission from ref 139. Copyright 2018 John Wiley & Sons, Inc.

In the case of thermal desorption, improved heating functions can reduce the redeposition of initially desorbed material¹⁴⁰ on the vacuum-transfer line walls.¹⁴¹ Additionally, inline ion sources can aid with material ionization for lower limits of detection. For example, AFM coupled to a custom-built electrospray ionization (ESI) stage is capable of detecting 250 nm craters in pure caffeine.¹²⁹ Several implementations using atmospheric pressure chemical ionization (APCI) stages have been used, based on modifications of the manufacturer’s ion source design¹³³ and custom APCI-based inline ionization stages.^136,141 Detection of ink dye components for spot-sampling yields 800 nm lateral resolution as well as 411 nm spatial resolution for P2VP and polystyrene PS polymers. For the multimodal imaging with an AFM–MS system, Ovchinnikova et al. illustrated a phase-separated PS/P2VP system using multiple imaging modalities provided by the AFM (topography and band excitation nanomechanical) followed by the chemical imaging of thermally desorbed material in the MS (Figure 10).¹⁴²

Figure 10.

Co-registered AFM (a) prepyrolysis topography image, (b) BE elastic modulus image, (c) postpyrolysis topography image, and (d) mass spectrometry chemical image for m/z 106, obtained from a ~500 nm thick thin film of phase-separated PS/P2VP blend. The color scale for the topography goes from dark to light, which is proportional to an increase in relative surface height. Highlighted ovals in panels b–d indicate areas where the AFM topography, elastic modulus, and mass spectrometry images differ in terms of the presence of P2VP. Reprinted with permission from ref 142. Copyright 2018 American Chemical Society.

In summary, multimodal platforms where mass spectrometry imaging is combined with other scanning-probe microscopy techniques allows a significant extension of capabilities for chemical characterization and correlation of chemical phenomena with structural and physical properties. In addition to classical optics and scanning-probe systems incorporated into combinatorial MS systems, tools featuring nonlinear optical microscopy components are gaining popularity.

NONLINEAR OPTICAL MICROSCOPY

Nonlinear optical spectroscopies have now been used for several decades to probe chemical and physical phenomena in systems spanning small molecules,^143–145 proteins,^146–148 complex materials,^149–151 and interfaces.^152–155 The foundation of any nonlinear method lies in multiple interactions between the sample and the incident radiation driven by the high intensity of a laser that is often pulsed. It is through these multiple light–matter interactions that access to ultrafast dynamics or surface-selective chemistry, which are not available using linear optical methods, is obtained. These methods thereby provide a direct view of the underlying physical and chemical processes at play. Despite the great advances in fundamental understanding, traditionally ensemble spectroscopic probes report spatially averaged information. In many cases, however, heterogeneity on the nano- and mesoscales is critical to the function and performance of these sample systems. As such, there has been a push to apply nonlinear optical methods to microscopic platforms to obtain both chemically selective and time-resolved information from spatially heterogeneous and chemically complex samples. By far, the most common form of nonlinear microscopy is two-photon fluorescence microscopy, which has found greatest utility in biological imaging applications.^156–159 This technique takes advantage of two-photon transitions to populate an electronic excited state from which a photon is emitted and recorded at a given spatial location. This approach is the nonlinear analog of confocal fluorescence microscopy but boasts superior penetrating power, reduced photobleaching, and intrinsic longitudinal sectioning.^156–158 These methods tend to rely on an assortment of stains and dyes that are designed to target specific regions of a system^160,161 (say, in a cellular membrane)¹⁶² and, thus, provide contrast based on where the labels are localized and can emit light. These approaches have also found utility in imaging domains and local ordering in light-harvesting materials without the need for labels.¹⁶³ While widely used, these methods rely on the emission of light, which, in many cases, necessitates staining procedures or the study of materials with appreciable quantum yields, thereby limiting the overall utility. To address these limitations, complementary suites of nonlinear optical microscopies have been developed to probe native species in complex heterogeneous samples without the need for chemical labels or dyes. These can be categorized as even-order techniques that are sensitive to the molecules at an interfacial monolayer or to chiral assemblies of molecules, e.g., second-harmonic generation (SHG)^92,164–170 and sum-frequency generation (SFG),^{101,171–175} and odd-ordered methods that probe molecular resonances in the bulk material, e.g., coherent anti-Stokes Raman scattering (CARS),^176–179 stimulated Raman scattering (SRS),^180,181 and transient absorption microscopy (TAM).^182–188 Note that this is not an exhaustive list but is rather a compilation of those most commonly found in the literature.

As an example, both CARS and SRS microscopies have revolutionized the way in which vibrational chemical images are taken. Traditional linear Raman microscopes rely on a weak Raman scattering process, which limits the time and fidelity at which a chemical map can be obtained. In contrast, using coherent nonlinear approaches, Raman signals can be greatly enhanced allowing for chemical images to be obtained at video rate^180,181 or for the dynamics of a single molecules tracked in time.¹⁷⁷ SHG imaging methods have been developed and applied to probe interfacial or chiralspecies. 92,164–170 Due to symmetry considerations, only species in noncentrosymmetric environments (i.e., interfaces) or chiral species can produce appreciable SHG signals, thus making it an ideal tool for imaging interfacial phenomena and biological systems. Through staining procedures, SHG methods can provide insight in local electric-field complex systems such as neurons.^165,166 Extension of SHG imaging to SFG microscopy has been reported in a several different geometries with samples studied ranging from biological systems to complex matrerials.^{101,171–175} Of particular note is the recent development of compressive sensing SFG microscopy, which leverages efficient means to acquisition hyperspectral images with relatively minor modifications to a conventional SFG spectrometer.^172,173 It is expected that approaches using compressive sensing will revolutionize the field of nonlinear microscopy by allowing chemical images to be obtained using established spectroscopic protocols while providing microscopic insight.

In parallel, approaches using intrinsic time-domain responses as a source of chemical contrast have appeared in the literature.^188–194 These approaches extend fluorescence lifetime imaging microscopy (FLIM) techniques,¹⁹⁵ in which the temporal responses of excited-states species can provide contrast between areas of disparate chemical speciation or environments while simultaneously providing access to excited-state photochemical dynamics. A distinguishing feature of transient absorption microscopies (also known as pump–probe microscopy)¹⁹² is the use of multiple laser pulses to excite and probe species at controllable ultrafast (femtoseconds to picoseconds) time delays to produce a 3D-image stack, as shown in Figure 11. By exciting and probing the system at very early time delays, access to a whole realm of information is obtained, including the dynamics of excitons, motions of charge carriers, and local chemical makeup, to name a few examples. Additionally, because the probe pulse carries the relevant optical signals, one does not rely on emission of the sample, thus allowing for native nonfluorescent samples to be imaged¹⁸⁸ at sub-diffraction limited spatial resolutions with temporal resolutions less than 100 fs.

Figure 11.

(a) Near-field pump–probe time traces recorded on different positions on a microcrystal (red, positions indicated in panel b), together with full time traces extracted from a series of spatiotemporal images (blue, images and positions shown in panel b). (b) Set of images showing spatial and temporal variations in the IMT dynamics of a single VO₂ microcrystal. (c) Line profiles for several pump–probe time delays across the microcrystal shown in panel b. Reprinted with permission from ref 196. Copyright 2018 American Chemical Society.

Pioneering work by the Warren group demonstrated this capability through selective probing and characterization of cancerous tissues.^193,194 By selectively exciting species and probing the ultrafast dynamics in time, they were able to differentiate between species having indistinguishable absorption spectra by using only the ultrafast responses. In this way, they were able to spatially localize the distributions of eumelanin and pheomelanin in skin lesions without chemical labels.¹⁹⁴ These approaches have even been applied to fine art for the characterization of pigments in paintings in a nondestructive manner.¹⁹⁷ TAM has also found numerous applications to materials systems in which insight into photophysical phenomena in spatially heterogeneous materials is generally lacking. For instance, work probing lead halide perovskites, also extensively studied by scanning probes,^198–201 used the temporal response at distinct spatial locations to characterize the photoexcited-state species that were created (Figure 12).¹⁹¹ This, and related work, demonstrated that a coexistence of charge carriers and excitons is present in these materials.^189–191 Other examples of the utility of TAM-based microscopies make use of spatially offset probe pulses^185,202,203 to track the motions of carriers as they move though a material in space and time. This powerful approach provides a movie of excited-state processes evolving on native time and length scales, which holds promise in understanding photo physics in spatially heterogeneous and chemically complex materials.

Figure 12.

TAM images acquired for the same area of the pristine (a) CH₃NH₃PbI₃, (b) CH₃NH₃PbI₃/PCBM, and (c) CH₃NH₃PbI₃/spiro-OMeTAD samples at five different delay time delays. Scale bars are 5 μm. The color bars encode the variation in TAM signal and sign. Adapted with permission from ref 204. Copyright 2018 American Chemical Society.

While TAM methods provide excellent sensitivity to ultrafast electronic processes, it is morphologically blind. This lack of morphological information can limit interpretations regarding photophysical processes based on challenges in correlating material structure to emission intensities or TAM images collected on different platforms. To circumvent this, multimodal approaches to TAM have been developed by combining linear photoluminescence (PL),²⁰⁵ transmission, and confocal reflectance microscopies into a single platform.²⁰⁶ By collecting a multidimensional data set using the same laser system at the same sample location, insight into how morphology and film thickness impact the ultrafast and exit channel dynamics (TAM and PL imaging, respectively) can be obtained. For instance, from a correlation analysis of four different optical modalities, it was shown that in mixed perovskite thin films, PL originates predominantly from the first few layers of the film, whereas the TAM measurements probe predominantly bulk excited-state processes.²⁰⁶

Through the combination of the various linear and nonlinear optical methods described above that use the same microscopic platform a more complete picture of the sample can be obtained than using any individual approach. That being said, challenges in processing and developing physical insight these large multidimensional data sets, often with different resolutions, bin sizes, etc., exist and remain to be addressed as chemical imaging methods continue to grow. Coupling of these optical methods to scanning probe^207,208 or electron microscopy methods^209,210 has also attracted attention and remains an avenue for continued development (for instance, using near-field optical excitation for the probing of the sample for the visualization and spectroscopic characterization of a vast variety of nanomaterials, from semiconducting nanoparticles to polymer thin films to the sensitive measurements of single molecules; see Figure 13). In addition, it can be conducted in non-contact mode, producing high-resolution measurements at ambient conditions. Such nonlinear optical properties of materials as nonlinear excited-state absorption and stimulated Raman vibrational transitions can be locally measured by these approaches. The ultimate goal would be to push the temporal and spatial limits of both optical and scanning probe- and electron-microscopy methods to address scientific questions at the extremes of space and time.

Figure 13.

(a) Schematic of the pump–probe excitation of silicon 2,3-naphthalocyanine (SiNc). (b) Time-resolved excited-state absorption measured with PiFM (solid dots) and with optical pump–probe microscopy (solid line). (c) Topography (top) and PiFM signal amplitude (bottom) of a nanocluster measured at different time delay settings of the probe pulse. Reprinted with permission from ref 211. Copyright 2018 American Chemical Society.

MULTIMODAL AND MULTIDIMENSIONAL DATA ANALYSIS

Over the last two decades, developments in the characterization equipment have significantly increased the size and the quality of data produced by experimental techniques.^212–217 Improved detectors, a rise in multimodal tools (machines capable of probing different material properties) and clever uses of metadata have underwritten the growth of collected and stored data. Interestingly, this trend in scientific data growth follows classification devised by IBM in 2012²¹⁸ for big data on the web, shown in Figure 14, and is reflected in a wide variety of common characterization techniques, e.g., scanning probe microscopy; with data volumes and computational complexity reflected in Figure 14b,c as well as scanning transmission electron microscopy, mass spectrometry, and many others. Terabytes of scientific data in biological and material sciences are now common beyond monolithic tools like accelerators and beamlines; the data are mixed between structured and unstructured, and many detectors and devices are moving to storing data streams as opposed to data batches. Unfortunately, analysis approaches have largely lagged behind this growth or remained static altogether.^219,220

Figure 14.

Data set size and computational power evolution. (a) IBM’s characterization of big data: volume, variety, velocity (V3). (b) Evolution of multidimensional data sets in scanning probe microscopy and their sizes over the past decade. Acronym list: BE, band excitation; SSPFM, switching spectroscopy piezoresponse force microscopy; TR-PFM, time-resolved piezoresponse force microscopy; BESSPFM, band excitation piezoresponse force microscopy; TR-BE, time-resolved band excitation; FORC, first-order reversal curve. (c) Typical processing and acquisition time (smaller value is better) on a laptop, desktop, and cluster for multidimensional data sets. Hardware configurations were assumed as follows: laptop, 4-core processor, 8 GB of RAM, integrated video, and 1 hard drive with approximately 1 TB of space; desktop, 12-core processor, 32 GB RAM, dedicated video, 2 hard drives, 4 TB of space; cluster, 10 nodes, each node with 8 processors at 8 cores, 20 GB of RAM, 160 GB storage space. Adopted with permission from ref 221 under an Open Access article distributed under the terms of the Creative Commons Attribution License. Copyright 2018 Springer.

Advances in machine learning brought significant developments to multiple areas of science and engineering, specifically image processing,²²² with autonomous vehicle technology perhaps leading the charge.²²³ While the aspects of image processing and machine learning have already begun appearing in areas of microscopy,^224–227 the underpinning aspects of what makes machine learning so attractive to scientists is the ability to make sense of seemingly disconnected pieces of data potentially coming from a wide variety of sources at various rates and resolutions and being reflective of different material properties.²²⁸

This is promising for chemical imaging, in which many of the collected data, or data descriptors, contain correlated information on structure and property of the specimen. While the analysis of these data has traditionally followed a qualitative and manual approach, the veracity and volume of data produced by modern multimodal instrumentation either preclude such methods or significantly hamper the processing rate. Nonetheless, manual or semiautomated approaches have proven to be successful in relatively high-volume data environments combining multimodal information to present a fuller picture of material composition and exhibited properties.^{142,229–232} More-advanced approaches, leveraging deep learning, are less common but are now gaining popularity.^233–235 Nevertheless, the penetration of machine learning into scientific and industrial areas reliant on multimodal tools, and combinatorial data approaches remain slow. In addition, machine learning enables high-throughput processing of experimental data from multiple sources through the data fusion (Figure 15). This approach boosts the capability to generate complex insights about composition and structure of the sample by improving the quality of the image as well as deriving connections between various sample properties on a fundamental level.

Figure 15.

Ion image measured in mouse brain, describing the distribution of m/z 778.5 [identified as lipid PE(P-40:4)] at 100 μm spatial resolution (top right) is integrated with an hematoxylin and eosin microscopy image measured from the same tissue sample at a 10 μm resolution (top left). By combining the information from both image types, the image fusion process can predict the ion distribution of m/z 778.5 at a 10 μm resolution (bottom). Reprinted with permission from ref 240. Copyright 2018 Nature Methods.

This trend reflects several challenges complicating the direct adaptability of data analytics to multimodal problems in science and engineering. While many in their respective fields have achieved a considerable degree of sophistication in the use of respective experimental, theoretical, and computational tools, the overlap between these communities is small. This problem is exacerbated by a lack of a common language and philosophy and will likely necessitate extensive cross-disciplinary training. Furthermore, big- and deep-data approaches, implying knowledge extraction from data, greatly benefit from universal, centralized, or distributed databases and repositories. This requires information exchange between researchers, requiring the development of infrastructure, the adoption of compatible and potentially universal data formats and addressing inevitable intellectual property and socio-cultural issues. Private entities like Citrine Informatics have already begun to fill these gaps proving that there is a viable pathway toward centralizing and standardizing scientific information.

Perhaps the most compelling evidence for feasibility of large-scale processing and machine learning in science and engineering comes from areas that have long embraced the potential of these tools and have shown their practicality. Data analytics have been long-established in the materials-modeling community, starting with chemometrics and on to computation and correlation of response functions with experimental data. In fact, the argument could be made that rapid advances in quantum density functional theory was the impetus for the concept of computational-based materials by design.²³⁶ Early efforts that utilized the power of supervised and unsupervised neural networks and genetic and evolutionary algorithms, along with graph theory and statistics-based methods, demonstrated capabilities, even though the data available at that time was modest compared with the volumes accessible today.^237,238 These success stories, advances in experimental techniques, and availability of high-quality information as well as notable advances in high-performance computing²³⁹ offer a clear road map to the required capabilities for better scientific data analysis and bridge technologies across various disciplines.

CONCLUSIONS

Chemical analysis at the nanoscale is critical to progress in the fields of biology, medicine, and material science. Complicated processes such as cellular signal transduction, pharmaceutical discovery, and trace-element characterization in nanoelectronics require nanometer-resolved multimodal chemical and physical analysis. This subsequently drives the need for analytical tools offering higher sensitivity and detailed chemical information coupled to high spatial resolution modes. The trend is beginning to be recognized by equipment manufacturers and is evidenced by the development of several platforms. Much attention has been focused on combining chemically sensitive techniques with high spatial resolution techniques such as scanning probes, optical microscopy, and electron and ion systems. Developing multimodal imaging approaches and interrogating multiple properties on a single platform circumvents many technical issues associated with sample preparation, transfer, and storage. Furthermore, these combinatorial techniques generate multidimensional data sets, which are expected to grow and contain data as a function of constantly increasing number of parameters such as time, temperature, bias, light, and other external stimuli. From the technical aspect, moving toward single platforms can significantly reduce the complexity associated with intermediary data processing and visualization steps as well as data provenance. Effective approaches to dimensionality reduction, scalable algorithms, high-performance computing, and cloud infrastructure still need to be widely and uniformly implemented for scientific use. Nonetheless, multimodal chemical-imaging systems already offer a glimpse of powerful capabilities offered by extracting additional information from the cross-correlating and combinatorial processing of the captured material signals. Going forward, we will, without a doubt, see an emergence of more-complex processing capabilities (likely through the use of support vector machines and supervised learning methods) as well as significant breakthroughs in multimodal hardware capable of capturing even more independent channels of information.

GLOSSARY

multimodal chemical imaging

combined independent characterization modalities capable of mapping concentration, or distribution of chemical species.

functional imaging

characterization techniques for detecting or measuring the distribution of physical parameters.

generative model

a model of the conditional probability of the observable X, given the target y.

co-registration

transforming data from multiple sources onto a single common coordinate system.

data velocity

the speed of data processing.

ACRONYMS

AFM

atomic force microscopy

APCI

atmospheric pressure chemical ionization

ATR

attenuated total reflection

BE

band excitation

BE-SSPFM

band-excitation piezoresponse force microscopy

BF-TEM

bright-field transmission electron microscopy

BMAA

β-N-methylamino-L-alanine

BSE

back-scattered electrons

CARS

coherent anti-Stokes Raman scattering

CNT

carbon nanotubes

EELS

electron energy loss spectroscopy

ESI

electrospray ionization

FLIM

fluorescence lifetime imaging microscopy

FORC

first-order reversal curve

FTIR

Fourier transform infrared spectroscopy

GCIB

gas cluster ion beam

GFIS

gas field ion sources

HIM

helium ion microscopy

HIPP

high-impact polypropylene

IR

infrared

IR s-SNOM

infrared vibrational scattering scanning near-field optical microscopy

LC–MS/MS

liquid chromatography-tandem mass spectrometry experiments

LMIG

liquid metal ion guns

MALDI

matrix-assisted laser desorption/ionization

MALDI-MS

matrix-assisted laser desorption/ionization mass spectrometry

MALDI-ToF-MSI

matrix-assisted laser desorption/ionization time-of-flight mass spectrometry imaging

MCP

multichannel plate

MRSI

magnetic resonance spectroscopic imaging

MS

mass spectrometry

MSI

mass spectrometry imaging

nc-AFM

non-contact atomic-force microscopy

NP-LDI MS

nanoparticle-assisted laser desorption ionization mass spectrometry

P2VP

poly(2-vinylpyridine)

P3HT

poly(3-hexylthiophene-2,5-diyl)

PC

phosphocholine

PEDOT

poly(3,4-ethylenedioxythiophene) polystyrenesulfonate

PiFM

photoinduced force microscopy

PIES

parallel ion electron spectrometry

PL

photoluminescence

PMMA

poly(methyl methacrylate

PS

polystyrene

PTIR

photothermal induced resonance

SEM

scanning electron microscopy

SFG

sum-frequency generation

SHG

second harmonic generation

SIMS

secondary ion mass spectrometry

SNOM

scanning near-field optical microscopy

SPM

scanning probe microscopy

SPM-MS

scanning probe microscopy mass spectrometry

SRS

stimulated Raman scattering

SSPFM

switching spectroscopy piezoresponse force microscopy

STEM

scanning transmission electron microscopy

STM

scanning tunneling microscopy

TAM

transient absorption microscopy

TEM

transmission electron microscopy

TERS

tip-enhanced Raman scattering

TR-BE

time-resolved band excitation

TR-PFM

time-resolved piezoresponse force microscopy