Unveiling the spatial distribution of molecular coherences at conical intersections by covariance X-ray diffraction signals

Significance

Monitoring the real-space motion of the electrons directly involved in ultrafast molecular dynamics is a challenging goal, hindered by the strong contributions of the remaining inactive electrons. Molecular quantum coherences can provide a direct window into elementary electronic dynamics, but they could not be separately imaged so far. Here, we demonstrate a time- and frequency-resolved X-ray diffraction technique, which can employ existing stochastic X-ray pulses to separately image the coherence contributions. The technique can selectively follow the relevant electrons involved in the ultrafast dynamics and identify the confined molecular regions covered by their motion. It enables readily feasible measurements and can be straightforwardly extended to full three-dimensional imaging via X-ray and electron diffraction.

Abstract

The outcomes and timescales of molecular nonadiabatic dynamics are decisively impacted by the quantum coherences generated at localized molecular regions. In time-resolved X-ray diffraction imaging, these coherences create distinct signatures via inelastic photon scattering, but they are buried under much stronger background elastic features. Here, we exploit the rich dynamical information encoded in the inelastic patterns, which we reveal by frequency-dispersed covariance ultrafast powder X-ray diffraction of stochastic X-ray free-electron laser pulses. This is demonstrated for the photoisomerization of azobenzene involving the passage through a conical intersection, where the nuclear wave packet branches and explores different quantum pathways. Snapshots of the coherence dynamics are obtained at high frequency shifts, not accessible with conventional diffraction measurements. These provide access to the timing and to the confined spatial distribution of the valence electrons directly involved in the conical intersection passage. This study can be extended to full three-dimensional imaging of conical intersections with ultrafast X-ray and electron diffraction.

The transient pathways of virtually all photophysical and photochemical processes in molecules are dictated by conical intersections (CoIns). These are degeneracy regions in electronic potential energy surfaces where electrons and nuclei move on comparable timescales and become strongly coupled (1, 2). Despite CoIns being ubiquitous in molecules, accessing them experimentally has been especially challenging (3, 4). Recent advances in extreme-ultraviolet (XUV) and X-ray light sources have significantly increased the experimental capability to target different properties of these elusive nonadiabatic passages. It is now possible to record the precise timings of CoIns with attosecond transient absorption spectroscopy (5, 6) or XUV photoelectron spectroscopy (7), measure the transport of electronic coherences through CoIns (8), and distinguish between coupled nuclear and electronic dynamics with ultrafast electron diffraction (9, 10). Techniques directly accessing the quantum coherences generated at CoIns have also been put forward (11–13).

Real-space imaging of the evolution of the electronic charge densities at CoIns is a long-standing dream. With (sub)femtosecond hard X-ray pulses from free-electron lasers (FELs) (14) and laser-driven plasmas (15), X-ray diffraction, a technique that provides spatial information for structure determination, has recently entered the ultrafast regime (16–21). While diffraction experiments at third-generation synchrotron sources are typically performed in crystals, high-brilliance FEL sources are now enabling ultrafast X-ray diffraction (UXD) from samples of reduced size, such as nanocrystals (22), gases of aligned molecules (23), and macromolecules (24), with prospects for single-molecule experiments (25).

In UXD, most X-ray photons are scattered elastically off electronic state densities, with contributions from all electrons in the molecule. This represents a challenge when monitoring CoIns, where most core and valence electrons are inactive, and only a few valence electrons are directly involved. Key temporal and spatial information about CoIn dynamics is encoded in the additional UXD inelastic patterns (19, 26, 27). These signatures directly stem from the localized quantum coherences generated at CoIns. They are scattered off transition charge densities, which map the few electrons involved in the CoIn passage without background contributions from the remaining inactive ones. However, observing these distinctive coherence signatures in the UXD signal has not been possible thus far. Frequency dispersing signals from existing stochastic broadband FEL pulses does not allow an effective separation of elastic and inelastic scattering. As a result, the coherence contributions are buried beneath the much stronger elastic scattering off state densities.

Here, by frequency dispersion of time-resolved X-ray diffraction signals, and further employing a covariance-based protocol (28–36) with state-of-the-art stochastic FEL pulses, we separate the inelastic signatures due to CoIn coherences from the dominant, less distinctive elastic features. This is exemplified for powder diffraction off randomly oriented nanocrystals of azobenzene molecules undergoing photoisomerization through a CoIn (27). The signal allows us to monitor the molecular coherences emerging during the nonadiabatic dynamics, revealing a confined distribution of transition charge densities representing small distances in the molecule. This reflects the $n{\pi }^{*}$ character of the excitation, with contributing orbitals localized around the nitrogen atoms. Implemented in a crystal, covariance UXD can offer background-free, full three-dimensional reconstruction of transition charge densities at CoIns. Our approach, based on energy dispersing the diffraction signal to isolate inelastic coherence contributions, can be analogously applied to ultrafast electron diffraction (9, 10, 37).

The effectiveness of the covariance UXD signal is demonstrated for the photoisomerization of azobenzene, a textbook photochemical switch with cis and trans isomers, as shown in Fig. 1A. Azobenzene can be switched selectively with high quantum yield between both geometries on a femtosecond timescale (39–41). With its derivates, azobenzene has found application in photopharmacology and optogenetics, to control the activity of pharmaceutical compounds (42) or neurons (43). Understanding the primary events of its transformation is thus of broad relevance.

Fig. 1.

Time- and frequency-resolved CUPXD signal of azobenzene molecules. (A) CUPXD measurement. (Left) An ensemble of molecules is excited by an off-resonant X-ray pulse with variable central time $T$ and wavevector ${\mathit{k}}_{\mathrm{X}}$. The pulse encounters azobenzene molecules undergoing photoisomerization from a cis to a trans geometry. The signal scattered along different ${\mathit{k}}_{\mathrm{s}}$ directions is recorded by a pixel array detector placed far from the sample. The time- and frequency-dependent X-ray diffraction signal is obtained by varying the X-ray pulse arrival time and spectrally dispersing the signal at every pixel point. The resulting rotationally averaged CUPXD signal (Eq. 5) is shown in arbitrary units for a pulse duration of 22 fs, time delay T = 50 fs, and (Bottom Right) ${\omega }_{\mathrm{R}}=0\hspace{0.17em}\mathrm{e}\mathrm{V}$ on the $q_x$–$q_y$ plane or (Top Right) frequency dispersed at $q=3.3$ ${\AA }^{-1}$. (B) Loop diagrams (38) for UXD off a nonstationary state of azobenzene molecules with the two electronic states $g$ (ground) and $e$ (excited). Population and coherence contributions are shown.

To monitor the evolution of the electronic charge densities $\widehat{\sigma }\left(\mathit{r}\right)$ during the CoIn passage, we assume the powder diffraction setup shown in Fig. 1A. Stochastic X-ray pulses, with variable central time $T$ and momentum ${\mathit{k}}_{\mathrm{X}}$, are scattered off randomly oriented nanocrystals of azobenzene molecules undergoing cis $\to$ trans photoisomerization. The spontaneously emitted photons are assumed to be dispersed in momentum ${\mathit{k}}_s$ and frequency ${\omega }_{\mathrm{s}}$ (44) by an array of frequency spectrometers far from the sample. Within each nanocrystal, owing to long-range structural order, the signal arises from the interference of X-ray photons scattered at pairs of molecular sites (19). Scanning the pulse arrival time results in the momentum-, frequency-, and time-resolved X-ray diffraction signal

$$\begin{array}{cc}\hfill & S\left(\mathit{q},{\omega }_{\mathrm{s}}^{\prime },T\right)\propto |{\widehat{\mathit{ϵ}}}_{\mathrm{s}}^{*}\cdot {\widehat{\mathit{ϵ}}}_{\mathrm{X}}{|}^2\hspace{0.17em}\frac{{\alpha }^3{\omega }_{\mathrm{s}}}{4{\pi }^2}\hfill \\ \hfill & \hspace{1em}\times {\left|\int \mathrm{d}t\hspace{0.17em}{A}_{\mathrm{X}}\left(t-T\right)\hspace{0.17em}{\mathrm{e}}^{\mathrm{i}{\omega }_{\mathrm{s}}^{\prime }\left(t-T\right)}\hspace{0.17em}\left[{\tilde{\sigma }}_{\mathrm{p}\mathrm{o}\mathrm{p}}\left(\mathit{q},t\right)+{\tilde{\sigma }}_{\mathrm{c}\mathrm{o}\mathrm{h}}\left(\mathit{q},t\right)\right]\right|}^2\hfill \\ \hfill & =|{\widehat{\mathit{ϵ}}}_{\mathrm{s}}^{*}\cdot {\widehat{\mathit{ϵ}}}_{\mathrm{X}}{|}^2\hspace{0.17em}\frac{{\alpha }^3{\omega }_{\mathrm{s}}}{4{\pi }^2}\hfill \\ \hfill & \hspace{1em}\times {\left|\int \frac{\mathrm{d}\omega }{2\pi }\hspace{0.17em}{Ã}_{\mathrm{X}}\left({\omega }_{\mathrm{s}}^{\prime }-\omega \right)\hspace{0.17em}{\mathrm{e}}^{-\mathrm{i}\omega T}\hspace{0.17em}\left[{\widetilde{\stackrel{~}{\sigma }}}_{\mathrm{p}\mathrm{o}\mathrm{p}}\left(\mathit{q},\omega \right)+{\widetilde{\stackrel{~}{\sigma }}}_{\mathrm{c}\mathrm{o}\mathrm{h}}\left(\mathit{q},\omega \right)\right]\right|}^2,\hfill \end{array}$$ [1]

with the diffraction momentum transfer $\mathit{q}={\mathit{k}}_{\mathrm{s}}-{\mathit{k}}_{\mathrm{X}}$, the frequency difference ${\omega }_{\mathrm{s}}^{\prime }={\omega }_{\mathrm{s}}-{\omega }_{\mathrm{X}}$, and the fine-structure constant $\alpha$ (see Materials and Methods for details). Atomic units are used throughout unless otherwise stated. $A_{\mathrm{X}}\left(t\right)$ and ${Ã}_{\mathrm{X}}\left(\omega \right)=\int \mathrm{d}t\hspace{0.17em}{A}_{\mathrm{X}}\left(t\right)\hspace{0.17em}{\mathrm{e}}^{\mathrm{i}\omega t}$ are the complex time and frequency envelopes of the pulse vector potential ${\mathit{𝒜}}_{\mathrm{X}}\left(\mathit{r},t\right)={\widehat{\mathit{ϵ}}}_{\mathrm{X}}{A}_{\mathrm{X}}\left(t-T\right)\hspace{0.17em}{\mathrm{e}}^{\mathrm{i}{\mathit{k}}_{\mathrm{X}}\cdot \mathit{r}}\hspace{0.17em}{\mathrm{e}}^{-\mathrm{i}{\omega }_{\mathrm{X}}\left(t-T\right)}$ with central frequency ${\omega }_{\mathrm{X}}$, while ${\widehat{\mathit{ϵ}}}_{\mathrm{X}}$ and ${\widehat{\mathit{ϵ}}}_{\mathrm{s}}$ are the polarization unit vectors of the incident and scattered X-ray photon, respectively. ${\tilde{\sigma }}_{\mathrm{p}\mathrm{o}\mathrm{p}}$ and ${\tilde{\sigma }}_{\mathrm{c}\mathrm{o}\mathrm{h}}$ are later defined as the population and coherence contributions to the charge density in reciprocal space (19), $\widehat{\tilde{\sigma }}\left(\mathit{q}\right)=\int {\mathrm{d}}^{3}r\hspace{0.17em}\widehat{\sigma }\left(\mathit{r}\right)\hspace{0.17em}{\mathrm{e}}^{-\mathrm{i}\mathit{q}\cdot \mathit{r}}$. We assume a powder diffraction setup, where X-ray pulses scatter off an ensemble of randomly oriented crystallites of dimensions between several hundreds of nanometers and a few micrometers (15). The charge densities $\widehat{\tilde{\sigma }}\left(q\right)=\int \mathrm{d}{\mathrm{\Omega }}_{\mathit{q}}/\left(4\pi \right)\hspace{0.17em}\widehat{\tilde{\sigma }}\left(\mathit{q}\right)$ accessed by the signal are thus rotationally averaged over the momentum space solid angle ${\mathrm{\Omega }}_{\mathit{q}}$, and only depend on the modulus $q=\left|\mathit{q}\right|$.

The charge density contributions from populations and coherences are exemplified in Fig. 1B for the two relevant electronic states in azobenzene photoisomerization. This is highlighted by expanding the wavefunction $\left|\mathrm{\Psi }\left(t\right)\right.⟩={\sum }_i{c}_i\left(t\right)\hspace{0.17em}\left|{\chi }_i\left(t\right)\right.⟩\hspace{0.17em}\left|{\varphi }_i\right.⟩$ in the basis $\left\{\left|{\varphi }_i\right.⟩\right\}$ of adiabatic electronic states, with time-dependent normalized nuclear wavepackets $\left|{\chi }_i\left(t\right)\right.⟩$ and state amplitudes $c_i\left(t\right)$. The resulting charge densities $⟨\widehat{\tilde{\sigma }}\left(q,t\right)⟩=⟨\mathrm{\Psi }\left(t\right)\left|\widehat{\tilde{\sigma }}\left(q\right)\right|\mathrm{\Psi }\left(t\right)⟩={\tilde{\sigma }}_{\mathrm{p}\mathrm{o}\mathrm{p}}\left(q,t\right)+{\tilde{\sigma }}_{\mathrm{c}\mathrm{o}\mathrm{h}}\left(q,t\right)$, with Fourier transform ${\widetilde{\stackrel{~}{\sigma }}}_{\mathrm{p}\mathrm{o}\mathrm{p}}\left(q,\omega \right)+{\widetilde{\stackrel{~}{\sigma }}}_{\mathrm{c}\mathrm{o}\mathrm{h}}\left(q,\omega \right)$, are then given by the sum of diagonal charge densities stemming from states’ populations (Fig. 1 B, i and ii),

$${\tilde{\sigma }}_{\mathrm{p}\mathrm{o}\mathrm{p}}\left(q,t\right)=\sum _i{\tilde{\sigma }}_{ii}\left(q,t\right)=\sum _i|c_i\left(t\right){|}^2\hspace{0.17em}⟨{\chi }_i\left(t\right)\left|{\widehat{\tilde{\sigma }}}_{ii}\left(q\right)\right|{\chi }_i\left(t\right)⟩,$$ [2]

and transition charge densities associated with the quantum coherences (Fig. 1 B, iii and iv),

$${\tilde{\sigma }}_{\mathrm{c}\mathrm{o}\mathrm{h}}\left(q,t\right)=\sum _{i,j\ne i}{\tilde{\sigma }}_{ji}\left(q,t\right)=\sum _{i,j\ne i}{c}_j^{*}\left(t\right)\hspace{0.17em}{c}_i\left(t\right)\hspace{0.17em}⟨{\chi }_j\left(t\right)\left|{\widehat{\tilde{\sigma }}}_{ji}\left(q\right)\right|{\chi }_i\left(t\right)⟩.$$ [3]

The modulus square in Eq. 1 results in the sum of three contributions, $S\left(q,{\omega }_{\mathrm{s}}^{\prime },T\right)=S_{\mathrm{p}\mathrm{o}\mathrm{p}}\left(q,{\omega }_{\mathrm{s}}^{\prime },T\right)+S_{\mathrm{c}\mathrm{o}\mathrm{h}}\left(q,{\omega }_{\mathrm{s}}^{\prime },T\right)+S_{\mathrm{h}\mathrm{e}\mathrm{t}}\left(q,{\omega }_{\mathrm{s}}^{\prime },T\right)$, which contain ${\tilde{\sigma }}_{\mathrm{p}\mathrm{o}\mathrm{p}}\left(q,t\right){\tilde{\sigma }}_{\mathrm{p}\mathrm{o}\mathrm{p}}\left(q,t^{\prime }\right)$, ${\tilde{\sigma }}_{\mathrm{c}\mathrm{o}\mathrm{h}}\left(q,t\right){\tilde{\sigma }}_{\mathrm{c}\mathrm{o}\mathrm{h}}\left(q,t^{\prime }\right)$, and their interference ${\tilde{\sigma }}_{\mathrm{p}\mathrm{o}\mathrm{p}}\left(q,t\right){\tilde{\sigma }}_{\mathrm{c}\mathrm{o}\mathrm{h}}\left(q,t^{\prime }\right)$, respectively. The interference contribution $S_{\mathrm{h}\mathrm{e}\mathrm{t}}\left(q,{\omega }_{\mathrm{s}}^{\prime },T\right)$ can be viewed as a heterodyne-detected coherence term, with the populations acting as a local oscillator. This is analogous to the setup employed in ref. 45, where the scattering off unexcited molecules served as local oscillator for the dynamical signal.

The advantage of directly accessing coherence dynamics without population background contributions is apparent in Fig. 2A, where we display the evolution of the charge densities in real and in momentum space. Ab initio nuclear wavepacket simulations (see Materials and Methods) of ultrafast azobenzene photoisomerization involving the CoIn passage (27), with the full inclusion of coupled nuclear and electronic degrees of freedom, were performed on the basis of accurate two-dimensional potential energy surfaces (46) containing the ground state $g$ and the electronically excited $n{\pi }^{*}$ state $e$. The system is prepared at t = 0 fs in the $e$ state by a vertical excitation, and the passage through the CoIn, taking place at approximately 100 fs, moves part of the populations to the $g$ state, thereby generating a coherence between $g$ and $e$. While our two-mode effective Hamiltonian captures the reactive pathway through the CoIn, additional effects like vibrational relaxation to other modes might come into play in a multimode treatment. It was shown recently that the coherence emerging at the CoIn can be appreciably large and long lived even in full-dimensional simulations of a much larger molecule with more decoherence pathways (47). We note that the diffraction signal in Eq. 1 assumes long-range order within each randomly oriented nanocrystal (22), such that the contribution from pairs of molecular sites dominates over the single-molecule one. From the molecular perspective, our azobenzene Hamiltonian is set up for the gas phase, and isomerization within a crystal will likely be different (48). From the signal perspective, the dynamics will be analogously captured by frequency-resolved UXD.

Fig. 2.

Photoisomerization dynamics of azobenzene. (A) (Top Left) Representative molecular structure of azobenzene at the reactive CoIn, with real-space transition charge density $\mathbf{R}\mathbf{e}\left\{{\sigma }_{\mathrm{c}\mathrm{o}\mathrm{h}}\left(\mathit{r},t\right)\right\}$ at t = 190 fs; and evolution of the rotationally averaged, momentum-space charge densities, with contributions from the populations (Top Right) $\left|{\tilde{\sigma }}_{ee}\left(q,t\right)\right|$ and (Bottom Left) $\left|{\tilde{\sigma }}_{gg}\left(q,t\right)\right|$ (Eq. 2), and (Bottom Right) the coherence $\left|{\tilde{\sigma }}_{\mathrm{c}\mathrm{o}\mathrm{h}}\left(q,t\right)\right|$ (Eq. 3), all in atomic units. The system is initially prepared in the $e$ state, with coherences and populations in the $g$ state emerging during the passage through the CoIn starting at approximately 100 fs. The spatial localization of the transition charge densities at the nitrogen atoms is reflected by the similar magnitude of ${\tilde{\sigma }}_{\mathrm{c}\mathrm{o}\mathrm{h}}\left(q,t\right)$ at different values of $q$. (B) Cuts along $q=3.3$ ${\AA }^{-1}$ of (Left) $\left|{\tilde{\sigma }}_{\mathrm{p}\mathrm{o}\mathrm{p}}\left(q,t\right)\right|$ and (Right) $\left|{\tilde{\sigma }}_{\mathrm{c}\mathrm{o}\mathrm{h}}\left(q,t\right)\right|$ in A (yellow, continuous), compared to the average temporal envelope of the stochastic pulse of duration $\tau =22\hspace{0.17em}\mathrm{f}\mathrm{s}$ and centered at T = 140 fs (blue, dashed). The pulse duration is short compared to the quasi-static populations, so that inelastic coherence scattering can be resolved in the CUPXD signal at large frequency shifts.

Fig. 2 A, Top Left shows the real-space transition charge density at t = 190 fs, with a clear localization at the nitrogen atoms stemming from the $n{\pi }^{*}$ character of the $e$ state. By highlighting the role of the valence electrons directly involved in the CoIn passage, transition charge densities carry more distinctive and localized information than state charge densities, which are spread through the entire molecule with strong contributions from inactive electrons. This is reflected in the rotationally averaged, momentum-space charge densities ${\tilde{\sigma }}_{\mathrm{p}\mathrm{o}\mathrm{p}}\left(q,t\right)$ and ${\tilde{\sigma }}_{\mathrm{c}\mathrm{o}\mathrm{h}}\left(q,t\right)$, also shown in Fig. 2A. The population contributions are stronger at small $q$, in contrast to the coherences which, albeit weaker overall, have similar magnitudes at varying momentum transfers and become more prominent at large $q$.

When an incident X-ray photon interacts with the nonstationary superposition state of Fig. 2A, it can exchange energy with the molecule and be inelastically scattered. The faster the molecular dynamics, the larger the X-ray inelastic frequency shift is. Although population terms are much stronger than coherences, their evolution is, in general, much slower (Fig. 2A), and they predominantly lead to elastic photon scattering. If the pulse provides the high spectral resolution needed to separate elastic and inelastic scattering, coherence dynamics can then be isolated. For coherent pulses, the spectral and temporal resolutions of the X-ray diffraction signal in Eq. 1 would be simultaneously determined by the pulse envelope $A_{\mathrm{X}}\left(t-T\right)$ and limited by Fourier uncertainty. To effectively detect inelastic coherence scattering, the pulse should cover an adequate range of coherence dynamics within its duration $\tau$, as illustrated in Fig. 2B. If the populations remain approximately constant within this time window, their contribution to inelastic scattering at large frequency shifts is minor, and the coherences are better resolved. However, existing FEL sources based on the self-amplified spontaneous emission (SASE) mechanism produce stochastic pulses with spiky intensity profiles. This generates ensembles of stochastic signals for each pulse realization which, upon averaging, do not possess the spectral resolution necessary to separate elastic and inelastic scattering.

To overcome this obstacle with existing X-ray FEL sources, we introduce the covariance ultrafast powder X-ray diffraction (CUPXD) signal

$$\begin{array}{cc}\hfill C\left(q,{\omega }_{\mathrm{s}1}^{\prime },{\omega }_{\mathrm{s}2}^{\prime },T\right)=& ⟨|{Ã}_{\mathrm{X}}\left({\omega }_{\mathrm{s}1}^{\prime }\right){|}^2\hspace{0.17em}S\left(q,{\omega }_{\mathrm{s}2}^{\prime },T\right)⟩\hfill \\ \hfill & -⟨|{Ã}_{\mathrm{X}}\left({\omega }_{\mathrm{s}1}^{\prime }\right){|}^2⟩⟨S\left(q,{\omega }_{\mathrm{s}2}^{\prime },T\right)⟩,\hfill \end{array}$$ [4]

obtained by averaging the product of each stochastic signal and the spectral intensity of the specific pulse generating it over independent pulse realizations. Covariance signals do not require additional measurements and can be obtained experimentally at the data processing stage. To simulate the performance of CUPXD, we model the X-ray pulse envelope $A_{\mathrm{X}}\left(t\right)=2\pi f\left(t\right)u\left(t\right)$ as the product of a broadband stochastic function $f\left(t\right)$ (see Materials and Methods) and a long (narrow-band) temporal gating envelope $u\left(t\right)={\mathrm{e}}^{-t^2/\left(2{\tau }^2\right)}/\sqrt{2\pi }$ of duration $\tau$. The model employed here was shown to reproduce the stochastic temporal and spectral spikes displayed by experimental SASE FEL pulses, with the average pulse duration $\tau$ and spectral bandwidth $\sigma$ independently set by $u\left(t\right)$ and $f\left(t\right)$, respectively (36, 49).

The average signal $⟨S\left(q,{\omega }_{\mathrm{s}}^{\prime },T\right)⟩$ calculated with the above model does not provide any frequency information (see Materials and Methods), and cannot spectrally separate elastic and inelastic contributions. In contrast, the CUPXD signal,

$$\begin{array}{cc}\hfill & C\left(q,{\omega }_{\mathrm{R}},T\right)\hfill \\ \hfill \propto & {\left|\int \mathrm{d}t\hspace{0.17em}{\mathrm{e}}^{-\frac{{\left(t-T\right)}^2}{{\tau }^2}}\hspace{0.17em}{\mathrm{e}}^{\mathrm{i}{\omega }_{\mathrm{R}}\left(t-T\right)}\hspace{0.17em}\left[{\tilde{\sigma }}_{\mathrm{p}\mathrm{o}\mathrm{p}}\left(q,t\right)+{\tilde{\sigma }}_{\mathrm{c}\mathrm{o}\mathrm{h}}\left(q,t\right)\right]\right|}^2\hfill \\ \hfill =& {\left|\int \frac{\mathrm{d}\omega }{2\pi }\hspace{0.17em}\sqrt{\pi }\tau \hspace{0.17em}{\mathrm{e}}^{-\frac{{\left({\omega }_{\mathrm{R}}-\omega \right)}^2{\tau }^2}{4}}\hspace{0.17em}{\mathrm{e}}^{-\mathrm{i}\omega T}\hspace{0.17em}\left[{\widetilde{\stackrel{~}{\sigma }}}_{\mathrm{p}\mathrm{o}\mathrm{p}}\left(q,\omega \right)+{\widetilde{\stackrel{~}{\sigma }}}_{\mathrm{c}\mathrm{o}\mathrm{h}}\left(q,\omega \right)\right]\right|}^2,\hfill \end{array}$$ [5]

recovers the same expressions as for a single coherent X-ray pulse, with $A_{\mathrm{X}}\left(t\right)$ now replaced by the average temporal envelope $|u\left(t\right){|}^2={\mathrm{e}}^{-t^2/{\tau }^2}/\left(2\pi \right)$, and the frequency difference ${\omega }_{\mathrm{s}}^{\prime }$ by the Raman frequency ${\omega }_{\mathrm{R}}={\omega }_{\mathrm{s}2}-{\omega }_{\mathrm{s}1}$. The average pulse duration thus acts as a control parameter providing the spectral resolution needed to separate elastic and inelastic contributions and, at the same time, the temporal resolution necessary to follow ultrafast molecular dynamics.

A representative section of the CUPXD signal in azobenzene is shown in Fig. 1A for an average pulse duration of $\tau =22\hspace{0.17em}\mathrm{f}\mathrm{s}$; others are presented in SI Appendix, Fig. S1. To highlight the information encoded in the diffraction patterns, we display, in Fig. 3, the time- and frequency-resolved CUPXD signal for a fixed, sufficiently large value of $q=3.3$ ${\AA }^{-1}$. In analogy with the three components already identified for $S\left(q,{\omega }_{\mathrm{s}}^{\prime },T\right)$ in Eq. 1, the covariance signal in Eq. 5 can be recast as the sum of three contributions, $C\left(q,{\omega }_{\mathrm{R}},T\right)=C_{\mathrm{p}\mathrm{o}\mathrm{p}}\left(q,{\omega }_{\mathrm{R}},T\right)+C_{\mathrm{c}\mathrm{o}\mathrm{h}}\left(q,{\omega }_{\mathrm{R}},T\right)+C_{\mathrm{h}\mathrm{e}\mathrm{t}}\left(q,{\omega }_{\mathrm{R}},T\right)$. The total signal $C\left(q,{\omega }_{\mathrm{R}},T\right)$ exhibits strong elastic scattering within a narrow frequency region around ${\omega }_{\mathrm{R}}=0\hspace{0.17em}\mathrm{e}\mathrm{V}$, mostly determined by contributions from quasi-stationary populations. These reflect the slow evolution of ${\tilde{\sigma }}_{\mathrm{p}\mathrm{o}\mathrm{p}}\left(q,t\right)$ displayed in Fig. 2B, which leads to narrow elastic spectral features in ${\widetilde{\stackrel{~}{\sigma }}}_{\mathrm{p}\mathrm{o}\mathrm{p}}\left(q,\omega \right)$. These are convolved with the broader Fourier transform ${\mathrm{e}}^{-{\omega }^2{\tau }^2/4}$ of the average pulse intensity, so that the spectral width of the population term $C_{\mathrm{p}\mathrm{o}\mathrm{p}}\left(q,{\omega }_{\mathrm{R}},T\right)$ is set by $1/\tau$. The faster coherence dynamics ${\tilde{\sigma }}_{\mathrm{c}\mathrm{o}\mathrm{h}}\left(q,t\right)$, also shown in Fig. 2B, are encoded in the inelastic scattering patterns at larger Raman frequencies ${\omega }_{\mathrm{R}}$. These features are broader than the pulse correlation bandwidth $1/\tau$, and can thus be distinguished in the coherence contribution $C_{\mathrm{c}\mathrm{o}\mathrm{h}}\left(q,{\omega }_{\mathrm{R}},T\right)$. The heterodyne term $C_{\mathrm{h}\mathrm{e}\mathrm{t}}\left(q,{\omega }_{\mathrm{R}},T\right)$ carries signatures of the coherences, magnified by interference with the larger population contribution. However, this amplification is limited to elastic scattering within a narrow region around ${\omega }_{\mathrm{R}}=0\hspace{0.17em}\mathrm{e}\mathrm{V}$, due to the narrow spectral width of ${\widetilde{\stackrel{~}{\sigma }}}_{\mathrm{p}\mathrm{o}\mathrm{p}}\left(q,\omega \right)$. We note that this heterodyne term $C_{\mathrm{h}\mathrm{e}\mathrm{t}}\left(q,{\omega }_{\mathrm{R}},T\right)$ is equal to the real part of the product between $\int \mathrm{d}t\hspace{0.17em}{\mathrm{e}}^{-{\left(t-T\right)}^2/{\tau }^2}\hspace{0.17em}{\mathrm{e}}^{\mathrm{i}{\omega }_{\mathrm{R}}\left(t-T\right)}\hspace{0.17em}{\tilde{\sigma }}_{\mathrm{p}\mathrm{o}\mathrm{p}}\left(q,t\right)$ and ${\left[\int \mathrm{d}t\hspace{0.17em}{\mathrm{e}}^{-{\left(t-T\right)}^2/{\tau }^2}\hspace{0.17em}{\mathrm{e}}^{\mathrm{i}{\omega }_{\mathrm{R}}\left(t-T\right)}\hspace{0.17em}{\tilde{\sigma }}_{\mathrm{c}\mathrm{o}\mathrm{h}}\left(q,t\right)\right]}^{*}$. For fixed $q$ and $T$, the ${\omega }_{\mathrm{R}}$ profile of $C_{\mathrm{h}\mathrm{e}\mathrm{t}}\left(q,{\omega }_{\mathrm{R}},T\right)$ and its positive or negative sign thus encode the phase of the molecular superposition state encountered by the X-ray pulse at a given time delay.

Fig. 3.

Temporally and spectrally resolved CUPXD signal in azobenzene. The signal $C\left(q,{\omega }_{\mathrm{R}},T\right)$ (Eq. 5) and its three components, $C_{\mathrm{p}\mathrm{o}\mathrm{p}}\left(q,{\omega }_{\mathrm{R}},T\right)$, $C_{\mathrm{h}\mathrm{e}\mathrm{t}}\left(q,{\omega }_{\mathrm{R}},T\right)$, and $C_{\mathrm{c}\mathrm{o}\mathrm{h}}\left(q,{\omega }_{\mathrm{R}},T\right)$, are displayed in arbitrary units for $q=3.3$ ${\AA }^{-1}$ and $\tau =22\hspace{0.17em}\mathrm{f}\mathrm{s}$ as a function of time delay $T$ and Raman frequency ${\omega }_{\mathrm{R}}$. Elastic scattering off quasi-stationary populations dominates the narrow spectral region around ${\omega }_{\mathrm{R}}=0\hspace{0.17em}\mathrm{e}\mathrm{V}$. At higher Raman frequencies, due to the drop in $C_{\mathrm{p}\mathrm{o}\mathrm{p}}\left(q,{\omega }_{\mathrm{R}},T\right)$, the inelastic signal allows access to the coherence dynamics with significantly reduced background.

At large Raman frequencies, elastic scattering from slowly varying populations is significantly inhibited, and the signal carries distinguishable, temporally resolved coherence signatures from $C_{\mathrm{h}\mathrm{e}\mathrm{t}}\left(q,{\omega }_{\mathrm{R}},T\right)$ and $C_{\mathrm{c}\mathrm{o}\mathrm{h}}\left(q,{\omega }_{\mathrm{R}},T\right)$. This is highlighted by the sections of the signal displayed in Fig. 4A. Up to ${\omega }_{\mathrm{R}}=0.1\hspace{0.17em}\mathrm{e}\mathrm{V}$, the signal is dominated by the strong elastic scattering off populations. At the intermediate Raman frequency ${\omega }_{\mathrm{R}}=0.15\hspace{0.17em}\mathrm{e}\mathrm{V}$, the role of the coherences starts to emerge, magnified in $C_{\mathrm{h}\mathrm{e}\mathrm{t}}\left(q,{\omega }_{\mathrm{R}},T\right)$ by the populations acting as local oscillator. At higher Raman frequency, ${\omega }_{\mathrm{R}}=0.2\hspace{0.17em}\mathrm{e}\mathrm{V}$, where scattering off populations is highly suppressed, $C_{\mathrm{h}\mathrm{e}\mathrm{t}}\left(q,{\omega }_{\mathrm{R}},T\right)$ and $C_{\mathrm{c}\mathrm{o}\mathrm{h}}\left(q,{\omega }_{\mathrm{R}},T\right)$ become comparable, and additional information can be extracted from the signal. For these high Raman frequencies, the inelastic coherence terms are strongest around 170 and 240 fs, providing direct access to the timing of the CoIn passage, and reflecting the faster coherence dynamics—hence stronger inelastic scattering—taking place during these time intervals (see, e.g., Fig. 2B). In Fig. 4B, analogous information is obtained by integrating $C\left(q,{\omega }_{\mathrm{R}},T\right)$ over frequency after having filtered out a central spectral region of width $2\mathrm{\Delta }{\omega }_{\mathrm{R}}$. In an experiment, a proper choice of $\mathrm{\Delta }{\omega }_{\mathrm{R}}$ will thus ensure that population contributions are removed, and the coherence dynamics can be clearly viewed.

Fig. 4.

Coherence contributions to the CUPXD signals in azobenzene. (A) Sections of the CUPXD signal (Eq. 5) of azobenzene molecules at $q=3.3$ ${\AA }^{-1}$ and for selected values of the Raman frequency ${\omega }_{\mathrm{R}}$. The total signal $C\left(q,{\omega }_{\mathrm{R}},T\right)$ (blue, continuous) and the three contributions, $C_{\mathrm{p}\mathrm{o}\mathrm{p}}\left(q,{\omega }_{\mathrm{R}},T\right)$ (yellow, dashed), $C_{\mathrm{h}\mathrm{e}\mathrm{t}}\left(q,{\omega }_{\mathrm{R}},T\right)$ (green, dotted), and $C_{\mathrm{c}\mathrm{o}\mathrm{h}}\left(q,{\omega }_{\mathrm{R}},T\right)$ (red, dot-dashed), are compared. (B) Filtered CUPXD signal at $q=3.3$ ${\AA }^{-1}$, obtained by integrating the CUPXD signal after having filtered out the spectral region $\left[-\mathrm{\Delta }{\omega }_{\mathrm{R}},\hspace{0.17em}\mathrm{\Delta }{\omega }_{\mathrm{R}}\right]$ of indicated width $\mathrm{\Delta }{\omega }_{\mathrm{R}}$. Filtered signals from $C\left(q,{\omega }_{\mathrm{R}},T\right)$ and their three contributions are displayed, with the same line styles used in A. For large Raman frequencies ${\omega }_{\mathrm{R}}$, lying outside the spectral width of elastic population terms, the population contribution $C_{\mathrm{p}\mathrm{o}\mathrm{p}}\left(q,{\omega }_{\mathrm{R}},T\right)$ is suppressed, and the coherence contributions become clearly visible.

By suppressing the elastic (population) background via frequency dispersion, the CUPXD signal offers a direct access to the evolution of the transition charge densities, with distinctive spatial information on the localized molecular events determining the CoIn passage. The elastic (${\omega }_{\mathrm{R}}=0\hspace{0.17em}\mathrm{e}\mathrm{V}$) CUPXD signal is shown in Fig. 5A. At each momentum transfer $q$, the signal is dominated by the elastic population scattering, whereas coherence contributions, both population heterodyne and homodyne, are significantly weaker. The signal is strongest around small momentum transfers, which reflects the underlying behavior of the populations in $q$ space (see, e.g., ${\tilde{\sigma }}_{ee}\left(q,t\right)$ and ${\tilde{\sigma }}_{gg}\left(q,t\right)$ in Fig. 2A) and the spread of the state densities throughout the entire molecule. In contrast, the inelastic CUPXD signal, displayed in Fig. 5B at ${\omega }_{\mathrm{R}}=0.2\hspace{0.17em}\mathrm{e}\mathrm{V}$, carries clear signatures of the temporal evolution of the transition charge densities. For time delays between 150 and 250 fs, additional features appear at large $q$, reflecting the homogeneous distribution of ${\tilde{\sigma }}_{\mathrm{c}\mathrm{o}\mathrm{h}}\left(q,t\right)$ in momentum space. As already seen in Fig. 4, these attributes are most intense at intervals of faster coherence dynamics, since they lead to stronger inelastic scattering. This appears by comparing the signal in Fig. 5B and the evolution of ${\tilde{\sigma }}_{\mathrm{c}\mathrm{o}\mathrm{h}}\left(q,t\right)$ in Fig. 2A.

Fig. 5.

CUPXD signals revealing reciprocal-space and real-space information on coherence dynamics in azobenzene. (A and B) The (A) elastic (${\omega }_{\mathrm{R}}=0\hspace{0.17em}\mathrm{e}\mathrm{V}$) and (B) inelastic (at ${\omega }_{\mathrm{R}}=0.2\hspace{0.17em}\mathrm{e}\mathrm{V}$) CUPXD signals $C\left(q,{\omega }_{\mathrm{R}},T\right)$ (Eq. 5), along with their three components $C_{\mathrm{p}\mathrm{o}\mathrm{p}}\left(q,{\omega }_{\mathrm{R}},T\right)$, $C_{\mathrm{h}\mathrm{e}\mathrm{t}}\left(q,{\omega }_{\mathrm{R}},T\right)$, and $C_{\mathrm{c}\mathrm{o}\mathrm{h}}\left(q,{\omega }_{\mathrm{R}},T\right)$, are displayed in arbitrary units as a function of time delay $T$ and momentum transfer amplitude $q$. The elastic signal is dominated by population contributions at small momentum transfers. In the inelastic signal at ${\omega }_{\mathrm{R}}=0.2\hspace{0.17em}\mathrm{e}\mathrm{V}$, coherence signatures at large momentum transfers appear between 150 and 250 fs, as indicated by the white arrow. (C and D) Spatial information encoded in $\left|G\left(r,{\omega }_{\mathrm{R}},T\right)\right|$ and $\left|G_{\mathrm{p}\mathrm{o}\mathrm{p}}\left(r,{\omega }_{\mathrm{R}},T\right)\right|$ (Eq. 6) for (C) elastic and (D) inelastic scattering. $G\left(r,{\omega }_{\mathrm{R}},T\right)$ contains contributions from the total charge density $⟨\widehat{\tilde{\sigma }}\left(q,t\right)⟩={\tilde{\sigma }}_{\mathrm{p}\mathrm{o}\mathrm{p}}\left(q,t\right)+{\tilde{\sigma }}_{\mathrm{c}\mathrm{o}\mathrm{h}}\left(q,t\right)$ (Eqs. 2 and 3), while $G_{\mathrm{p}\mathrm{o}\mathrm{p}}\left(r,{\omega }_{\mathrm{R}},T\right)$ is calculated from population-only terms. Elastic scattering is dominated by the populations, whereas coherence features appear at small values of $r$ for inelastic scattering, as highlighted by the white arrows. This reflects the underlying coherence dynamics within confined molecular regions at the nitrogen atoms (see also Fig. 2A).

After inverting the CUPXD signal from momentum to real space, the inelastic contributions offer clear insight into the spatial distribution of the transition charge densities, highlighting the valence electrons directly involved in the CoIn dynamics. Fig. 5 C and D depicts the modulus of the spatial Fourier transform

$$G\left(r,{\omega }_{\mathrm{R}},T\right)=\int \frac{\mathrm{d}q}{2\pi }\hspace{0.17em}{\mathrm{e}}^{\mathrm{i}qr}\left|\int \mathrm{d}t\hspace{0.17em}{\mathrm{e}}^{-\frac{{\left(t-T\right)}^2}{{\tau }^2}}\hspace{0.17em}{\mathrm{e}}^{\mathrm{i}{\omega }_{\mathrm{R}}\left(t-T\right)}\hspace{0.17em}⟨\widehat{\tilde{\sigma }}\left(q,t\right)⟩\right|,$$ [6]

obtained from the square root of $C\left(q,{\omega }_{\mathrm{R}},T\right)$. For powder diffraction, the variable $r$ conjugated to $q$ spans the distribution of real-space distances within the molecule. For comparison, we also display $G_{\mathrm{p}\mathrm{o}\mathrm{p}}\left(r,{\omega }_{\mathrm{R}},T\right)$, calculated by including only scattering population amplitudes in Eq. 6, with the substitution $⟨\widehat{\tilde{\sigma }}\left(q,t\right)⟩\to {\tilde{\sigma }}_{\mathrm{p}\mathrm{o}\mathrm{p}}\left(q,t\right)$. Fig. 5C depicts $G\left(r,{\omega }_{\mathrm{R}},T\right)$ for the elastic scattering case of ${\omega }_{\mathrm{R}}=0\hspace{0.17em}\mathrm{e}\mathrm{V}$. In this case, the role and localization of the transition charge densities do not appear, as they are hidden by the much stronger state densities, stemming from all electrons in the molecule. In contrast, by frequency dispersing $G\left(r,{\omega }_{\mathrm{R}},T\right)$ and accessing inelastic scattering, the spatial spread of the coherences and its variation with time are apparent. For ${\omega }_{\mathrm{R}}=0.2\hspace{0.17em}\mathrm{e}\mathrm{V}$, Fig. 5D exhibits new features at small molecular distances $r$, which reflect the localization of the transition charge densities at the nitrogen atoms due to the $n{\pi }^{*}$ character of the excitation (Fig. 2A). This allows direct access to the primary events involved in the CoIn passage, revealing the evolution and localized nature of the molecular coherences.

To summarize, we have shown how covariance UXD of existing, stochastic FEL pulses can provide direct access to electron densities around CoIns. This is achieved by separating the inelastic and elastic scattering, and thus revealing the distinct coherence signature from the dominating population background. We model the correlation properties of existing stochastic X-ray FEL pulses generated by the SASE mechanism, and employ a covariance-based analysis to recover the joint spectral and temporal resolutions needed for CoIn detection and hidden by the stochasticity of the pulses. The coherence contribution appears more spread in $q$ space than the populations, reflecting the localized nature of the transition charge densities at the nitrogen atoms. The CUPXD signal shows that coherence contributions originate at small distances within the molecule, but does not identify where the coherences are located. In crystals, the covariance diffraction signal will feature Bragg peaks and can yield full three-dimensional information on the localization and dynamics of the transition charge densities.

While the stochastic model used here aimed at reproducing the spiky temporal and spectral profiles of SASE FEL pulses, other models could be utilized to simulate, for example, shaped X-ray pulses available at FEL sources (50). By engineering suitable stochastic X-ray pulses, it may be possible to highlight desired molecular dynamics features. A high temporal and spectral resolution could be alternatively obtained by postprocessing time-resolved diffraction snapshots. In ref. 20, the Fourier transform of the time-dependent X-ray diffraction signal was employed to separate contributions from different molecular modes. For nonadiabatic dynamics, Wigner or frequency-resolved optical gating–like spectrograms of the time-resolved diffraction signal could be used instead (12), as shown in SI Appendix, Fig. S2, requiring, however, attosecond FEL pulses (51, 52). This differs from the covariance UXD signals presented here, where the frequency information is directly obtained in the experiment through frequency dispersion. Our protocol for the separation of elastic and inelastic scattering can be straightforwardly extended to ultrafast electron diffraction (9, 10). Similar to UXD, the electron diffraction signal contains most valuable contributions from mixed elastic and inelastic scattering (37). In contrast to UXD, however, electrons are diffracted from the total (electron and nuclear) charge density in the molecule. Dispersing the electron diffraction signal as a function of energy, and thereby isolating the inelastic scattering contribution, will provide additional imaging information on CoIns.

Materials and Methods

Momentum-, Frequency-, and Time-Resolved X-ray Diffraction Signal.

The diffraction signal in Eq. 1 involves the off-resonant scattering of an X-ray pulse off a molecular sample. This is characterized by the minimal-coupling light–matter interaction Hamiltonian in the rotating-wave approximation

$$\begin{array}{cc}\hfill {Ĥ}_{\mathrm{i}\mathrm{n}\mathrm{t}}& =\frac{1}{2}\int \hspace{0.17em}{\mathrm{d}}^{3}r\hspace{0.17em}\widehat{\sigma }\left(\mathit{r}\right)\hspace{0.17em}|\widehat{\mathit{𝒜}}\left(\mathit{r}\right)+{\widehat{\mathit{𝒜}}}^{†}\left(\mathit{r}\right){|}^2\hfill \\ \hfill & =\int {\mathrm{d}}^{3}r\hspace{0.17em}\widehat{\sigma }\left(\mathit{r}\right)\hspace{0.17em}{\widehat{\mathit{𝒜}}}^{†}\left(\mathit{r}\right)\cdot \widehat{\mathit{𝒜}}\left(\mathit{r}\right),\hfill \end{array}$$ [7]

where $\widehat{\sigma }\left(\mathit{r}\right)$ is the charge density operator, and

$$\widehat{\mathit{𝒜}}\left(\mathit{r}\right)=\sum _{\mathit{k},\lambda }{\widehat{\mathit{ϵ}}}_{\mathit{k}}^{\lambda }\hspace{0.17em}\sqrt{\frac{2\pi }{V_{\mathrm{q}}{\omega }_{\mathit{k}}}}\hspace{0.17em}{\mathrm{e}}^{\mathrm{i}\mathit{k}\cdot \mathit{r}}\hspace{0.17em}{â}_{\mathit{k},\lambda }$$ [8]

is the vector potential operator. $V_{\mathrm{q}}$ is the quantization volume, and the sum runs over the modes $\mathit{k}$ and polarization indices $\lambda$, with associated annihilation operators ${â}_{\mathit{k},\lambda }$, polarization vectors ${\widehat{\mathit{ϵ}}}_{\mathit{k}}^{\lambda }$, and mode frequencies ${\omega }_{\mathit{k}}$. The wavevector- and frequency-resolved diffraction signal $S\left({\mathit{k}}_{\mathrm{s}},{\omega }_{\mathrm{s}}\right)$ is defined as the integrated rate of change of the number of photons of wavevector ${\mathit{k}}_s$, frequency ${\omega }_{\mathrm{s}}$, and polarization ${\widehat{\mathit{ϵ}}}_{\mathrm{s}}$, within a differential solid angle $\mathrm{d}{\mathrm{\Omega }}_{\mathrm{s}}$ and differential frequency interval $\mathrm{d}{\omega }_{\mathrm{s}}$,

$$S\left({\mathit{k}}_{\mathrm{s}},{\omega }_{\mathrm{s}}\right)\mathrm{d}{\mathrm{\Omega }}_{\mathrm{s}}\mathrm{d}{\omega }_{\mathrm{s}}=\int \mathrm{d}t\hspace{0.17em}⟨\frac{\mathrm{d}{\widehat{N}}_{\mathrm{s}}}{\mathrm{d}t}⟩.$$ [9]

Here, ${\widehat{N}}_{\mathrm{s}}$ is the number operator of photons in the signal mode ${\mathit{k}}_{\mathrm{s}}$. The derivative can be calculated by Heisenberg equations of motion and using the commutator between ${\widehat{N}}_{\mathrm{s}}$ and the light–matter interaction Hamiltonian of Eq. 7. The expectation values are performed on both molecular and field degrees of freedom. The latter are calculated assuming coherent states, and the vector-potential operator $\widehat{\mathit{𝒜}}\left(\mathit{r}\right)$ is replaced by the classical field ${\mathit{𝒜}}_{\mathrm{X}}\left(\mathit{r},t\right)$. In general, the diffraction signal contains contributions originating 1) at a single molecular site ${\mathit{R}}_n$, associated with the two-point matter correlation function $⟨\widehat{\sigma }\left(\mathit{r}-{\mathit{R}}_n,t\right)\widehat{\sigma }\left({\mathit{r}}^{\prime }-{\mathit{R}}_n,t^{\prime }\right)⟩$, or 2) from the cooperative interference of X-ray photons scattered at pairs of molecular sites, given by products of expectation values $⟨\widehat{\sigma }\left(\mathit{r}-{\mathit{R}}_n,t\right)⟩⟨\widehat{\sigma }\left({\mathit{r}}^{\prime }-{\mathit{R}}_{n^{\prime }},t^{\prime }\right)⟩$ independently carried out at different molecular centers (19). For a sample of oriented molecules with long-range order, such as in a crystal, the cooperative two-molecule contribution is dominant, leading to Eq. 1. The proportionality sign therein accounts for a multiplication factor encoding the number of molecules in the samples and their long-range intermolecular structure (19).

Covariance UXD Signals Generated by a Stochastic Pulse.

Numerical techniques have long been applied to simulate chaotic fields available experimentally (49, 53–55). The envelope $A_{\mathrm{X}}\left(t\right)=2\pi f\left(t\right)u\left(t\right)$ of the classical stochastic field ${\mathit{𝒜}}_{\mathrm{X}}\left(\mathit{r},t\right)$ is modeled by the product of a stochastic broadband function $f\left(t\right)$ and a temporal gating function $u\left(t\right)$ of duration $\tau$. The function $f\left(t\right)=\int \mathrm{d}\omega /\left(2\pi \right)\tilde{g}\left(\omega \right){\mathrm{e}}^{\mathrm{i}\phi \left(\omega \right)}$ is defined in terms of the broadband real spectral envelope $\tilde{g}\left(\omega \right)$, modeling the large average pulse bandwidth $\sigma$, and a stochastic phase $\phi \left(\omega \right)$, with correlation length $\mathrm{\Lambda }$. The latter is obtained by interpolating a set of independent random variables ${\phi }_k$ at the discrete frequencies ${\omega }_k=k\mathrm{\Lambda }$. Each ${\phi }_k$ is uniformly distributed in the interval $\left[-\pi ,\pi \right]$ with probability density function

$$P\left({\phi }_k\right)=\left\{\begin{array}{cccc}\hfill & \frac{1}{2\pi },\hfill & \hfill & \text{if}-\pi \le {\phi }_k\le \pi ,\hfill \\ \hfill & 0,\hfill & \hfill & \text{otherwise}.\hfill \end{array}\right.$$ [10]

Satisfying the condition $\mathrm{\Lambda }\ll 1/\tau \ll \sigma$ allows one to model the stochastic temporal and spectral spikes displayed by experimental SASE FEL pulses, as shown in ref. 49. The more general case of stochastic phases uniformly distributed in a generic interval $\left[-a,a\right]$, with $a\le \pi$, was recently investigated in ref. 36.

The ensemble-averaged signals are calculated in terms of two- and four-point field correlation functions,

$$F_2\left({\omega }_1,{\omega }_2\right)\doteq ⟨{Ã}_{\mathrm{X}}^{*}\left({\omega }_1\right){Ã}_{\mathrm{X}}\left({\omega }_2\right)⟩,$$ [11]

$$F_4\left({\omega }_1,{\omega }_2,{\omega }_3,{\omega }_4\right)\doteq ⟨{Ã}_{\mathrm{X}}^{*}\left({\omega }_1\right){Ã}_{\mathrm{X}}\left({\omega }_2\right){Ã}_{\mathrm{X}}^{*}\left({\omega }_3\right){Ã}_{\mathrm{X}}\left({\omega }_4\right)⟩.$$ [12]

To leading (second) order in $\left(\mathrm{\Lambda }\tau \right)$, for the stochastic pulse model used here, these are given by (36)

$$F_2\left({\omega }_1,{\omega }_2\right)\approx \tilde{g}*\left({\omega }_1\right)\hspace{0.17em}\tilde{g}\left({\omega }_2\right)\hspace{0.17em}\mathrm{\Lambda }\sqrt{\pi }\tau \hspace{0.17em}{\mathrm{e}}^{-\frac{{\left({\omega }_1-{\omega }_2\right)}^2{\tau }^2}{4}},$$ [13]

$$\begin{array}{cc}\hfill F_4\left({\omega }_1,{\omega }_2,{\omega }_3,{\omega }_4\right)\approx & \tilde{g}*\left({\omega }_1\right)\hspace{0.17em}\tilde{g}\left({\omega }_2\right)\hspace{0.17em}\tilde{g}*\left({\omega }_3\right)\hspace{0.17em}\tilde{g}\left({\omega }_4\right)\hspace{0.17em}{\mathrm{\Lambda }}^2\pi {\tau }^2\hfill \\ \hfill & \times \left({\mathrm{e}}^{-\frac{{\left({\omega }_1-{\omega }_2\right)}^2{\tau }^2}{4}}\hspace{0.17em}{\mathrm{e}}^{-\frac{{\left({\omega }_3-{\omega }_4\right)}^2{\tau }^2}{4}}\right.\hfill \\ \hfill & +\left.{\mathrm{e}}^{-\frac{{\left({\omega }_1-{\omega }_4\right)}^2{\tau }^2}{4}}\hspace{0.17em}{\mathrm{e}}^{-\frac{{\left({\omega }_2-{\omega }_3\right)}^2{\tau }^2}{4}}\right)+\cdots .\hfill \end{array}$$ [14]

The average pulse spectral and temporal intensities are independently set by $\tilde{g}\left(\omega \right)$ and $u\left(t\right)$,

$$⟨|{Ã}_{\mathrm{X}}\left(\omega \right){|}^2⟩=F_2\left(\omega ,\omega \right)=\mathrm{\Lambda }\sqrt{\pi }\tau |\tilde{g}\left(\omega \right){|}^2,$$ [15]

$$\begin{array}{cc}\hfill ⟨|A_{\mathrm{X}}\left(t\right){|}^2⟩=& \int \frac{\mathrm{d}{\omega }_1}{2\pi }\int \frac{\mathrm{d}{\omega }_2}{2\pi }\hspace{0.17em}{F}_2\left({\omega }_1,{\omega }_2\right)\hspace{0.17em}{\mathrm{e}}^{-\mathrm{i}\left({\omega }_2-{\omega }_1\right)t}\hfill \\ \hfill =& \mathrm{\Lambda }\sqrt{\pi }\sigma |u\left(t\right){|}^2.\hfill \end{array}$$ [16]

Without loss of generality, we focus on the limit of extremely broadband pulses, $\tilde{g}\left(\omega \right)\to 1$.

Since the diffraction signal is quadratic in the amplitude of the field, the signal averaged over independent pulse realizations can be expressed in terms of $F_2\left({\omega }_1,{\omega }_2\right)$ and reads

$$\begin{array}{cc}\hfill ⟨S\left(\mathit{q},{\omega }_{\mathrm{s}}^{\prime },T\right)⟩=& |\widehat{\mathit{ϵ}}{*}_{\mathrm{s}}\cdot {\widehat{\mathit{ϵ}}}_{\mathrm{X}}{|}^2\hspace{0.17em}\frac{{\alpha }^3{\omega }_{\mathrm{s}}}{4{\pi }^2}\hfill \\ \hfill & \times \int \frac{\mathrm{d}\omega }{2\pi }\int \frac{\mathrm{d}{\omega }^{\prime }}{2\pi }\hspace{0.17em}\mathrm{\Lambda }\sqrt{\pi }\tau \hspace{0.17em}{\mathrm{e}}^{-\frac{{\left(\omega -{\omega }^{\prime }\right)}^2{\tau }^2}{4}}\hfill \\ \hfill & \times {\mathrm{e}}^{-\mathrm{i}\left(\omega -{\omega }^{\prime }\right)T}\hspace{0.17em}⟨\widehat{\widetilde{\stackrel{~}{\sigma }}}\left(-\mathit{q},-{\omega }^{\prime }\right)⟩⟨\widehat{\widetilde{\stackrel{~}{\sigma }}}\left(\mathit{q},\omega \right)⟩.\hfill \end{array}$$ [17]

It offers temporal resolution determined by the pulse duration $\tau$, but it does not provide any frequency-resolved information. The CUPXD signal in Eq. 5 is obtained from Eq. 4 and the last line in Eq. 1 via the two- and four-point field correlation functions in Eqs. 13 and 14.

Wave Packet Simulations.

To model azobenzene photoisomerization, we perform exact nuclear wavepacket simulations according to the time-dependent Schrödinger equation in the gas phase

$$i\hslash \frac{\partial }{\partial t}\psi =Ĥ\psi =\left[{\widehat{T}}_q+\widehat{V}\right]\psi .$$ [18]

We use two reactive nuclear degrees of freedom that describe the isomerization pathway. The first coordinate is the carbon–nitrogen–nitrogen–carbon (CNNC) torsion angle connecting the cis and trans minima at 0° and 180°, respectively. The second coordinate is one of the two CNN bending angles, while the other one remains fixed. Symmetry breaking of these two angles is necessary to reach the minimum energy CoIn along the isomerization pathway. Detailed high-level potential energy surfaces in this nuclear space were reported in refs. 27 and 46. After impulsive excitation of the vibrational S₀ ground-($g$-)state wavefunction to the excited ($e$) state S₁, the Chebychev propagation scheme (56) was used for numerical propagation with a time step of 0.05 fs on a discretized spatial grid with 600 points in CNNC and 256 in CNN. Impulsive excitation of 100% of the population is an ideal case that might not be feasible experimentally, leading to background of elastic g-state scattering in the diffraction patterns. This is not an issue here, since frequency dispersion of the signal generated with the stochastic FEL pulse separates the coherence contribution from the background. The G-matrix formalism (57) was used to set up the kinetic energy operator ${\widehat{T}}_q$ in $M=2$ reactive coordinates $r$ and $s$ as described in refs. 58 and 59,

$${\widehat{T}}_q\simeq -\frac{{\hslash }^2}{2m}\sum _{r=1}^M\sum _{s=1}^M\frac{\partial }{\partial q_r}\left[G_{rs}\frac{\partial }{\partial q_s}\right],$$ [19]

with the G matrix computed via its inverse elements

$${\left(G^{-1}\right)}_{rs}=\sum _{i=1}^{3N}{m}_i\frac{\partial x_i}{\partial q_r}\frac{\partial x_i}{\partial q_s}.$$ [20]

To absorb the parts of the wavepacket exiting the grid along CNN, and thus avoid artifacts, a Butterworth filter (60) was used. In S₀, the filter was also employed at 0° and 360° of torsion, to absorb the parts of the wavepacket that have reached the product minimum.

Data Availability

All study data are included in the article and SI Appendix.