Anisotropic spin fluctuations in detwinned FeSe

Abstract

Superconductivity in FeSe emerges from a nematic phase that breaks four-fold rotational symmetry in the iron plane. This phase may arise from orbital ordering, spin fluctuations or hidden magnetic quadrupolar order. Here we use inelastic neutron scattering on a mosaic of single crystals of FeSe, detwinned by mounting on a BaFe₂As₂ substrate to demonstrate that spin excitations are most intense at the antiferromagnetic wave vectors Q_AF = (±1, 0) at low energies E = 6–11 meV in the normal state. This two-fold (C₂) anisotropy is reduced at lower energies, 3–5 meV, indicating a gapped four-fold (C₄) mode. In the superconducting state, however, the strong nematic anisotropy is again reflected in the spin resonance (E = 3.6 meV) at Q_AF with incommensurate scattering around 5–6 meV. Our results highlight the extreme electronic anisotropy of the nematic phase of FeSe and are consistent with a highly anisotropic superconducting gap driven by spin fluctuations.

High-transition-temperature superconductivity in copper- and iron-based materials emerges from their antiferromagnetic (AF) ordered non-superconducting parent compounds¹. While the parents of copper oxide superconductors are Mott insulators with a simple chequerboard AF structure¹, most iron pnictide parent materials exhibit a tetragonal-to-orthorhombic structural transition at T_s (<295 K) and form twin domains before ordering antiferromagnetically at T_N (T_s ≥ T_N) (ref. ²). Therefore, we must detwin iron pnictides to measure their intrinsic electronic properties below T_s. By applying a uniaxial pressure along one axis of the orthorhombic lattice to detwin the sample, an in-plane resistivity anisotropy has been observed in strained iron pnictides BaFe_2−xT_xAs₂ (where T is Co or Ni) above T_s (refs. ^3,4). The resistivity anisotropy has been ascribed to an electronic nematic phase that spontaneously breaks the rotational symmetry while preserving the translation symmetry of the underlying lattice, and is established in the temperature regime below T_s and above T_N (refs. ^5,6). Below T_N, the AF structure is collinear, consisting of columns of antiparallel spins along the orthorhombic a_o axis and parallel spins along the b_o axis with an in-plane AF ordering wave vector Q_AF = (±1, 0) in reciprocal space².

The highly unusual iron-based superconductor FeSe exhibits an orthorhombic structural distortion and superconductivity without static AF order (Fig. 1a)^7–9. Although the nematic phase in FeSe is established below T_s (≈90 K)⁸, it has been argued that its nematic order and superconductivity are induced by orbital fluctuations (Fig. 1b)^10–14, forming a sign-preserving s⁺⁺-wave electron pairing, and therefore it would be fundamentally different from other iron-based superconductors¹⁵. Alternatively, the absence of static AF order in FeSe has been interpreted as evidence for a quantum paramagnet arising from the d-orbital spin-1 localized iron moments^16,17. Here, the nematic phase is driven by magnetic frustration due to competition between low-energy spin fluctuations associated with AF collinear order and those associated with various types of staggered order¹⁸. Third, the nematic superconductivity in FeSe without AF order may arise from a frustration-induced nematic quantum spin liquid state with melted AF order¹⁹. This model predicts a dramatic suppression of the magnetic spectral weight at Q = (0, ±1) in a detwinned sample, and explains the observed superconducting gap anisotropy by angle-resolved photoemission spectroscopy^20–22 and scanning tunnelling microscopy (STM)^23–25 experiments by an orbital-dependent Hund coupling¹⁹. Fourth, the nematic order may arise from a hidden magnetic quadrupolar order^26,27. Finally, the nematic phase and superconductivity in FeSe have also been described in terms of itinerant electrons interacting among quasinested hole–electron Fermi surfaces^28,29, as in other iron-based superconductors³⁰. In this picture, the electronic correlation effect is taken into account using orbital-dependent quasiparticle weights^24,31. Without electron correlation effects, spin fluctuations in the nematic phase below T_s exhibit only a minor C₄ asymmetry. Including correlations in the theoretical calculations renders the spin fluctuations highly C₂ symmetric with negligible weight at (0, ±1), and a neutron spin resonance exhibited only at Q_AF = (±1, 0) driven by the d_yz orbitals³¹. Approaches based on localized models with magnetic quadrupolar order have also predicted a strong suppression of low-energy (0, ±1) intensity²⁷.

Fig. 1 |. Crystal structure, Fermi surface and neutron scattering of FeSe.

a, Crystal structure of FeSe, where blue and orange colours mark Fe and Se positions, respectively. The red arrows indicate the uniaxial strain direction applied through detwinned BaFe₂As₂. a_o and b_o are the orthorhombic lattice parameters (double-headed black arrows) in the nematic phase. Grey dashed lines are guides for the eye. b, Hole–electron Fermi surfaces of the tight-binding model for FeSe (ref. ²⁴). The colour indicates the orbital character of the Fermi surfaces, where red, green and blue indicate d_xz, d_yz and d_xy orbitals of the Fe atom. Fermi surface nesting of Γ → X and Γ → Y corresponds to (1, 0) and (0, 1) in reciprocal lattice units, where (H, K) = (q_xa_o/2π, q_yb_o/2π) are in-plane Miller indices of the orthorhombic lattice, respectively (Methods). c, S(Q, E) integrated around (1, 0) above (T = 10 K) and below (T = 2 K) T_c (≈8 K) on twinned FeSe (Supplementary Fig. 5). The vertical error bars indicate the statistical errors of 1 s.d. d, Schematic diagram of the sample arrangement. FeSe samples are glued on large single crystals of BaFe₂As₂ under a uniaxial pressure of about 20 MPa (refs. ^39,40). e, Wave-vector scans of nuclear (2, 0, 0) and (0, 2, 0) Bragg peaks of FeSe on an assembly of BaFe₂As₂ single crystals at different temperatures. The dashed lines indicate the single-Gaussian components of the fitting. f,g, Schematic illustrations of the magnetic scattering at (1, 0) (f) and (0, 1) (g) in the normal and superconducting (SC) states estimated from the twinned and detwinned samples (Supplementary Fig. 10). The shaded region in f is (0, 1) data from g.

In recent inelastic neutron scattering (INS) experiments on twinned FeSe^18,32,33, well-defined low-energy (E < 15 meV) spin fluctuations are found at Q_AF = (±1, 0) and its twin-domain positions (0, ±1) in the nematic phase below T_s. On cooling below the superconducting transition temperature T_c, ≈ 8 K, a neutron spin resonance, a key signature of unconventional superconductivity¹, appears at E ≈ 4 meV and sharply peaks at the (±1, 0) and (0, ±1) positions^32,33. Figure 1c shows the energy dependence of the magnetic scattering S(E) integrated around Q_AF obtained from our high-resolution INS experiments (Methods). In the normal state, the magnetic scattering is gapless above E = 0.5 meV and increases in intensity with increasing energy (Fig. 2a). In addition to having a weak peak around E ≈ 3.2 meV, we find that the scattering changes from well-defined commensurate peaks centred around Q_AF below E = 3.625 ± 0.125 meV (Fig. 2b, c) to a peak with a flattish top at E = 5.625 ± 0.125 meV (Fig. 2d). On cooling to below T_c in the superconducting state, the spin excitation spectra open a gap below E ≈ 2.5 meV (Fig. 2e,f), form a commensurate resonance at E = 3.6 meV (Fig. 2g) and exhibit ring-like incommensurate scattering at E = 5.25 ± 0.075 meV (Fig. 2f). The dispersive ring-like incommensurate resonance is also seen in hole-doped Ba_0.67K_0.33(Fe_1−xCo_x)₂As₂ superconductors³⁴.

Fig. 2 |. Low-energy spin fluctuations in twinned FeSe below and above T_c.

a,e, Two-dimensional images of a wave-vector and energy dependence of spin fluctuations at T = 10 K (a) and T = 2 K (e). b–d,f,g, Wave-vector dependence of spin fluctuations in the (H, K) plane at energies E = 1 ± 0.25 meV (b,f), E = 3.6 ± 0.125 meV (c,g) and E = 5.62 ± 0.125 meV (d). Cuts along the [H, 0] and [1, K] directions with a width of ±0.04 reciprocal lattice units show a scattering peak with a flattish top near (1, 0). h, E = 5.25 ± 0.075 meV. Incommensurate scattering is clearly seen through the identical cuts along the [H, 0] and [1, K] directions. This feature is missed in previous work^18,32,33 due to the small incommensurability and narrow energy range. a–d, T = 10 K; e–h, T = 2 K. Solid lines are fits with the sum of two Gaussians to the data. Colour bars indicate scattering intensity in arbitrary unit (a.u.). The vertical error bars indicate statistical errors of 1 s.d.

Although these results on twinned FeSe suggest that spin fluctuations play an important role in the superconductivity of FeSe, they provide no information on the possible orbital-selective nature of the fluctuations, which may lead to a highly anisotropic electron pairing state^{19,31,35–38}. From STM quasiparticle interference measurements on single-domain (detwinned) FeSe, where the Fermi surface geometry of electronic bands can be determined in the nematic phase, sign-reversed superconducting gaps are found at the hole (Γ or Q = (0, 0)) and electron (X or Q_AF = (1, 0)) Fermi surface states derived from d_yz orbitals of the Fe atoms along the orthorhombic a_o-axis direction (Fig. 1a,b)²⁴. Moreover, similar STM measurements show that the same orbital-selective self-energy effects are already present in the normal state of FeSe above T_c (ref. ²⁵).

If superconductivity in FeSe arises from quasiparticle excitations between hole and electron pockets (Fig. 1b) that are indeed orbital selective^24,25, detwinned crystals should exhibit a strong anisotropy of the low-energy spin excitations. In particular, it is expected that the neutron spin resonance associated with s^± superconductivity^32,33 should only occur along the orthorhombic a_o-axis direction at Q_AF = (±1, 0) in detwinned FeSe, as the orbital-selective superconducting gap with the d_yz orbital character is large for scattering vectors along the a_o axis²⁴. Similarly, orbital-dependent Hund’s coupling in a nematic quantum spin liquid of FeSe can also induce a large superconducting gap and spin excitation anisotropy¹⁹. To test these hypotheses, we used INS to study the low-energy spin fluctuations in detwinned FeSe (Fig. 1d,e). In the normal state, spin fluctuations from 6 to 11 meV are centred around Q_AF with negligible intensity at (0, ±1), thus exhibiting a pronounced C₂ rotational symmetry as predicted by these theoretical approaches^19,27,31. By contrast, for energies between 3 and 5 meV the spin fluctuations have a C₄ rotational symmetry magnetic component as shown in the schematic illustration in Fig. 1f,g, which is based on combining experimental evidence from multiple instruments (Supplementary Fig. 10), possibly corresponding to a localized mode in both wave vector and energy. On cooling below T_c, the resonance only appears at Q_AF = (1, 0) (Fig. 1f,g), consistent with the STM observation that superconducting gaps are extremely anisotropic, with minima at the tips of the elliptical pockets. Therefore, while the normal-state C₄ rotational symmetry magnetic component in the 3–5 meV range is not anticipated, the anisotropic superconductivity-induced resonance is consistent with theoretical expectations^19,24.

To detect anisotropic spin fluctuations by INS^32,33, one needs to co-align hundreds of single-crystal FeSe samples. These are grown by the chemical vapour transport method and are about 1–3 mm² in size and a few micrometres in thickness (Methods)⁹. Therefore, the most difficult part of carrying out INS experiments on FeSe is to simultaneously detwin hundreds of samples. In previous work on iron pnictides, we were able to completely detwin large (of the order of 0.5–1 cm² by a few millimetres in thickness) single crystals of BaFe₂As₂ using a mechanical uniaxial pressure device^39,40. By gluing many oriented FeSe samples on uniaxially pressured BaFe₂As₂ as shown schematically in Fig. 1d, we were able to simultaneously detwin many FeSe single crystals required for INS experiments (Supplementary Fig. 2). Figure 1e shows the temperature dependence of rocking scans along the [H, 0, 0] and [0, K, 0] directions on multiple FeSe on BaFe₂As₂ assemblies. Below T_s ≈ 90 K, we see a clear splitting of the lattice constants. By comparing the scattering intensities of the (2, 0, 0) and (0, 2, 0) nuclear Bragg peaks, we find that the FeSe sample assembly has a detwinning ratio of η = [I(2, 0, 0)_o − I(0, 2, 0)_o]/[I(2, 0, 0)_o + I(0, 2, 0)_o] ≈ 50% at 2 K (Fig. 1e), where I(2, 0, 0)_o and I(0, 2, 0)_o are the observed Bragg peak intensities at (2, 0, 0) and (0, 2, 0), respectively, below T_s (Supplementary Figs. 3 and 4).

To understand the effect of detwinning FeSe, we first need to determine the wave-vector and energy dependence of the magnetic scattering S(Q, E) in twinned samples (Methods and Supplementary Fig. 5). Figure 2a,e shows the energy dependence of the magnetic scattering along the [1, K] direction above and below T_c, respectively. In the normal state at T = 10 K, the scattering is gapless above E = 0.5 meV and exhibits a weak peak around E = 3.2 meV (Fig. 2a). The spin excitations are centred around Q_AF = (1, 0) at E = 1 ± 0.25 meV (Fig. 2b) and 3.625 ± 0.125 meV (Fig. 1c). At E = 5.625 ± 0.125 meV, the spin excitations have a flattish top as revealed by wave-vector cuts along the [H, 0] and [1, K] directions (Fig. 1d). In the superconducting state at T = 2 K, a superconductivity-induced spin gap opens below E ≈ 2.5 meV and a resonance forms around E = 3.6 meV (Figs. 1c and 2e). This is confirmed by the vanishing signal at E = 1 ± 0.25 meV (Fig. 2f) and enhanced magnetic scattering at 3.625 ± 0.125 meV (Fig. 2g). In addition, the resonance is clearly centred at the commensurate Q_AF = (1, 0) position (Fig. 2g). However, on increasing the energy to E = 5.25 ± 0.075 meV, we see clear incommensurate ring-like magnetic scattering centred around Q_AF = (1, 0), as confirmed by wave-vector cuts along the [H, 0] and [1, K] directions (Fig. 2h). The incommensurate scattering intensity in the superconducting state is higher than that in the normal state, suggesting that it is part of the dispersive resonance. In previous work, a dispersive ring-like neutron spin resonance has been seen in the hole-doped BaFe₂As₂ family of materials, where the incommensurate scattering has been ascribed to quasiparticle excitations from mismatched hole and electron Fermi surfaces³⁴.

Figure 3 summarizes the energy evolution of the normal-state spin fluctuations at Q_AF = (1, 0) and (0, 1) in the (H, K) plane in partially detwinned FeSe. Since our FeSe single crystals are mounted on surfaces of BaFe₂As₂, one should also see spin fluctuations from BaFe₂As₂ at approximately the same position in reciprocal space. However, the spin waves in BaFe₂As₂ are gapped below about 10 meV in the low-temperature AF ordered state^41,42, meaning that spin fluctuations at Q_AF ≈ (1, 0) and (0, 1) below 10 meV must originate from FeSe. Figure 3a,b shows constant-energy cuts in the (H, K) plane for energy transfers of E = 3.5 ± 0.5 and 4.5 ± 0.5 meV, respectively, in the normal state at T = 12 K. We see clear evidence for magnetic scattering at Q_AF ≈ (1, 0) and (0, 1) with about the same strength (Supplementary Fig. 6a–d), suggesting a possible mode that has C₄ rotational symmetry in the normal state. On increasing energies to E = 6 ± 1 and 8 ± 1 meV, the scattering at Q_AF ≈ (1, 0) becomes much stronger than that at (0, 1), suggesting that spin fluctuations become highly C₂ symmetric at these energies (Fig. 3c,d). To confirm these results, we carried out energy scans at Q_AF ≈ (1, 0) and (0, 1) from 2.5 meV to 11 meV as shown in Fig. 3e (Supplementary Fig. 6e). From 6 meV to 11 meV, magnetic scattering at (1, 0) increases in intensity with increasing energy approximately twice as fast as the increase of magnetic scattering at (0, 1). Figure 3f shows wave-vector scans approximately along the [1, K] and [H, 1] directions at E = 8 meV (see E = 3.6 meV data in Supplementary Fig. 6f). The scattering intensity at (1, 0) dominated the signal while spin fluctuations at (0, 1) are only one-third of those at (1, 0). After taking into account the finite η of the FeSe samples (Supplementary Information), there is almost no magnetic scattering at (0, 1) above the background. These results are consistent with Fig. 3c,d, suggesting that the spin fluctuations between 6 and 10 meV are strongly C₂ symmetric.

Fig. 3 |. Normal-state spin fluctuations in detwinned FeSe.

a–d, Two-dimensional images of spin fluctuations at E = 3.5 ± 0.5 meV (a), E = 4.5 ± 0.5 meV (b), E = 6 ± 1 meV (c) and E = 8.5 ± 1 meV (d). The data are collected at T = 12 K using a MAPS chopper spectrometer with incident neutron energy of E_i = 38 meV along the c axis and are folded to improve statistics. The scattering near wave vector (1, 1) is background and not magnetic in origin (Supplementary Figs. 6–8). Colour bars in a–d indicate scattering intensity in arbitrary units (a.u.). e, Energy dependence of the scattering at (1, 0) and (0, 1) above background at T = 11 K. The positions of signal and background are marked as large and small spots in the inset. f, Wave-vector scans at E = 8 meV along the [1, K] (green) and [H, 1] directions at T = 11 K. Linear backgrounds have been subtracted from the data. The vertical error bars indicate the statistical errors of 1 s.d.

To confirm that spin fluctuations in FeSe for energies below 5 meV have a C₄ component as suggested in Fig. 3a,b and determine the impact of superconductivity (Supplementary Figs. 7 and 8), we carried out constant-energy and constant-wave-vector scans at (1, 0) and (0, 1) using a cold neutron triple-axis spectrometer (Methods). Figure 4a,b shows temperature difference plots below (T = 2 K) and above (T = 12 K) T_c as a function of energy at (1, 0) and (0, 1), respectively. In previous work on twinned samples, superconductivity has been found to induce a neutron spin resonance appearing below T_c at (1, 0) and (0, 1) around E ≈ 3.6 meV (Fig. 1c)^32,33. While Fig. 4a shows clear evidence for the resonance at E ≈ 3.6 meV, with intensity reduction (negative scattering) below the mode indicating opening of a spin gap^32,33, an identical temperature difference plot at (0, 1) in Fig. 4b yields no observable temperature difference across T_c, and therefore no superconductivity-induced resonance or spin gap. Figure 4c,d shows wave-vector scans along the [H + 1, 0] and [0, K + 1] directions, respectively, at E = 3.6 meV. In the normal state (T = 12 K), we see well-defined peaks centred at (1, 0) and (0, 1), consistent with Fig. 3a,b. On cooling below T_c, the scattering at (1, 0) increases in intensity and forms a resonance (Fig. 4c), while it does not change across T_c at (0, 1) (Fig. 4d). Figure 4e,f shows the same data after correcting for background scattering and η. Similarly to Fig. 4c,d, we again find that superconductivity induces a C₂-symmetric resonance on a background of approximately C₄-symmetric normal-state magnetic scattering (Supplementary Fig. 9). Thus, it is the highly anisotropic pairing state of FeSe that drives the C₂-symmetric magnetic scattering at these energies below T_c. Figure 1f,g summarizes the key results of our INS experiments on detwinned FeSe. The deviation of magnetic scattering intensity ratio at (1, 0) and (0, 1) from 3:1 provides convincing evidence for the existence of an unexpected mode. In the normal state, spin fluctuations have approximate C₄ symmetry near the resonance energy but become C₂ symmetric for energies above 6 meV. On entering the superconducting state, a resonance with C₂ symmetry is formed at Q_AF (Fig. 1f and Supplementary Fig. 10).

Fig. 4 |. Effect of superconductivity on low-energy spin fluctuations of detwinned FeSe.

a,b, Difference of the scattering in the superconducting state (below T_c) and the normal state plotted as a function of energy for momentum transfers (1, 0) (a) and (0, 1) (b). The peak seen at E ≈ 3.6 meV in a marks the neutron spin resonance. The solid blue and dashed red lines are guides to the eye. c, Wave-vector scans below and above T_c at E = 3.6 meV and (1, 0). d, Similar scans at (0, 1). e,f, (1, 0) (e) and (0, 1) (f) scans with background subtracted and detwinning ratio corrected (Supplementary Fig. 9). The solid lines in c–f are Gaussian fits to the data before and after linear background subtraction. The vertical error bars indicate the statistical errors of 1 s.d.

To achieve a theoretical understanding of the experimental results presented above, we start from an itinerant five-band model that quantitatively matches the low-energy electronic structure of FeSe in its nematic state^9,21,24,36 and compute the magnetic scattering S(Q,E)∝χ′′(Q,E) (1∕ −e^−E/k_BT ) where the imaginary part of dynamic susceptibility χ′′(Q, E) is calculated using a standard random phase approximation formulation^28,29,31,36. The spectral function at the Fermi level is presented in Fig. 5a. As illustrated in Fig. 5c,e, this ‘plain vanilla’ approach completely fails, as is evident, for example, from the presence of scattering close to (1, 1), and a negligible (1, 0)–(0, 1) anisotropy. The latter properties can be traced to an improper balance of the three most important scattering channels (Fig. 5a). However, electronic interactions and associated self-energy effects are known to be important in FeSe, constituting an example of a Hund metal²⁵. Important properties of Hund’s metals include the existence of orbital-dependent mass renormalizations^43–45, and an associated redistribution of the relative importance of different orbital-dependent scattering channels in the spin susceptibility⁴⁶.

Fig. 5 |. Theoretical calculations of the spin fluctuations in detwinned FeSe.

a,b, Map of the spectral function at zero energy for our model of the electronic structure of FeSe in the fully coherent case with quasiparticle weights Z_ℓ=1 (a) and the orbital-selective case where reduced quasiparticle weights (Z_ℓ<1) (b) weaken the spin fluctuations at a certain momentum transfer (blue and red arrows) while the spin fluctuations stemming mostly from the d_yz orbital become dominant (green arrow). The grey bar for a–b represents the intensity of the spectral function. c, For the fully coherent model, one obtains in the normal state large contributions to the dynamical structure factor close to (±1, ±1) from the d_xy orbital and an almost identical but weaker contribution at (±1, 0) and (0, ±1). The colour code for c–d shown in the colour bar indicates the calculated intensity maps of the magnetic scattering in arbitrary units (a.u). e, The susceptibility integrated around these momentum transfer vectors shows the same trend at low energies. d,f, In contrast, the orbital-selective model yields spin fluctuations at low energies that are dominated by peaks at (±1, 0) in the low energy range shown. f,g, The enhancement of the spin fluctuations at (±1, 0) in the superconducting state is clearly seen when plotted as a function of energy (f) and as an intensity map in momentum–energy space (g). h, The spin fluctuations at (1, 0) are dominated by the contributions of the d_yz orbital.

A simple means to incorporate the important effects of such orbital selectivity is through the introduction of orbital-dependent quasiparticle weights Z_ℓ<1 (refs. ^31,36), leading to a modified bare susceptibility ${\tilde{\chi }}_{{\mathcal{l}}_1{\mathcal{l}}_2{\mathcal{l}}_3{\mathcal{l}}_4}^0(Q,E)$ given by

$${\tilde{\chi }}_{{\mathcal{l}}_1{\mathcal{l}}_2{\mathcal{l}}_3{\mathcal{l}}_4}^0(Q,E)=\sqrt{Z_{{\mathcal{l}}_1}{Z}_{{\mathcal{l}}_2}{Z}_{{\mathcal{l}}_3}{Z}_{{\mathcal{l}}_4}}{\chi }_{{\mathcal{l}}_1{\mathcal{l}}_2{\mathcal{l}}_3{\mathcal{l}}_4}^0(Q,E).$$ (1)

In agreement with theoretical expectations^43–45,47, and earlier detailed studies of tunnelling spectroscopy^24,25, we apply the hierarchy Z_xy < Z_xz < Z_yz, which shifts the relative importance of the dominant scattering vectors, as illustrated in Fig. 5b, and thereby modifies the magnetic scattering. As seen in Fig. 5d, the d_xy-dominated (1, 1) scattering is strongly reduced (because Z_xy is the smallest), and the degree of C₄-symmetry breaking as seen by the difference in the scattering intensities at (±1, 0) versus (0, ±1) is strongly enhanced (because Z_xz < Z_yz), as seen explicitly by the dashed lines in Fig. 5f (Supplementary Fig. 1).

In the superconducting state, we employ a gap structure identical to that of refs. ^24,36, which is known to faithfully describe the gap in FeSe, and recalculate the bare susceptibility accordingly^28,36. When entering the highly anisotropic superconducting state, generated by the orbital-selective spin fluctuations^24,36, a neutron resonance is exhibited solely at the (±1, 0) position as seen from Fig. 5f,g, in agreement with experiments. The associated neutron resonance is highly orbital selective with predominant d_yz character, as seen by the orbital-resolved spin susceptibilities plotted in Fig. 5h. Therefore, both the very strong C₄-symmetry breaking in the 5–10 meV range and the unidirectional neutron resonance observed experimentally are captured by the itinerant orbital-selective scenario.

This approach, however, does not provide an explanation for the emergence of the localized approximately C₄-symmetric spin excitations near E = 3 meV as shown in Figs. 1f,g and 3a,b. There are several possible scenarios for this remarkable discovery. First, it is possible that self-energy effects in FeSe have a considerably more complicated functional form that cannot be simply captured by including energy- and momentum-independent Z-factors. Second, there is a possibility of impurity-generated low-energy spectral weight similar to the case of cuprates, where vortices and disorder have been shown to generate localized modes in a restricted low-energy regime^48–50. A counter-argument to the disorder-based scenario, however, is the high quality of the FeSe crystals used in the current experiment (Supplementary Fig. 2).

Finally, if some of the spin excitations in FeSe arise from a local moment quantum paramagnet^16,17,19, the C₂-symmetric AF collinear order competes with the C₄-symmetric Néel order across the nematic ordering temperature T_s (ref. ¹⁸). In this picture, the C₄-symmetric low-energy magnetic excitations with spin-wave ring-like features in detwinned FeSe may simply be the remnant of the localized moment not directly associated with Fermi surface nesting and itinerant electrons.

Regardless of the microscopic origin of the C₄ spin excitations, our data support the notion that the spin fluctuations in the nematic phase of FeSe are, generally, highly anisotropic, and are consistent with superconductivity being driven by spin fluctuations arising mainly from the d_yz orbital states. Our measurements highlight the need for a quantitative understanding of the extreme spin anisotropy, as well as the emergence of C₄-symmetric magnetic excitations at the very lowest energies. Progress in this direction may well shed new light on the role of electronic correlations in FeSe in particular, and the origin of unconventional superconductivity in interacting systems in general.

Online content

Any methods, additional references, Nature Research reporting summaries, source data, statements of code and data availability and associated accession codes are available at https://doi.org/10.1038/s41563-019-0369-5.

Methods

Experimental setups.

Elastic neutron experiments were carried out on the HB-3A four-circle diffractometer at the High Flux Isotope Reactor, Oak Ridge National Laboratory, United States, to first check if the method works well in detwinning FeSe on a single piece of BaFe₂As₂ (Supplementary Fig. 3). HB-3A uses a silicon monochromator and a scintillator-based two-dimensional (H, K, L) = (q_xa_o/2π, q_yb_o/2π, q_zc/2π) in reciprocal lattice units using the orthorhombic lattice notation for FeSe, where a_o ≈ 5.33 Å, b_o ≈ 5.31 Å, c = 5.486 Å (ref. ⁹).

Our INS experiments on twinned samples were done on the MACS cold triple-axis spectrometer at NIST Center for Neutron Research at Gaithersburg, MD. The MACS spectrometer has a double focusing pyrolytic graphite (PG(002)) monochromator and multiple detectors. We used fixed scattered neutron energy E_f = 3.7 meV with a BeO filter after the sample and a Be filter before the monochromator for energy transfers below E = 1.5 meV.

Our INS experiments on detwinned samples were carried out on the PANDA cold neutron and PUMA thermal neutron triple-axis spectrometers at MLZ, Garching, Germany³⁹, and on the MAPS time-of-flight chopper spectrometer at ISIS, Rutherford-Appleton laboratory, Didcot, United Kingdom⁴⁰.

For PANDA experiments, a double-focused pyrolytic graphite (PG(002)) monochromator and analyser with E_f = 5.1 meV were used with collimations of none–40′–40′–none for inelastic measurements. For elastic measurements, we used E_f = 4.39 meV with collimations of 80′–80′–80′–80′. For thermal neutron measurements on PUMA, we used E_f = 14.69 meV with double focusing monochromator and analyser and no collimators. For MAPS neutron time-off-light measurements, we used an incident beam energy of E_i = 38 meV with the incident beam along the c axis of the crystal.

Sample growth and preparation.

The high-quality FeSe single crystals used in the experiments were grown by a chemical vapour transport method. Fe and Se powder were sealed in quartz tubes with KCl–AlCl₃ flux. The growth took 28 d in a temperature gradient from 330 °C to 400 °C. Typical samples are 1 × 1 mm² in area and less than 0.1 mm in thickness. The square BaFe₂As₂ crystals were grown using a flux method⁴⁰. They were aligned using a Laue camera and cut along the tetragonal [1, 1, 0] and [1, −1, 0] directions with a high-precision wire saw. Since single crystals of FeSe have one natural edge 45° rotated from the orthorhombic a_o direction, we can use an optical method to co-align FeSe on the surface of BaFe₂As₂. Given our intent to measure spin excitations in detwinned FeSe, we aligned and glued (with CYTOP type M) about 300 small pieces of FeSe single crystals on many pieces of large BaFe₂As₂ single crystals (Supplementary Figs. 1 and 3).