Jahn–Teller coupling to moiré phonons in the continuum model formalism for small-angle twisted bilayer graphene

Abstract

We show how to include the Jahn–Teller coupling of moiré phonons to the electrons in the continuum model formalism which describes small-angle twisted bilayer graphene. These phonons, which strongly couple to the valley degree of freedom, are able to open gaps at most integer fillings of the four flat bands around the charge neutrality point. Moreover, we derive the full quantum mechanical expression of the electron–phonon Hamiltonian, which may allow accessing phenomena such as the phonon-mediated superconductivity and the dynamical Jahn–Teller effect.

Introduction

The discovery of superconductivity first in small-angle twisted bilayer graphene (tBLG) [1–4], and later in trilayer [5] and double bilayer graphene [6], has stimulated an intense theoretical and experimental research activity. In these systems, the twist angle tunes a peculiar interference within a large set of energy bands, compressing energy levels to form a set of extremely narrow bands around charge neutrality [7–10]. In twisted bilayer graphene, these flat bands (FBs) have a bandwidth of the order of $\approx 10-20$ meV and are isolated in energy by single-particle band gaps of the order of 30–50 meV. Superconductivity is observed upon doping such narrow bands, often surrounding insulating states at fractional fillings that contradict the metallic behaviour predicted by band structure calculations. The observed phenomenology of these insulating states, which turn metallic above a threshold Zeeman splitting or above a critical temperature, suggests that they might arise from a weak-coupling Stoner or CDW band instability driven by electron–electron and/or electron–phonon interactions, rather than from the Mott’s localisation phenomenon in the presence of strong correlations. This is further supported by noting that the effective on-site Coulomb repulsion U must be identified with the charging energy of the supercell, which can be as large as tens of nanometers at small angles, projected onto the flat bands. If screening effects due to the gates and to the other bands are taken into account, the actual value of U is estimated of the order of few meVs, suggesting that tBLG might not be more correlated than a single graphene sheet [11]. On the contrary, there are evidences that the coupling to the lattice is instead anomalously large if compared with the FBs bandwidth. For instance, ab initio DFT-based calculations fail to predict well-defined FBs separated from other bands, unless atomic positions are allowed to relax [7, 9, 12–15], in which case gaps open that are larger than the FBs bandwidth. Further evidences supporting a sizeable electron–phonon coupling come from transport properties [16–18], but also from direct electronic structure calculations.

Specifically, in Ref. [8] it was shown that a pair of optical phonon modes are rather strongly coupled to the FBs and thus might play an important role in the physics of tBLG. These modes, which have a long wavelength modulation on the same moiré scale, have been dubbed as “moiré phonons” and recently observed experimentally [19]. However, the large number of atoms contained in the small-angle unit cell of twisted bilayer graphene (more than 11000) makes any calculation more involved than a simple tight-binding one, rather tough, if not computationally impossible. In this paper, we try to cope with such problem by implementing the effect of these phonon modes on the band structure in the less computationally demanding continuum model of Ref. [20]. This method can serve as a suitable starting point for BCS [21, 22], Hartree-Fock [23–26] and many other calculations, which may involve both phonons and correlations. The work is organised as follows. In Sect. 2, we derive the Bistritzer and MacDonald continuum model for twisted bilayer graphene. In Sect. 3, we implement in the continuum model formalism the effect of a static atomic displacement. By using lattice deformation fields which are similar to the Jahn–Teller moiré phonon modes of Ref. [8], we show how the band structure and the density of states of the system evolve as a function of the lattice distortion intensity. Finally, 4 is devoted to concluding remarks.

Continuum model Hamiltonian

We start by introducing the Bistritzer–MacDonald continuum model for twisted bilayer graphene [20] and recall that the single-layer Dirac Hamiltonian is, around the $\mathbf{K}$ and ${\mathbf{K}}^{\prime }=-\mathbf{K}$ valleys, respectively,

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill {\widehat{H}}_{\mathbf{k}\sim \mathbf{K}}& \equiv {\widehat{H}}_{\mathbf{k}}^{+1}=-v(\mathbf{k}-\mathbf{K})·({\sigma }_x,{\sigma }_y),\hfill \\ \hfill {\widehat{H}}_{\mathbf{k}\sim -\mathbf{K}}& \equiv {\widehat{H}}_{\mathbf{k}}^{-1}=-v(\mathbf{k}+\mathbf{K})·(-{\sigma }_x,{\sigma }_y),\hfill \end{array}\end{array}$$ 1

namely

$$\begin{array}{cc}\hfill {\widehat{H}}_{\mathbf{k}}^{\zeta }& =-v(\mathbf{k}-\zeta \mathbf{K})·(\zeta {\sigma }_x,{\sigma }_y),\hfill \end{array}$$ 2

with $\zeta =\pm 1$ the valley index, and where the Pauli matrices ${\sigma }_a$ act on the two component wavefunctions, each component referring to one of the two sites per unit cell that form the honeycomb lattice, which we shall hereafter denote as sublattices A and B.

Our analysis must start by defining the specific twisted bilayer graphene we shall investigate, and by setting some conventional notations. We assume that the twisted bilayer is obtained by rotating two AA-stacked layers ($i=1,2$) by an opposite angle $\pm \varphi$ with respect to the centre of two overlapping hexagons, see Fig. 1a, where $tan\varphi =1/[(2k+1)\sqrt{3}]$, with large positive integer k. With this choice, the moiré pattern forms a superlattice, which is still honeycomb and endowed by a ${\text{D}}_6$ space group symmetry [7] that is generated by the threefold rotation ${\text{C}}_{3z}$ around the z-axis perpendicular to the bilayer and the twofold rotations around the in-plane x- and y-axes, ${\text{C}}_{2x}$ and ${\text{C}}_{2y}$, respectively.

Fig. 1.

a Two graphene layers twisted with respect to the centre of two overlapping hexagons by an angle $2\varphi$. b Symmetries of twisted bilayer graphene which are those of the group ${\text{D}}_6$. Two- (${180}^{\circ }$) and threefold (${120}^{\circ }$) rotations with respect to the z-axis denoted as $C_{2z}$ and $C_{3z}$ as well as twofold rotations with respect to the x- and y-axis denoted as $C_{2y}$ and $C_{2x}$. c The monolayer Brillouin zones that are folded in the mini-Brillouin zone (MBZ) of the twisted heterostructure. d Mini-Brillouin zone of twisted bilayer graphene with reciprocal lattice vectors $\mathbf{A}$ and $\mathbf{B}$. The high-symmetry points and paths are highlighted

The corresponding mini-Brillouin zone (MBZ) has reciprocal lattice vectors

$$\begin{array}{c}\hfill \mathbf{A}={\mathbf{G}}_2^{(2)}-{\mathbf{G}}_2^{(1)},\mathbf{B}={\mathbf{G}}_1^{(1)}-{\mathbf{G}}_1^{(2)}.\end{array}$$ 3

where ${\mathit{G}}_{1/2}^{(1)}={\mathcal{R}}_{+\varphi }({\mathit{G}}_{1/2})$ and ${\mathit{G}}_{1/2}^{(2)}={\mathcal{R}}_{-\varphi }({\mathit{G}}_{1/2})$ are the reciprocal lattice vectors of each layer after the twist, with ${\mathcal{R}}_{\varphi }$ the rotation operator by an angle $\varphi$.

The Dirac nodes of each monolayer are, correspondingly, ${\mathcal{R}}_{\pm \varphi }(\mathbf{K})\equiv {\mathbf{K}}_{\pm \varphi }$ for the valley we shall denote as $\zeta =+1$, and ${\mathcal{R}}_{\pm \varphi }({\mathbf{K}}^{\prime })\equiv {\mathbf{K}}_{\pm \varphi }^{\prime }$ for the other valley, $\zeta =-1$. With our choices, ${\mathbf{K}}_{+\varphi }$ and ${\mathbf{K}}_{-\varphi }^{\prime }$ fold into the same point ${\mathbf{K}}_2$ of the MBZ, as well as ${\mathbf{K}}_{-\varphi }$ and ${\mathbf{K}}_{+\varphi }^{\prime }$ into the point ${\mathbf{K}}_1$, see Fig. 1.

We introduce the (real) Wannier functions derived by the $p_z$ orbital of each carbon atom:

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill {\varphi }_{1\alpha {\mathbf{R}}^{(1)}}(\mathbf{r})& \equiv \varphi (\mathbf{r}-{\mathbf{R}}^{(1)}-{\mathbf{r}}_{\alpha }^{(1)}-\frac{{\displaystyle {\mathit{d}}_{\perp }}}{{\displaystyle 2}}),\hfill \\ \hfill {\varphi }_{2\alpha {\mathbf{R}}^{(2)}}(\mathbf{r})& \equiv \varphi (\mathbf{r}-{\mathbf{R}}^{(2)}-{\mathbf{r}}_{\alpha }^{(2)}+\frac{{\displaystyle {\mathit{d}}_{\perp }}}{{\displaystyle 2}}),\hfill \end{array}\end{array}$$ 4

where ${\mathit{d}}_{\perp }=(0,0,d)$, with d the interlayer distance, ${\mathbf{R}}^{(i)}$ label the positions of the unit cells in layer $i=1,2$, while ${\mathbf{r}}_{\alpha }^{(i)}$ the coordinates with respect to ${\mathbf{R}}^{(i)}$ of the two sites within each unit cell, $\alpha =A,B$ denoting the two sublattices. From the Wannier functions, we build the Bloch functions

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill {\psi }_{1\alpha \mathbf{k}}(\mathbf{r})& =\frac{{\displaystyle 1}}{{\displaystyle \sqrt{N}}}\sum _{{\mathbf{R}}^{(1)}}{\text{e}}^{-i\mathbf{k}·({\mathbf{R}}^{(1)}+{\mathbf{r}}_{\alpha }^{(1)})}{\varphi }_{1\alpha {\mathbf{R}}^{(1)}}(\mathbf{r}),\hfill \\ \hfill {\psi }_{2\alpha \mathbf{k}}(\mathbf{r})& =\frac{{\displaystyle 1}}{{\displaystyle \sqrt{N}}}\sum _{{\mathbf{R}}^{(2)}}{\text{e}}^{-i\mathbf{k}·({\mathbf{R}}^{(2)}+{\mathbf{r}}_{\alpha }^{(2)})}{\varphi }_{2\alpha {\mathbf{R}}^{(2)}}(\mathbf{r}).\hfill \end{array}\end{array}$$ 5

Conventionally, one assumes the two-centre approximation [20], so that, if $V_{\perp }(\mathbf{r})$ is the interlayer potential, then the interlayer hopping

$$\begin{array}{cc}\hfill \int \mathrm{d}\mathbf{r}{\varphi }_{1\alpha {\mathbf{R}}^{(2)}}(\mathbf{r})V_{\perp }(\mathbf{r}){\varphi }_{2\beta {\mathbf{R}}^{(1)}}(\mathbf{r})& \simeq T_{\perp }({\mathbf{R}}^{(2)}+{\mathbf{r}}_{\alpha }^{(2)}-{\mathbf{R}}^{(1)}-{\mathbf{r}}_{\beta }^{(1)}),\hfill \end{array}$$ 6

depends only on the distance between the centres of the two Wannier orbitals. We define $T_{\perp }(\mathbf{q})$, the Fourier transform of $T_{\perp }(\mathbf{r})$:

$$\begin{array}{c}\hfill T_{\perp }(\mathbf{r})=\frac{{\displaystyle 1}}{{\displaystyle N}}\sum _{\mathbf{q}}{\text{e}}^{i\mathbf{q}·\mathbf{r}}{T}_{\perp }(\mathbf{q}),\end{array}$$ 7

where $\mathbf{r}$ and $\mathbf{q}$ are vectors in the x–y plane. Hereafter, all momenta are assumed also to lie in the x–y plane.

The interlayer hopping between an electron in layer 1 with momentum $\mathbf{p}$ and one in layer 2 with momentum $\mathbf{k}$ is in general a matrix ${\widehat{T}}_{\mathbf{k}\mathbf{p}}$, with elements $T_{\mathbf{k}\mathbf{p}}^{\alpha \beta }$, $\alpha ,\beta =A,B$, which, through equations (5), (6) and (7), reads explicitly

$$\begin{array}{cc}\hfill T_{\mathbf{k}\mathbf{p}}^{\alpha \beta }& =\frac{{\displaystyle 1}}{{\displaystyle N}}\sum _{{\mathbf{R}}^{(2)}{\mathbf{R}}^{(1)}}{\text{e}}^{-i\mathbf{k}·({\mathbf{R}}^{(2)}+{\mathbf{r}}_{\alpha }^{(2)})}{\text{e}}^{i\mathbf{p}·({\mathbf{R}}^{(1)}+{\mathbf{r}}_{\beta }^{(1)})}{T}_{\perp }({\mathbf{R}}^{(2)}+{\mathbf{r}}_{\alpha }^{(2)}-{\mathbf{R}}^{(1)}-{\mathbf{r}}_{\beta }^{(1)})\hfill \\ \hfill & =\sum _{\mathbf{q}}{T}_{\perp }(-\mathbf{q})\frac{{\displaystyle 1}}{{\displaystyle N^2}}\sum _{{\mathbf{R}}^{(2)}{\mathbf{R}}^{(1)}}{\text{e}}^{-i(\mathbf{k}+\mathbf{q})·({\mathbf{R}}^{(2)}+{\mathbf{r}}_{\alpha }^{(2)})}{\text{e}}^{i(\mathbf{p}+\mathbf{q})·({\mathbf{R}}^{(1)}+{\mathbf{r}}_{\beta }^{(1)})}\hfill \\ \hfill & =\sum _{\mathbf{q}}{T}_{\perp }(-\mathbf{q})\sum _{{\mathbf{G}}^{(2)}{\mathbf{G}}^{(1)}}{\delta }_{\mathbf{k}+\mathbf{q},-{\mathbf{G}}^{(2)}}{\delta }_{\mathbf{p}+\mathbf{q},-{\mathbf{G}}^{(1)}}{\text{e}}^{i{\mathbf{G}}^{(2)}·{\mathbf{r}}_{\alpha }^{(2)}}{\text{e}}^{-i{\mathbf{G}}^{(1)}·{\mathbf{r}}_{\beta }^{(1)}}\hfill \\ \hfill & =\sum _{{\mathbf{G}}^{(2)}{\mathbf{G}}^{(1)}}{T}_{\perp }(\mathbf{k}+{\mathbf{G}}^{(2)}){\delta }_{\mathbf{k}+{\mathbf{G}}^{(2)},\mathbf{p}+{\mathbf{G}}^{(1)}}{\text{e}}^{i{\mathbf{G}}^{(2)}·{\mathbf{r}}_{\alpha }^{(2)}}{\text{e}}^{-i{\mathbf{G}}^{(1)}·{\mathbf{r}}_{\beta }^{(1)}}.\hfill \end{array}$$ 8

Since we are interested in the low energy physics, $\mathbf{k}$ and $\mathbf{p}$ must be close to the corresponding Dirac points, namely ${\mathbf{K}}_{\varphi }$ and ${\mathbf{K}}_{\varphi }^{\prime }$ for $\mathbf{p}$ in layer 1, while ${\mathbf{K}}_{-\varphi }$ and ${\mathbf{K}}_{-\varphi }^{\prime }$ for $\mathbf{k}$ in layer 2. Therefore, ${\widehat{T}}_{\mathbf{k}\mathbf{p}}$ can in principle couple to each other states of different layers within the same valley or between opposite valleys. Since $T_{\perp }(\mathbf{q})$ decays exponentially with $q=|\mathbf{q}|$ [20], the leading terms are those with the least possible $|\mathbf{k}+{\mathbf{G}}^{(2)}|$ compatible with momentum conservation $\mathbf{k}+{\mathbf{G}}^{(2)}=\mathbf{p}+{\mathbf{G}}^{(1)}$. At small twist angle $\varphi$, only the intra-valley matrix elements, $\mathbf{p}\sim \mathbf{k}$, are sizeable, while the inter-valley ones are negligibly small, despite opposite valleys of different layers fold into the same point of the MBZ. For instance, if $\mathbf{p}\simeq {\mathbf{K}}_{+\varphi }$ and $\mathbf{k}\simeq {\mathbf{K}}_{-\varphi }^{\prime }$, momentum conservation requires very large ${\mathbf{G}}^{(1)}=(2k+1)({\mathbf{G}}_2^{(1)}-{\mathbf{G}}_1^{(1)})$ and ${\mathbf{G}}^{(2)}=(2k+1)({\mathbf{G}}_2^{(2)}-{\mathbf{G}}_1^{(2)})$, thus an exponentially small $T_{\perp }(\mathbf{k}+{\mathbf{G}}^{(2)})$. The effective decoupling between the two valleys implies that the number of electrons within each valley is to high accuracy a conserved quantity, thus an emergent valley $U_v(1)$ symmetry [20, 33] that causes accidental band degeneracies along high-symmetry lines in the MBZ [8, 33].

We can therefore just consider the intra-valley inter-layer scattering processes. We start with valley $\zeta =+1$ and thus require that $\mathbf{k}$ is close to ${\mathbf{K}}_{-\varphi }={\mathbf{K}}_1$ and $\mathbf{p}$ close to ${\mathbf{K}}_{+\varphi }={\mathbf{K}}_2$, see Fig. 1d. Since the modulus of $\mathbf{k}\sim {\mathbf{K}}_1$ is invariant under $C_{3z}$ rotations, where $C_{3z}({\mathbf{K}}_1)={\mathbf{K}}_1-{\mathbf{G}}_1^{(2)}$ and $C_{3z}^2({\mathbf{K}}_1)={\mathbf{K}}_1-{\mathbf{G}}_2^{(2)}$, maximisation of $T_{\perp }(\mathbf{k}+{\mathbf{G}}^{(2)})$ compatibly with momentum conservation leads to the following conditions, see Eq. (3),

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill \mathbf{p}& =\mathbf{k},\hfill \\ \hfill \mathbf{p}& =\mathbf{k}-{\mathbf{G}}_1^{(2)}+{\mathbf{G}}_1^{(1)}=\mathbf{k}+\mathit{B},\hfill \\ \hfill \mathbf{p}& =\mathbf{k}-{\mathbf{G}}_2^{(2)}+{\mathbf{G}}_2^{(1)}=\mathbf{k}-\mathit{A}.\hfill \end{array}\end{array}$$ 9

Upon defining $T(\mathbf{k})\equiv t_{\perp }$, and using Eq. (9) to evaluate the phase factors in (8), we finally obtain

$$\begin{array}{c}\hfill {\widehat{T}}_{\mathbf{k}\mathbf{p}}^{\zeta =+1}={\delta }_{\mathbf{p},\mathbf{k}}{\widehat{T}}_1+{\delta }_{\mathbf{p},\mathbf{k}+\mathit{B}}{\widehat{T}}_2+{\delta }_{\mathbf{p},\mathbf{k}-\mathit{A}}{\widehat{T}}_3,\end{array}$$ 10

where we explicitly indicate the valley index $\zeta$, and

$$\begin{array}{c}\hfill {\widehat{T}}_1=t_{\perp }\left(\begin{array}{cc}1& 1\\ 1& 1\end{array}\right),{\widehat{T}}_2=t_{\perp }\left(\begin{array}{cc}1& {\omega }^{\ast }\\ \omega & 1\end{array}\right),{\widehat{T}}_3=t_{\perp }\left(\begin{array}{cc}1& \omega \\ {\omega }^{\ast }& 1\end{array}\right),\end{array}$$ 11

with $\omega ={\text{e}}^{2\pi i/3}$.

We now focus on the other valley, $\zeta =-1$, and take $\mathbf{k}$ close to ${\mathbf{K}}_{-\varphi }^{\prime }=-{\mathbf{K}}_2$, and $\mathbf{p}$ to ${\mathbf{K}}_{\varphi }^{\prime }=-{\mathbf{K}}_1$, see Fig. 1d. In this case, Eq. (9) is replaced by

$$\begin{array}{c}\hfill \mathbf{p}=\mathbf{k},\mathbf{p}=\mathbf{k}-\mathit{B},\mathbf{p}=\mathbf{k}+\mathit{A},\end{array}$$ 12

and

$$\begin{array}{c}\hfill {\widehat{T}}_{\mathbf{k}\mathbf{p}}^{\zeta =-1}={\delta }_{\mathbf{p},\mathbf{k}}{\widehat{T}}_1^{\ast }+{\delta }_{\mathbf{p},\mathbf{k}-\mathit{B}}{\widehat{T}}_2^{\ast }+{\delta }_{\mathbf{p},\mathbf{k}+\mathit{A}}{\widehat{T}}_3^{\ast }.\end{array}$$ 13

Let us briefly discuss how one can take into account lattice relaxation, which is known to shrink the energetically unfavourable AA regions enlarging the Bernal-stacked triangular domains in the moiré pattern [7, 12, 14, 15, 27, 28, 30, 31]. As a consequence, the inter- and intra-sublattice hopping processes acquire different amplitudes, which is taken into account by modifying the operators ${\widehat{T}}_i$ in Eq. (11) according to

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill {\widehat{T}}_1& \to T_1(u,u^{\prime })=\left(\begin{array}{cc}u& u^{\prime }\\ u^{\prime }& u\end{array}\right)=u{\sigma }_0+u^{\prime }{\sigma }_x,\hfill \\ \hfill {\widehat{T}}_2& \to {\widehat{T}}_2(u,u^{\prime })=\left(\begin{array}{cc}u& u^{\prime }{\omega }^{\ast }\\ u^{\prime }\omega & u\end{array}\right)=u{\sigma }_0+u^{\prime }(cos\frac{{\displaystyle 2\pi }}{{\displaystyle 3}}{\sigma }_x+sin\frac{{\displaystyle 2\pi }}{{\displaystyle 3}}{\sigma }_y),\hfill \\ \hfill {\widehat{T}}_3& \to {\widehat{T}}_3(u,u^{\prime })=\left(\begin{array}{cc}u& u^{\prime }\omega \\ u^{\prime }{\omega }^{\ast }& u\end{array}\right)=u{\sigma }_0+u^{\prime }(cos\frac{{\displaystyle 2\pi }}{{\displaystyle 3}}{\sigma }_x-sin\frac{{\displaystyle 2\pi }}{{\displaystyle 3}}{\sigma }_y),\hfill \end{array}\end{array}$$ 14

with u generally smaller than $u^{\prime }$.

We conclude by showing how this formalism allows recovering the untwisted case, where ${\mathbf{G}}_{1/2}^{(1)}={\mathbf{G}}_{1/2}^{(2)}$, so that, through Eq. (3), $\mathit{A}=\mathit{B}=\mathbf{0}$, and therefore

$$\begin{array}{c}\hfill {\widehat{T}}_{\mathbf{k}\mathbf{p}}\underset{\varphi \to 0}{⟶}{\delta }_{\mathbf{p},\mathbf{k}}({\widehat{T}}_1+{\widehat{T}}_2+{\widehat{T}}_3)=3t_{\perp }{\delta }_{\mathbf{p},\mathbf{k}}\left(\begin{array}{cc}1& 0\\ 0& 1\end{array}\right),\end{array}$$ 15

which is what one would expect from an AA-stacked bilayer.

A more convenient representation

For our purposes, it is actually more convenient to use the alternative representation of the Hamiltonian derived in Ref. [32]. We translate ${\mathbf{K}}_{\varphi }^{\prime }=-{\mathbf{K}}_1$ so that it falls on ${\mathbf{K}}_{-\varphi }={\mathbf{K}}_1$, and similarly, ${\mathbf{K}}_{-\varphi }^{\prime }=-{\mathbf{K}}_2$ on ${\mathbf{K}}_{\varphi }={\mathbf{K}}_2$, see Fig. 1d. This implies that the diagonal parts of the Hamiltonian ${\widehat{H}}_{\zeta }^{(i)}(\mathbf{k})$, where $i=1,2$ is the layer index and $\zeta =\pm 1$ the valley one, become simply

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill {\widehat{H}}_{+1}^{(2)}(\mathbf{k})& =-v(\mathbf{k}-{\mathbf{K}}_1)·({\sigma }_x,{\sigma }_y),\hfill \\ \hfill {\widehat{H}}_{-1}^{(1)}(\mathbf{k})& =-v(\mathbf{k}-{\mathbf{K}}_1)·(-{\sigma }_x,{\sigma }_y)=v(\mathbf{k}-{\mathbf{K}}_1)·{\mathit{\sigma }}^T,\hfill \\ \hfill {\widehat{H}}_{+1}^{(1)}(\mathbf{k})& =-v(\mathbf{k}-{\mathbf{K}}_2)·({\sigma }_x,{\sigma }_y),\hfill \\ \hfill {\widehat{H}}_{-1}^{(2)}(\mathbf{k})& =-v(\mathbf{k}-{\mathbf{K}}_2)·(-{\sigma }_x,{\sigma }_y)=v(\mathbf{k}-{\mathbf{K}}_2)·{\mathit{\sigma }}^T.\hfill \end{array}\end{array}$$ 16

Following Ref. [32], we define a set of vectors

$$\begin{array}{cc}\hfill \mathbf{Q}& =\{{\mathbf{Q}}_A,{\mathbf{Q}}_B\}=\left\{\begin{array}{c}{\mathbf{Q}}_A={\mathbf{K}}_1+n\mathbf{A}+m\mathbf{B},\hfill \\ {\mathbf{Q}}_B={\mathbf{K}}_2+n\mathbf{A}+m\mathbf{B},\hfill \end{array}\right)\hfill \end{array}$$ 17

which span the vertices of the MBZs, where ${\mathbf{Q}}_A$, black circles in Fig. 2, correspond to valley $\zeta =+1$ in layer 2 and valley $\zeta =-1$ in layer 1, while ${\mathbf{Q}}_B$, red circles in Fig. 2, correspond to valley $\zeta =+1$ in layer 1 and valley $\zeta =-1$ in layer 2. In addition, we define

$$\begin{array}{c}\hfill {\mathbf{q}}_1={\mathbf{K}}_1-{\mathbf{K}}_2{\mathbf{q}}_2={\mathbf{q}}_1+\mathbf{B}{\mathbf{q}}_3={\mathbf{q}}_1-\mathbf{A}.\end{array}$$ 18

Next, we redefine the momenta for layers 1 and 2 as, respectively,

$$\begin{array}{c}\hfill \mathbf{p}-{\mathbf{K}}_2\to {\mathbf{k}}^{\prime }-{\mathbf{Q}}_B\mathbf{k}-{\mathbf{K}}_1\to {\mathbf{k}}^{\prime }-{\mathbf{Q}}_A,\end{array}$$ 19

thus

$$\begin{array}{c}\hfill \mathbf{k}={\mathbf{k}}^{\prime }+{\mathbf{K}}_1-{\mathbf{Q}}_A\mathbf{p}={\mathbf{k}}^{\prime }+{\mathbf{K}}_2-{\mathbf{Q}}_B,\end{array}$$ 20

so that the selection rules transform into

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill \mathbf{p}& =\mathbf{k}\Rightarrow {\mathbf{Q}}_A-{\mathbf{Q}}_B={\mathbf{q}}_1\Rightarrow {\mathbf{Q}}_B={\mathbf{Q}}_A-{\mathbf{q}}_1,\hfill \\ \hfill \mathbf{p}& =\mathbf{k}+\mathbf{B}\Rightarrow {\mathbf{Q}}_A-{\mathbf{Q}}_B={\mathbf{q}}_1+\mathbf{B}\Rightarrow {\mathbf{Q}}_B={\mathbf{Q}}_A-{\mathbf{q}}_2,\hfill \\ \hfill \mathbf{p}& =\mathbf{k}-\mathbf{A}\Rightarrow {\mathbf{Q}}_A-{\mathbf{Q}}_B={\mathbf{q}}_1-\mathbf{A}\Rightarrow {\mathbf{Q}}_B={\mathbf{Q}}_A-{\mathbf{q}}_3.\hfill \end{array}\end{array}$$ 21

With those definitions, and denoting the conserved momentum ${\mathbf{k}}^{\prime }$ as $\mathbf{k}$, the Hamiltonian of valley $\zeta$ now reads

$$\begin{array}{cc}\hfill {\widehat{H}}_{\mathbf{Q}{\mathbf{Q}}^{\prime }}^{\zeta }(\mathbf{k})& ={\delta }_{\mathbf{Q}{\mathbf{Q}}^{\prime }}v\zeta (\mathbf{k}-\mathbf{Q})·({\sigma }_x,\zeta {\sigma }_y)\hfill \\ \hfill & +\sum _{i=1}^3({\delta }_{{\mathbf{Q}}^{\prime }-\mathbf{Q},{\mathbf{q}}_i}+{\delta }_{\mathbf{Q}-{\mathbf{Q}}^{\prime },{\mathbf{q}}_i}){\widehat{T}}_i^{\zeta }(u,u^{\prime }),\hfill \end{array}$$ 22

where

$$\begin{array}{cc}\hfill {\widehat{T}}_1^{\zeta }(u,u^{\prime })& =u{\sigma }_0+u^{\prime }{\sigma }_x,\hfill \\ \hfill {\widehat{T}}_2^{\zeta }(u,u^{\prime })& =u{\sigma }_0+u^{\prime }(cos\frac{2\pi }{3}{\sigma }_x+\zeta sin\frac{2\pi }{3}{\sigma }_y),\hfill \\ \hfill {\widehat{T}}_3^{\zeta }(u,u^{\prime })& =u{\sigma }_0+u^{\prime }(cos\frac{2\pi }{3}{\sigma }_x-\zeta sin\frac{2\pi }{3}{\sigma }_y)={\sigma }_x{\widehat{T}}_2^{\zeta }(u,u^{\prime }){\sigma }_x.\hfill \end{array}$$ 23

In particular,

$$\begin{array}{cc}\hfill {\widehat{T}}_i^{-\zeta }(u,u^{\prime })& ={\sigma }_x{\widehat{T}}_i^{\zeta }(u,u^{\prime }){\sigma }_x.\hfill \end{array}$$ 24

One can further simplify the notation introducing the Pauli matrices ${\tau }_a$, $a=0,x,y,z$, with ${\tau }_0$ the identity, that act in the valley subspace, and thus write

$$\begin{array}{cc}\hfill {\widehat{H}}_{\mathbf{Q}{\mathbf{Q}}^{\prime }}(\mathbf{k})& ={\delta }_{\mathbf{Q}{\mathbf{Q}}^{\prime }}v{\tau }_z(\mathbf{k}-\mathbf{Q})·\Omega \mathit{\sigma }\Omega \hfill \\ \hfill & +{\tau }_0\sum _{i=1}^3({\delta }_{{\mathbf{Q}}^{\prime }-\mathbf{Q},{\mathbf{q}}_i}+{\delta }_{\mathbf{Q}-{\mathbf{Q}}^{\prime },{\mathbf{q}}_i})\Omega {\widehat{T}}_i(u,u^{\prime })\Omega ,\hfill \end{array}$$ 25

where ${\widehat{T}}_i(u,u^{\prime })\equiv {\widehat{T}}_i^{+1}(u,u^{\prime })$, and $\Omega$ is the real unitary operator

$$\begin{array}{cc}\hfill \Omega & ={\sigma }_x\frac{{\displaystyle 1-{\tau }_z}}{{\displaystyle 2}}+{\sigma }_0\frac{{\displaystyle 1+{\tau }_z}}{{\displaystyle 2}}={\sigma }_x{\text{P}}_{\zeta =-1}+{\sigma }_0{\text{P}}_{\zeta =+1},\hfill \end{array}$$ 26

being P${}_{\zeta }$ the projector onto valley $\zeta$, which actually interchanges sublattice A with B in the valley $\zeta =-1$. Applying the unitary operator $\Omega$, we thus obtain

$$\begin{array}{cc}\hfill \Omega {\widehat{H}}_{\mathbf{Q}{\mathbf{Q}}^{\prime }}(\mathbf{k})\Omega & \to {\widehat{H}}_{\mathbf{Q}{\mathbf{Q}}^{\prime }}(\mathbf{k})={\delta }_{\mathbf{Q}{\mathbf{Q}}^{\prime }}v{\tau }_z(\mathbf{k}-\mathbf{Q})·\mathit{\sigma }\hfill \\ \hfill & +{\tau }_0\sum _{i=1}^3({\delta }_{{\mathbf{Q}}^{\prime }-\mathbf{Q},{\mathbf{q}}_i}+{\delta }_{\mathbf{Q}-{\mathbf{Q}}^{\prime },{\mathbf{q}}_i}){\widehat{T}}_i(u,u^{\prime }),\hfill \end{array}$$ 27

which has the advantage of having a very compact form. For convenience, we list the action of $\Omega$ applied to $\sigma$ and $\tau$ operators,

$$\begin{array}{cc}\hfill \Omega {\sigma }_x\Omega & ={\sigma }_x,\Omega {\sigma }_y\Omega ={\sigma }_y{\tau }_z,\Omega {\sigma }_z\Omega ={\sigma }_z{\tau }_z,\hfill \\ \hfill \Omega {\tau }_x\Omega & ={\tau }_x{\sigma }_x,\Omega {\tau }_y\Omega ={\tau }_y{\sigma }_x,\Omega {\tau }_z\Omega ={\tau }_z.\hfill \end{array}$$ 28

In this representation, any symmetry operation $\text{G}\in D_6$ corresponds to a transformation

$$\begin{array}{cc}\hfill \widehat{H}(\mathbf{k})& ={\widehat{D}}^{†}(\text{G})\widehat{H}(\text{G}(\mathbf{k}))\widehat{D}(\text{G}),\hfill \end{array}$$ 29

whose explicit expressions are given in Ref. [32].

Fig. 2.

The lattice in momentum space of Ref. [32]. The ${\mathbf{Q}}_A$ and ${\mathbf{Q}}_B$ vectors span the lattice formed by the $\mathbf{K}$ points of the two twisted monolayer Brillouin zones

Perturbation induced by a static atomic displacement

We now move to derive in the continuum model the expression of the perturbation induced by a collective atomic displacement. Under a generic lattice deformation, the in-plane atomic positions ${\mathbf{x}}_{i\alpha }$ change according to

$$\begin{array}{cc}\hfill {\mathbf{x}}_{i\alpha }\equiv {\mathbf{R}}_i+{\mathbf{r}}_{i\alpha }& \to {\mathbf{R}}_i+{\mathbf{r}}_{\alpha }+{\mathit{u}}_i({\mathbf{x}}_{i\alpha })={\mathbf{x}}_{i\alpha }+{\mathit{u}}_i({\mathbf{x}}_{i\alpha }),\hfill \end{array}$$ 30

where i is now labelling a generic unit cell position. Since the phonon modes we are going to study involve only in-plane atomic displacements, we assume that z-coordinate of each carbon atom does not vary. It follows that a generic potential in the two-centre approximation and at linear order in the displacement reads

$$\begin{array}{cc}& T({\mathbf{x}}_{i\alpha }-{\mathbf{x}}_{j\beta },z_{i\alpha }-z_{j\beta })\to T({\mathbf{x}}_{i\alpha }-{\mathbf{x}}_{j\beta },z_{i\alpha }-z_{j\beta })\hfill \\ \hfill & +\mathit{W}({\mathbf{x}}_{i\alpha }-{\mathbf{x}}_{j\beta },z_{i\alpha }-z_{j\beta })·({\mathit{u}}_i({\mathbf{x}}_{i\alpha })-{\mathit{u}}_j({\mathbf{x}}_{j\beta })).\hfill \end{array}$$ 31

We further neglect the dependence on z, which we will take into account by distinguishing at the end between different scattering channels, intra- and inter-layers, so that:

$$\begin{array}{cc}\hfill \mathit{W}(\mathbf{r})& =\frac{{\displaystyle 1}}{{\displaystyle N}}\sum _{\mathbf{q}}{\text{e}}^{i\mathbf{q}·\mathbf{r}}\mathit{W}(\mathbf{q})=\mathbf{\nabla }T(\mathbf{r})=i\frac{{\displaystyle 1}}{{\displaystyle N}}\sum _{\mathbf{q}}\mathbf{q}T(\mathbf{q}){\text{e}}^{i\mathbf{q}·\mathbf{r}},\hfill \end{array}$$ 32

namely

$$\begin{array}{c}\hfill \mathit{W}(\mathbf{q})=i\mathbf{q}T(\mathbf{q})=i\mathbf{q}T(q),\end{array}$$ 33

assuming, as before, that $T(\mathbf{q})$ depends only on $q=|\mathbf{q}|$.

Jahn–Teller moiré phonon modes

Reference [8] pointed out the existence of a pair of high-frequency optical modes at the $\mathit{\Gamma }$ point of the MBZ, which are extremely efficient in lifting the valley degeneracies observed in the band structure. These phonon modes are schematically drawn in Fig.3a, and they both share the same modulation on the moiré length scale. However, microscopically, they both look as the well-known in-plane optical phonon modes of graphene at $\mathbf{K}$, which transform as the $A_1$ and $B_1$ irreducible representations, see Fig.3b. These two irreducible representations differ by the fact that $B_1$ is odd with respect to ${\text{C}}_{2z}$ and ${\text{C}}_{2y}$, while $A_1$ is even with respect to all symmetries of the ${\text{D}}_6$ space group.

Fig. 3.

a Real space phonon amplitude of the pair of Jahn–Teller phonon modes found in Ref. [8]. These modes, often referred to as “moiré modes”, vibrate mostly within the AA regions involving no atomic movement at all within the Bernal (AB/BA)-stacked regions. b Sketch of the microscopic vibration of these modes. The atomic movement is in-plane and behaves as two irreducible representations ($A_1$ and $B_1$) of the group ${\text{D}}_6$. c Q-vectors which connect inequivalent valleys in different layers. These vectors has been used to approximate the vibration of the modes in a and b, see (34)

Although the complexity of these modes is hard to capture by a simple analytical expression, their effect on the band structure can be well approximated introducing the following deformation fields

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill {\mathit{u}}_c^{(a)}({\mathbf{x}}_{\alpha }^{(a)})& =\sum _{i=1}^3\sum _{j=0}^2{\mathit{u}}_{\alpha }^{(a)}({\mathbf{Q}}_{\mathit{ij}})cos(({\mathbf{R}}^{(a)}+{\mathbf{r}}_{\alpha }^{(a)})·{\mathbf{Q}}_{\mathit{ij}}),\hfill \\ \hfill {\mathit{u}}_s^{(a)}({\mathbf{x}}_{\alpha }^{(a)})& =\sum _{i=1}^3\sum _{j=0}^2{\mathit{v}}_{\alpha }^{(a)}({\mathbf{Q}}_{\mathit{ij}})sin(({\mathbf{R}}^{(a)}+{\mathbf{r}}_{\alpha }^{(a)})·{\mathbf{Q}}_{\mathit{ij}}),\hfill \end{array}\end{array}$$ 34

where ${\mathbf{Q}}_{\mathit{ij}}$ are the k-vectors connecting different valleys and depicted in Fig. 3c, while $a=1,2$ is the layer index. Since the transformation ${\text{C}}_g$ ($g=3z,2x$) is a symmetry operation even in the distorted lattice, we have that

$$\begin{array}{cc}\hfill {\text{C}}_g({\mathbf{x}}_{\alpha }^{(a)})& ={\mathbf{x}}_{\beta }^{(b)}⟶{\text{C}}_g({\mathit{u}}^{(a)}({\mathbf{x}}_{\alpha }^{(a)}))={\mathit{u}}^{(b)}({\mathbf{x}}_{\beta }^{(b)}).\hfill \end{array}$$ 35

By noting that the set of momenta $\{{\mathbf{Q}}_{\mathit{ij}}\}$ is invariant under ${\text{C}}_{3z}$ and ${\text{C}}_{2x}$, it immediately follows that, for $g=3z,2x$

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill {\text{C}}_g({\mathit{u}}_{\alpha }^{(a)}({\mathbf{Q}}_{\mathit{ij}}))& ={\mathit{u}}_{\beta }^{(b)}({\text{C}}_g({\mathbf{Q}}_{\mathit{ij}})).\hfill \\ \hfill {\text{C}}_g({\mathit{v}}_{\alpha }^{(a)}({\mathbf{Q}}_{\mathit{ij}}))& ={\mathit{v}}_{\beta }^{(b)}({\text{C}}_g({\mathbf{Q}}_{\mathit{ij}})).\hfill \end{array}\end{array}$$ 36

On the contrary, $\{{\mathbf{Q}}_{\mathit{ij}}\}$ is not invariant under $C_g$ ($g=2y,2z$), and the phonon modes are either even ($A_1$) or odd ($B_1$) under these symmetries. Therefore, recalling that ${\text{C}}_{2z}$ exchanges the two sublattices,

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill {\text{C}}_{2z}({\mathit{u}}_A^{(a)}({\mathbf{Q}}_{\mathit{ij}}))& =-{\mathit{u}}_A^{(a)}({\mathbf{Q}}_{\mathit{ij}})=\pm {\mathit{u}}_B^{(a)}({\mathbf{Q}}_{\mathit{ij}}),\hfill \\ \hfill {\text{C}}_{2z}({\mathit{v}}_A^{(a)}({\mathbf{Q}}_{\mathit{ij}}))& =-{\mathit{v}}_A^{(a)}({\mathbf{Q}}_{\mathit{ij}})=\mp {\mathit{v}}_B^{(a)}({\mathbf{Q}}_{\mathit{ij}}).\hfill \end{array}\end{array}$$ 37

If we choose

$$\begin{array}{cc}\hfill {\mathit{u}}_A^{(a)}({\mathbf{Q}}_{\mathit{ij}})& ={\mathit{u}}_B^{(a)}({\mathbf{Q}}_{\mathit{ij}}),{\mathit{v}}_A^{(a)}({\mathbf{Q}}_{\mathit{ij}})={\mathit{v}}_B^{(a)}({\mathbf{Q}}_{\mathit{ij}}),\hfill \end{array}$$ 38

then the cosine distortion in (34) ${\mathit{u}}_c^{(a)}({\mathbf{x}}_{\alpha }^{(a)})$ transforms as $B_1$, while the sine one, ${\mathit{u}}_s^{(a)}({\mathbf{x}}_{\alpha }^{(a)})$, as $A_1$. They both can be shortly written as

$$\begin{array}{cc}\hfill {\mathit{u}}^{(a)}({\mathbf{x}}_{\alpha }^{(a)})& =\sum _{i=1}^3\sum _{j=0}^2[{\mathit{u}}^{(a)}({\mathbf{Q}}_{\mathit{ij}}){\text{e}}^{i({\mathbf{R}}^{(a)}+{\mathbf{r}}_{\alpha }^{(a)})·{\mathbf{Q}}_{\mathit{ij}}}+c.c.],\hfill \end{array}$$ 39

where ${\mathit{u}}^{(a)}({\mathbf{Q}}_{\mathit{ij}}){}^{\ast }\equiv {\mathit{u}}^{(a)}(-{\mathbf{Q}}_{\mathit{ij}})$ and ${\mathit{u}}^{(a)}({\mathbf{Q}}_{\mathit{ij}})$ is real for the $B_1$ distortion and imaginary for $A_1$.

We end by pointing out that $\mathit{W}(\mathbf{q})$ satisfies

$$\begin{array}{cc}\hfill {\text{C}}_g(\mathit{W}(\mathbf{q}))& =\mathit{W}({\text{C}}_g(\mathbf{q})),\hfill \end{array}$$ 40

for all symmetry operations of the lattice, in particular

$$\begin{array}{cc}\hfill {\text{C}}_{2z}(\mathit{W}(\mathbf{q}))& =-\mathit{W}(\mathbf{q})=\mathit{W}({\text{C}}_{2z}(\mathbf{q}))=\mathit{W}(-\mathbf{q})=\mathit{W}{(\mathbf{q})}^{\ast }.\hfill \end{array}$$ 41

Phonon-induced Hamiltonian matrix elements

A lattice distortion involving the $A_1$ or $B_1$ phonons generates a matrix element between layer a momentum $\mathbf{k}\sim {\mathbf{K}}^{(a)}$ and layer b momentum $\mathbf{p}\sim -{\mathbf{K}}^{(b)}$, where we recall that ${\mathbf{K}}^{(1)}={\mathbf{K}}_2$ and ${\mathbf{K}}^{(2)}={\mathbf{K}}_1$ in Fig. 1d:

$$\begin{array}{cc}& W(a,\mathbf{k},\alpha ;b,\mathbf{p},\beta )=\frac{{\displaystyle 1}}{{\displaystyle N^2}}\sum _{\mathbf{q}{\mathbf{Q}}_{\mathit{ij}}}\sum _{{\mathbf{R}}^{(a)}{\mathbf{R}}^{(b)}}\mathit{W}(-\mathbf{q})·[{\mathit{u}}^{(a)}({\mathbf{Q}}_{\mathit{ij}}){\text{e}}^{-i(\mathbf{k}+\mathbf{q}-{\mathbf{Q}}_{\mathit{ij}})·({\mathbf{R}}^{(a)}+{\mathbf{r}}_{\alpha }^{(a)})}{\text{e}}^{i(\mathbf{p}+\mathbf{q})·({\mathbf{R}}^{(b)}+{\mathbf{r}}_{\beta }^{(b)})}\hfill \\ \hfill & -{\mathit{u}}^{(b)}({\mathbf{Q}}_{\mathit{ij}}){\text{e}}^{-i(\mathbf{k}+\mathbf{q})·({\mathbf{R}}^{(a)}+{\mathbf{r}}_{\alpha }^{(a)})}{\text{e}}^{i(\mathbf{p}+\mathbf{q}+{\mathbf{Q}}_{\mathit{ij}})·({\mathbf{R}}^{(b)}+{\mathbf{r}}_{\beta }^{(b)})}]\hfill \\ \hfill & =\sum _{\mathbf{q}{\mathbf{Q}}_{\mathit{ij}}}\sum _{{\mathbf{G}}^{(a)}{\mathbf{G}}^{(b)}}\mathit{W}(-\mathbf{q})·[{\delta }_{-\mathbf{q},\mathbf{k}-{\mathbf{Q}}_{\mathit{ij}}+{\mathbf{G}}^{(a)}}{\delta }_{-\mathbf{q},\mathbf{p}+{\mathbf{G}}^{(b)}}{\mathit{u}}^{(a)}({\mathbf{Q}}_{\mathit{ij}}){\text{e}}^{i{\mathbf{G}}^{(a)}·{\mathbf{r}}_{\alpha }^{(a)}-i{\mathbf{G}}^{(b)}·{\mathbf{r}}_{\beta }^{(b)}}\hfill \\ \hfill & -{\delta }_{-\mathbf{q},\mathbf{k}+{\mathbf{G}}^{(a)}}{\delta }_{-\mathbf{q},\mathbf{p}+{\mathbf{Q}}_{\mathit{ij}}+{\mathbf{G}}^{(b)}}{\mathit{u}}^{(b)}({\mathbf{Q}}_{\mathit{ij}}){\text{e}}^{i{\mathbf{G}}^{(a)}·{\mathbf{r}}_{\alpha }^{(a)}-i{\mathbf{G}}^{(b)}·{\mathbf{r}}_{\beta }^{(b)}}].\hfill \end{array}$$ 42

We can readily follow the same steps outlined in Sect. 2 to identify the ${\mathbf{G}}^{(a)}$ and ${\mathbf{G}}^{(b)}$ reciprocal lattice vectors that enforce momentum conservation and maximise the matrix element $W(-\mathbf{q})=W(q)$. Therefore, we shall not repeat that calculation and jump directly to the results.

The lattice distortion introduces a perturbation both intra-layer and inter-layer. The former, in the representation introduced in Sect. 2.1, has the extremely simple expression:

$$\begin{array}{cc}\hfill \delta {\widehat{H}}_{x(y)}^{||}(\mathbf{k}){}_{\mathbf{Q}{\mathbf{Q}}^{\prime }}& ={\tau }_{x(y)}\sum _{i=1}^3({\delta }_{{\mathbf{Q}}^{\prime }-\mathbf{Q},{\mathbf{q}}_i}+{\delta }_{\mathbf{Q}-{\mathbf{Q}}^{\prime },{\mathbf{q}}_i}){\widehat{T}}_i(g,g^{\prime })\equiv {\tau }_{x(y)}\delta {\widehat{H}}_{\mathbf{Q}{\mathbf{Q}}^{\prime }}^{||},\hfill \end{array}$$ 43

where ${\tau }_x$ refers to the $A_1$ mode, ${\tau }_y$ to the $B_1$ one, and the matrices ${\widehat{T}}_i(g,g^{\prime })$ have the same expression as those in Eq. (14), with u and $u^{\prime }$ replaced, respectively, by g and $g^{\prime }$.

The inter-layer coupling has a simpler expression, since, as we many times mentioned, the opposite valleys in different layers fold on the same momentum in the MBZ, and thus, the coupling is diagonal in $\mathbf{Q}$ and ${\mathbf{Q}}^{\prime }$ and reads

$$\begin{array}{cc}\hfill \delta {\widehat{H}}_{x(y)}^{\perp }{(\mathbf{k})}_{\mathbf{Q}{\mathbf{Q}}^{\prime }}& ={\delta }_{\mathbf{Q},{\mathbf{Q}}^{\prime }}\gamma {\sigma }_0{\tau }_{x(y)}\equiv {\tau }_{x(y)}\delta {\widehat{H}}_{\mathbf{Q}{\mathbf{Q}}^{\prime }}^{\perp }.\hfill \end{array}$$ 44

As before, ${\tau }_x$ and ${\tau }_y$ refers to the $A_1$ and $B_1$ modes, respectively.

It is worth remarking that, because of the transformation (26), which exchanges the sublattices in the valley $\zeta =-1$, the diagonal elements of the matrices ${\widehat{T}}_i(g,g^{\prime })$ in (43) and ${\sigma }_0$ in (44) refer to the opposite sublattices, while the diagonal elements to the same sublattice, right the opposite of the unperturbed Hamiltonian (27).

Let us rephrase the above results in the second quantisation and introducing the quantum mechanical character of the phonon mode. In the continuum model, a plane wave with momentum $\mathbf{k}+\mathbf{G}$, where $\mathbf{G}=n\mathit{A}+m\mathit{B}$ is a reciprocal lattice vector of the MBZ, in layer $i=1,2$, valley $\zeta =+1$ and with sublattice components described by a two-component spinor ${\chi }_{\mathbf{k}+\mathbf{G}}$, can be associated with a two-component spinor operator according to

$$\begin{array}{c}\hfill {\text{e}}^{i(\mathbf{k}+\mathbf{G})·\mathbf{r}}{\chi }_{\mathbf{k}+\mathbf{G}}\stackrel{}{\to }{\Psi }_{+1,\mathbf{k}+\mathbf{G}}^{(i)}.\end{array}$$ 45

For any $\mathbf{G}$, we can write, see Eq. (17),

$$\begin{array}{cc}\hfill \mathbf{G}& ={\mathbf{K}}_1-{\mathbf{Q}}_A={\mathbf{K}}_2-{\mathbf{Q}}_B,\hfill \end{array}$$ 46

and thus define

$$\begin{array}{cc}\hfill {\Psi }_{+1,\mathbf{k}+\mathbf{G}}^{(1)}& ={\Psi }_{+1,\mathbf{k}+{\mathbf{K}}_2-{\mathbf{Q}}_B}^{(1)}\equiv {\Psi }_{\mathbf{k},{\mathbf{Q}}_B,+1},\hfill \\ \hfill {\Psi }_{+1,\mathbf{k}+\mathbf{G}}^{(2)}& ={\Psi }_{+1,\mathbf{k}+{\mathbf{K}}_1-{\mathbf{Q}}_A}^{(1)}\equiv {\Psi }_{\mathbf{k},{\mathbf{Q}}_A,+1}.\hfill \end{array}$$ 47

We note that ${\mathbf{K}}_{+\varphi }+{\mathbf{K}}_{-\varphi }\equiv {\mathbf{G}}_k=(2k+1)(\mathit{A}+\mathit{B})$, which allows us defining the operators in valley $\zeta =-1$ as

$$\begin{array}{cc}\hfill {\Psi }_{-1,\mathbf{k}+\mathbf{G}-{\mathbf{G}}_k}^{(1)}& ={\Psi }_{-1,\mathbf{k}-{\mathbf{K}}_{+\varphi }-{\mathbf{Q}}_A}^{(1)}\equiv {\sigma }_x{\Psi }_{\mathbf{k},{\mathbf{Q}}_A,-1},\hfill \\ \hfill {\Psi }_{-1,\mathbf{k}+\mathbf{G}-{\mathbf{G}}_k}^{(2)}& ={\Psi }_{+1,\mathbf{k}-{\mathbf{K}}_{-\varphi }-{\mathbf{Q}}_B}^{(2)}\equiv {\sigma }_x{\Psi }_{\mathbf{k},{\mathbf{Q}}_B,-1},\hfill \end{array}$$ 48

where, in accordance with our transformation in Eq. (26), we interchange the two sublattices in valley $\zeta =-1$ through ${\sigma }_x$. We note that the mismatch momentum ${\mathbf{G}}_k$ is just what is provided by the phonon modes. Absorbing the valley index into two additional components of the spinors, and introducing back the spin label, the second quantised Hamiltonian can be written in terms of four-component spinor operators ${\Psi }_{\mathbf{k}\sigma ,\mathbf{Q}}$, where, $\mathbf{Q}={\mathbf{Q}}_A$ refer to layer 2 if the valley index $\zeta =+1$ and layer 1 if $\zeta =-1$, while $\mathbf{Q}={\mathbf{Q}}_B$ to layer 1 if $\zeta =+1$, and layer 2 if $\zeta =-1$.

Next, we introduce a two-component dimensionless variable ${\mathbf{q}}_{\mathbf{0}}=(q_1,q_2)$, and its conjugate one, ${\mathbf{p}}_{\mathbf{0}}=(p_1,p_2)$, where $q_1$ and $q_2$ are the phonon coordinates of the $A_1$ and $B_1$ modes at $\mathit{\Gamma }=\mathbf{0}$, respectively. Using the above-defined operators, the full quantum mechanical Hamiltonian reads

$$\begin{array}{cc}\hfill H& =\sum _{\mathbf{k}\mathbf{Q}\sigma }[{\Psi }_{\mathbf{k}\sigma \mathbf{Q}}^{†}{\widehat{H}}_{\mathbf{Q}{\mathbf{Q}}^{\prime }}(\mathbf{k}){\Psi }_{\mathbf{k}\sigma {\mathbf{Q}}^{\prime }}+{\Psi }_{\mathbf{k}\sigma \mathbf{Q}}^{†}({\mathit{q}}_{\mathbf{0}}·\mathit{\tau }\delta {\widehat{H}}_{\mathbf{Q}{\mathbf{Q}}^{\prime }}){\Psi }_{\mathbf{k}\sigma {\mathbf{Q}}^{\prime }}]\hfill \\ \hfill & +\frac{{\displaystyle {\omega }_{\mathbf{0}}}}{{\displaystyle 2}}({\mathbf{p}}_{\mathbf{0}}·{\mathbf{p}}_{\mathbf{0}}+{\mathbf{q}}_{\mathbf{0}}·{\mathbf{q}}_{\mathbf{0}}),\hfill \end{array}$$ 49

where ${\omega }_{\mathbf{0}}$ is the phonon frequency, equal for both $A_1$ and $B_1$ modes, $\mathit{\tau }=({\tau }_x,{\tau }_y)$, and

$$\begin{array}{cc}\hfill {\widehat{H}}_{\mathbf{Q}{\mathbf{Q}}^{\prime }}(\mathbf{k})& ={\delta }_{\mathbf{Q}{\mathbf{Q}}^{\prime }}v{\tau }_z(\mathbf{k}-\mathbf{Q})·\mathit{\sigma }+{\tau }_0\sum _{i=1}^3({\delta }_{{\mathbf{Q}}^{\prime }-\mathbf{Q},{\mathbf{q}}_i}+{\delta }_{\mathbf{Q}-{\mathbf{Q}}^{\prime },{\mathbf{q}}_i}){\widehat{T}}_i(u,u^{\prime }),\hfill \\ \hfill \delta {\widehat{H}}_{\mathbf{Q}{\mathbf{Q}}^{\prime }}& =\delta {\widehat{H}}_{\mathbf{Q}{\mathbf{Q}}^{\prime }}^{||}+\delta {\widehat{H}}_{\mathbf{Q}{\mathbf{Q}}^{\prime }}^{\perp }=\sum _{i=1}^3({\delta }_{{\mathbf{Q}}^{\prime }-\mathbf{Q},{\mathbf{q}}_i}+{\delta }_{\mathbf{Q}-{\mathbf{Q}}^{\prime },{\mathbf{q}}_i}){\widehat{T}}_i(g,g^{\prime })+{\delta }_{\mathbf{Q},{\mathbf{Q}}^{\prime }}\gamma .\hfill \end{array}$$ 50

We observe that the Hamiltonian (49) still possesses a valley $U_v(1)$ symmetry, with generator

$$\begin{array}{cc}\hfill J_z& =\frac{{\displaystyle 1}}{{\displaystyle 2}}\sum _{\mathbf{k}\mathbf{Q}\sigma }{\Psi }_{\mathbf{k}\sigma \mathbf{Q}}^{†}{\sigma }_0{\tau }_z{\Psi }_{\mathbf{k}\sigma \mathbf{Q}}+{\mathbf{q}}_{\mathbf{0}}\wedge {\mathbf{p}}_{\mathbf{0}}\equiv T_z+L_z,\hfill \end{array}$$ 51

where $T_z$ is half the difference between the number of electrons in valley $\zeta =+1$ and the one in valley $\zeta =-1$, while $L_z$ is the angular momentum of the phonon mode. The Hamiltonian (49) actually realises a $e\otimes E$ Jahn–Teller model.

It is straightforward to generalise the above result to an atomic displacement modulated with the wave vectors ${\mathbf{Q}}_{\mathit{ij}}+\mathit{P}$, where $\mathit{P}\in \text{MBZ}$. Since ${\mathbf{Q}}_{\mathit{ij}}$ are multiples of the MBZ reciprocal lattice vector, such displacement is at momentum $\mathit{P}$ and can be considered as the previous one at $\mathit{\Gamma }$, as shown in Fig. 3, on top of which we add an additional incommensurate long wavelength component. Since $\mathit{P}$ is tiny as compared to the vectors ${\mathbf{Q}}_{\mathit{ij}}$, we shall assume that the displacement has the same expression of Eq. (39), with the only difference that

$$\begin{array}{cc}\hfill {\text{e}}^{\pm i({\mathbf{R}}^{(a)}+{\mathbf{r}}_{\alpha }^{(a)})·{\mathbf{Q}}_{\mathit{ij}}}& \stackrel{}{\to }{\text{e}}^{\pm i({\mathbf{R}}^{(a)}+{\mathbf{r}}_{\alpha }^{(a)})·({\mathbf{Q}}_{\mathit{ij}}\pm \mathit{P})}.\hfill \end{array}$$ 52

The full quantum mechanical Hamiltonian becomes

$$\begin{array}{cc}\hfill H& =\sum _{\mathbf{k}\mathbf{Q}\sigma }{\Psi }_{\mathbf{k}\sigma \mathbf{Q}}^{†}{\widehat{H}}_{\mathbf{Q}{\mathbf{Q}}^{\prime }}(\mathbf{k}){\Psi }_{\mathbf{k}\sigma {\mathbf{Q}}^{\prime }}+\sum _{\mathbf{k}\mathbf{Q}\mathit{P}\sigma }{\Psi }_{\mathbf{k}\sigma \mathbf{Q}}^{†}({\mathit{q}}_{-\mathit{P}}·\mathit{\tau }\delta {\widehat{H}}_{\mathbf{Q}{\mathbf{Q}}^{\prime }}(\mathit{P})){\Psi }_{\mathbf{k}+\mathit{P}\sigma {\mathbf{Q}}^{\prime }}\hfill \\ \hfill & +\frac{{\displaystyle 1}}{{\displaystyle 2}}\sum _{\mathit{P}}{\omega }_{\mathit{P}}({\mathbf{p}}_{\mathit{P}}·{\mathbf{p}}_{-\mathit{P}}+{\mathbf{q}}_{\mathit{P}}·{\mathbf{q}}_{-\mathit{P}}),\hfill \end{array}$$ 53

where $\delta {\widehat{H}}_{\mathbf{Q}{\mathbf{Q}}^{\prime }}(\mathit{P})$ is the same as $\delta {\widehat{H}}_{\mathbf{Q}{\mathbf{Q}}^{\prime }}$ in Eq. (50) with $\mathit{P}$-dependent constants $g_{\mathit{P}}$, $g_{\mathit{P}}^{\prime }$ and ${\gamma }_{\mathit{P}}$, invariant under the little group at $\mathit{P}$. In this general case, the generator of $U_v(1)$ reads

$$\begin{array}{cc}\hfill J_z& =\frac{{\displaystyle 1}}{{\displaystyle 2}}\sum _{\mathbf{k}\mathbf{Q}\sigma }{\Psi }_{\mathbf{k}\sigma \mathbf{Q}}^{†}{\sigma }_0{\tau }_z{\Psi }_{\mathbf{k}\sigma \mathbf{Q}}+\sum _{\mathit{P}}{\mathbf{q}}_{\mathit{P}}\wedge {\mathbf{p}}_{-\mathit{P}}.\hfill \end{array}$$ 54

Frozen phonon band structure

We can perform a frozen phonon calculation neglecting the phonon energy, last term in Eq. (49), and fixing $\mathbf{q}=(q_1,q_2)$ to some constant value. Because of the $U_v(1)$ symmetry, what matters is just the modulus q of $\mathbf{q}$. In practice, we have taken $\mathbf{q}=(1,0)$ and studied the band structure varying the coupling constants g, setting $g^{\prime }=g/10$ and $\gamma =g/2.5$, and assuming the following parameters: $ħv/a_0=2.1354eV$ [29]; $u=0.0761eV$ and $u^{\prime }=0.1031eV$ [14]. This choice fits well the microscopic tight-binding calculations in Ref. [8]. As shown in Fig. 4, as soon as the frozen phonon terms are turned on, all the degeneracies in the band structure arising due to the valley symmetry are lifted. This occurs with a set of avoided crossings which move from $\mathbf{K}\to \mathbf{M}$ and from $\mathbf{K}\to \mathit{\Gamma }$ (Fig. 4b). In particular, the crossings that move from $\mathbf{K}\to \mathbf{M}$ eventually meet at $\mathbf{M}$, forming (six) Dirac nodes, which then move towards $\mathit{\Gamma }$ (Fig. 4c). Finally, at a threshold value of g, a gap opens at the charge neutrality point (Fig. 4d). Such gap keeps increasing as the deformation amplitude increases.

Fig. 4.

Band structure of $2\varphi =1.{08}^{\circ }$ twisted bilayer graphene with increasing frozen phonon deformation intensity g. The other parameters used in this calculation are: $u=0.0761$, $u^{\prime }=0.1031$, $g^{\prime }=g/10$ and $\gamma =g/2.5$. a Undistorted case. b Slightly distorted lattice. Two small avoided crossings occur along the ${\mathbf{K}}_1\to \mathit{\Gamma }$ and $\mathbf{M}\to \mathit{\Gamma }$ lines. c By further increasing the distortion intensity, the four bands further separate. The avoided crossing along ${\mathbf{K}}_1\to \mathit{\Gamma }$ persists, while a genuine crossing occurs along $\mathit{\Gamma }\to \mathbf{M}$. d A gap at charge neutrality has finally opened

Moiré phonons at $\mathbf{M}$

The phonon modes considered in the previous section were at position $\mathit{\Gamma }$ of the MBZ, thus preserving the periodicity of the moiré superlattice. As pointed out in Ref. [8] and shown before, these modes are able to open a gap in the band structure only at charge neutrality. Gap opening at different commensurate fillings requires freezing finite momentum phonons [8]. Here, we consider a multicomponent distortion which involves the modes at the three inequivalent $\mathbf{M}$ points in the MBZ:

$$\begin{array}{c}\hfill {\mathbf{M}}_1=\frac{{\displaystyle \mathit{A}}}{{\displaystyle 2}},{\mathbf{M}}_2={\text{C}}_{3z}({\mathbf{M}}_1)=\frac{{\displaystyle \mathit{B}}}{{\displaystyle 2}},{\mathbf{M}}_3={\text{C}}_{3z}({\mathbf{M}}_2)=-\frac{{\displaystyle \mathit{A}+\mathit{B}}}{{\displaystyle 2}}.\end{array}$$ 55

Freezing a multiple distortion at all these points reduces by a quarter the Brillouin zone, see Fig. 5, which has now the reciprocal lattice vectors

$$\begin{array}{c}\hfill {\mathit{A}}^{\prime }={\mathbf{M}}_1,{\mathit{B}}^{\prime }={\mathbf{M}}_2.\end{array}$$ 56

Since ${\mathbf{M}}_i$, $i=1,2,3$, are tiny as compared to the vectors ${\mathbf{Q}}_{\mathit{ij}}$ introduced in the previous section, we can make the same assumption (52) that leads to the Hamiltonian (53); namely, assume that the displacement induced by the multiple distortion has the same expression of Eq. (39), with the only difference that

$$\begin{array}{c}\hfill {\text{e}}^{i({\mathbf{R}}^{(a)}+{\mathbf{r}}_{\alpha }^{(a)})·{\mathbf{Q}}_{\mathit{ij}}}⟶\sum _{n=1}^6{\text{e}}^{i({\mathbf{R}}^{(a)}+{\mathbf{r}}_{\alpha }^{(a)})·({\mathbf{Q}}_{\mathit{ij}}+{\mathbf{p}}_n)}\end{array}$$ 57

where

$$\begin{array}{c}\hfill {\mathbf{p}}_1=-{\mathbf{p}}_4={\mathit{M}}_1={\mathit{A}}^{\prime },{\mathbf{p}}_2=-{\mathbf{p}}_5={\mathit{M}}_2={\mathit{B}}^{\prime },{\mathbf{p}}_3=-{\mathbf{p}}_6={\mathit{M}}_3=-{\mathit{A}}^{\prime }-{\mathit{B}}^{\prime }\end{array}$$ 58

are the additional long wavelength modulation vectors on top of the leading short wavelength ones at ${\mathbf{Q}}_{\mathit{ij}}$. The vectors ${\mathbf{q}}_i$ defined in Eq. (18) can be written in terms of the new reciprocal lattice vectors ${\mathit{A}}^{\prime }$ and ${\mathit{B}}^{\prime }$ as

$$\begin{array}{c}\hfill {\mathbf{q}}_1=\frac{{\displaystyle 2}}{{\displaystyle 3}}({\mathit{A}}^{\prime }-{\mathit{B}}^{\prime }),{\mathbf{q}}_2=\frac{{\displaystyle 2}}{{\displaystyle 3}}({\mathit{A}}^{\prime }+2{\mathit{B}}^{\prime }),{\mathbf{q}}_3=\frac{{\displaystyle 2}}{{\displaystyle 3}}(-2{\mathit{A}}^{\prime }-{\mathit{B}}^{\prime }).\end{array}$$ 59

Both ${\mathbf{p}}_i$ and ${\mathbf{q}}_i$, $i=1,2,3$, are shown in Fig. 5. Considering all momenta $\mathbf{k}$ within the new BZ, the light blue hexagon in Fig. 5, and assuming that, besides the multicomponent distortion at $\mathbf{M}$, there is still a distortion at $\mathit{\Gamma }$, the Hamiltonian can be written again as a matrix ${\widehat{H}}_{\mathbf{Q}{\mathbf{Q}}^{\prime }}(\mathbf{k})$, which now reads

$$\begin{array}{cc}\hfill {\widehat{H}}_{\mathbf{Q}{\mathbf{Q}}^{\prime }}(\mathbf{k})=& {\delta }_{\mathbf{Q},{\mathbf{Q}}^{\prime }}v{\tau }_z(\mathbf{k}-\mathbf{Q})·\mathit{\sigma }+{\tau }_0\sum _{i=1}^3({\delta }_{{\mathbf{Q}}^{\prime }-\mathbf{Q},{\mathbf{q}}_i}+{\delta }_{\mathbf{Q}-{\mathbf{Q}}^{\prime },{\mathbf{q}}_i}){\widehat{T}}_i(u,u^{\prime })\hfill \\ \hfill & +\gamma {\delta }_{\mathbf{Q},{\mathbf{Q}}^{\prime }}{\tau }_x+{\tau }_x\sum _{i=1}^3({\delta }_{{\mathbf{Q}}^{\prime }-\mathbf{Q},{\mathbf{q}}_i}+{\delta }_{\mathbf{Q}-{\mathbf{Q}}^{\prime },{\mathbf{q}}_i}){\widehat{T}}_i(g,g^{\prime })\hfill \\ \hfill & +\alpha {\tau }_x\sum _{i=1}^6({\delta }_{{\mathbf{Q}}^{\prime }-\mathbf{Q},{\mathbf{p}}_i}+{\delta }_{\mathbf{Q}-{\mathbf{Q}}^{\prime },{\mathbf{p}}_i})+{\tau }_x\sum _{i=1}^3\sum _{j=1}^6({\delta }_{{\mathbf{Q}}^{\prime }-\mathbf{Q},{\mathbf{q}}_i+{\mathbf{p}}_j}\hfill \\ \hfill & +{\delta }_{\mathbf{Q}-{\mathbf{Q}}^{\prime },{\mathbf{q}}_i+{\mathbf{p}}_j}){\widehat{T}}_i(a,a^{\prime }),\hfill \end{array}$$ 60

Fig. 5.

Reduced Brillouin zones with the multicomponent distortion at $\mathbf{M}$, shown as coloured hexagons, while the four times larger hexagon drawn with a dashed line is the undistorted MBZ. The vectors ${\mathbf{p}}_i$, black arrows, and ${\mathbf{q}}_i$, blue arrow, for $i=1,\cdots ,3$, are also shown. Note that ${\mathbf{p}}_1={\mathit{A}}^{\prime }$ and ${\mathbf{p}}_2={\mathit{B}}^{\prime }$ are also the reciprocal lattice vectors of the reduced Brillouin zone. The black and red circles are the positions of the sublattice vectors ${\mathbf{Q}}_A$ and ${\mathbf{Q}}_B$, respectively, defined in Eq. (61)

where the matrices ${\widehat{T}}_i(x,x^{\prime })$ are those in Eq. (14), though they depend on different sets of parameters, $(u,u^{\prime })$, $(g,g^{\prime })$ and $(a,a^{\prime })$. The crucial difference with respect to the Hamiltonian (50) with only the $\mathit{\Gamma }$-distortion is that the $\mathbf{Q}$ vectors span now the sites of the honeycomb lattice generated by the new fourfold-smaller Brillouin zone; hence, they are defined through

$$\begin{array}{cc}\hfill \mathbf{Q}& =\{{\mathbf{Q}}_A^{\prime },{\mathbf{Q}}_B^{\prime }\}=\left\{\begin{array}{c}{\mathbf{Q}}_A^{\prime }=\frac{{\displaystyle {\mathit{A}}^{\prime }-{\mathit{B}}^{\prime }}}{{\displaystyle 3}}+n{\mathit{A}}^{\prime }+m{\mathit{B}}^{\prime },\hfill \\ {\mathbf{Q}}_B^{\prime }=-\frac{{\displaystyle {\mathit{A}}^{\prime }-{\mathit{B}}^{\prime }}}{{\displaystyle 3}}+n{\mathit{A}}^{\prime }+m{\mathit{B}}^{\prime },\hfill \end{array}\right)\hfill \end{array}$$ 61

and shown in Fig. 5 as black and red circles, respectively, and must not be confused with those in Eq. (17). In Fig. 6, we show the density of states around neutrality of the Hamiltonian (60). The first two cases correspond to undistorted and $\Gamma$-only distorted structures, while the third panel involves also the M multicomponent distortion. As can be seen, a gap now opens at the partial filling of one electron per unit cell. As it was shown in Ref. [8], other phonons or combinations of them can open gaps at any integer filling of the four electronic flat bands.

Fig. 6.

Density of states around charge neutrality obtained with the Hamiltonian (60) with $g^{\prime }=g/10$, $\gamma =g/2.5$, $a^{\prime }=a/10$ and $\alpha =a/2.5$. Gaps are highlighted in red, and the corresponding filling factor is $\nu$. a) The undistorted lattice density of states. b) The density of states obtained by deforming the lattice with only the distortion at $\mathit{\Gamma }$, which opens a gap at charge neutrality. c) Density of states obtained using both the $\Gamma$ and the $\mathbf{M}$ multicomponent distortions. Here, a gap opens at filling of one electron (three holes with respect to neutrality) per unit cell

Conclusions

We have shown that the moiré phonons of Ref. [8], which are coupled to the valley degrees of freedom of the electrons so to realise an $E\otimes e$ Jahn–Teller model, can be successfully implemented in the continuum model formalism of small-angle twisted bilayer graphene. This method is more manageable than the realistic tight-binding modelling of Ref. [8], whose results have been here reproduced with much less effort. In addition, the continuum model formalism has the great advantage of providing a full quantum mechanical expression of the electron–phonon Hamiltonian, which may allow going beyond the simple frozen phonon calculation of [8], and thus describing phenomena like a dynamical Jahn–Teller effect and the phonon-mediated superconductivity.