Modulation of Radiative Heat Transfer at the Nanoscale via Topological Polaritons in Twisted van der Waals Crystals

ABSTRACT

Twisted layers of α‐MoO₃ support phonon polaritons whose propagation can be adjusted by the twist angle, a concept known as ‘twistoptics’. Although emergent in the field of nano‐optics, the application to heat transfer has lagged behind, particularly regarding near‐field radiative heat transfer (NFRHT), which is important for thermal management in nanodevices and remains insufficiently explored. Here, we report the role of twistoptics in NFRHT, demonstrating that the heat flux between two separated twisted α‐MoO₃ bilayers can be monotonically increased by simply increasing the twist angle. Interestingly, this modulation is explained by the emergence of topological transitions from open (hyperbolic) to closed (elliptical) polaritonic dispersions. This phenomenon is further demonstrated by considering α‐MoO₃ gapped trilayers, which show greater flexibility in regulating the NFRHT due to the emergence of a wider variety of topological transitions. Based on these findings, we propose an experimental scenario where the NFRHT between a nanoparticle and a closely spaced twisted α‐MoO₃ bilayer can be modulated by a factor of 3 by simply adjusting the twist angle. This work provides theoretical guidance for the modulation of NFRHT using twistoptics, making an important step toward the development of twisted thermotics.

Twisted α‐MoO₃ layers support phonon polaritons whose propagation depends on the twist angle, enabling control of near field radiative heat transfer. Increasing the angle yields a monotonic rise in heat flux, driven by topological transitions from open hyperbolic to closed elliptical dispersions.

1. Introduction

Twisted stacks of van der Waals (vdW) crystals have emerged as an exceptional material platform for the discovery and controlled study of exotic optical phenomena [1, 2, 3, 4, 5, 6, 7, 8, 9]. A recent example is PhP canalisation in twisted bilayers and trilayers of the vdW semiconductor α‐MoO₃ [10]. This phenomenon, characterised by the unidirectional propagation of the energy flux at the nanoscale, has its origin in the emergence of optical topological transitions, defined as the transition of polariton propagation from hyperbolic to circular (or elliptical), or vice versa [11, 12]. Beyond canalisation, recent studies show that twisted α‐MoO₃ bilayers can also modulate hybrid polaritons, intrinsic chirality, Cherenkov radiation, and spontaneous emission, enriching the broader picture of twist‐engineered light–matter interactions [13, 14, 15, 16]. It is striking that the study of topological polaritons has predominantly focused on their propagative electromagnetic nature, while their role in radiative heat transfer, especially in the near field, remains largely unexplored. When the separation distance between two objects is comparable or smaller than the characteristic wavelength of thermal radiation, forward and backward evanescent waves can couple to each other, thus establishing pathways for photon tunnelling. This process gives rise to the near‐field radiative heat transfer (NFRHT) phenomenon, which can exhibit values significantly higher than those dictated by the blackbody limit stipulated by Planck's law of traditional radiation [17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29]. Importantly, this enhancement can be leveraged in various applications, such as thermophotovoltaic devices, thermal rectification, and thermal switches [30, 31, 32, 33, 34, 35, 36, 37, 38].

In particular, NFRHT between twisted structures separated by a vacuum gap has been addressed in the past [39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49], observing that the radiative heat flux shows a consistent decrease with increasing twist angle for monolayers (maximum when they are equally oriented) or a prominent enhancement near a given twist angle for bilayers. Such enhancement has been attributed to the excitation of surface plasmon polaritons undergoing a topological transition in their propagation. Despite these initial studies, key aspects for the optimisation of NFRHT using twisted layers remain to be elucidated, such as the precise role of the twist angle or the use of structures with more than two polaritonic thin layers.

In the present study, we introduce twisted van der Waals (vdW) multilayers composed of pairs of bilayers or pairs of trilayers separated by a vacuum gap as a material platform to efficiently modulate the heat transfer between them. Specifically, the results obtained show a marked monotonic increase of the NFRHT with increasing twist angle, reaching a maximum when the gapped bilayers, or gapped trilayers, are orthogonally aligned. By studying the resulting NFRHT as a function of layer thickness, we show that this finding originates from the occurrence of polaritonic topological transitions towards circular, or elliptical, dispersions (characterised by a closed isofrequency curve, IFC [1], in momentum space), which exhibit a higher density of optical states (DOS). Based on these results, we propose an experimental scheme in which the heat flux between a nanoparticle and a closely spaced twisted bilayer (50 nm) can be modulated up to a factor of 3 by simply varying the twist angle. These results pave the way for the development of nanodevices capable of efficient and reliable heat management, with potential applications in advanced thermal regulation systems.

Figure 1a shows a schematic representation of the initially proposed NFRHT system based on gapped bilayers. The emitter and receiver, consisting of mirror symmetric twisted α‐MoO₃ bilayers with thickness t, are considered in vacuum and separated a distance d = 50 nm. The bottom layers (labelled 1 and 4) are twisted simultaneously with an angle β. The radiative heat flux between the α‐MoO₃ layers is calculated based on the fluctuation‐dissipation theory [17].

$$P=\frac{1}{8{\pi }^3}{\int }_0^{\infty }\left[\mathrm{\Theta }\left(\omega ,T_{\mathrm{e}}\right)-\mathrm{\Theta }\left(\omega ,T_{\mathrm{r}}\right)\right]d\omega {\int }_0^{2\pi }{\int }_0^{\infty }\xi \left(\omega ,k,\varphi \right)kdkd\varphi ,$$ (1)

where $\mathrm{\Theta }\left(\omega ,T\right)=\mathrm{\hslash }\omega /\left[\exp \left(\mathrm{\hslash }\omega /k_{\mathrm{B}}T\right)-1\right]$ is the mean energy of a Planck oscillator at frequency ω and temperature T. $\xi \left(\omega ,k,\varphi \right)$ is the energy transfer coefficient (ETC), which is given by the following equation:

$$\xi \left(\omega ,k,\varphi \right)=\left\{\begin{array}{l}\text{Tr}\left[\left(\mathbf{I}-{\mathbf{R}}_{12}^{\ast }{\mathbf{R}}_{12}-{\mathbf{T}}_{12}^{\ast }{\mathbf{T}}_{12}\right)\mathbf{D}\left(\mathbf{I}-{\mathbf{R}}_{34}^{\ast }{\mathbf{R}}_{34}-{\mathbf{T}}_{34}^{\ast }{\mathbf{T}}_{34}\right){\mathbf{D}}^{\ast }\right],k<k_0\\ \text{Tr}\left[\left({\mathbf{R}}_{12}^{\ast }-{\mathbf{R}}_{12}\right)\mathbf{D}\left({\mathbf{R}}_{34}-{\mathbf{R}}_{34}^{\ast }\right){\mathbf{D}}^{\ast }\right]e^{-2\left|k_z\right|d},k>k_0\end{array}\right.,$$ (2)

where $k=\sqrt{k_x^2+k_y^2}$ is the surface parallel wavevector, and k ₀ = ω/c is the wavevector in vacuum. $k_z=\sqrt{k_0^2-k^2}$ is the surface perpendicular wavevector, * signifies the complex conjugate, R _12/34 and T _12/34 represent the reflection/transmission coefficients of the emitter and receiver, and $\mathbf{D}={\left(\mathbf{I}-{\mathbf{R}}_{12}{\mathbf{R}}_{34}{e}^{2i{k}_{z}d}\right)}^{-1}$. Throughout this work, the coordinate axes are aligned with the main crystalline axes of the upper layer (labelled 2 and 3), that is, the x (y, z) axis is fixed to the [100] ([001], [010]) direction of the upper α‐MoO₃ layer. Specifically, changes in twist angles are understood as a rotation of the lower layer with respect to the [100] direction of the upper α‐MoO₃ layer.

FIGURE 1.

(a) Schematic representation of the NFRHT between a twisted α‐MoO₃ bilayer (emitter) set at a temperature T _e = 320 K and a mirror‐symmetric structure (receiver) spaced a distance d = 50 nm apart and set at a temperature T _r = 300 K. The thickness of all α‐MoO₃ layers is the same, as indicated by t. (b) NFRHT calculated as a function of twist angle β for different layer thicknesses t.

Figure 1b shows the calculated radiative heat flux between the gapped α‐MoO₃ bilayers as a function of the layer thickness t and the twist angle β. We observe that for the thicker layers (t = 100 nm and t = 200 nm), the radiative heat flux hardly varies with the twist angle. However, when the layer thickness is reduced to 20 nm, we observe a clear increase of NFRHT with β up to a value of 50°. This effect is most prominent when considering 10 nm thick layers, for which the NFRHT exhibits a clear maximum at β = 90° (corresponding to a 17% increase). These results convincingly demonstrate a twist angle‐dependent NFRHT for thin vdW layers, opening the door to efficient heat transfer control. Note that these results differ significantly from previous calculations on purely 2D material platforms, such as graphene gratings [38], which show the enhancement effect of the topological transitions of the surface state at the photonics magic angle.

To better explore the physical origin of these results, particularly in the case of thin layers, we plot the NFRHT power density as a function of frequency, P _ω . The resulting graphs for t = 10 nm (Figure 2a) show high values in three well‐defined spectral bands (labelled I, II, and III), which correspond exactly to those in which α‐MoO₃ exhibits negative permittivity [50] (normally referred to as Reststrahlen bands, RB) and, therefore, supports surface PhPs. Interestingly, the spectral distribution of P _ω varies significantly with β within RBI and RBII. Specifically, it evolves from a relatively constant value for β = 0° to show a double peak function for β = 30° and 60° and, finally, a maximum for β = 90° (at frequencies of 651 and 894 cm⁻¹ in RB I and RB II, respectively). Notably, the spectral concentration at the maxima for β = 90° is accompanied by an increase in the power density of up to a factor of 7.2. On the other hand, P _ω remains practically unchanged within RB III, showing in all cases a single peak at ω = 998 cm⁻¹.

FIGURE 2.

(a) NFRHT as a function of frequency for twist angles β = 0°, β = 30°, β = 60°, and β = 90° (blue, red, orange, and violet curves, respectively). The thicknesses t of both α‐MoO₃ layers are 10 nm. (b–m) Energy transfer coefficient $\xi \left(\omega ,k,\varphi \right)$ in k‐space for β = 0°, β = 30°, β = 60°, β = 90° (row 1–4, respectively) at frequencies ω = 651 cm⁻¹ (b–e), ω = 894 cm⁻¹ (f–i), and ω = 998 cm⁻¹ (j–m). The green lines indicate the IFCs of the bilayer α‐MoO₃. The cyan and white dashed lines indicate the IFCs of the top layer (i.e., without twist) and bottom layer (with twist) of the twisted bilayer α‐MoO₃.

To interpret these results, we calculate the energy transfer coefficient $\xi \left(\omega ,k,\varphi \right)$ (defined in Equation (2)) as a function of twist angle β and frequency (contour plots in Figure 2b–m). Note that this coefficient takes values in the range of 0–1 and accounts for p‐polarised waves (the contribution from s‐polarised waves is negligible). For the maximum of P _ω at ω = 651 cm⁻¹, that is, within the RBI, we observe that $\xi$ shows a hyperbolic contour that transitions to a square‐like curve as β increases, accompanied by a sharp increase in intensity. Interestingly, for β = 60°, the propagation of hyperbolic PhPs, indicated by their isofrequency contour IFC (green line in Figure 2d), becomes highly directional or canalised [1, 2, 3, 4] (flattened IFC). This polariton propagation enhances the radiative heat flux (colour plot) along a narrow angular region in k‐space. When β = 90°, the distribution of energy transfer coefficients in k‐space takes on a quasi‐square shape, as shown in Figure 2e. The value of energy transfer coefficients approaches 1, indicating highly efficient radiative heat flux transfer through hyperbolic PhPs between the pairs of bilayers. Furthermore, the energy transfer coefficients cover a broader range in k‐space compared to other twist angles, resulting in the highest radiative heat flux after integration. However, the total near‐field radiative heat flux is given by integrating the spectral heat flux over all frequencies and all parallel wavevectors. Thus, even though ξ at the resonance frequency changes significantly, that resonance band contributes only a portion of the full spectral integral. This same behaviour of ξ as a function of β is observed for the maximum of P _ω at ω = 894 cm⁻¹, that is, within RB II, as shown in Figure 2f–i. As the twist angle increases, hyperbolic PhPs undergo a topological transition from an open (hyperbolic) to a closed (elliptical) IFC (green lines), resulting in a gradual enhancement of the heat flux. Although hyperbolic contours are often expected to outperform elliptical ones in NFRHT, the comparison is subtle and strongly parameter‐dependent. As shown by Liu and Zhang [51], losses and geometric cutoffs limit high‐k contributions, so hyperbolicity offers no universal benefit. Because ref. [51] compares different geometries, its cases are not directly comparable. In our study, only the twist angle varies, allowing a fair comparison between hyperbolic and elliptical regimes.

In sharp contrast, for the maximum of P _ω at ω = 998 cm⁻¹, the energy transfer coefficient in k‐space does not undergo any topological transition as a function of the twist angle, resulting in a minimal variation of the total heat flux (Figure 2j–m).

Taken together, these results unambiguously demonstrate: (i) the appearance of a topological transition in gapped bilayers as a function of the twist angle and (ii) that this transition involves a modulation and eventual enhancement of $\xi$ for a closed contour, which explains the sharp increase in P _ω (Figure 2a). Note that the same conclusions can be derived when considering bulky bottom media, as shown in Figures S2 and S3. It is important to note that the mechanism in plate–plate systems differs fundamentally from plate‐mediated nanoparticle interactions. For planar slabs, heat transfer is obtained by integrating the energy transmission coefficient over k‐space, whereas for nanoparticles above a plate, it is mainly governed by mode matching between the plate's polariton dispersion and the particles' positions [44, 49]. Consequently, a twist‐induced topological change affects NFRHT in a distinct manner.

Interestingly, Figure 2b–m reveals two separated branches of ξ at low momentum values. This observation seems to indicate the formation of two distinct polaritonic modes as a result of the electromagnetic coupling between the gapped bilayers (note that surface polaritons decay exponentially in the direction perpendicular to their propagation and this decay is proportional to 1/k). This is corroborated by observing that the IFCs for isolated twisted bilayers (green lines) lie between these branches, indicating that the origin of the latter is rooted in the presence of, and therefore coupling with, another twisted bilayer. The formation of symmetric–antisymmetric coupled modes has been reported previously for the case of surface polaritons in SiC [52]. However, to the best of our knowledge, there are no reports of such coupled modes for the case of hyperbolic volume polaritons, nor of their dependence on the twist angle.

To potentially expand the capabilities of twisted gapped stacks for controlling NFRHT, we further study α‐MoO₃ gapped trilayers (Figure 3a), which has been shown to exhibit a large variety of polaritonic topological transitions [2]. As before, we consider layers of the same thickness t, whereas the twist angles, β ₁₂ and β ₁₃, are defined now between the [001] crystal direction of the top layer and the [001] crystal direction of the middle and bottom layers, respectively. By calculating the normalised radiative heat flux η [defined as (P − P _min)/P _min] between the trilayers (Figure 3b), we observe a clear dependence on both twist angles. Specifically, η is minimal for β ₁₂ = β ₁₃ = 0°, and reaches a maximum for β ₁₂ = β ₁₃ = 90°, with an overall enhancement of more than 20%. To explain this result, we plot in Figure 3c the energy transfer coefficient $\xi$ in k‐space as a function of twist angles for the maximum of P _ω (see Figure S5) at ω = 633 cm⁻¹. For the minimum of η at β ₁₂ = β ₁₃ = 0°, the contour of $\xi$ is hyperbolic, corresponding to the excitation of hyperbolic PhPs as shown by the IFC contour (green lines). As the twist angle increases, $\xi$ and the polaritonic IFC gradually transition toward a closed contour. Specifically, for the three cases corresponding to β ₁₂ = 0° and β ₁₃ = 90°, β ₁₂ = 90° and β ₁₃ = 0°, β ₁₂ = 90° and β ₁₃ = 90°, $\xi$ exhibits closed elliptical contours, which are similar to the polaritonic IFCs previously observed for twisted trilayers [2]. More importantly, in these three cases $\xi$ appears noticeably broader in k‐space and exhibits a maximum value (close to 1), which in turn leads to an enhanced radiative heat flux.

FIGURE 3.

(a) Schematic representation of the NFRHT between a twisted α‐MoO₃ trilayer (emitter) set at a temperature T _e = 320 K and a mirror‐symmetric structure (receiver) separated by a distance d = 50 nm and set at a temperature T _r = 300 K. The thickness of all α‐MoO₃ layers is the same (t = 5 nm). (b) The normalised NFHT varies with twist angles β ₁₂ and β ₁₃. The thicknesses of both α‐MoO₃ layers are 5 nm. (c) The energy transfer coefficients distribution in wavevector space. The frequency is ω = 633 cm⁻¹. The twist angles are β ₁₃ = 0°, β ₁₃ = 30°, β ₁₃ = 60°, β ₁₃ = 90° corresponding to rows 1–4, and β ₁₂ = 0°, β ₁₂ = 30°, β ₁₂ = 60°, β ₁₂ = 90° corresponding to columns 1–4. The green lines are the IFCs of the α‐MoO₃ trilayer. The cyan, white, and magenta dashed lines are the IFCs of the top layer (without twist), middle layer (with twist), and bottom layer (with twist) in the twisted trilayer.

To further highlight the potential of topological transitions in twisted stacks to modulate the NFRHT, we study the case of an α‐MoO₃ bilayer in close proximity to a nanoparticle (NP). First, we calculate the spectral distribution of the local density of states (LDOS) of the α‐MoO₃ bilayer for different twist angles (Figure 4a). We observe that it closely resembles that of the spectral radiative heat flux shown in Figure 2a, varying significantly with the twist angle and showing a strong peak in the RB I at ω = 637 cm⁻¹. Next, we consider a NP with a strong LDOS response at this maximum, which is obtained by designing its dielectric function.

$$\epsilon \left(\omega \right)=1-\frac{{\omega }_p^2}{{\omega }^2+i\mathrm{\Gamma }\omega }.$$ (3)

FIGURE 4.

(a) LDOS of the α‐MoO₃ bilayer as a function of frequency for different twist angles β = 0°, β = 30°, β = 60°, and β = 90° (blue, red, orange, and violet curves, respectively). (b) NFRHT between the α‐MoO₃ bilayer and a NP as a function of twist angle. Inset: schematic representation of the system considered involving a twisted α‐MoO₃ bilayer, set as emitter at a temperature T _e = 320 K, and a NP, set as receiver at a temperature T _r = 300 K, separated by a distance d = 50 nm. The thickness of all α‐MoO₃ layers is the same (t = 5 nm). Imaginary part of the Green's function distribution in k‐space at ω = 637 cm⁻¹ for β = 0° (c), β = 30° (d), β = 60° (e), β = 90° (f). The green lines indicate the IFCs of the α‐MoO₃ bilayer. The cyan and white dashed lines indicate the IFCs of the top layer (without twist) and bottom layer (with twist) of the twisted bilayer.

Specifically, we ensure that the NP resonance at ε(ω) = −2 coincides spectrally with the maximum of LDOS in the bilayer. Disregarding losses, this occurs for ω _p  = 1103 cm⁻¹ and Γ = 0.01ω _p . Finally, we calculate the NFRHT between the bilayer and the NP for a separation distance of 50 nm (Figure 4b). We observe that the total radiative heat flux P increases monotonically with the twist angle, reaching a value up to 3 times higher for β = 90° compared to β = 0°. To investigate the mechanism of this large enhancement, we examine the imaginary part of the Green's function (proportional to the radiative heat flux) for the bilayer‐NP system in k‐space (at the resonance frequency). The Green's function can be obtained from the following equation [53]:

$${\mathbf{G}}_{\mathbf{k}}\left(\omega ,k_x,k_y\right)=e^{i{k}_{z}z}\left[\left(r_{\text{ss}}{\mathbf{e}}_{s+}+r_{\text{ps}}{\mathbf{e}}_{p+}\right){\mathbf{e}}_{s-}^{\mathrm{T}}+\left(r_{\text{sp}}{\mathbf{e}}_{s+}+r_{\text{pp}}{\mathbf{e}}_{p+}\right){\mathbf{e}}_{p-}^{\mathrm{T}}\right],$$ (4)

where ${\mathbf{e}}_{s\pm }={\left[\begin{array}{ccc}\frac{k_y}{k_0}& \frac{-k_x}{k_0}& 0\end{array}\right]}^{\mathrm{T}}$ and ${\mathbf{e}}_{p\pm }=\frac{-1}{k_0}{\left[\frac{\pm k_z{k}_x}{k_0}\frac{\pm k_z{k}_y}{k_0}-\sqrt{k_x^2+k_y^2}\right]}^{\mathrm{T}}$.

The resulting plots are shown in Figure 4c–f. When β = 0°, Im(G) shows a hyperbolic contour with upper and lower openings, closely aligning with the polaritonic IFC of the α‐MoO₃ bilayer (dashed lines), indicating the direct relation between heat transfer and the excitation of hyperbolic PhPs. As the twist angle increases, Im(G) also increases. When β = 60°, the two hyperbolic branches of the polaritonic IFC are almost parallel at large wavevectors, resembling a canalisation phenomenon [46]. Finally, when β = 90°, Im(G) shows a closed contour resembling a circle, and the maximum magnitude. These results corroborate the key role of topological transitions for achieving modulation and eventual enhancement of NFRHT in twistoptics.

In conclusion, our results demonstrate the potential of twisted van der Waals layers to modulate and enhance the radiative heat flux at the nanoscale. These capabilities are explained by the emergence of polaritonic topological transitions (from hyperbolic to closed IFCs) as a function of the twist angle, as demonstrated for gapped bilayers and trilayers of the anisotropic vdW materials α‐MoO₃. Based on these findings, we propose an experimental setup in which the heat flux between a nanoparticle and nearby twisted α‐MoO₃ bilayer can be modulated up to a factor of 3. Our findings on topological transitions in NFRHT are broadly relevant to materials supporting in‐plane hyperbolic polaritons. The effect depends on anisotropy, losses, and geometry, and should occur in the infrared, where thermal radiation is most significant. Overall, this work introduces twistoptics as an efficient route for heat management applications.

Author Contributions

Yang Hu: methodology, data curation, investigation, validation, visualization, writing – original draft, funding acquisition. José Álvarez‐Cuervo: investigation, writing – review & editing. Enrique Terán‐García: visualization, writing – review & editing. Xiuquan Huang: supervision, visualization, writing – review & editing. Pablo Alonso‐González: investigation, visualization, funding acquisition, supervision, writing – review & editing.

Conflicts of Interest

The authors declare no conflicts of interest.

Supporting information

Supporting Information S1

Data Availability Statement

Data will be made available at a reasonable request.