Implicitly Restarted Lanczos Enables Chemically Accurate Shallow Neural Quantum States

Abstract

The variational optimization of neural quantum states (NQS) is hindered by the slow convergence and numerical instability of conventional optimizers. We address this bottleneck by introducing the implicitly restarted Lanczos (IRL) method as the core engine for NQS training. Our approach recasts the parameter update problem into a Hermitian eigenvalue problem that IRL solves efficiently, automatically determining the optimal descent direction and step size without hyperparameter tuning. We demonstrate that IRL enables shallow NQS architectures to achieve extreme precision in just 3–5 optimization steps, dramatically outperforming existing methods, including minimum-step stochastic reconfiguration (MinSR). For the N₂ dissociation curve, IRL successfully trains a shallow NQS to match full configuration interaction accuracy, validating its robust convergence in strongly correlated regimes. This work establishes IRL as an efficient and robust second-order optimization strategy for variational quantum models.

Introduction

The emergence of neural quantum states (NQS) has provided a powerful new, scalable paradigm for representing quantum many-body wave functions. − Highly accurate predictions from NQSwhich are intrinsically flexible trial wave functions optimized via variational − or diffusion − Monte Carlo (VMC/DMC)are contingent upon both the NQS architecture’s sufficient expressive power and the effective optimization of its variational parameters. However, despite the architectural flexibility of NQS, achieving chemically accurate results faces significant optimization hurdles. Conventional first-order stochastic methods, such as stochastic gradient descent (SGD) and Adam, are widely used but suffer from painfully slow convergence, high sensitivity to hyperparameters, and numerical instabilities in flat energy landscapes. Conversely, more powerful second-order algorithmsincluding the Newton, approximate Newton, linear method (LM), − and stochastic reconfiguration (SR) − methodsare generally more efficient but suffer from a major limitation in the modern NQS era. These methods are currently constrained by their need to explicitly build and store large matrices, such as the quantum geometric tensor (QGT) S and the effective Hamiltonian H _eff, whose size scales quadratically with the number of variational parameters ( $\mathcal{O}(P^2)$ ). Given that modern NQS typically employ tens of thousands to millions of parameters, this quadratic scaling renders these second-order approaches computationally intractable due to both immense memory consumption and prohibitive wall-clock time.

Recent advances have sought to mitigate this scalability issue. − Notably, the minimum-step stochastic reconfiguration (MinSR) method bypasses the $\mathcal{O}(P^2)$ memory bottleneck by operating in the smaller sampling dimension, making it applicable to larger NQS architectures. However, as we demonstrate in this work, MinSR requires thousands of optimization steps to reach moderate accuracy, highlighting the need for further improvements in convergence efficiency.

The LM and SR optimization methods, which are foundational second-order techniques in NQS, reduce the optimization task to solving either a system of linear equations or a linear eigenvalue problem. The matrices involved (S and H _eff) are determined by stochastic sampling within the VMC framework. The essential difficulty, as discussed, is that the dimension of these matrices equals the square of the number of variational parameters (P), making their explicit construction and storage computationally infeasible when P scales up, a common occurrence in modern NQS. To overcome this critical bottleneck, we propose solving the central linear algebra problems of these optimization methods by utilizing iterative Krylov subspace algorithms. Crucially, these solvers do not require the matrices to be built explicitly. Instead, they only require the efficient evaluation of matrix-vector products (MVPs), which we demonstrate to be far less computationally expensive than actually constructing full matrices. In the context of VMC, this approach is made particularly efficient by evaluating these MVPs on-the-fly during the stochastic sampling process. Since each sampled configuration contributes an outer product to the overall matrix, operating by the matrices during sampling allows us to compute the required MVPs without ever needing to store the large $\mathcal{O}(P^2)$ matrices, enabling us to treat massive NQS efficiently.

In this work, we will demonstrate this approach by using implicitly restarted Lanczos (IRL) to improve the SR method. Instead of directly approximating the inverse of the ill-conditioned QGT via iterative linear solvers, we exploit the mathematical equivalence between the SR update and the generalized eigenvalue problem (GEVP): H _eff v = λS v. We introduce IRL as the robust, second-order solver for this GEVP. The IRL method efficiently extracts the smallest eigenvalue λ_min and its corresponding eigenvector v _min from a small, dynamically generated Krylov subspace. This approach bypasses the ill-conditioning of S and provides a step direction v _min whose magnitude is intrinsically determined by the energy difference λ_min, thereby eliminating the need for external learning rate tuning (η). As shown in Table (below), this IRL-based framework achieves 10^–10–10^–12 kcal/mol precision in just 3–5 steps across diverse molecular systemsa dramatic improvement over MinSR, which requires 160–3000 steps to reach only 10^–4–10^–6 kcal/mol accuracy. We demonstrate that the IRL-GEVP framework transforms NQS optimization into a numerically stable and highly efficient learning paradigm. This unprecedented stability and efficiency enable shallow NQS architectures to achieve performance previously restricted to deep or massive networks. Our results show that this approach reliably and consistently reaches chemical accuracy (and often far surpasses it), successfully bridging the long-standing gap between the theoretical promise of NQS and their practical utility in computational quantum science.

1. Performance Comparison of the Neuronal Quantum State (NQS) across Molecular Systems .

	System				IRL-NQS (Ours)		SL-NQS		MinSR-NQS		CCSD(T)
Molecule	N 0	N e	PVCs	N p	Steps	ε abs	Steps	ε abs	Steps	ε abs	ε abs
H2	4	2	4	7.2e2	2	4.6e-12(1)	2	6.1e-5(3)	160	3.2e-6(5)	9.8e-7
LiH	12	4	2.3e2	2.7e3	3	1.9e-11(2)	31	1.7e-5(4)	700	8.5e-6(8)	3.4e-3
H2O	14	10	4.4e2	3.3e3	5	1.3e-10(1)	100	8.0e-4(2)	2000	4.2e-5(6)	7.6e-2
H2O*	24	10	2.1e5	8.5e3	8	1.1e-10(2)	150	4.5e-4(3)	3000	6.5e-5(7)	3.2e-3
H2S	22	18	3.0e3	6.5e3	5	2.5e-10(2)	100	7.9e-4(3)	2400	8.3e-5(9)	4.2e-2
NH3	16	10	3.1e3	4.0e3	5	2.2e-11(1)	100	9.7e-4(4)	2400	9.1e-5(8)	1.4e-1
CH4	20	10	1.6e4	4.8e3	5	2.8e-10(3)	100	9.2e-3(5)	2600	8.4e-5(7)	1.5e-1
N2	20	14	1.4e4	5.6e3	5	3.3e-10(3)	100	7.2e-5(4)	2400	7.8e-5(6)	1.4
LiF	20	12	4.4e4	5.6e3	5	1.3e-10(2)	100	8.0e-4(3)	2600	8.1e-5(8)	6.3e-2
LiCl	28	20	1.0e6	9.6e3	5	4.8e-10(4)	100	2.3e-3(2)	3000	2.9e-4(5)	2.5e-1
Li2O	30	14	4.1e7	1.1e4	5	5.8e-10(5)	100	2.9e-3(3)	3000	3.2e-4(6)	2.7e-1
Convergence Speed					2–8 steps		2–150 steps		160–3000 steps

^aFor each system, we report the total orbitals (N _o), electrons (N _e), number of physically valid configurations (PVCs), and network parameters (N _p). ε _abs = |E _NQS – E _FCI|. The shallow NQS trained with our implicitly restarted Lanczos (IRL) method achieves extreme numerical precision (ε _abs in kcal/mol) of 10^–10–10^–12 kcal/mol in only 2–8 optimization steps, outperforming standard Lanczos (SL), MinSR-optimized NQS, and CCSD­(T). For H₂O, we include additional results (H₂O*) with the cc-pVDZ basis set. All other calculations used the STO-3G basis set. Statistical uncertainties (one standard error) are shown in parentheses as the uncertainty in the last digit(s).

Results

The IRL Framework for Robust Second-Order Optimization

The variational optimization of a neural quantum state |ψ_θ⟩ aims to find parameters θ that minimize the energy E(θ) = ⟨ψ_θ|H|ψ_θ⟩/​⟨ψ_θ|ψ_θ⟩. The SR update rule is derived from a first-order expansion of the imaginary-time evolution, leading to a linear system for the parameter update Δθ:

$$S\mathrm{\Delta }\theta =-\eta F$$ 1

where F is the energy gradient vector, S is the QGT, and η is a learning rate. Solving this system via iterative methods like Conjugate Gradient (CG) is prone to the numerical instabilities caused by the ill-conditioned matrix S.

Our central methodological contribution is to reformulate this optimization task into a Hermitian eigenvalue problem. By deliberately replacing the quantum geometric tensor S with the identity matrix I, a design choice we term identity preconditioning, we simplify the generalized eigenvalue problem that arises in the stochastic reconfiguration framework into a standard Hermitian form. Specifically, the optimal parameter update can be obtained by minimizing the Rayleigh quotient of an effective Hamiltonian H̃ (see Methods for a detailed construction). Under this approximation, the update rule transforms into

$$\mathrm{H̃}v=\lambda v$$ 2

The lowest eigenvector v _min of this problem yields the optimal parameter update Δθ ∝ v _min. Crucially, this transformation automatically determines the ideal energy descent step and direction, eliminating the manual tuning of the learning rate hyperparameter and enhancing numerical stability. While discarding the full metric information on the parameter space, which forms the theoretical foundation of natural gradient methods, this simplification proves effective in practice when parameters are not strongly linearly dependent, with the IRL process iteratively refining the descent direction. A detailed discussion of its implications and limitations is provided in the Methods section.

We solve this eigenvalue problem using Algorithms 1 and 2. As illustrated schematically in Figure , IRL iteratively builds a Krylov subspace in the parameter space, efficiently and stably converging to the extremal eigenvectors without the need for explicit matrix inversion or the severe loss of orthogonality that plagues the standard Lanczos (SL) method, particularly on high-dimensional, noisy optimization landscapes.

1.

Schematic of the IRL-optimized NQS optimization framework. (a) The traditional first-order stochastic optimization landscape (e.g., Adam) is noisy and requires careful tuning of the learning rate (η). (b) Our approach reformulates the second-order parameter update as an eigenvalue problem in the neural network’s parameter space. (c) The IRL method stably solves this problem, finding the optimal descent direction (red arrow) and step size automatically, enabling robust navigation of the optimization landscape.

Efficiency and Numerical Stability of IRL Optimization

To fundamentally assess the efficiency and stability of our proposed IRL optimizer, we first visualized its optimization trajectory against conventional methods in the calculation of the F₂ molecule. Here, we define the absolute error ε _abs = |E – E _FCI|, where E is the energy obtained by a given method (NQS, CCSD­(T), etc.) and E _FCI is the reference energy from full configuration interaction (FCI). For NQS, E = ⟨ψ _θ |Ĥ|ψ _θ ⟩/⟨ψ _θ |ψ _θ ⟩ is the variational energy expectation value of the ansatz. As shown in Figure A, the variational energy error landscape ε _abs forms a complex, high-dimensional surface in parameter space. The trajectory of the conventional first-order Adam optimizer (Figure B, C) is tortuous and protracted, requiring over 10,000 steps to approach the global minimum and exhibiting characteristic instability. The SL method, while offering a more direct descent path, fails to converge after 50 steps due to the accumulation of numerical instability and loss of orthogonality. In stark contrast, the IRL optimizer demonstrates a near-ideal descent trajectory, navigating efficiently and robustly to the global minimum in merely 5 steps. This dramatic difference underscores IRL’s capability to overcome the critical challenges of noise and ill-conditioning that plague other second-order iterative methods.

2.

Optimization landscape and trajectories for different NQS training methods, with a base neural network size of 5,622 parameters. (A) Three-dimensional representation of the variational energy error surface ε _abs as a function of two shifted hyperparameters θ₁ and θ₂, with the global minimum corresponding to FCI accuracy (the origin (0,0) is shifted to align with the global minimum for visualization clarity). (B) Two-dimensional projection with optimization paths for Adam, standard Lanczos (SL), and implicitly restarted Lanczos (IRL) methods. (C) Convergence behavior showing IRL reaching the global minimum in only 5 steps (5.1 ms), compared to 10,000 Adam steps (91.2 s) and 50 SL steps (0.272 s) that fail to converge. Data are from F₂ molecule calculations. (D) Performance scaling with the model size for the F₂ molecule. As the network dimension (e.g., number of hidden units or layers) increases, Adam’s optimization behavior exhibits a clear scaling trend. In contrast, both SL and IRL demonstrate robust stability, showing minimal dependence on the model’s scale. All calculations were performed with a fixed random seed of 111.

The practical impact is further quantified by the total wall-clock time required to achieve the highest possible accuracy for the F₂ molecule. The Adam optimizer required 91.2 s to complete 10,000 steps and failed to reach the global minimum. The SL method completed 50 steps in 0.272 s (a 335× speed-up over Adam) but also failed. Remarkably, the IRL optimizer achieved the global minimum in only 5.1 ms, demonstrating a total runtime speed-up of approximately 17,900-fold over Adam’s procedure.

This superior convergence efficiency is validated by its scaling behavior with model size (N _p) (Figure D). The performance of the first-order Adam optimizer exhibits a strong dependence on N _p , adhering to a distinct N _p scaling law, indicating that achieving high accuracy with larger networks becomes progressively more difficult. Conversely, both SL and IRL demonstrate remarkable robustness, showing no significant scaling relationship with N _p. This independence from the model’s complexity is a hallmark of a numerically stable and physically grounded optimization strategy, crucial for scaling NQS to more complex systems in quantum machine learning.

Shallow NQS Achieve Unparalleled Accuracy and Efficiency

We conducted a comprehensive benchmark across diverse molecules to quantify the performance of our IRL-optimized shallow NQS. Table summarizes the results, emphasizing the absolute error (ε _abs), convergence speed (Steps), and required network complexity (Parameters, N _p). The benchmark spans systems with up to ∼4.1 × 10⁷ physically valid configurations (PVCs), with additional results for H₂O* using the larger cc-pVDZ basis set to demonstrate basis set scalability. Here, “shallow NQS” refers to architectures with limited depth (typically one or two hidden layers), which contrast with deep networks that require many layers to achieve expressive power. The specific neural network architecture used in this work follows the autoregressive framework developed in our previous studies, which has been demonstrated to efficiently represent quantum many-body states while maintaining tractable sampling and probability computation.

The statistical uncertainties associated with these energy estimates are reported in Table in parentheses, indicating one standard error in the last digit(s). For IRL and SL-NQS, we used N _s = 10⁵ Monte Carlo samples per optimization step, yielding statistical uncertainties on the order of 10^–12–10^–10 kcal/mol across all tested systems. For MinSR-NQS, a larger sample size of N _s = 10⁶ per step was required to ensure stable gradient estimates, resulting in uncertainties ranging from 10^–7 to 10^–5 kcal/mol. Importantly, the extreme precision reported in the manuscript (e.g., <10^–12 kcal/mol for H₂) refers to the mean energy averaged over the final 50 optimization steps; the associated statistical uncertainty falls well within this threshold, confirming that the claimed accuracy is statistically significant.

The key finding is the dual advantage of extreme precision and parameter efficiency (compactness). For every system tested, our IRL-optimized shallow NQS consistently achieves extreme precision, with errors on the order of 10^–10 to 10^–12 kcal/mol relative to the full configuration interaction (FCI). This accuracy surpasses not only standard Lanczos (SL) and minimum-step stochastic reconfiguration (MinSR) optimized NQS, but also high-level nonvariational quantum chemistry methods like coupled-cluster singles-doubles with perturbative triples (CCSD­(T)).

The superiority over MinSR is particularly striking: while MinSR requires thousands of optimization steps (160–3000 steps) and yields errors in the 10^–4 to 10^–6 kcal/mol range, our IRL method achieves 10^–10 to 10^–12 kcal/mol precision in merely 2–8 steps. This dramatic improvement stems from IRL’s implicit restart technique, which effectively mitigates the stagnation issues that plague both SL and MinSR approaches in large-scale optimization landscapes.

More importantly for machine intelligence, this accuracy is achieved using orders of magnitude fewer parameters (N _p) than the large NQS optimized with Adam (e.g., 718 parameters vs 150,000 for H₂). This confirms that the robust second-order optimization provided by IRL is highly effective at exploiting the expressive power of shallow architectures, avoiding the common machine learning strategy of needing to exponentially increase the model size to compensate for poor optimization.

Furthermore, the IRL optimizer converges robustly in just 3 to 5 steps for most molecules, irrespective of system size or complexity, even for those with up to ∼4.1 × 10⁷ PVCs. The consistency holds when scaling to larger basis sets: for H₂O* with cc-pVDZ (24 orbitals, 2.1 × 10⁵ PVCs), IRL maintains 10^–10 kcal/mol precision in 8 steps, while MinSR requires 3000 steps with errors 2 orders of magnitude higher. This systemic stability and efficiency demonstrate that IRL provides a robust and scalable learning framework for high-dimensional variational problems, fundamentally outperforming stochastic reconfiguration-based optimizers such as MinSR that struggle with slow convergence and higher error floors.

Robust Optimization in the Strong Correlation Regime: Bond Breaking

A stringent test for any high-dimensional machine learning model used in a physical simulation is its stability and convergence in regions characterized by strong correlation, such as chemical bond dissociation. In optimization terms, strong correlation corresponds to a highly nonconvex, multimodal, and severely ill-conditioned loss landscape, , posing an extreme challenge for standard optimization routines. We applied our IRL-NQS approach to map the bond dissociation curve of the N₂ molecule, a classic multireference benchmark. Figure presents the total energy as a function of the N–N bond length, comparing our shallow IRL-NQS results with Hartree–Fock, CCSD­(T), and the near-exact full configuration interaction (FCI) reference where available.

3.

Bond dissociation curve of the N₂ molecule computed using various electronic structure methods, including Hartree–Fock, CCSD­(T), FCI, and the IRL-NQS approach introduced in this work. The total energy (in Hartree) is plotted as a function of the N–N bond length (in Å). The IRL-NQS results demonstrate close agreement with high-accuracy FCI calculations across the dissociation range. The STO-3G basis is used.

The comparison highlights the well-documented limitations of single-reference methods. Hartree–Fock fails qualitatively, showing an unphysical energy increase at large separations. CCSD­(T), while highly accurate near equilibrium, diverges at larger bond lengths (R > 2.5R _e ) due to its inability to capture multireference electronic structure. In contrast, our IRL-NQS results demonstrate near-perfect agreement with FCI across the entire dissociation range (Figure ). This result primarily validates the expressivity of our shallow, physically motivated ansatz and the correctness of the IRL implementation, demonstrating its capability to train variational models in strongly correlated regimes rather than implying uniqueness as the sole optimizer capable of achieving such accuracy.

Beyond accuracy, the computational efficiency of the IRL becomes particularly evident in this strong correlation setting. As shown in Table , for the N₂ system, IRL achieves 3.3 × 10^–10 kcal/mol precision in just 5 optimization steps, whereas MinSR requires 2,400 steps to reach 7.8 × 10^–5 kcal/mol errora nearly 3 orders of magnitude reduction in iteration count. This efficiency gain is especially pronounced in strongly correlated regimes where the optimization landscape becomes increasingly rugged, demonstrating IRL’s robust convergence properties where MinSR exhibits significant slowdown.

Discussion

We have introduced a novel, robust second-order optimization paradigm for high-dimensional variational machine learning models by seamlessly integrating the IRL method into the SR framework. This algorithmic breakthrough directly resolves the long-standing critical issues of numerical instability and hyperparameter sensitivity that have curtailed the practical application of NQS to high-precision quantum chemistry.

The performance of our approach is rooted in the superior numerical properties of IRL. By reformulating the ill-conditioned natural gradient linear system SΔθ = −F into a stable Hermitian eigenvalue problem, IRL achieves two critical objectives: (i) The method intrinsically extracts the optimal energy descent direction and ideal step size, effectively eliminating the demanding manual tuning of the learning rate, a major bottleneck in all stochastic methods. (ii) The implicit restarting mechanism actively filters spurious eigenvectors (noise) and maintains the orthogonality of the Krylov basis vectors, preventing the convergence collapse typically seen when conventional iterative solvers encounter the highly ill-conditioned QGT. This stability is particularly crucial for training shallow network architectures, allowing them to fully realize their expressivity without relying on excessive parameter counts to compensate for poor optimization.

Our systematic benchmarking confirms that this strategy represents a fundamental algorithmic advance, moving beyond incremental improvements in the convergence rate. The reliable attainment of subchemical accuracy (ε _abs ≈ 10^–10–10^–12 kcal/mol) in just 3 to 5 stepscoupled with the ability to use shallow networkssignifies that the ultimate precision of NQS is primarily limited by the optimization method itself, not necessarily the architectural size. This dual gain in accuracy and parameter efficiency is the key to scaling NQS to larger, chemically relevant systems where both computational cost and model complexity are primary constraints. Furthermore, the demonstrated success in the N₂ bond breaking curve validates IRL’s generalizability to the strong correlation (multireference) regime, where standard single-reference methods like CCSD­(T) fail.

Despite these advantages, it is important to acknowledge the scalability limitations of the IRL framework when it is applied to extremely large parameter regimes (p ≈ 10⁶). Although the matrix-free implementation avoids explicit storage of H _eff and thus circumvents $\mathcal{O}(p^2)$ memory requirements, the computational cost per iteration remains dominated by the matrix-vector products (MVPs) required for each Lanczos step. Each MVP scales as $\mathcal{O}(p·N_s)$ , where N _s ≈ 10⁵ is the number of Monte Carlo samples, leading to a total cost per IRL iteration of $\mathcal{O}(m·p·N_s)$ for a Krylov subspace of dimension m = 20. For p ≈ 10⁶, this becomes computationally demanding, even with our optimized implementation. Several promising directions could mitigate this limitation in future work: (i) distributed computing strategies that parallelize the MVP operations across multiple nodes; (ii) more aggressive restarting schemes that reduce the required Krylov subspace dimension; (iii) GPU acceleration to leverage massive parallelism in the MVP calculations; and (iv) adaptive sampling techniques that reduce N _s as optimization progresses. Exploring these avenues represents an important direction for scaling IRL-NQS to even larger systems.

The IRL optimization framework offers a solid foundation for future variational research. Immediate applications include extending its reach to challenging chemical problems such as transition metal chemistry, excited state calculations, and molecular dynamics in nonequilibrium geometries, all of which demand the type of robust, high-fidelity convergence demonstrated here. More broadly, the success of leveraging advanced numerical linear algebra techniques, specifically the stable Krylov subspace projection, opens a new direction for optimizing complex high-dimensional machine learning models in physics and other scientific domains. Future algorithmic extensions could explore alternative Krylov subspace methods like the Davidson method or block-Lanczos methods to further exploit problem-specific structures and potentially achieve linear scaling for even larger parameter spaces.

Conclusion

In conclusion, our work bridges the long-standing gap between the theoretical expressivity of neural network quantum states and their practical utility, establishing IRL-NQS as a competitive, reliable, and highly efficient tool for ab initio electronic structure theory that sets a new standard for high-precision variational optimization.

Methods

The IRL for Wave Function Optimization

The foundation of our robust optimization strategy for NQS is the seamless integration of the IRL method within the SR framework. This innovation provides a numerically stable and efficient alternative to conventional iterative linear solvers plagued by ill-conditioning, primarily by reformulating the ill-posed parameter update problem into a well-defined Hermitian eigenvalue problem.

From SR to a Stable Eigenvalue Problem

The standard SR method is a second-order optimization technique that computes the steepest descent direction Δθ in the Riemannian manifold of the NQS. This involves solving the ill-conditioned linear system:

$$\mathbf{S}\mathrm{\Delta }\mathit{\theta }=-\eta \mathbf{F}$$ 3

where S is the QGT and F is the energy gradient.

We circumvent the numerical instability of inverting S by adopting the Linear Method (LM), which is asymptotically equivalent to SR but rooted in the Rayleigh–Ritz variational principle. The LM seeks the optimal update vector v = Δθ by minimizing the energy of the linearized wave function ψ′ in the local tangent space:

$${\psi }^{\prime }(\mathit{v})\approx \psi (\mathit{\theta })+\sum _{i=1}^p{v}_i\frac{\partial \psi (\mathit{\theta })}{\partial {\theta }_i}=\sum _{j=0}^p{c}_j{\chi }_j$$ 4

Here, c = [1, v ₁, ..., v _p ]^T is the coefficient vector, and {χ _j } _j=0 are the basis functions {ψ, ∂_θ1 ψ, ..., ∂_{θ p} ψ}. The problem is to minimize the Rayleigh quotient E[ψ′]:

$$E[{\psi }^{\prime }]=\lambda =\frac{⟨{\psi }^{\prime }|Ĥ|{\psi }^{\prime }⟩}{⟨{\psi }^{\prime }|{\psi }^{\prime }⟩}=\frac{{\mathit{c}}^{†}\mathit{H̃}\mathit{c}}{{\mathit{c}}^{†}\mathit{S̃}\mathit{c}}$$ 5

where H̃ ~ _ij = ⟨χ _i |Ĥ|χ _j ⟩ and S̃ ~ _ij = ⟨χ _i |χ _j ⟩ are the full (p + 1) × (p + 1) Hermitian matrices.

To find the minimum λ, we apply the variational condition by taking the derivative of E[ψ′] with respect to c ^† and c and setting them to zero:

$$\frac{\partial \lambda }{\partial {\mathit{c}}^{†}}=0\mathrm{and}\frac{\partial \lambda }{\partial \mathit{c}}=0$$ 6

Since the two resulting equations are equivalent under a conjugate transpose operation, they provide the same solution set for c . Therefore, when deriving the GEVP, it is sufficient to impose only one of the two independent variational conditions ( $(\frac{\partial \lambda }{\partial \mathit{c}}=0\mathrm{or}\frac{\partial \lambda }{\partial {\mathit{c}}^{†}}=0)$ to obtain the complete GEVP form. The choice to use $\frac{\partial \lambda }{\partial {\mathit{c}}^{†}}=0$ is a standard mathematical convention. Using the quotient rule and matrix derivatives (N = c ^† H̃ ~ c , D = c ^† S̃ ~ c ):

$$\frac{1}{D^2}[D\frac{\partial N}{\partial {\mathit{c}}^{†}}-N\frac{\partial D}{\partial {\mathit{c}}^{†}}]=0$$ 7

Substituting $\frac{\partial N}{\partial {\mathit{c}}^{†}}=\mathit{H̃}\mathit{c}$ and $\frac{\partial D}{\partial {\mathit{c}}^{†}}=\mathit{S̃}\mathit{c}$ :

$$D(\mathit{H̃}\mathit{c})-N(\mathit{S̃}\mathit{c})=0$$ 8

Substituting N/D = λ (and canceling D), we arrive at the GEVP:

$$\mathit{H̃}\mathit{c}=\lambda \mathit{S̃}\mathit{c}$$ 9

The optimal update is given by eigenvector c _min corresponding to the smallest eigenvalue λ_min.

To simplify the problem, we adopt the standard VMC convention that the NQS wave function is normalized, ⟨ψ|ψ⟩ = 1. This ensures that the current energy expectation is E(θ) = ⟨ψ|Ĥ|ψ⟩. To reduce the problem to the p × p parameter subspace and ensure λ_min represents the energy reduction (ΔE) rather than the absolute energy (E _new), we employ the energy centering technique. We define the centered Hamiltonian operator Ĥ′ = Ĥ – E(θ)Î.

In the parameter subspace spanned by the derivatives {|∂_{θ i} ψ⟩}, the effective Hamiltonian matrix H _eff is defined using the centered operator Ĥ′ = Ĥ – E(θ):

$$[{\mathbf{H}}_{\mathrm{eff}}{]}_{ij}=⟨{\partial }_{{\theta }_i}\psi |{Ĥ}^{\prime }|{\partial }_{{\theta }_j}\psi ⟩=⟨{\partial }_{{\theta }_i}\psi |Ĥ|{\partial }_{{\theta }_j}\psi ⟩-E(\mathit{\theta })⟨{\partial }_{{\theta }_i}\psi |{\partial }_{{\theta }_j}\psi ⟩$$ 10

The energy expectation value E(θ) = ⟨ψ|Ĥ|ψ⟩/⟨ψ|ψ⟩ is computed using variational Monte Carlo. By introducing a complete set of configurations, this expectation can be expressed as a weighted average over the configuration space. We define the local energy E _loc(x) for a configuration x as

$$E_{\mathrm{loc}}(\mathbf{x})=\frac{Ĥ\psi (\mathbf{x})}{\psi (\mathbf{x})}$$ 11

The energy expectation then takes the form of a Monte Carlo average of the local energy with respect to probability distribution P(x) ∝ |ψ­(x)|²: E(θ) = ⟨E _loc⟩ _P . This local quantity is essential, as it allows us to formulate the derivative of the energy, and subsequently the H _eff matrix, in a form computable through stochastic sampling. By employing the logarithmic derivative vector O, where O _i (x) = ∂_{θ i} ln ψ­(x; θ), this relation (eq ) can be compactly written in matrix form:

$${\mathbf{H}}_{\mathrm{eff}}=⟨\mathbf{O}|{Ê}_{\mathrm{loc}}|\mathbf{O}⟩-E(\mathit{\theta })\mathbf{S}$$ 12

where S _ij = ⟨O _i |O _j ⟩ is the QGT. This matrix H _eff is equivalent to the definition using connected correlation functions (subscript c), [H _eff] _ij = ⟨∂_{θ i} ψ|Ĥ|∂_{θ j} ψ⟩ _c , confirming that H _eff contains only the fluctuations and correlations between the parameter space and the local energy.

This rigorous transformation leads to the final compact GEVP in parameter space where v is the eigenvector:

$${\mathbf{H}}_{\mathrm{eff}}\mathit{v}=\lambda \mathbf{S}\mathit{v}$$ 13

The solution to eq , λ_min, strictly represents the minimum value of the Rayleigh quotient when restricted to the parameter subspace:

$$\lambda =\underset{\mathit{v}}{\min }\frac{{\mathit{v}}^{†}{\mathbf{H}}_{\mathrm{eff}}\mathit{v}}{{\mathit{v}}^{†}\mathbf{S}\mathit{v}}$$ 14

By substituting the definition of the energy-centered effective Hamiltonian H _eff into the expression for λ_min:

$${\lambda }_{\min }=\underset{\mathit{v}}{\min }\frac{{\mathit{v}}^{†}(⟨\mathbf{O}|Ĥ|\mathbf{O}⟩-E(\mathit{\theta })\mathbf{S})\mathit{v}}{{\mathit{v}}^{†}\mathbf{S}\mathit{v}}$$ 15

$${\lambda }_{\min }=\underset{\mathit{v}}{\min }\left(\frac{{\mathit{v}}^{†}⟨\mathbf{O}|Ĥ|\mathbf{O}⟩\mathit{v}}{{\mathit{v}}^{†}\mathbf{S}\mathit{v}}\right)-E(\mathit{\theta })$$ 16

We define the term within the parentheses as E _new:

$$E_{\mathrm{new}}=\underset{\mathit{v}}{\min }\left(\frac{{\mathit{v}}^{†}⟨\mathbf{O}|Ĥ|\mathbf{O}⟩\mathit{v}}{{\mathit{v}}^{†}\mathbf{S}\mathit{v}}\right)$$ 17

E _new represents the lowest energy attainable by the updated wave function ψ′( v ) within the linear tangent space defined by the current parameters θ.

Consequently, λ_min is strictly the difference between this local minimum energy and the current energy expectation:

$${\lambda }_{\min }=E_{\mathrm{new}}-E(\mathit{\theta })$$ 18

The physical interpretation of λ_min requires careful consideration. Within the local tangent space of the variational manifold where the linear approximation holds, λ_min represents the predicted energy reduction ΔE for an optimal update along the direction v _min. However, this prediction is rigorously valid only within a sufficiently small neighborhood where the linearization remains accurate. Since λ_min directly provides the energy difference scale, the method automatically determines an intrinsic step size, eliminating the need for manual learning rate tuning. The eigenvector v _min defines the descent direction, while the eigenvalue λ_min implicitly provides the necessary magnitude information.

To ensure that the update remains within the trust region, where the linear approximation is reliable, our IRL framework employs an implicit regularization strategy. By restricting the optimization to a low-dimensional Krylov subspace of dimension m = 20, we effectively limit the step to directions and magnitudes that are well-resolved by the local tangent space. This confinement naturally prevents overly aggressive steps that would extend beyond the valid region of the linear approximation. Additionally, the implicitly restarted Lanczos process refines the search direction iteratively, allowing the optimizer to correct any deviations caused by stepping outside the linearization regime. In practice, we observe that the step sizes derived from λ_min consistently remain within the trust region, as evidenced by the monotonic energy decrease and stable convergence behavior across all tested systems.

As the optimization progresses and the current energy E(θ) approaches the exact ground state energy E _exact, the locally minimized energy E _new also converges toward E _exact. Consequently, the energy difference λ_min = E _new – E(θ) approaches zero, naturally satisfying the convergence criterion.

To leverage the efficiency and numerical stability of the IRL method, we simplify the GEVP by replacing the metric S with the identity matrix I for the dominant descent direction and converting it into a standard Hermitian eigenvalue problem (SHEP):

$${\mathbf{H}}_{\mathrm{eff}}\mathit{v}=\lambda \mathit{v}$$ 19

The optimal parameter update is then determined by eigenvector v _min corresponding to the smallest eigenvalue λ_min:

$$\mathrm{\Delta }\mathit{\theta }\propto {\mathit{v}}_{\min }$$ 20

The physically meaningful λ_min automatically provides the optimal step size, eliminating the challenging hyperparameter tuning required by traditional SR.

The Standard Lanczos Process and Implicit Restart

We solve the SHEP (eq ) using the IRL algorithm. IRL builds upon the standard Lanczos (SL) process, a projection method used to construct an orthonormal basis V _m = [ v ₁, ..., v _m ] for the m-dimensional Krylov subspace:

$${\mathcal{K}}_m({\mathbf{H}}_{\mathrm{eff}},{\mathit{v}}_1)=\mathrm{span}\{{\mathit{v}}_1,{\mathbf{H}}_{\mathrm{eff}}{\mathit{v}}_1,...,{\mathbf{H}}_{\mathrm{eff}}^{m-1}{\mathit{v}}_1\}$$ 21

The SL process uses a three-term recurrence relation to project H _eff onto this subspace, yielding a small, symmetric tridiagonal matrix ${\mathit{T}}_m\in {\mathbb{R}}^{m\times m}$ :

$${\mathit{T}}_m=\left[\begin{array}{ccccc}{\alpha }_1& {\beta }_1& & & \\ {\beta }_1& {\alpha }_2& {\beta }_2& & \\ & {\beta }_2& \ddots & \ddots & \\ & & \ddots & {\alpha }_{m-1}& {\beta }_{m-1}\\ & & & {\beta }_{m-1}& {\alpha }_m\end{array}\right]$$ 22

which satisfies the projection relation H _eff V _m = V _m T _m + β _m v _m+1 e _m . The eigenvalues of T _m are the Ritz values, which approximate the eigenvalues of H _eff.

The SL process is susceptible to numerical instability in finite-precision arithmetic, resulting in the loss of basis vector orthogonality and the appearance of spurious ghost eigenvalues. IRL addresses this by implicitly applying a polynomial filter to the Krylov subspace. This filter, defined by the unwanted Ritz values {ν₁, ..., ν_μ} (typically those farthest from λ_min), effectively suppresses undesired eigencomponents:

$$\psi ({\mathbf{H}}_{\mathrm{eff}})=({\mathbf{H}}_{\mathrm{eff}}-{\nu }_1\mathbf{I})({\mathbf{H}}_{\mathrm{eff}}-{\nu }_2\mathbf{I})···({\mathbf{H}}_{\mathrm{eff}}-{\nu }_{\mu }\mathbf{I})$$ 23

Instead of explicitly computing this polynomial, IRL achieves the filtering robustly by performing μ steps of the shifted QR algorithm on the small matrix T _m . This procedure, guaranteed by the implicit Q theorem, , is mathematically equivalent to restarting the Lanczos process with an optimally filtered starting vector, ensuring numerical stability and guaranteed convergence to the extremal eigenpair v _min.

Practical Matrix-Free Implementation

For NQS optimization, the effective Hamiltonian matrix H _eff is never explicitly constructed, as its memory requirement scales prohibitively as $\mathcal{O}(p^2)$ for p parameters. Instead, the Implicitly Restarted Lanczos (IRL) algorithm operates entirely through matrix-free computations, requiring only the matrix-vector product H _eff v at each iteration.

The i-th component of this product is defined theoretically as [H _eff v ] _i = ∑ _j=1 [H _eff] _ij v _j . Using the definition of the effective Hamiltonian matrix element [H _eff] _ij = $\mathbb{E}[O_i^{*}(E_{\mathrm{loc}}-E(\mathit{\theta }))O_j]$ and the linearity of the expectation operator, the product is estimated stochastically via Monte Carlo sampling:

$${[{\mathbf{H}}_{\mathrm{eff}}\mathit{v}]}_i\approx \frac{1}{N_s}\sum _{n=1}^{N_s}{O}_i^{*}({\mathbf{x}}_n)(E_{\mathrm{loc}}({\mathbf{x}}_n)-E(\mathit{\theta }))\left(\sum _{j=1}^p{O}_j({\mathbf{x}}_n)v_j\right)$$ 24

where $E_{\mathrm{loc}}({\mathbf{x}}_n)=\frac{Ĥ{\psi }_{\mathit{\theta }}({\mathbf{x}}_n)}{{\psi }_{\mathit{\theta }}({\mathbf{x}}_n)}$ is the local energy and $O_i({\mathbf{x}}_n)=\frac{\partial \ln {\psi }_{\mathit{\theta }}({\mathbf{x}}_n)}{\partial {\theta }_i}$ is the log-derivative for sample x _n .

A naive reading of eq might suggest $\mathcal{O}(N_s·p^2)$ complexity, as it involves sums over both samples and parameter pairs. However, the key to achieving linear scaling lies in the observation that the inner summation ∑ _j=1 O _j (x _n )v _j depends only on the sample x _n and the vector v and is completely independent of the outer index i. This allows us to restructure the computation into two sequential steps. First, for each Monte Carlo sample x _n , we precompute a scalar quantity C _n = (E _loc(x _n ) – E(θ))​(∑ _j=1 O _j (x _n )v _j ). The inner sum ∑ _j O _j v _j is evaluated once per sample at a cost of $\mathcal{O}(p)$ , leading to a total of $\mathcal{O}(N_s·p)$ operations for this step. Once these C _n values are obtained, they are reused to assemble all p components of the result via [H _eff v ] _i ≈ $\frac{1}{N_s}\sum _{n=1}^{N_s}{O}_i^{*}({\mathbf{x}}_n)C_n$ , which requires another $\mathcal{O}(N_s·p)$ operations. Because the scalar C _n is computed only once per sample and then reused across every i, we entirely avoid the $\mathcal{O}(p^2)$ scaling that would result if the inner sum were naively recomputed for each parameter index. The total computational complexity is therefore $\mathcal{O}(N_s·p)$ , linear in both the number of samples and the number of parameters, making large-scale NQS optimization tractable.

We configure the IRL algorithm to target only the lowest eigenpair (k = 1) with a typical Krylov subspace dimension of m = 20. A stringent convergence tolerance (τ = 10^–12) is enforced on the residual norm, ensuring the extreme chemical accuracy necessary for fundamental scientific applications.

Turning to computational cost, we analyze the complexity in terms of Monte Carlo samples (N _s ) and network parameters (p). For first-order optimizers such as Adam or SGD, a single step requires estimating the energy gradient F = 2Re­(⟨O ^† E⟩ _c ), which scales as $\mathcal{O}(N_s·p)$ . The core operation of IRL, the matrix-vector product H _eff v , achieves the same $\mathcal{O}(N_s·p)$ scaling through the result reuse strategy described above. In practice, we observe that the stochastic evaluation of [H _eff v ] requires slightly more computation than the gradient F, quantified by C _IRL ≈ α·C _Adam with an overhead factor typically α ∈ [2, 5].

Despite this moderate per-step overhead, the total computational cost Cost_Total = Steps × Cost_Per‑step overwhelmingly favors IRL. To reach chemical accuracy, Adam often requires approximately 50,000 steps, while IRL converges in just 3–5 steps (see Table ). This dramatic reduction in iterations leads to a significant net gain:

$$\frac{{\mathrm{Cost}}_{\mathrm{Total}}^{\mathrm{IRL}}}{{\mathrm{Cost}}_{\mathrm{Total}}^{\mathrm{Adam}}}\approx \frac{5\times {\mathbf{C}}_{\mathrm{IRL}}}{50000\times {\mathbf{C}}_{\mathrm{Adam}}}\approx \frac{5·\alpha }{50000}\ll 1$$

demonstrating that the low iteration count of IRL far outweighs its moderately higher per-step cost, making it the superior optimizer for high-accuracy NQS computations.

The complete IRL procedure is summarized in Algorithm 1, which highlights the explicit construction of the Krylov subspace ${\mathcal{K}}_m$ through the Lanczos process.

The Identity Preconditioning Strategy: Numerical Rationale and Design Principles

The variational optimization of NQS relies on second-order methods, which generally aim to solve the GEVP associated with the LM or SR update: H _eff v = λS v. A primary numerical challenge in this approach, particularly in large-scale NQS and strongly correlated regimes, is the severe ill-conditioning of the QGT, S. This ill-conditioning necessitates the use of external, heuristic regularization (S ← S + ϵI) for traditional iterative linear solvers (like CG) used in SR, a process that introduces its own instability and requires careful hyperparameter (ϵ) tuning.

Our central methodological innovation, the IRL framework, strategically replaces the QGT S with the identity matrix I. This simplification transforms the generalized eigenvalue problem into a numerically robust standard Hermitian eigenvalue problem (SHEP): H _eff v = λ v . We emphasize that this is a deliberate algorithmic design choice rather than a geometric approximationwe do not claim that S is close to I in any mathematical sense. Indeed, given the ill-conditioned nature of S in strongly correlated regimes, such a claim would be untenable. Instead, this substitution is motivated by a pragmatic numerical strategy: by preconditioning the optimization with the identity, we circumvent the numerical instabilities associated with inverting or regularizing a near-singular S.

Importantly, this identity preconditioning preserves the fixed points of the optimization dynamics. At any stationary point of the energy landscape, the effective Hamiltonian satisfies H _eff = 0, as it derives from the energy gradient. Consequently, both the original generalized eigenvalue problem H _eff v = λS v and the simplified standard eigenvalue problem H _eff v = λ v yield the same trivial solution: λ = 0 with an arbitrary eigenvector v . The resulting parameter update Δ θ ∝ λ v _min vanishes identically at these points, regardless of the metric choice. Thus, while the preconditioning modifies the optimization trajectory, it does not alter the set of attainable stationary points. This ensures that any ground state solution accessible via the exact natural gradient dynamics remains accessible under the identity preconditioned dynamics, provided the optimization converges.

This identity preconditioning yields three critical advantages. First, it eliminates the QGT’s ill-conditioning as a computational barrier, providing the IRL solver with a stable environment for extracting extremal eigenvectors without requiring heuristic regularization parameters. Second, by solving the SHEP, the optimal parameter update direction v _min and its intrinsic magnitude (derived from λ_min) are determined automatically, entirely removing the need for manual tuning of learning rate hyperparametersa significant source of instability in conventional methods. Third, this simplification, when combined with the matrix-vector product (MVP) strategy, ensures that the overall complexity of the second-order update remains scalable to millions of parameters.

However, this design choice necessarily entails a trade-off. By discarding the metric information encoded in S, we deviate from the theoretical foundation of the exact natural gradient methods. In principle, this could lead to suboptimal update directions in regimes where the curvature of the parameter manifold deviates significantly from the Euclidean metricsuch as in deeper network architectures or when using larger basis sets that introduce more complex parameter correlations. The approximation S → I might, in such cases, fail to fully capture the geometry of the variational manifold, potentially resulting in slower convergence or less accurate minima compared to an exact second-order method.

To assess the practical impact of this trade-off under more challenging conditions, we performed additional benchmark calculations for H₂O using the larger cc-pVDZ basis set (24 orbitals, 2.1 × 10⁵ PVCs), as reported in Table . Despite the increased basis set size and corresponding growth in Hilbert space complexity, our IRL-based framework maintained 10^–10 kcal/mol precision in just 8 optimization steps. This result demonstrates that, at least for systems of this scale, the identity preconditioning strategy remains highly effective. The continued success likely stems from the fact that the dominant curvature directions captured by H _eff remain sufficiently aligned with the true natural gradient descent direction in these regimes, even after discarding the explicit S metric.

Nevertheless, we anticipate that as NQS continue to scale to deeper architectures and larger basis sets, incorporating more sophisticated treatments of S may become necessary for optimal performance. The empirical success of our method on systems up to 30 orbitals (4.1 × 10⁷ PVCs) and with cc-pVDZ basis sets suggests that the current simplification remains effective across a wide range of tractable systems. However, identifying the precise boundary where S → I begins to degrade performance represents an important direction for future investigation.

The empirical success of the IRL frameworkdemonstrated by its robust convergence and consistent high precision on challenging, strongly correlated systems such as the N₂ dissociation curvevalidates this design choice as a highly effective pragmatic strategy for achieving chemical accuracy in NQS optimization.

Wave Function Ansatz

We employ a neural network variational ansatz formulated in the second-quantized framework to represent the many-electron wave function. The electronic Hamiltonian in this representation is expressed as

$$Ĥ=\sum _{pq}{h}_{pq}{ĉ}_p^{†}{ĉ}_q+\frac{1}{2}\sum _{pqrs}{g}_{pqrs}{ĉ}_p^{†}{ĉ}_q^{†}{ĉ}_r{ĉ}_s$$ 25

where h _pq and g _pqrs are one- and two-electron integrals, and ĉ _p (ĉ _p ) are fermionic creation (annihilation) operators for a given spin-orbitalp.

The many-body wave function is expanded in the Fock space as

$$|\psi ⟩=\sum _{\mathit{n}}\psi (\mathit{n})|\mathit{n}⟩$$ 26

where | n ⟩ = |n ₁, n ₂, ..., n _M ⟩ denotes a basis state in the occupation number representation for a system with M spin-orbitals, and n _i ∈ {0, 1} due to the Pauli exclusion principle. The complex coefficient ψ­( n ) is the wave function amplitude for configuration n .

In our approach, a neural network serves as a variational function approximator for ψ­( n ), which we denote as ψ _θ ( n ), parametrized by θ. The network takes the occupation number vector n as input and outputs a complex number, which we decompose into its amplitude and phase:

$${\psi }_{\mathit{\theta }}(\mathit{n})=\sqrt{p_{\mathit{\theta }}(\mathit{n})}{\mathrm{e}}^{i{\varphi }_{\mathit{\theta }}(\mathit{n})}$$ 27

Here, p _θ ( n ) = |ψ _θ ( n )|² defines a probability distribution over the Hilbert space, and ϕ _θ ( n ) is the configuration-dependent phase.

Autoregressive Modeling and Physical Constraints

By the product rule of probability, any joint distribution p _θ ( n ) can be factorized exactly into a product of conditional distributions:

$$p_{\mathit{\theta }}(\mathit{n})=\prod _{i=1}^M{p}_{\mathit{\theta }}(n_i|{\mathit{n}}_{<i})$$ 28

where n _<i = (n ₁, n ₂, ..., n _i–1) denotes the occupation numbers of all spin-orbitals preceding the i-th orbital. This factorization allows for efficient and exact sampling of configurations n sequentially from i = 1 to M, crucial for variational Monte Carlo calculations.

To ensure the physical validity of the sampled configurations, we enforce conservation of the total number of spin-up electrons (N ^↑) and spin-down electrons (N ^↓) directly within the autoregressive sampling process. For each step i in the sequential sampling, the conditional probability p _θ (n _i | n _<i ) is dynamically adjusted to a restricted conditional probability p̃ _θ (n _i | n _<i ) based on the current electron counts:

$${\mathrm{p̃}}_{\mathit{\theta }}(n_i|{\mathit{n}}_{<i})=\{\begin{array}{ll}0& \mathrm{if}{\mathcal{N}}^{\uparrow }({\mathit{n}}_{<i},n_i)>N^{\uparrow }\mathrm{or}{\mathcal{N}}^{\downarrow }({\mathit{n}}_{<i},n_i)>N^{\downarrow }\\ p_{\mathit{\theta }}(n_i|{\mathit{n}}_{<i})& \mathrm{otherwise}\end{array}$$ 29

Here, ${\mathcal{N}}^{\uparrow }({\mathit{n}}_{<i},n_i)$ and ${\mathcal{N}}^{\downarrow }({\mathit{n}}_{<i},n_i)$ are the cumulative counts of spin-up and spin-down electrons after considering the i-th orbital. The restricted probabilities are subsequently renormalized. This procedure guarantees that every generated sample n belongs to the subspace with fixed particle numbers (N ^↑, N ^↓), drastically reducing the effective size of the Hilbert space and improving the learning efficiency.

The authors declare no competing financial interest.

Published as part of Precision Chemistry special issue “Quantum Landscape of Precision Chemistry”.