Memory efficient Lanczos

check this out! Luke Fleed's blog post

Though it is not user friendly yet, however, memory is a major concern in ED, so might be good to keep in mind.

Conclusion. If you are using Krylov.jl for a symmetric matrix (e.g., using cg, minres, or symmlq), you will not suffer from the memory explosion described in the blog post screenshot. You can safely solve massive systems (like the 500 million variable example mentioned) on a GPU without running out of RAM.

This is an excellent question. Using this specific "Two-Pass" Lanczos approach for Quantum Many-Body Exact Diagonalization (ED) is actually a very standard and effective strategy, specifically when you are hitting the limits of your RAM.

Here is a breakdown of how this applies to your ED work, how it compares to the standard "on-the-fly" options, and why the comments in the screenshot (specifically Antoine's critique) might be misleading for your specific use case.

In the context of the blog post and ED, the comparison is between:

• Single-Pass (Standard): You perform Lanczos steps. You store every vector in memory (or on disk). At the end, you diagonalize the small tridiagonal matrix to get the ground state coefficients, and compute the linear combination of the stored vectors to get the full wavefunction .

• Two-Pass (The approach in the post):

  • Pass 1: You run the Lanczos iteration to build the tridiagonal matrix (energy), but you discard the vectors as you go (keeping only the last 3).
  • Pass 2: Once you have the coefficients for the ground state, you run the exact same Lanczos iteration again from the same random seed. This time, you generate the vectors "on the fly" and accumulate them into the final eigenvector.

Antoine argues the method is "useless" because short recurrence relations exist for (Linear Systems) and (Time Evolution).

However, in Quantum ED, you are usually solving the Eigenvalue Problem ( ) to find the Ground State.

• There is no "short recurrence" to magically produce the Ground State eigenvector without the basis vectors.
• You must reconstruct the vector basis.
• Therefore, you are forced to choose between Storage (Single-Pass) or Recomputation (Two-Pass).

In Many-Body ED, the Hamiltonian is typically defined "on the fly" (matrix-free) because the matrix is too sparse/large to store. The bottleneck is storing the Krylov Vectors (the state vectors).

Here is the trade-off analysis:

Use the Two-Pass (Post) method if:

• Memory is your bottleneck: Your Hilbert space is large (e.g., spins), and storing Lanczos vectors would crash your RAM.
• CPU is cheap: You have plenty of compute time but limited memory bandwidth/capacity.
• You need the Eigenvector: If you only need the Energy, you never need the two-pass method. If you need to measure observables ( ), you need the vector, making Two-Pass essential for large systems.

Use the Standard (Single-Pass) method if:

• System size is small: If all your Lanczos vectors fit in RAM, Single-Pass is 2x faster.
• You are doing Time Evolution: As noted, if calculating , there are Krylov subspace tricks that don't require recomputing the whole basis in the same way.

The "Cute trick" discussed in the image is actually the standard industry practice for large-scale Exact Diagonalization codes (like those used in Condensed Matter Physics).

While Antoine is right that it's unnecessary for solving linear systems ( ), it is critical for finding the Ground State Eigenvector when the Hilbert space is too large to store the Krylov subspace.