When Windowed Anderson Acceleration Converges Faster

Windowed Anderson acceleration AA($m$) combines at most $m+1$ recent iterates by least squares. For a real affine contraction $x\mapsto Wx+b$ with $q=\|W\| \in(0,1)$, we determine exactly when AA($m$) converges from every starting point with an asymptotic factor below $q$. We give a polynomial criterion for this improvement and an equivalent spectral condition: there is no probability distribution on the phases of the eigenvalues of modulus $q$ whose first $m$ cosine moments are $q,q^2,\ldots,q^m$. If such a distribution exists, we construct a starting point from which AA($m$) reproduces fixed-point iteration at every step. Otherwise, over a fixed number of consecutive steps, every run contracts the residual norm by a common factor strictly smaller than the corresponding power of $q$. The polynomial criterion also gives explicit convergence-rate bounds with constants independent of the dimension. When $q$ equals the spectral radius, this characterizes improvement over the asymptotic convergence factor of fixed-point iteration. For general affine contractions, the asymptotic factor never exceeds the numerical radius of $W$, and a related moment condition gives a uniform upper bound strictly below that radius.

Source: arXiv:2610.30073v1 - http://arxiv.org/abs/2610.30073v1 PDF: https://arxiv.org/pdf/2610.30073v1 Original Link: http://arxiv.org/abs/2610.30073v1