SGD with Vanishing Correlated Gradient Errors (technique notes)

# Concise takeaways - Analyze cumulative weights K_{T,s}=sum_{t=s}^T W_{t,s}, not only E||b_t||^2. Under the PR hypotheses the effective covariance is S+Gamma+C_0 K^T+K C_0^T. - For eta_t=D t^{-alpha}, 1/2<alpha<1, an error bound O(t^{-beta}) needs beta>max(1/2,2alpha-1) for the stated nonlinear PR remainder estimates, plus the weight limits, maximal-weight condition and cumulatively negligible remainder. - With zero innovation cross-covariance, the running mean contributes 2Q. For fixed c>1/2 the c/t recursion contributes 2c/(2c-1) Q although its one-step asymptotic error coefficient is c^2/(2c-1). - For normalized deterministic scalar historical weights, Gamma_T-Q=(Q/T)sum_s(k_{T,s}-1)^2 is positive semidefinite. This is a sharp bound within that estimator class, not an oracle lower bound for arbitrary algorithms. - Initialize b_1=epsilon_1 and use gamma_t=a t^{-rho}, 0<a<=1, 1/2<rho<1. Abel summation gives cumulative-weight flatness O(T^{rho-1}) and Gamma_T->Q; downstream PR also requires rho>2alpha-1. Single-step MSE is slower, of order t^{-rho} when Q is nonzero. - Probability convergence of the cumulative remainder alone does not identify the actual covariance limit; the manuscript separately states the stronger L2 requirement. Experiments illustrate asymptotic formulas and are not proofs.