schema_version: 1
title: SGD with Vanishing Correlated Gradient Errors (technique notes)
abstract: For strongly convex smooth objectives and deterministic weighted historical
  gradient errors, the manuscript proves mean-square and almost-sure convergence and
  a Polyak–Ruppert CLT governed by cumulative weights and joint innovation covariance.
  Normalized deterministic scalar historical estimators have cumulative covariance
  at least Q; a polynomial-gain recursion attains this bound under the stated exponent
  conditions despite slower one-step MSE decay. The lower bound concerns this specified
  linear estimator class and, for the downstream comparison, zero innovation cross-covariance.
field: Optimization
license: CC-BY-4.0
type: paper
authors:
- eggtea
keywords:
- stochastic gradient descent
- correlated gradient errors
- Polyak–Ruppert averaging
- variance lower bound
main: main.pdf
takeaways: takeaways.md
verification:
- type: review
  path: verification/review.md
  status: materials-present-not-independently-verified
  model: Codex (GPT-6; current-session non-independent check)
  date: '2026-10-17'
  scope: Limited current-session claim/assumption and selected-proof consistency check
    of main.pdf SHA-256 f02ce29f4b79672a3b23bc88c7d29279ef1316bddd2c8bff355b2e239cafc0ca;
    includes executed formula checks; not an independent full mathematical review.
references:
- title: Acceleration of stochastic approximation by averaging
  authors: B. T. Polyak; A. B. Juditsky
  doi: 10.1137/0330046
- title: A convergence theorem for non negative almost supermartingales and some applications
  authors: H. Robbins; D. Siegmund
  doi: 10.1016/B978-0-12-604550-5.50015-8
- title: 'On linear stochastic approximation: Fine-grained Polyak–Ruppert and non-asymptotic
    concentration'
  authors: W. Mou; C. J. Li; M. J. Wainwright; P. L. Bartlett; M. I. Jordan
  url: https://proceedings.mlr.press/v125/mou20a.html