stochastic-gradient-descent

Information-Theoretic Generalization Bounds for Stochastic Gradient Descent with Predictable Virtual Noise

This paper introduces predictable history-adaptive virtual perturbations to enhance information-theoretic generalization bounds for Stochastic Gradient Descent. This new approach allows perturbation covariances to dynamically depend on past SGD history, addressing limitations of existing methods that require fixed covariances.