Generalization Guarantees on Data-Driven Tuning of Gradient Descent with Langevin Updates

This paper introduces the Langevin Gradient Descent (LGD) algorithm for convex regression problems, proving that optimal hyperparameter configurations achieve the Bayes' optimal solution. The work also provides generalization guarantees for meta-learning LGD's optimal hyperparameters, with a pseudo-dimension bound of O(dh).