This paper introduces the Langevin Gradient Descent (LGD) algorithm for convex regression problems, proving that optimal hyperparameter configurations achieve the Bayes' optimal solution. The work also provides generalization guarantees for meta-learning LGD's optimal hyperparameters, with a pseudo-dimension bound of O(dh).
This paper introduces -DPO, a direct preference optimization method using a ratio reward margin, to address the challenge of hyperparameter tuning in SimPO. The research analyzes SimPO and reformulates the preference objective to improve interpretability across datasets with varying reward gap structures.
This paper derives scaling rules for Gated Delta Networks to address the computational demands of training and scaling Large Language Models. Experiments validate that these configurations enable stable learning-rate transfer across various model widths, unlike standard parametrization.