The paper develops a sharp pseudospectral theory for coupled gradient descent systems with block-triangular Jacobians, relevant to bilevel optimization and adversarial training. It provides bounds for transient amplification and characterizes critical coupling thresholds for spectral instability.
This paper challenges the conventional view that flat minima inherently lead to better generalization, showing that function-preserving reparameterization can drastically alter a minimum's perceived sharpness. It introduces "weakness"—a reparameterization-invariant measure based on what the network does—as the actual driver of generalization, proving its minimax optimality and correlation with PAC-Bayes bounds.
This research proposes distinguishing between capability elicitation and capability creation in large language model post-training. It argues that elicitation reweights existing behaviors within a model's accessible support, while creation changes that support itself, developing this through a free-energy view.
This content explores notes on Kullback-Leibler Divergence and its relationship with the concept of Likelihood. It covers fundamental principles of information theory and statistical inference relevant to AI.