Are Flat Minima an Illusion?

This paper challenges the conventional view that flat minima inherently lead to better generalization, showing that function-preserving reparameterization can drastically alter a minimum's perceived sharpness. It introduces "weakness"—a reparameterization-invariant measure based on what the network does—as the actual driver of generalization, proving its minimax optimality and correlation with PAC-Bayes bounds.