Authors: Jörg Franke, Michael Hefenbrock, Gregor Koehler, Frank Hutter Introduction We present in our recent preprint a novel approach to parameter regularization for deep learning: Constrained Parameter Regularization (CPR). It is an alternative to traditional weight decay. Instead of applying a constant penalty uniformly to all parameters, we enforce an upper bound on a statistical […]
New Horizons in Parameter Regularization: A Constraint Approach
Posted on December 12, 2023 by Jörg Franke