to standardize: when you need zero mean and unit variance for gradient-based optimization

Proximal gradient methods for learning

Proximal gradient descent efficiently handles non-differentiable L1 regularization by combining gradient descent with a proximity operator

batch size affects generalization: larger batches find sharper minima

Larger batch sizes lead to sharper minima, enhancing generalization by providing more accurate gradient estimates

natural gradient descent does: preconditions with inverse Fisher matrix

Natural gradient descent optimizes using the Fisher information matrix's inverse as the metric

Boosting (machine learning)

Boosting reduces bias in ML models

AdaGrad's learning rate decays to zero

AdaGrad adjusts learning rate by accumulating squared gradients, causing it to decay to zero as denominator grows exponentially

score matching does: learns the gradient of the log-density without normalizing

Matching score learns gradient of log-density without normalizing