expected calibration error (ECE) measures: gap between confidence and accuracy

calibration means: a model predicting 80% should be correct 80% of the time

Calibration refers to a model's accuracy in predicting outcomes 80% of the time

Brier score

Brier score measures mean squared error of probability predictions

word error rate (WER) measures: edit distance between predicted and reference transcriptions

Word Error Rate (WER) measures the edit distance between predicted and reference transcriptions

Bias vs variance: high bias = underfitting, high variance = overfitting

High bias = underfitting, high variance = overfitting

Binomial proportion confidence interval

Binomial proportion confidence interval estimates success probability

log-loss / cross-entropy loss penalizes: confident wrong predictions more heavily

Log-loss penalizes confident incorrect predictions more heavily