What importance sampling does: reweights samples from proposal to estimate target expectation

What rejection sampling does: samples from target by accepting/rejecting proposals

Rejection sampling generates samples from a target distribution by accepting or rejecting proposals based on a comparison with a uniform distribution

What score matching does: learns the gradient of the log-density without normalizing

Score matching approximates log-density gradients for variational inference without normalization

How does score matching utilize the Fisher Information Matrix to learn the parameters of a probabilistic model without normalizing the score?

Score matching estimates parameters by minimizing the Kullback-Leibler divergence between empirical and model score distributions

What AWQ does differently — activation-aware weight quantization preserves important weights

AWQ quantizes weights while preserving critical activation values for neural network efficiency

What sufficient statistics are — compress data without losing information about θ

Sufficient statistics for θ are those that capture all necessary information to estimate the parameter

What weight tying does in language models: shares embedding and output projection matrices

Language models use tied weights to share embedding and output projection matrices, enhancing parameter efficiency