Bootstrapping samples with replacement to estimate distributions
Image: Rembrandt, Public domain, via Wikimedia Commons
Resampling (statistics)
Bootstrapping samples with replacement to estimate distributions
Resampling methods, including bootstrapping, involve creating new samples from an observed sample. Bootstrapping specifically uses resampling with replacement to estimate the distribution of a statistic.
Example
From a sample of 100 test scores, we repeatedly draw 100 scores with replacement to create new samples, then calculate the mean for each sample to estimate the distribution of the sample mean.
Bootstrapping helps estimate the variability and distribution of a statistic without relying on strong parametric assumptions.
Related concepts
Boosting (machine learning)
Boosting reduces bias in ML models
importance sampling does: reweights samples from proposal to estimate target expectation
Importance sampling reweights samples from a proposal distribution to estimate the expectation under a target distribution
GraphSAGE does: samples and aggregates a fixed-size neighborhood
GraphSAGE samples and aggregates a fixed-size neighborhood
Metropolis–Hastings algorithm
Metropolis-Hastings algorithm samples from difficult distributions
denoising score matching does: learns to denoise, which equals learning the score
Denoising score matching learns to denoise by estimating the score (gradient of log probability) of data distributions
batch size affects generalization: larger batches find sharper minima
Larger batch sizes lead to sharper minima, enhancing generalization by providing more accurate gradient estimates
One email a day: 5 concepts + the 5 stories that matter →
Swipe through 100 ML concepts daily
Open TickerNews