PCA vs t-SNE: PCA preserves global variance linearly, t-SNE preserves local structure nonlinearly

t-SNE preserves local structure

t-SNE preserves local structure by converting distances to probabilities and minimizing Kullback-Leibler divergence

Dimensionality reduction

Dimensionality reduction transforms high-dimensional data into low-dimensional space while preserving meaningful properties

Principal component analysis

Eigenvectors point along maximum variance

to standardize: when you need zero mean and unit variance for gradient-based optimization

Standardize when zero mean and unit variance are required for gradient-based optimization

Eigenvalues and eigenvectors

Eigenvectors are unchanged in direction by a linear transformation

2024 in hip-hop

LoRA rank r controls model capacity and parameters