Eigenvectors point along maximum variance
Principal component analysis
Eigenvectors point along maximum variance
Eigenvectors in PCA represent the directions of maximum variance in the data, revealing the principal axes.
PCA transforms data into a new coordinate system where principal components capture the largest variation. These components are orthogonal unit vectors that best fit the data while minimizing squared perpendicular distances.
The principal components form an orthonormal basis, making individual dimensions uncorrelated. This basis simplifies data visualization and analysis by focusing on the most significant variation directions.
Example
In a dataset of student grades, PCA might reveal that the first principal component captures the variance due to overall academic performance, while the second captures variance related to participation in extracurricular activities.
Understanding principal axes through eigendecomposition helps in reducing dimensionality and identifying key patterns in data.
Related concepts
Eigenvalues and eigenvectors
Eigenvectors are unchanged in direction by a linear transformation
Ordinary least squares
OLS minimizes squared differences
PCA vs t-SNE: PCA preserves global variance linearly, t-SNE preserves local structure nonlinearly
PCA: Linear variance preservation, t-SNE: Nonlinear local structure preservation
List of unsolved problems in mathematics
Random points in high dimensions are nearly equidistant due to the uniform distribution of volume in high-dimensional space
sinusoidal position encoding works: each dimension has a different frequency
Sinusoidal position encoding assigns unique frequencies to each dimension, enabling the model to distinguish positions effectively
ill-conditioned matrices cause numerical instability: small input changes → large output changes
Ill-conditioned matrices amplify input perturbations, leading to significant output variability
One email a day: 5 concepts + the 5 stories that matter →
Swipe through 100 ML concepts daily
Open TickerNews