iterative methods (CG, GMRES) do: solve Ax=b without explicitly inverting A

second-order methods (Newton's) converge faster but are expensive: O(n³) per step

Second-order methods converge faster due to quadratic convergence but are expensive due to O(n³) per iteration

Finite element method

Runge-Kutta method improves Euler by providing higher-order accuracy with k₁,k₂,k₃,k₄

Euler method

Euler method approximates ODE solution with y_{n+1} = y_n + h·f(y_n)

Master theorem (analysis of algorithms)

Master theorem solves T(n) = aT(n/b) + f(n) recurrences

approximation algorithms guarantee: solution within factor α of optimal

Approximation algorithms guarantee a solution within a factor α of the optimal solution

natural gradient descent does: preconditions with inverse Fisher matrix

Natural gradient descent optimizes using the Fisher information matrix's inverse as the metric