ReLU and Leaky ReLU

Write the contrastive loss function for SimCLR

Contrastive loss function: L = (1/2N) Σ [max(0, margin - y_i * (z_i - z_j))^2 + max(0, y_i * (z_i - z_j) - margin)^2]

Write the triplet loss formula: max(d(a,p) - d(a,n) + margin, 0)

Triplet loss formula: max(d(a,p) - d(a,n) + margin, 0)

Write the Bellman equation for reinforcement learning

Bellman equation: V(s) = max_a [R(s,a) + γ Σ P(s'|s,a) V(s')]

Cross-entropy

Cross-entropy loss equation: H(p, q) = -Σ(p(x) * log(q(x)))

Gradient descent

Gradient descent weight update equation: w := w - α * ∇J(w)

Adam optimizer weight update with m and v terms

Adam optimizer weight update: w_t = w_{t-1} - α * m_t / (sqrt(v_t) + ε)