to write a fused softmax kernel in Triton: load row, compute max, subtract, exp, sum, divide

`output = exp(row - max_val) / sum(exp(row - max_val))`

Image: RepRapPro, CC BY 3.0, via Wikimedia Commons

to write a fused softmax kernel in Triton: load row, compute max, subtract, exp, sum, divide

`output = exp(row - max_val) / sum(exp(row - max_val))`

Related concepts

One email a day: 5 concepts + the 5 stories that matter →

Swipe through 100 ML concepts daily

Open TickerNews