the revelation principle says: any mechanism can be converted to a truthful one

Curry–Howard correspondence

Proofs are programs, types are propositions

mechanism design does: designs rules so rational agents produce desired outcomes

Mechanism design: Creates rules ensuring rational agents achieve intended outcomes

Neural scaling law

Chinchilla scaling law: optimal model size scales linearly with compute budget

CAP theorem states: you can have at most 2 of consistency, availability, partition tolerance

CAP theorem: A system can achieve at most two out of Consistency, Availability, Partition Tolerance

eventual consistency means: all replicas converge to the same state given enough time

Eventual consistency: All replicas converge to the same state given enough time

the tokenizer's special tokens do: [CLS], [SEP], [PAD], [MASK] have specific roles

[CLS] marks the start of input, [SEP] denotes separation, [PAD] fills space, [MASK] hides words for prediction