constitutional AI does: model critiques and revises its own outputs using principles

MoE models have more parameters but similar compute cost

MoE models distribute parameters across k experts, reducing active experts' compute cost

Large language model

LLMs can generate, summarize, translate, and analyze text in many contexts

Curry–Howard correspondence

Proofs are programs, types are propositions

Prompt engineering

The GenAI model learns tasks from examples in the prompt

weight tying does in language models: shares embedding and output projection matrices

Tying reduces the number of parameters by sharing embedding and output projection matrices

Reasoning model

RLMs excel in logic, math, and programming tasks