\arg\min

Argmin (the input that minimises)

Operator: it does something

In plain English

Not the minimum value, but where it occurs. $\min_x f(x)$ is the lowest value of $f$ ; $\arg\min_x f(x)$ is the $x$ that achieves it. Training a model is, in principle, an argmin over parameters: find the $\theta$ that makes the loss smallest. In practice, training finds a good-enough $\theta$ , not provably the best one. $\arg\max$ is the mirror image (e.g. greedy decoding picks the argmax token).

As code

const xs = [-2, -1, 0, 1, 2, 3];
const f = (x) => (x - 1) ** 2;
const minValue = Math.min(...xs.map(f));      // min:    0
const argmin = xs.reduce((b, x) => (f(x) < f(b) ? x : b)); // argmin: 1

Where the analogy bends: Exhaustive search over a list. Real argmins over billions of parameters are approximated by gradient descent.

See it

Learning rate η0.35

\arg\min_\theta L

is the location of the lowest point, not the lowest value itself. Gradient descent is one way to search for it.

What it means, field by field

AI / ML

Training objective θ* = argmin_θ L(θ); greedy decoding = argmax over next-token probabilities.

Physics

Systems at low temperature settle toward energy minima; the principle of least action selects paths that make the action stationary.

Biology

Optimality models (e.g. optimal foraging) posit behaviours that maximise a payoff. Whether evolution actually achieves the optimum is an empirical question.

Mathematics

Optimisation; argmin may be a set (several minimisers) or not exist.

In a research paper

Compute-optimal allocation (Hoffmann et al. 2022, "Training Compute-Optimal Large Language Models", arXiv:2203.15556).

(N^{*}, D^{*}) = \arg\min_{N,D\ :\ 6ND = C} L(N, D)

For a fixed compute budget C, which model size and token count give the lowest loss?

Common misreadings

Confusing argmin with min, i.e. the location with the value.
Assuming a trained network sits at the argmin. Training finds a low point, not a certified global minimum.
Assuming nature "optimises" because a quantity can be written as an argmin.

Connected symbols

\mathcal{L}

Loss function (calligraphic L)

\nabla

Gradient operator (nabla, del)

\theta

Theta

\eta

Eta

Where it appears on this site

§ 6. The Mathematics Inside Modern AI