Skip to content
Science, DecodedContext Jamming

Rosetta Stone · 12 of 30

η\eta

Eta

Naming convention: meaning depends on the field

In plain English

In modern ML papers, the learning rate: how far to move along the negative gradient on each update. Too small and training crawls; too large and it overshoots and diverges. The convention is strong in ML, but physicists use η\eta for viscosity or efficiency, and in cosmology it is conformal time.

As code

const eta = 0.01;                       // learning rate
theta = theta.map((t, j) => t - eta * grad[j]);

See it

η\eta sets the step size. If it is too small the steps crawl, and if it is too large they bounce across the valley or blow up.

What it means, field by field

AI / ML

Learning rate / step size; usually scheduled (warmup, decay) in real training runs.

Physics

Viscosity of a fluid; efficiency of an engine; the metric tensor η_μν in special relativity.

Biology

Occasionally efficiency or a rate constant; no universal convention.

In a research paper

Stochastic gradient descent with a learning-rate schedule η_t (used, with Adam variants, throughout LLM training).
θt+1=θt−ηt ∇θL(θt)\theta_{t+1} = \theta_t - \eta_t\,\nabla_\theta \mathcal{L}(\theta_t)

Each update steps downhill by an amount set by the current learning rate.

Common misreadings

  • Thinking η is learned. It is a hyperparameter chosen (or scheduled) by the engineer.

  • Assuming a single learning rate describes production training. Real runs use schedules and adaptive optimisers.

Connected symbols

∇\nabla Gradient operator (nabla, del)θ\theta Thetaarg⁡min⁡\arg\min Argmin (the input that minimises)L\mathcal{L} Loss function (calligraphic L)

Where it appears on this site

∂ Gradient descent update§ 6. The Mathematics Inside Modern AI