\|\mathbf{x}\|

Norm (length of a vector)

Operator: it does something

In plain English

The size of a vector. The usual (L2, Euclidean) norm is Pythagoras extended to any number of dimensions: $\|\mathbf{x}\| = \sqrt{\sum_i x_i^2}$ . A subscript changes the recipe: $\|\mathbf{x}\|_1$ sums absolute values; $\|\mathbf{x}\|_\infty$ takes the largest entry. Norms appear in regularisation (penalising large weights), in normalisation layers, and in distances between points.

As code

const norm = (x) => Math.sqrt(x.reduce((a, xi) => a + xi * xi, 0));
norm([3, 4]);   // 5

See it

Angle between‖b‖ = 2.50

\|\mathbf{x}\| = \sqrt{\sum_i x_i^2}

is a vector’s length, Pythagoras in any number of dimensions.

What it means, field by field

Mathematics

L1, L2, L∞ norms; distance ‖x − y‖; matrix norms.

AI / ML

Weight decay λ‖θ‖²; gradient clipping by norm; L1 sparsity penalties in sparse autoencoders.

Physics

Magnitude of a force or velocity; normalisation of a quantum state, ‖ψ‖ = 1.

In a research paper

Sparse autoencoder objective in interpretability research (cf. Bricken et al. 2023; Templeton et al. 2024): reconstruct activations while keeping features sparse.

\mathcal{L} = \|\mathbf{x} - \hat{\mathbf{x}}\|_2^2 + \lambda \|\mathbf{f}\|_1

Reconstruct the activation x faithfully, but pay a penalty proportional to how many features f are active.

Common misreadings

Confusing ‖x‖ (norm) with |x| (absolute value of a number or determinant of a matrix).
Ignoring the subscript. L1 and L2 penalties have very different effects (L1 encourages exact zeros).

Connected symbols

\mathbf{a}\cdot\mathbf{b}

Dot product (or multiplication)

\sum

Summation (capital sigma)

\lambda

Lambda

\mathcal{L}

Loss function (calligraphic L)

Where it appears on this site

§ 6. The Mathematics Inside Modern AI