Rosetta Stone · 27 of 30
Dot product (or multiplication)
Operator: it does something
In plain English
Between two numbers, is just multiplication. Between two vectors, it is the dot product: multiply matching entries and add them up. . Geometrically it is : large and positive when the vectors point the same way, zero when perpendicular, negative when opposed. Attention scores, similarity search and every neuron’s weighted input are dot products.
As code
const dot = (a, b) => a.reduce((acc, ai, i) => acc + ai * b[i], 0);
dot([1, 2, 3], [4, 5, 6]); // 32 : a Σ in disguiseSee it
What it means, field by field
Inner product; also written ⟨a, b⟩ or aᵀb.
Work W = F·d (only the force component along the motion counts).
Attention scores q·k; cosine similarity for embeddings; each neuron computes w·x + b.
Weighted sums in models of neuronal input. The biological neuron is far more complicated than this abstraction.
In a research paper
How aligned two embedding directions are, ignoring their lengths: 1 means identical direction, 0 unrelated.
Common misreadings
Assuming · always means dot product. Between scalars it is ordinary multiplication.
Treating a high dot product as "meaning similarity" without normalising. Long vectors give big dot products.