Skip to content
← All mechanisms

A small rank. A different space.

Change the adapter rank and watch its outputs open from a line into a plane and a volume.

Edit this visualization

Preparing the 3D view…

Constrain the correction

The base weight W stays frozen. A maps the input into r coordinates; B maps those coordinates back into the output space. Only the adapter correction BAx is shown.

Open another direction

In this three-output toy model, A selects the first r input coordinates and B contains the first r coordinate basis vectors. At rank one the outputs lie on a line, at rank two on a plane, and at rank three in a volume.

Read the limit

The points are fixed synthetic inputs, not measured model activations. In real training B also learns, so its subspace can change. The α/r scale, dropout, and frozen Wx contribution are omitted.

The equation behind the motion

h = Wx + BAx; rank(BA) ≤ r

Synthetic inputs with explicit geometric assumptions. Rotate the view and inspect the model above.

Read the primary source