Rotate by position; only the distance survives the dot product.
query at , key at , score
RoPE rotates the query and the key by angles proportional to their positions. Shift both tokens along the sequence and the angle between them, and so their score, never changes. Only the distance m − n matters. Real models use many pairs of dimensions, each turning at its own rate.