The most frequent pair merges, again and again, into tokens.
Vocabulary symbols, corpus tokens
BPE builds a vocabulary by repeatedly merging the most frequent pair of adjacent symbols. Pairs are counted across the corpus and weighted by how often each word occurs; the amber pair is the next merge and the purple tokens are ones merges have already made. Step through the ten merges and watch frequent words such as “newest” collapse into a single token.