DeepSeek V4.1-Flash

DeepSeek V4.1-Flash is a concrete example of compressed sparse attention, not a synonym for FlashAttention or linear attention. Its model card describes a causal encoder–decoder arrangement: decoder global K/V are projected from final encoder hidden states rather than independently derived from each decoder layer's hidden states.

This differs from the 2017 translation Transformer's bidirectional source encoder. Similar architectural names do not imply identical masks or execution schedules.

Compressed Sparse Attention 2 assigns layers Full, Reindex or Reuse modes to share memory and reuse sparse selections. A hierarchical indexer narrows later candidate searches. FP4 main-cache storage and local-window replay address other parts of the memory problem. The diagram separates these mechanisms instead of treating all of them as “linear attention.”

The model card reports 552B backbone parameters, with 8B active per token during prefill and 16B during decode; Engram parameters are reported separately. Active parameters describe work selected per token, not the full checkpoint size.

A useful decomposition is:

$\text{serving memory} = \text{resident weights} + \text{global KV} + \text{local state} + \text{runtime buffers}$

Compressing cache entries changes their byte size. Selecting fewer entries changes what a query reads. Reusing entries across layers changes duplication. None of those alone proves linear total runtime: indexing, projections, feed-forward work and scheduling still matter.

Use explicit units when comparing reported cache figures. This arithmetic example deliberately uses an invented 1,024-byte-per-token budget, not a model measurement.

bytes_per_token = 1024  # Hypothetical budget, not DeepSeek telemetry.
tokens = 1_000_000
cache_gib = bytes_per_token * tokens / 2**30
print(round(cache_gib, 3))
# Add weights, local state and runtime allocations separately.

A deployment estimate needs the actual checkpoint and serving engine. Quantization metadata, replication and temporary allocations can invalidate an estimate based only on a headline cache rate.

Is sparse selection the same as compressing K/V? No. Selection chooses entries; compression changes their representation. A system can do either or both.

Why distinguish prefill and decode? Prefill processes an input prefix; decode produces additional tokens. Architectural work and cache reuse can differ between the two phases. One throughput number cannot characterize both.

Does “Flash” identify the kernel? No. Read the attention description and implementation. Compare FlashAttention with linear attention.

A system reduces stored K/V bytes but leaves every query reading all past entries. Has it necessarily reduced the number of pairwise score computations? No. Memory compression and interaction sparsity are separate claims.

Source: DeepSeek's official V4.1-Flash model card. Architecture claims reviewed October 2026; diagrams show data dependencies, not measured timing or independently reproduced performance.