Loading your workspace…
Loading your workspace…
Cold checks
These are not recall prompts for terminology. Each one starts with a failure, regression, or ambiguous experiment and asks you to isolate the cause before reaching for a fix.
No account required.
Your written answer stays in the page.
Only your self-assessment is saved in this browser.
During a preference-optimization run, training preference accuracy keeps rising. Blind human evaluation and factuality peak early, then fall. Responses also become longer.
Work the problem
A run moves from 8 to 64 GPUs. Per-device batch size and gradient accumulation stay unchanged, the loss uses mean reduction, and the learning-rate schedule is still expressed in epochs. Throughput improves, but the loss curve and final quality move.
Work the problem
A new pretraining mixture produces a large gain on a public benchmark but almost no gain on private task data. The benchmark questions and common solution writeups existed online before the crawl cutoff.
Work the problem
You must explain why tiled attention can be exact even though each query tile sees the key and value blocks one at a time and the softmax denominator is not known in advance.
An interrupted mixed-precision training job reloads the model and optimizer successfully. The first resumed batch has a different loss from an uninterrupted control run, even on the same hardware.
Offline retrieval recall says the supporting document is in the top five. Production answers remain wrong, and adding more retrieved chunks sometimes makes them worse.
Joint training improves a high-resource generation task while a low-resource classification task falls below its single-task baseline. Aggregate validation loss still looks healthy.
A serving stack meets its median latency target at steady load. When short chats and long-context requests share the queue, P99 time-to-first-token and inter-token latency both spike.
A multi-node DDP job runs for hours and then hangs. GPU utilization drops on some ranks while others remain inside a collective. Restarting with the same seed reproduces the hang near the same batch.