Chips Kept Buying Bandwidth but DeepSeek-V4.1-Flash Stopped Spending It

The Bottleneck Moved: Two Ledgers, the SRAM Predicament, and How DeepSeek-V4.1-Flash Rewrote Inference

Same GB200 NVL72 rack, same accounting, 128K context, three generations of DeepSeek models, theoretical decode throughput: DeepSeek-V3 delivers 928 tokens per second per GPU, DeepSeek-V4-Flash 46,964, DeepSeek-V4.1-Flash 83,363. The rack did not change. The chips did not change. The model changed twice, and the number went up 90×.

That 90× is what this article sets out to explain.

In July, Jarvis and I did something rather tedious. We took a single token's journey, from entering the model to the moment the next token comes out, and split every step into two ledgers: how much you compute, and how much you move. Then we priced each step in time against the spec sheet of every chip we could find. The report ran to 2,800 lines and its conclusion fit in one sentence: under almost any reasonable production configuration, decode is bandwidth-bound. You are waiting for bytes to arrive; the FLOPs finished long ago. On September 10, DeepSeek released V4.1-Flash, and I fed its architecture back into the same ledger. For the contexts agents actually use, up to 128K, decode flipped to compute-bound. At 1M context, the thing that binds turned out to be HBM capacity. At no context length did bandwidth become the tightest constraint.

The article has three parts. The first explains what the two ledgers are and why they decide everything. The second uses them to explain why machines that keep their weights in SRAM, Cerebras and Groq, struggle to compete head-on with HBM machines. The third looks at what V4.1-Flash actually changed, and why I think it may qualify as a shift in paradigm. I try not to use that word. This time I will spell out both the reason for using it and where it stops applying.

Part One: Two Ledgers. Compute, Bytes, and Whichever Is Slower Wins

1.1 Two costs per token

A large language model generates text in two phases. Prefill takes every token you typed, pushes the whole batch through every layer in parallel, and produces the first output token. Decode then emits one token at a time. Both phases use the same weights, and their accounting could not be more different.

The compute ledger is simple. In a dense weight matrix, every parameter gets multiplied once and added once per token, so the FLOPs per token come to roughly 2× the active parameter count. Prefill handles thousands of tokens at once; the weights are read from HBM a single time and serve the whole batch, which makes it compute-leaning by nature. Decode re-reads the entire weight set for every token it emits, and re-reads the KV Cache of every token seen so far, all to compute one token's worth of FLOPs. It is movement-leaning by nature.

The byte ledger has two entries. Weights are the fixed item: the bigger the model, the more you move. The KV Cache is the item that grows linearly with context: every extra token in history adds one more record to read per step. Whichever is larger is the main item.

The two ledgers reduce to one rule: the lower bound on any step's time is max(FLOPs ÷ compute, bytes ÷ bandwidth). Work out how long the math takes, how long the data takes, and keep the longer one. To know which side you are on, one ratio suffices. It is called arithmetic intensity, the number of FLOPs you get to perform per byte read from HBM. Every chip has its own balance point, peak compute divided by HBM bandwidth, which the industry calls the ridge point. A workload whose arithmetic intensity sits below the chip's ridge point leaves the chip waiting for data; above it, the chip is waiting for FLOPs.

The two ledgers on one chart: below a chip's ridge you wait for bytes, above it you wait for FLOPs

source: Jason & Jarvis inference ledger (Model × Hardware Chemistry report, July 2026; DeepSeek-V4.1-Flash ledger, September 2026)

Ridge points for the mainstream chips, on an FP8 basis: H200 412, H100 591, GB200 625, Vera Rubin VR200 875. The numbers climb generation over generation, then barely rise at Rubin, because HBM4 bandwidth jumped from 8 to 22 TB/s, a 175% increase that outran the gain in compute. The scarce input NVIDIA is buying more of is bandwidth. Its own spec sheet says so.

1.2 Running the ledger on one H100

An abstract rule needs a concrete example underneath it. Take a 52.6B-parameter dense model, 64 layers, standard multi-head attention, FP8 weights, BF16 KV, on a single H100, with an 8K-token input and a 512-token output.

Each decode step needs only 0.071 ms of compute but 20.99 ms of data movement, a gap of 300×. During generation, this chip spends 99.7% of its time waiting for data. Inside those 20.99 ms, FFN weights account for 48.9%, historical KV 25.2%, QKV projection weights 18.3%. At short context, what you move is mostly weights. Every additional token of history adds 2.10 MB of KV to read, and at roughly 25K tokens the KV stream overtakes the weight stream and becomes the main item.

The theoretical speed comes out to 47.65 tokens per second. Single-sequence decode has an arithmetic intensity of only 1 to 2 FLOP per byte, 100 to 1,000 times below the ridge point of any HBM chip. Swap in GQA, MLA, and sparse attention one at a time and the KV entry shrinks, but the weight stream keeps a floor around 15.7 ms per token. This was the July report's central conclusion: compressing KV removes the slope that grows with context, but it does not by itself flip decode to compute-bound. Part Three will come back and overturn half of it.

1.3 Why bandwidth is harder to raise than compute

If the bottleneck is movement, the obvious question is why nobody simply builds more bandwidth. The answer is a geometric law that has nothing to do with process node. Compute scales with chip area, which grows as the square of the edge length. The off-chip bandwidth that feeds that compute has to enter and exit through PHYs placed along the chip's edge, so it scales with the edge itself. Double the edge and compute goes up 4×, bandwidth 2×, and the ratio of bandwidth to compute halves. From 2012 to 2022, NVIDIA GPU FLOPS rose 80× while bandwidth rose 17×, by Google's own accounting.

Trying to route around off-chip bandwidth with on-chip SRAM does not work either. The area of a high-density SRAM bit cell stopped shrinking after TSMC's N5, at 0.021 µm², and N3E, N2, and A16 are all stuck at that same number. Logic keeps shrinking, SRAM does not, and SRAM's share of an advanced wafer keeps creeping up.

Cerebras's Wafer Scale Engine is the most vivid specimen of this law. WSE-1 carried 18 GB on-wafer, WSE-2 40 GB, WSE-3 44 GB. The latest generation added only 10% while logic transistors grew by about 50% over the same period. A single Blackwell Ultra package, meanwhile, carries 288 GB of HBM.

From first principles, the ideal hardware for a large model is a chip big enough to hold all the parameters and the entire KV Cache on-die, so that nothing ever has to be moved off it. That chip cannot be built. A lithography reticle exposes at most 858 mm² in one shot, yield falls with area, and SRAM no longer shrinks. So the whole industry is doing the same thing: stitching many chips into one "logically large chip" with interconnect. NVLink joins 72 GPUs into a single domain that presents itself to software as one chip. Huawei wrote the same idea as a formula: system performance equals supernode spec times single-chip spec.

1.4 The direction of the reaction is set by both sides

The most interesting thing about the two ledgers is that the outcome is decided jointly by the model and the hardware. The hardware side supplies the ridge point, the HBM capacity, the interconnect bandwidth. The model side supplies how many FLOPs each token needs, how many bytes each step moves, how much capacity each sequence occupies. Keep the chip and change the attention mechanism, and the bottleneck moves. Keep the model and change the hardware route, and it moves again.

That is why I call this intersection chemistry. In the public markets, the people who follow models rarely run the hardware numbers, and the people who follow hardware rarely track model architecture. Nearly every question that matters right now sits at the seam between the two. The next two parts are two cross-sections of that seam: first a bet on the hardware side, then a rewrite on the model side.

Part Two: Why SRAM Machines Struggle to Compete with HBM Machines

2.1 Two routes, one shared starting point

Inference chips split into two routes by where the weights live. The HBM route is NVIDIA, TPU, Trainium, Maia: weights and KV live in off-chip HBM and are fed into the chip at 3.35 to 22 TB/s. The SRAM route is Cerebras and Groq: the weights are pinned into on-chip SRAM, and the on-chip bandwidth is in a different class altogether. A single Groq LP30 LPU has 150 TB/s of on-chip bandwidth; a Cerebras WSE-3 wafer has 21 PB/s.

Part One just said decode is bandwidth-bound. By that logic the SRAM route ought to win, and the products seem to confirm it. Cerebras's official figures put Llama-405B at 969 tokens per second per user and gpt-oss-120B at 3,000, with Artificial Analysis independently measuring 1,822. OpenAI signed a 750 MW purchase agreement on the strength of it, now worth more than $20bn. The speed is real.

The trouble is three walls that have nothing to do with speed.

2.2 The first wall: it does not fit, so capacity dictates the chip count

Everything about an SRAM machine starts with whether the weights fit. The July report's reference model is a 526B-parameter MoE, 526 GB of weights in FP8. A Groq LPU has 500 MB of SRAM, so holding the weights alone takes 1,052 chips before you serve anyone. A Cerebras wafer holds 44 GB; 12 wafers hold the weights exactly, with zero budget left for KV, and zero users served.

Could the weights spill into off-chip DRAM? The ledger says that is a cliff. Prefetching one 3.94 GB expert from the DRAM expansion tier of a Groq rack takes 79 ms; reading the same bytes on-chip takes 26 µs, 3,000× faster. The ratio is on-chip bandwidth divided by off-chip bandwidth, independent of expert size; a real DeepSeek-class 44 MB expert gives the same 3,000×.

The only way around the cliff is to pin every weight into SRAM, which means stacking racks. Cerebras's solution is more extreme still: push the pipeline all the way to "one layer per wafer," 64 wafers in a chain, each holding one layer's 8.2 GB of weights and keeping 34.8 GB for that layer's KV.

2.3 The second wall: step off the chip and bandwidth drops 100×

The on-chip bandwidth advantage evaporates at the chip boundary. Cerebras has 21 PB/s on the wafer and 150 GB/s off it, a factor of 140,000. Groq has 150 TB/s on the chip, about 1.25 TB/s one-way chip-to-chip, a factor of 120, and 400 Gbps across racks, another 25×. Compare the HBM machines: GB200 goes from 8 TB/s of HBM to 0.9 TB/s of one-way NVLink, a factor of only 9, and 72 chips share that domain.

SRAM machines win by 1,000× on the chip and give it all back at the first boundary

source: vendor specs compiled in the July 2026 Model × Hardware Chemistry report; one-way NVLink = half the official bidirectional figure

This cliff rewrites the menu of parallelism outright. An HBM machine can run expert parallelism inside the NVLink domain, spreading a MoE's experts across 72 chips, with all 128 all-to-all exchanges per step across 64 layers costing about 0.5 ms in total. An SRAM machine cannot leave the rack; a 400 Gbps pipe across racks will not carry all-to-all traffic, and the only parallelism left is a deep pipeline: weights parked layer by layer along a chain of chips, tokens marching forward through the layers.

A Cerebras token crosses 64 chassis boundaries per generated step, each with a fixed 5 to 10 µs of latency. A Groq token hops 3 to 10 chip-to-chip links per layer at 0.5 to 1 µs per hop. More than 97% of Cerebras's TPOT goes into these fixed delays, and Groq's is likewise dominated by inter-layer latency once the KV is compressed. None of that time has anything to do with bandwidth, or with compute.

2.4 The third wall: capacity is a third kind of constraint, and it caps concurrency

Compute and bandwidth cap the time per step. Capacity caps how many users you can serve at once. Whatever SRAM is left after the weights is how much KV you can hold. Six Groq racks, 1,536 chips, leave a 222 GB KV pool after the weights. With standard multi-head attention, one 8K sequence of the reference model needs 18.25 GB of KV, so the system serves 12 users. With MLA compression, 345. The equivalent figures for 64 Cerebras wafers are 122 and 3,468.

What makes the capacity constraint so stark is that when it binds, no pipe is full. The compute sits almost idle, the on-chip bandwidth sits idle, and the machine simply cannot take another customer. Admitting one more costs 18.25 GB or 0.64 GB of on-chip space, and neither compute nor bandwidth has anything to do with it.

Long context shoves this wall in your face. KV grows linearly with context. Stretch the context from 8.7K to 400K, and six Groq racks in the MLA tier fall from 345 concurrent sessions to 7: an \$11M system serving seven agent sessions at a time. In the standard-attention tier, nothing fits at all from 200K up. Adding racks is the only way out, and each \$1.85–1.9M rack adds only 128 GB, which works out to \$469K per concurrent 400K session, 56× the \$8.3K on a GB200 NVL72. Cerebras's one-layer-per-wafer layout has some resilience, since a single sequence can reach 1M, but concurrency collapses from 3,468 to 75.

Capacity runs out first: concurrent sessions each system can hold versus context, on the 526B reference MoE

source: Jason & Jarvis inference ledger (July 2026 report, §5.4 and §5.6)

There is an unlucky pairing here. High interactivity is the SRAM machine's home turf and long context is its weak spot, and agent workloads bind the two into the same request. A coding agent wants hundreds of tokens per second of response and 200K or more of context at the same time. Cerebras's production deployments cluster at short context, and the bigger the model the slower the per-user speed, 3,000 tokens per second for GPT-OSS-120B versus 750 for GPT-5.6 Sol. That is consistent with the ledger.

2.5 The economics: what multiple must a fast token sell at to break even

Convert the three walls into money, with GB200 NVL72 as the 1.00× baseline at $3.0M per rack and 0.12 MW:

Cerebras bought real speed, and the capital bill rose faster than the speed did. To break even at equal lifetime and utilization, its tokens must sell for at least 1.1–2.3× what a GB200 token fetches. Against the next-generation VR200 that becomes 1.8–3.6×, and in the batch tier for 2T-class models, 5–10×. Those three numbers are the bar the SRAM route has to clear commercially. Groq's ledger is harder still to fill with a speed premium: its capital per unit of throughput runs 3.4 to 9.8× the baseline.

Two things must be said in fairness. First, throughput per megawatt is not a natural weakness of SRAM; Cerebras's 2.8–5.4M tokens per second per MW is comparable with NVL72's 5.8M in the same tier, and the gap is concentrated in capital and in utilization risk. Second, OpenAI really is paying for speed, but the contract price per megawatt shows no fast-token premium, and the bar remains a business proposition still waiting to be proven.

2.6 Can it be repaired

Turn the same ledger around and ask how far each machine would have to be patched to get back onto the price-performance line. On the SRAM path, each Groq LPU would need to go from 0.5 GB to 1.5–2 GB, and each Cerebras wafer to 46–86 GB. No planar design can hold any of those parity points; every one of them lands in territory that requires 3D stacking.

The network path is more blunt. Unlocking expert parallelism requires roughly 1 µs per hop, injection bandwidth in the terabytes per second per chip, and predictable determinism. Put those lines together and you have written the spec sheet of an NVLink domain.

Followed to its logical end, that path means conceding that the KV must live in someone else's HBM while you handle only weight residency, which is to say becoming a coprocessor in somebody else's rack. NVIDIA has already walked Groq down that road (last December's licensing deal was the starting point): the LPX rack is a rack-level companion, and the next-generation LP40 adopts the NVLink protocol and stacked memory, co-designed with the Feynman platform. Cerebras remains outside the system, and what it has to prove is that fast tokens can command a premium, something its public pricing has yet to show. My July call was that the window for SRAM machines to compete against the mainstream as standalone systems is 2026 to 2028.

This part compresses to one sentence: make one stave the longest in the world, and the barrel still holds water only to the shortest. Groq pushed on-chip bandwidth to 6.8× the R200's and then got pinned by 400 Gbps across racks. Cerebras pushed on-chip bandwidth three orders of magnitude above the fastest HBM and then got pinned by 150 GB/s off the wafer. Point optimization gets stuck off-chip.

Part Three: DeepSeek-V4.1-Flash, or the Model Rewrote Its Own Ledger

The first two parts rest on the premise that decode is bandwidth-bound. That premise comes from a fact on the model side: each token moves too many bytes per step. V4.1-Flash changed exactly that fact.

3.1 What it did: compress the notebook to 890 bytes per token

Start with the model itself. V4.1-Flash has 552B backbone parameters, plus 196B of Engram lookup-table memory kept in host memory. It activates 8B parameters per token in prefill and 16B in decode, supports 1M context natively, and takes multimodal input. All of its tension sits in one number: the global KV Cache costs 890 bytes per token. DeepSeek-V1 needed 389,120 bytes, DeepSeek-V3.2 48,068, V4-Flash 3,514, and this generation 890, a 3.9× compression over its predecessor and 437× over V1. A single 1M-context sequence occupies 0.83 GB of HBM in this generation versus 3.27 GB in the last.

Global KV Cache bytes per token across DeepSeek generations (right) and agentic benchmark results (left)

source: DeepSeek-AI, DeepSeek-V4.1-Flash Technical Report (September 2026), Figure 1

The compression comes from three layers, each attacking a different cost.

Saving in the architecture. The 40-layer network is split into a 20-layer causal encoder and a 20-layer decoder, an arrangement called CED. The decoder's 20 layers no longer derive their global KV from their own hidden states; they project it directly from the encoder's layer-20 output, each with its own projection matrices.

Think of an open-book exam. The old way read the book through all 40 layers, taking a fresh set of notes at each one. The new way reads carefully through the first 20, and the notes for the back 20 are derived from layer 20's result, cutting the reading work in half.

The second layer is CSA2, which has a roomful of people share one set of meeting minutes. The encoder's 18 layers keep only 3 copies of KV; the decoder's 20 layers keep only 1. Each layer retains its own query vector and its own short-term memory of the most recent 128 tokens; what is shared is only the thick notebook.

The third layer is the hierarchical sparse indexer. The decoder's first full layer scores every position and, along the way, shortlists 16,384 candidate positions; the layers behind it pick their own top-512 from that pool only. However long the context grows, the scoring cost of the later layers stays put.

Overall architecture of V4.1-Flash: a 20-layer causal encoder and a 20-layer decoder, with the decoder's global KV projected directly from the encoder output

source: DeepSeek-AI, DeepSeek-V4.1-Flash Technical Report (September 2026), architecture overview

Saving in precision. The main KV is stored in MXFP4, one E4M3 scaling factor per 16 channels, dropping NVFP4's second, global scaling factor. The justification is a numerical bound you can check yourself. The format represents values up to 2,688, while a normalized 512-channel KV latent cannot exceed 22.6 in absolute value on any single channel, and the largest value observed in training is about 10. It is the cleanest argument in the paper: the conclusion rests on a bound rather than on "the experiments looked fine."

Saving in deployment. In the previous generation's deployment, the sliding-window attention KV took up nearly half of the persistent cache while being useful only within the few-minute window of one live session. This generation simply does not store it. When needed, it replays only the most recent 128 tokens, turning a catastrophic cache miss into a cheap degradation. The persistent cache shrinks to about 1/8 of the previous generation's.

The product of the three layers is in the report's Figure 2: stretching context from 4K to 1M, a 256× increase, raises single-token decode FLOPs by only a quarter. The marginal cost of decoding at long context is close to zero.

3.2 How strong it is, and where it loses

With about one third of DeepSeek-V4-Pro's parameters and one quarter of its activated parameters, it scores 74.2% on DeepSWE v1.1, a hair above Claude Opus 5's 74.0%. On Terminal-Bench 2.1 it posts 90.6% against Opus-5's 89.1%. On CyberGym it leads every model in the comparison at 88.1%. Its Codeforces rating is 3,471, above V4-Pro's 3,348. On the skilled-trade tasks of everyday coding, terminal operation, and office automation, it has caught up with the frontier closed models and in places passed them.

Where it loses is equally clear. Humanity's Last Exam: 36.8 versus Opus-5's 56.3, a gap of nearly 20 points. Terminal-Bench 3.0: 30.0% versus 43.3%. The base model trails V4-Pro by 12.9 points on SimpleQA. On tasks that demand expert domain knowledge and the hardest reasoning, the distance remains obvious. This matches the piece I wrote in April, What a Model Doesn't Know Tells You How Big It Is: factual knowledge does not compress, and a model with two thirds fewer parameters is going to pay for it on knowledge-heavy questions. The report waves this cost away with "fully on par." I am not going to wave it away.

One sentence in the report matters more than any benchmark. It states plainly that post-training contains no algorithmic novelty: the recipe is the standard SFT, RL, and distillation, and almost all of the gains come from systematic improvements in the scale, diversity, and verifiability of synthetic data and environments. If that holds, the competitive focus of this phase lies in the engineering of building tasks, environments, and verifiers at scale.

3.3 Back into the two ledgers: bandwidth is no longer the tightest constraint

Now feed V4.1-Flash's parameters into Part One's ledger and line it up against V3 and V4-Flash on the same basis.

Two ledgers across three DeepSeek generations: V4.1-Flash flattens both, and throughput follows

source: Jason & Jarvis inference ledger; V3 at FP8 weights, V4 and V4.1 at FP4 expert weights, all Roofline theoretical values

The left panel is the compute ledger. V3 uses dense MLA attention, which performs 34 MFLOP per token for every position in history, so at 1M context a single token costs 35.7 TFLOP. V4.1-Flash goes from 39 GF at 4K to just 50 GF at 1M.

The middle panel is the byte ledger, the KV each sequence reads per step. At 128K, V3 reads 4.6 GB, V4-Flash 68 MB, V4.1-Flash 36.8 MB. The formula is 180 bytes times context length plus a fixed 13 MB. Only the indexers of the 4 full layers scan the whole context; every other layer's read per step is independent of context length. The right panel is what those two ledgers produce on the same GB200 rack, which is the set of numbers this article opened with.

Reading the arithmetic intensity: at 8K context, V4.1-Flash computes 39 GF and reads 14.7 MB per token, 2,655 FLOP per byte; at 128K, 1,095; at 1M, 246. The mainstream chips' ridge points sit between 412 and 875.

The crossover point differs by chip: 168K on VR200, 264K on GB200, 284K on H100. So across the 32K to 128K range that agents live in, as long as HBM can hold enough batch to saturate the compute, decode is compute-bound on every mainstream HBM chip. The sentence at the end of Part One, "compressing KV does not by itself flip decode to compute-bound," held for July's reference model. That model still read 622 MB per sequence per step, and the batch could not grow. V4.1-Flash reads 14.7 MB per sequence per step, so the batch grows and keeps growing until compute saturates first.

At 1M context the arithmetic intensity falls back below the ridge, and you would expect a return to bandwidth-bound. The ledger's answer is capacity instead.

Which constraint binds first: V4.1-Flash on GB200 across context lengths, with the bandwidth line crossing the capacity line only once, at 12K

source: Jason & Jarvis inference ledger (DeepSeek-V4.1-Flash ledger, September 2026, §4.4)

The reason has two halves. On the chip side, the time to read one full pass of HBM equals capacity divided by bandwidth. For GB200 that is 186 GB over 8 TB/s, 23 ms; for H100, 24 ms; for VR200, 14 ms. All of them sit in the same range as the 25 ms interactivity budget. Vendors deliberately size bandwidth so that one step is just enough to read the whole HBM once.

On the model side, V4.1-Flash re-reads only 22% of its resident KV each step. The 720 bytes of main KV are fetched only as top-512 per layer, effectively cold data; only the 180 bytes of indexer keys are scanned in full every step, the hot data. Divide the two, and the concurrency that bandwidth allows is always 3 to 5.6× the concurrency that capacity allows. HBM fills up first, and bandwidth runs at 20% to 30%.

V3, a dense model, re-reads 100%; its capacity and bandwidth lines nearly coincide, and both constraints top out together. In fairness, V4-Flash had already pushed the re-read fraction down to 13%, and bandwidth was not its tightest constraint either. What V4.1 changed relative to its predecessor is the other two ledgers: resident bytes cut to a quarter, and compute that no longer grows with context.

That is the precise meaning of "moved" in the title. At short context, compute binds. At long context, capacity binds. On this model, bandwidth is never the tightest constraint at any context length.

3.4 After the move, every chip's ledger changed

There are four direct consequences.

Throughput per chip is nearly ten times higher. At 128K context with TPOT under 25 ms, a GB200 NVL72 reaches a theoretical 83.4K tokens per second per chip, against 9.7K for July's reference model in the same setting. Applying the 0.5–0.7 efficiency band back-calculated from DeepSeek-V3 measurements, that is 42K–58K per chip and 3.0M–4.2M per rack.

Converted to hardware cost on a TCO basis, an output token at 128K context costs about \$0.01 per million on GB200, \$0.03–0.05 at 1M, and an input token \$0.004. Against the official price list, V4.1-Flash output tokens are \$0.60 per million off-peak and \$1.20 at peak, and cache-hit input is \$0.003–0.006. The distance between price and hardware cost is 60 to 120×, and that distance holds utilization, queueing, recomputation on cache misses, and gross margin.

A single node can serve it. Expert weights in FP4 come to only 289 GB. Eight H20s in one node hold the model with a 355 GB KV pool to spare, enough to keep about 3,000 sequences resident at 128K context. A single H20 node reaches 51K tokens per second in theory, 26K–36K after calibration, roughly twice the 14.8K that V3 measured on an H800 node in its day, on a chip with 15% of the H800's compute. A chip with 15% of its rival's compute produced twice the node throughput at 32× the context length.

At 1M context, capacity is a linear formula for output. Concurrency per chip equals usable HBM divided by 936 MB: 286 sessions on a 288 GB chip, 177 on 186 GB, 64 on 80 GB. GB300, VR200, and TPU 8i tie in this tier because all three carry 288 GB of HBM. VR200's 20 TB/s shows up here as per-user speed, 189 tokens per second against GB200's 128, while the session count looks only at capacity. The extra 100 GB that GB300 holds over GB200 buys only 3% more throughput at 128K, and 62% more concurrency at 1M.

Sessions per chip at 1M context (top) and per-user speed at that load (bottom)

source: Jason & Jarvis inference ledger (DeepSeek-V4.1-Flash ledger, September 2026, table A1-1024K)

Short context is actually more expensive. Activated parameters in decode rose from V4-Flash's 13B to 16B, so per-chip throughput at 8K is 0.5–0.9× V4-Flash's. The two break even at 32K; V4.1 pulls ahead to 1.2–2.3× at 128K and 2.2–3.7× at 1M. The gains from KV compression only cash out at long context. For short conversations it is a net step backward.

Solid lines V4.1-Flash, dashed V4-Flash: behind at 8K, even at 32K, ahead by 2 to 4× at 1M

source: Jason & Jarvis inference ledger (DeepSeek-V4.1-Flash ledger, September 2026, table D)

Intuition raises one more worry: with an encoder only half as deep, does time to first token suffer? Break TTFT into its four components and compute each, and the conclusion is the reverse. A fresh 1M prompt on eight H800s takes 2.7 seconds on V4.1-Flash versus 8.9 on V4-Flash; one agent turn of 2K new tokens plus a 1M prefix reloaded from host memory takes 46 ms versus 126. Its real costs lie elsewhere, in three places. Short context is more expensive; with three approximations stacked, the same prompt can yield different outputs depending on where the cache hit lands; and Engram requires 196 GB of free host memory on every machine.

Part Four: Why It May Be a Paradigm Shift

I try not to use the word "paradigm." Here it refers to one specific thing. For the past two years, the chemistry between models and hardware ran in one direction: the hardware bought bandwidth on the model's behalf. HBM3e to HBM4, NVLink domains from 8 to 72, Rubin CPX carving prefill out onto GDDR7, Cerebras and Groq moving the weights into SRAM outright. DeepSeek has been pushing the other way since MLA in 2024, with the model cutting its own bytes per step. V4.1-Flash pushed that direction to the point where, on the ledger, bandwidth no longer binds at any context length; the constraint was handed to compute and to capacity.

This reversal of direction carries four implications.

The variance in the numerator comes from the model; the variance in the denominator comes from the chip. Inference ROI is throughput divided by capital plus power. The July report already worked this out: on the denominator side, CAPEX and power across vendors differ by 4.7 to 6.8×, while on the numerator side, changing the attention mechanism alone can move things 55×. This time the same play was staged with real models. On one GB200 rack at 128K context, swapping the model from V3 to V4.1-Flash is 90×; swapping the chip from GB200 to VR200 is 2.7×. Whoever holds the model-side variables holds the larger share of the variance in ROI.

At long context, HBM capacity is worth more than HBM bandwidth. When I wrote Small by Design or by Default in March, the memory formula constrained how large a model you could load. It has come back in a different form: serviceable concurrency at 1M context is usable HBM divided by 936 MB, almost independent of bandwidth. For NVIDIA, that means GB300's extra 100 GB now has a priceable use on long-context agent workloads. For TPU 8i, it is why 288 GB of HBM puts it level with GB300. For HBM makers, what this generation compresses is the bytes per sequence; what it cannot compress is the demand that "more sessions" places on capacity.

Weak chips get a second life, and half of my April judgment needs correcting. When I wrote the V4 Pro piece, I said that each successive DeepSeek model would carry one more engineering concession than the last, because the ceiling had not moved. The ceiling has indeed not moved; eight H20s carry the same HBM they always did.

But what V4.1-Flash did is different from walking up to the wall. It changed which wall matters. The concessions are still there, the short-context cost, the regression on knowledge-heavy tasks, the nondeterminism introduced by approximation, and I have not skipped one of them above. Put the concessions together, though, and what they bought is a chip with 15% of the H800's compute delivering twice the node throughput at 128K context. That is a path I did not have in my April arithmetic.

DeepSeek's own choice is blunter than my correction. The official pricing page states that V4.1-Flash comprehensively surpasses V4 Pro in performance, cost, and speed, and that from September 14 all V4 Pro requests are routed to V4.1-Flash and billed at the Flash price. The model that walked up to the 1,128 GB ceiling at 1.6T in April was benched five months later by a 552B model.

For SRAM machines, V4.1-Flash solved one problem and deleted one argument. Using the method of table 4a in the July report, I ran a rough estimate of six Groq racks serving V4.1-Flash. This is my extension, not a figure from the report. With 552 GB of FP8 weights pinned into 768 GB of SRAM, the KV pool is about 196 GB, enough for roughly 209 sessions at 1M context and about 1,600 at 128K. The single-digit-concurrency death zone is gone. That is the good half.

But capital per unit of throughput still runs 2 to 14× GB200's, a gap of the same size as on the old model, because V4.1-Flash lifted both sides' output together. And the on-chip bandwidth these machines are built around has lost its opponent on a model where bandwidth no longer binds. What remains is one product: per-user speed of 2,000–5,000 tokens per second (rough, uncalibrated) against a calibrated 180–310 on HBM racks. The commercial proposition of the fast-token premium has not changed at all. What has vanished is the backdrop in which "long context collapses every machine's output by 40×": GB200 drops only 3.7× from 128K to 1M.

There is one more implication I am not confident about, and can only file as something to watch. V4.1-Flash is the first technical report I have read that writes a three-tier memory hierarchy into the architecture itself. Global KV lives in HBM; sliding-window KV lives in a distributed pool of host DRAM; persistent KV lives on SSD; Engram lives in host memory and is prefetched over RDMA. The model has started to design itself around the memory hierarchy.

If that path holds, the next binding item is already visible in outline. Of the 2.2 ms in a single-stream decode step, 1.32 ms is the fixed overhead of kernel launches, and host memory capacity and PCIe bandwidth have become deployment constraints. The chip boundary is about to move once more, this time toward the CPU side.

Three falsifiable readings. First, if DeepSeek publishes inference-system data as it did in the V3 era, look at single-node H800 decode throughput at 128K context. My predicted range is 47K–66K tokens per second, and 26K–36K for a single H20 node. If measurements come in more than 30% below the bottom of the range, the fixed-overhead or load-balancing assumptions were wrong.

Second, price. V4.1-Flash launched with output tokens at \$0.60–1.20 per million, against \$1.98–3.96 for V4 Pro. If this article's output ledger holds, Flash has room to cut further. If it has not dropped another notch within a year, there is something in the production efficiency or the business choices that the ledger does not see.

Third, data on the joint degradation of the three stacked approximations under real traffic, which the report itself admits it does not yet have.

The last sentence of the July report was: the winner is whoever can hold all six design variables in one hand, not whoever makes any single one of them the longest. At the time I assumed the hand holding the variables would belong to a hardware company. Two months later, the lab with the weakest hardware in its hand held the model-side variables tightly enough to change which hardware-side variable matters.

On the next chip's spec sheet, which number will a model architecture render irrelevant first?