The Ceiling Didn't Move — They Walked to It: The1.6TEngineering Bill Inside DeepSeek V4

A month ago I wrote a piece called Small by Design or by Default? What a Memory Formula Reveals About China's AI Ceiling. The conclusion was simple: on 8×H200 — currently the strongest inference configuration Chinese labs can get at any meaningful scale, with the qualifier that 8-card B300 servers do trickle through the market while 72-card rack-scale systems remain off-limits inside China — the memory-constraint formula for MoE inference draws a fairly sharp ceiling. The economically viable parameter cap sits around 0.6T. Stretch every string to its limit (PD disaggregation across 18 nodes, paired with MLA-style latent attention compressing the KV cache) and the theoretical ceiling reaches 1.6T— paid for with a 29× cross-node bandwidth penalty.

Four weeks later, DeepSeek V4 Pro shipped with 1.6Ttotal parameters.

That number isn't a coincidence. It is the line.

But after reading the technical report, SemiAnalysis's testing run on InferenceX, and the zero-day adaptation video from Huawei's CANN team, "a Chinese lab ran into the ceiling" turns out not to be the most interesting part of this story. The real question runs in another direction: what did they pay to walk to 1.6T— and can those costs be brushed off with the phrase "engineering optimization"?

My answer is no. This time really is different from before.

A Short Recap — What That Line Looks Like

If you haven't read the prior piece, the formula itself fits in a single line:

Here d is the share of total parameters occupied by the MoE shared trunk (embeddings, attention, router, output head) — roughly equal to the active ratio (V4 Pro is 49B /1.6T ≈3%); N is the GPU count. The intuition: expert weights can be sharded across GPUs via expert parallelism, but the shared trunk has to be replicated in full on every card — that piece is a "fixed tax" that doesn't get diluted by N.

Plug in 8×H200 (141GB per card, d=3%): denominator ≈0.151, theoretical loading ceiling ≈935B in FP8 precision. After deducting the 40–50% of HBM that KV cache and activations typically consume, the economic range lands at 0.6T. Treat every card as a dedicated EP server, layer in PD disaggregation across 18nodes, push attention through MLA-style latent compression to drive KV cache to its limit — and the theoretical ceiling, still in FP8, can be stretched to 1.6T, paid for with a 29× cross-node bandwidth penalty.

That is exactly the slot V4 Pro's 1.6T fell into. But two clarifications need to come first, or the rest will get muddled:

First, the assumption used in the prior article to derive the 1.6T extreme — "PD across 18nodes + full-network MLA" — doesn't fully overlap with V4's actual path, but it isn't entirely disjoint either, so it needs unpacking. V4's attention is Hybrid Attention: 30 layers of CSA + 31 layers of HCA alternating (Pro configuration). The CSA half is in fact built on top of MLA — queries go through latent compression (V4 Pro's d_c =1,536), KV is shared in MQA fashion, and the MLA "absorption" operation is preserved; on top of that, a Lightning Indexer is layered to perform sparse top-k selection (top-k=1024). The other HCA half does not use MLA absorption, opting instead for 128× heavy compression plus a sliding window. So the more precise statement: V4 shrinks full-network MLA down to half the layers and lets HCA take the other half — which sidesteps the other leg of my original assumption, multi-node PD disaggregation (SemiAnalysis's CI matrix explicitly logs disagg-false; this is a single-node deployment).

Second, the original formula defaults to FP8 precision (1B parameters ≈1GB), but V4 Pro switched the weight storage directly to native FP4 —1B parameters ≈0.5GB — halving bytes per parameter. That effectively releases 2× headroom in both the shared trunk and the expert shards simultaneously, lifting what was an FP8 ceiling of 935B straight up to roughly ~ 1.87T. This precision lever, combined with Hybrid Attention's compression of the KV cache, is the real machinery that lets 1.6T fit on a single 8×H200 node. Note that H200 captures only the FP4 storage benefit, not the compute benefit — Hopper has no native FP4 cube, so every GEMM must dequantize to FP8 first — a caveat the throughput-collapse section below will return to settle.

Putting both together: the assumed path in the original article was "FP8 + full-network MLA + PD across18nodes, multi-node"; V4's actual path is "FP4 storage + half MLA (CSA) + half non-MLA heavy compression (HCA) + single-node." Both are ways of arranging optimizations to fit1,128GB of HBM — just different arrangements. V4 trades "half the MLA" for "half HCA" on the attention side, squeezes one more cut out of precision via FP4, and swaps multi-node for single-node simplicity on deployment. The fact that they converge on the same number underscores that this wall's binding constraint is not any specific compression technique or precision choice — it is the1,128GB HBM hard cap itself, and the cascade of engineering concessions that follow once you press up against it.

Fitting 1.6T Inside Means Flipping Three Switches at Once

V4 Pro really does run on a single 8×H200 box (the SemiAnalysis InferenceX repo PR #1130CI run spells it out: single-node, tp8 ep8, disagg-false), and it does so by stacking three things together.

First is native FP4 QAT. The V4 Pro release ships its checkpoint directly in FP4 — not post-training quantized, but trained that way. Expert weights (90%+ of total parameters) compress from a theoretical 1.6TB (in FP8) down to roughly 720GB. As the vLLM blog confirms in plain language: "the model ships with native fp4 MoE weights."

Second is Hybrid Attention. The V3.2 baseline carries 83.9GiB of KV cache per sequence at 1M context; V4 Pro carries only 9.62GiB — an 8.7× reduction. Architecturally it is30layers of c4a (1/4KV compression, top-k 1024) alternating with31layers of c128a (1/128KV compression, no absorption). At 1M context the c128a layers produce only about 8K compressed tokens; complexity drops from O(N^2) to roughly O(N \cdot k/128). That recasts KV cache from "the headline killer at long context" into a relatively minor budget item — the cleanest engineering progress I've seen on this track in the past six months.

The Three Layer Types of Hybrid Attention, and the Layer Distribution Across V4 Flash / Pro

source: Bilibili - Huawei CANN team (Xu Ke, Huan), DeepSeek V4 Lands on Ascend: Hybrid Attention and CANN Training/Inference Optimization Walkthrough (2026-04-24)

Third is vLLM's KV cache memory packing. The c4a / c128a / SWA caches with three different compression ratios are funneled into 3page-size buckets, paired with a kernel-fusion trio (compressor + RMSNorm + RoPE fusion at1.4–3×; inverse RoPE + FP8 quant fusion at 2–3×; fused Q norm + KV RoPE + K insert at10–20×) — between them they push single-node Hopper throughput to where it sits today.

All three together: 1.6T really does live inside1,128GB of HBM.

DeepSeek V4 Model Card — 1.6T Total /49B Activated

source: SemiAnalysis, The Coding Assistant Breakdown: More Tokens Please (2026-04-25)

It sounds like an engineering miracle until you read the first line of the vLLM blog, which gives the game away: their single-node recipe is designed for 8×B200 /8×B300, and the H200 version is a SemiAnalysis-authored backport that comes with three explicit concessions noted in writing: H200 has no FP4 cube, so the indexer cache moves to FP8/BF16, eating extra HBM; max-model-len is cut from native1M down to800K to free KV headroom; prefix caching is disabled, surrendering the throughput lever that56%context cache hit rate provided in the V3 deployment.

The ledger:

Almost no slack.

But This Isn't a Pareto Improvement — It's a Compromise

The key to grasping this isn't engineering, it's economics. A Pareto improvement means no dimension gets worse; some get better. A compromise means trading degradation along certain axes to keep others alive. Every step of V4 Pro's path onto H200 does the latter.

The most direct cost is throughput. SemiAnalysis's zero-day measurement on H200 shows V4 Pro at roughly ~150 tok/sec/GPU @20 tok/sec interactivity. V3 in the same 8K-in/1K-out regime ran 1,300–2,300 tok/sec/GPU. A roughly 10× throughput regression — and not the kind a few weeks of optimization will reverse. FP4 → FP8 dequant in front of every GEMM, active params jumping from V3's 37B to 49B, the lower arithmetic intensity inherent in TP8 EP8 single-node, and prefix caching itself being disabled — these reasons compound and structurally fix the ceiling.

The downstream implication of that number bites harder. Tang Jie of Zhipu put it bluntly in a recent talk: "DSA's whole point is to drive throughput up; you have to open up large EP. Speed can drop to 20 tok/s. For chat that's fine; for coding, the dev's floor is 40 tok/s — anything below 40 and they'll start cursing." V4 Pro's interactivity sits squarely below that line — SemiAnalysis measured 20 tok/sec/user. On the hardware Chinese labs can actually deploy, this 1.6T model fails to serve well the very scenario most likely to be its killer use case: the coding agent.

DeepSeek V4 Pro Throughput vs. Interactivity Across GPU Configurations

source: SemiAnalysis, The Coding Assistant Breakdown: More Tokens Please (2026-04-25)

The second cost is more discreet. It is hidden in the training data.

V4 Pro's training-token-to-total-parameter ratio sits at 33T /1.6T = 20.6:1. That happens to be exactly the optimum the2022Chinchilla paper proposed. Looks like classic practice — until you put it next to the2026frontier. Llama 3 ran 1,875:1; Gemma 3 270M ran 22,223:1; the past two years have seen everyone push toward "severely over-trained" regimes, repeatedly verifying that even at100× over-train, models still haven't converged. The University of Warsaw MoE scaling paper does suggest large MoE models should use fewer tokens proportionally, not more — but the token-to-active-parameter ratio should still be far higher than V4 Pro's 33T /49B =673:1.

V4 Pro stuck at 20:1 doesn't read as unwillingness — it reads as inability. The technical report itself shows its hand: there's an entire section titled "Mitigating Training Instability" with two patches bolted on — Anticipatory Routing (decouple the synchronous updates between backbone and routing networks; routing uses historical parameters to suppress loss spikes — at the cost of about 20% extra wall-clock); and SwiGLU clamping (clamp SwiGLU's linear branch to [-10,10] and the gating branch's upper bound to 10). Only with both patches did the model finish training. Inside that stability budget, piling on more tokens scales training cost and failure rate non-linearly — so stopping at20:1conservatively is a constrained-stability second-best, not a deliberate design choice.

The third cost is rarely raised in Chinese-language coverage: the multimodal option has been quietly closed off.

V4 is a text-only model. That sounds like a neutral statement until you frame it against today's industry picture. Anthropic's Opus 4.7, released in the same month, has visibly tilted RL training objectives toward visual front-end styling work; SWE-bench Multimodal is now a mandatory metric. Gemini 3.1 Pro and GPT-5.5 are both natively multimodal. Apple's Vision Intelligence chose Gemini — not any open-source option — to power its camera integration. Mark Zuckerberg, in a Meta AI strategy interview, said it directly: "DeepSeek is good on text, but it's currently a text-only model — multimodal capability simply doesn't exist in the DeepSeek model."

This isn't an oversight. It's the cost of an architectural choice. V4 Pro spent its entire compression budget on text — FP4 quantization, c128× KV compression, sparse attention with top-k =1,024— methods that work on text tokens specifically because natural language has redundancy to squeeze. Visual tokens are dense; the information they carry is more uniformly distributed. Apply this compression ratio to them directly and reconstruction quality collapses. Native multimodal would require expanding the vocabulary into the millions, switching from int8/ FP4back to int16to preserve precision, and using wider attention heads instead of c128-style aggressive compression — every one of these moves the opposite direction from what V4 Pro did to fit inside H200.

Put another way: to get 1.6T to fit inside 1,128GB of HBM, DeepSeek "specialized" the model architecturally into a text-only inference engine. That trade-off is one-way — adding multimodal afterward isn't a fine-tune, it's a retrain.

Multi-Node Doesn't Rescue Much Either

I had Jarvis calibrate a simplified multi-node model against SemiAnalysis's 150 tok/sec/GPU (256 experts top-8, 2 all-to-all calls per layer, inter-node IB ~50GB/s +25μs/call latency floor). On the single-node baseline it produced 163 tok/sec/GPU — about 9% off the empirical 150, close enough to trust.

The cost-per-Mtok shape it swept out (at $2.5/GPU/hour):

The absolute numbers run low (industry H200 serving prices are in the $0.5–1/Mtok range), but the relative shape holds. 16 GPUs is more expensive than the single node — the single-node baseline already employs the extreme measures of "cut context, kill prefix caching" to fit weights, while at 16 GPUs the weight pressure eases but each card is left with only 8.2GB of KV budget, far from enough to support a large batch. 32 GPUs is the sweet spot — WideEP unlocks super-linear scaling for the first time. Past128GPUs, things invert again — cross-node IB becomes the new bottleneck.

WideEP Layout:256Experts Evenly Distributed Across64GPUs

source: SemiAnalysis, InferenceX v2 — NVIDIA Blackwell vs AMD vs Hopper (2026-02-17)

The physics is written into the boundary of the NVLink domain: intra-node NVLink bandwidth is 9× inter-node IB. 8 GPUs fills the NVLink domain; 16 GPUs forces the first IB crossing — that is the source of the cost jump. Blackwell's GB200 NVL72 expands the NVLink domain to 72 precisely to push that cliff out further — all 72 cards live inside one NVLink domain; the cliff doesn't appear until the next rack. Hopper can't replicate this because the NVLink scale-up domain is physically fixed.

Does Huawei's choice of 16P deployment on Ascend 950DT route around this? Crunching the numbers: Ascend 950DT has 144GB per card (3GB more than H200), so 8 cards can comfortably hold the model. 16P isn't forced — it's a WideEP preference. The 950DT's UB scale-up is 2TB/s per card (twice H200's NVLink, though the specs are fishy); 16cards still sit inside one UB domain, never hitting the IB boundary. That hardware property makes it more comfortable than H200 in the 16–32 card range — but past 64 cards, it still hits the same wall, just slightly further out. Ascend 950DT mass production has slipped to 2026Q4, by which point NVIDIA will already be running Vera Rubin VR200 — the per-chip gap will widen from today's 1.3× to closer to 17×.

From "Feeling Out the Frontier" to "Performing Rites in a Sealed Shell"

Stack everything together and the character of DeepSeek's work in this generation has shifted from the previous one.

In the V2 / V3 / R1 era, DeepSeek consistently lived at the efficiency frontier — MLA conserved more KV cache than contemporary GQA at long context; their FP8 training precision was directly absorbed into the Hopper ecosystem; DeepEP became a built-in dependency for vLLM. That was a stretch where what they did was something every lab had to study — expanding the boundary. When Lex Fridman talked to the SemiAnalysis team about DeepSeek, the recurring phrase was: "they figured it out in the public first… maybe even first in some cases."

V4 is different. FP4 QAT is genuinely good work; Hybrid Attention is no less hardcore. But the destination for all of it is the same — getting a model 2.5× larger than V3 to limp along inside a hardware budget far tighter than V3's deployment. Throughput regressed 10×; context shrank 20%; prefix caching shut off; training token-to-parameter retreated to Chinchilla; multimodal foreclosed. Each individually is a defensible engineering trade-off; together they tell a different story — this isn't pushing the frontier, it's racing the wall.

Engineering for trade-off, not for frontier. That is the qualitative difference between V3 and V4.

The good news is that open-source token compression and DSA push the entire Chinese LLM ecosystem's cost curve down a notch — these techniques will diffuse into GLM, Qwen, MiniMax, and the rest within 1–2 model cycles, while also validating that domestic hardware will be able to host frontier inference once Ascend 950 super-nodes ship at scale. But the same analyst note carries a quietly damning line: "V4 Pro's throughput is currently limited by Huawei's compute availability." That constraint won't dissolve with open-sourcing — it becomes the same wall every next-generation Chinese lab will have to dance around.

The Line Didn't Move — They Walked Precisely Up to It

Back to the question at the start: did DeepSeek V4 break the ceiling?

No.1.6T was the extreme value for 8×H200 given in the prior piece; DeepSeek used the three hard skills of FP4 QAT + Hybrid Attention + vLLM packing to compress the model itself into the last inch of slack against the wall. The costs scattered across small-looking metrics — 150 tok/sec/GPU throughput collapse, 200K of context shaved off, prefix caching disabled, training ratio stuck at Chinchilla 20:1, multimodal optionality quietly closed, the 16-GPU-is-actually-slower curve.

None of these is headline material on its own; together they tell the same story: V4 isn't a Pareto improvement, it's constraint-driven optimization. From "feeling out the frontier" in the V2-V3 era to "performing rites in a sealed shell" in V4 — the first creates, the second rearranges. That shift is more worth remembering than 1.6T itself.

The next event that genuinely moves this line won't be another Chinese lab open-sourcing a 2T model. It will be Ascend 950DT / Atlas 950 SuperPoD entering large-scale production, native MXFP4 stabilizing on domestic software stacks, and — crucially — Chinese hardware acquiring scale-up interconnect on the same order as NVLink. Until then, every successive DeepSeek model will carry one more engineering concession than the last.

V4 sits pinned against the ceiling. But the ceiling itself hasn't moved.