He Ran a 1-Trillion-Parameter LLM on 768GB of Secondhand Optane — The VRAM Wall Just Fell

While everyone was arguing about whether a 24GB GPU can "really" run a frontier model, a homelabber quietly answered a much more interesting question: what if the barrier to running a trillion-parameter LLM at home isn't VRAM at all — it's just what you're willing to buy secondhand? Redditor APFrisco got Moonshot AI's Kimi K2.5 — a 1-trillion-parameter Mixture-of-Experts model — running locally at roughly 4 tokens per second on a used Intel Xeon Gold 6246, a modest 12GB GPU, and 768GB of dirt-cheap, decommissioned Intel Optane DIMMs scavenged from data-center junk piles.

There's no new chip here, no driver unlock, no secret quantization. The recipe is old server hardware, a discontinued memory module, and llama.cpp's memory-tiering flags. And in 2026 — the year DRAM prices went vertical and cloud GPU rates climbed up to 21% — it might be the most subversive local-AI build of the year.

The build: 768GB of decommissioned memory

The hardware reads like a 2021 server-room clearance bin:

The total memory pool is over 900GB for an outlay that wouldn't buy one crisis-priced 48GB consumer GPU module set. Tom's Hardware flagged the setup after it circulated on r/LocalLLaMA, noting the enthusiast used a hybrid GPU/CPU methodology with llama.cpp and shoehorned the model's routing components into the 12GB GPU using the llama.cpp --override-tensor flag.

How MoE tiering beats the VRAM wall

Dense models activate every parameter for every token — a 70B dense model genuinely needs ~40GB of fast memory to run well, which is why local running of dense frontier models has been a rich-person's hobby. Mixture-of-Experts models break that rule. Kimi K2.5 has 1 trillion total parameters but activates only ~32B per token — the router picks a handful of "expert" networks and the other 968B sit idle for that step.

That asymmetry is what llama.cpp's tiering flags exploit. Community guides to MoE offloading describe the standard recipe: keep the always-used tensors — attention, router, shared experts — in the small-but-fast GPU, and leave the hundreds of expert FFN weight blocks in whatever cheap memory you have. Only the ~32B active slice actually streams per token, so the slow tier is touched a measured fraction of the time.

Push the idea far enough and capacity stops being a hardware spec and becomes a storage problem — the exact mental shift this build embodies. The community has been circling toward it all year (the --n-cpu-moe / -ot recipes that produced famous 5× speedups for over-VRAM models earlier in 2026), but APFrisco's Optane rig is the first time we've seen the tier-2 pool filled with discontinued server flash-RAM hybrid modules instead of ordinary DRAM.

The economics: Optane is the cheapest capacity on Earth

Here's why this build is timed so well. Intel killed Optane Persistent Memory in 2022, but data centers bought it in the tens of billions of dollars and are only now scrapping those platforms in volume. The result, per one used-memory broker's price sheet, is that 100/200-series DCPMM modules trade at roughly $0.40–$1.20 per GB — a 128GB stick for about $50–$100. eBay listings routinely show used 128GB Optane DIMMs in the $80 range.

Now do the comparison that matters. Filling ~768GB of tier-2 capacity with new DDR5 at October 2026's crisis pricing — the regime we documented in the memory price crisis post — means several thousand dollars for the same capacity. Even used DDR4 RDIMMs trade well above the Optane floor per GB. The build turns the scariest 2026 cost curve (DRAM $/GB) into a fixed clearance price on a supply glut.

• The capacity angle: if you're building any memory-tiered MoE rig, used Optane is the cheapest way to make the tier-2 pool huge. Check Optane DCPMM & server memory listings on Amazon →
• The platform angle: used Cascade Lake Xeon workstations and rack servers (the required host) are clearance-priced. Shop used Xeon platforms on Amazon →

The catch: bandwidth, platforms, and 4 tok/s reality

Let's be honest about what this rig is. Four tokens per second is slow. It's conversational — think "model thinking out loud" pace — but it's not the snappy 30+ tok/s decode you'd expect from a 24GB GPU running an 8B-class dense model. And prompt processing (ingesting your whole document or codebase on every turn) is typically the bigger wall for memory-tiered setups: every input token must flow through the full active expert set.

The Optane tier also has a real platform tax you can't buy your way out of:

• App Direct mode requires specific Intel Xeon Scalable (Cascade Lake / Cooper Lake era) boards with 12 DIMM slots per socket — consumer platforms need not apply. Tyan and Supermicro boards dominate the used listings.
• Optane DIMM bandwidth is lower than DDR4/DDR5 — it was designed as a capacity tier, not a bandwidth tier. That's precisely why the MoE trick makes it workable: only ~3% of the model streams per token.
• It's a dead-end platform. No next-gen CPU upgrade path, no DDR5 bridging, and Intel has formally exited the memory business. You're buying into a closed ecosystem at clearance prices.

The homelab playbook

If you want to chase this experiment yourself, the playbook is straightforward — no exotic gear, just unusual choices:

• Start with the platform, not the model. Look for a used LGA3647 or LGA4189 workstation with 12+ DIMM slots per socket — that's your Optane carrier. The RTX 3060 12GB tier of GPU is genuinely all the rig used; check current 12GB GPU pricing on Amazon →
• Budget the two memory tiers deliberately. Fast tier (DDR4 ECC) for what's touched constantly — attention, router, KV cache. Bulk tier (Optane or just lots of cheap DIMMs) for expert weights. Tiering beats a single expensive pool.
• Use llama.cpp's surgical offload flags. -ngl for the base offload, then --override-tensor or --n-cpu-moe to pin routing/shared tensors on the GPU while experts stay in system memory. Iterate placement — tensor placement is the entire ballgame.
• Size expectations by use case. ~4 tok/s serves chat, summarization, background agents, RAG over a document. It does not serve real-time coding or multi-user serving. Decide which problem you're solving before you spend.
• Consider the non-Optane variant first. If you already own a big-DDRAM consumer box (say 128–192GB), the same llama.cpp recipe on an 8B–30B-active MoE (gpt-oss, Qwen3, Llama-4 Scout class) delivers the same architectural lesson at zero extra spend — often 15–30 tok/s.

The bottom line: the VRAM wall just turned into a budget problem. When only 32B of a trillion parameters needs to be fast, and llama.cpp can stream the rest from clearance-rack Optane, the ceiling on home-runnable models moved from "what fits in a 24GB card" to "how many DIMM slots you can fill for $500." Meanwhile the cloud side repriced upward and DRAM went vertical. 2026's most interesting AI hardware trend isn't a new chip — it's software learning to live in the memory basement. And the basement is stocked.

FAQ

Can you really run a trillion-parameter LLM at home? Yes — not by loading it all in VRAM, but by exploiting MoE sparsity. Kimi K2.5 activates ~32B of its 1T parameters per token, so llama.cpp keeps routing/attention tensors on a 12GB GPU and streams expert weights from cheap system memory tiers. This exact build hit ~4 tok/s.

Why Optane instead of regular DDR4 or DDR5? Capacity per dollar: discontinued Optane DCPMM trades at ~$0.40–$1.20/GB secondhand (~$50–$100 per 128GB stick), versus several thousand dollars for equivalent DDR5 capacity at 2026 crisis pricing. Its lower bandwidth is fine when only ~3% of the model streams per token.

Is 4 tokens/second usable? For single-user chat, RAG, and agents that think between output bursts — yes. Not for snappy interactive coding or high-volume serving, and prompt processing is usually the bigger bottleneck than decode.

Should I buy used Optane DCPMM in 2026? Only if you have a Cascade Lake/Cooper Lake era board with 12 DIMM slots per socket (Tyan, Supermicro). It's a dead-end platform — but with no cheaper $/GB on the market for tier-2 capacity, it's unbeatable for experimental memory-tiering rigs.

Disclosure: This post contains affiliate links. We may earn a commission on purchases at no extra cost to you. See our affiliate disclosure for details.