On July 15, 2026, Thinking Machines Lab — the startup founded by former OpenAI CTO Mira Murati — released Inkling, its first foundation model trained from scratch. The headline number is staggering: 975 billion total parameters. But the number that actually matters for anyone building local AI infrastructure is 41 billion. That is how many parameters Inkling activates per token, thanks to a Mixture-of-Experts architecture that routes each token to just 6 of 256 expert networks plus 2 shared experts. The result is a model with the knowledge capacity of a near-trillion-parameter system but the inference cost of something a fraction of its size.
This is the story of 2026's open-weights frontier. Not the closed API wars between OpenAI, Anthropic, and Google — those are settled into a predictable cadence of incremental releases. The real action is in open weights, where a new generation of MoE models is collapsing the gap between frontier capability and local deployability. Inkling is the latest and most dramatic example. If you build AI infrastructure, run inference at the edge, or maintain a homelab GPU rig, the implications are immediate. Here is what changed, why the architecture matters, and what it means for your hardware decisions.
Inkling's Architecture: 975B Parameters, 41B Active
Inkling is a 66-layer decoder-only transformer with a sparse Mixture-of-Experts feed-forward backbone. Each token is routed to 6 of 256 experts, plus 2 shared experts that fire on every token. This is not a novel architecture — DeepSeek, Mistral, and Nvidia's Nemotron line have all used MoE — but the scale and the ratio are notable. At 975B total parameters with only 41B active, Inkling has an activation ratio of roughly 4.2%, meaning 95.8% of its parameters sit idle during any single forward pass. Those dormant experts are not wasted; they encode specialized knowledge that the router calls on demand. But from a compute perspective, you are effectively running a 41B model, not a 975B one.
The benchmark results validate the approach. Thinking Machines reports that Inkling matches Nvidia's Nemotron 3 Ultra — previously the largest American open-weights model at 550B parameters — on Terminal Bench 2.1 while using roughly one-third the thinking tokens. On coding and knowledge tasks, it lands in the same tier as the best open models available. It is multimodal, accepting text and image inputs. And it ships under Apache 2.0, meaning you can use it commercially, modify it, fine-tune it, and distribute derivatives without negotiating a license.
A smaller variant, Inkling-Small, was released alongside the full model, targeting developers who need the architecture's efficiency without the full storage footprint. Both models require an inference framework — SGLang, vLLM, or TokenSpeed — for GPU deployment, and the model card explicitly calls out these as the supported runtimes. This is not a model you load into Ollama with a single command. It is a production-grade system that expects production-grade tooling.
What 41B Active Means for Your Hardware
Here is where the architecture decision becomes a hardware decision. A dense 975B model in FP16 would need roughly 1.95 terabytes of VRAM — an H200 cluster, not a homelab. But because Inkling only activates 41B parameters per token, the inference compute footprint is comparable to a 41B dense model. The catch is that all 975B parameters still need to live in memory — the experts are selected at runtime, so the full weight set must be resident. This is the fundamental trade-off of MoE: you trade compute for memory capacity.
In practice, this means Inkling's full-precision deployment remains a data-center-class workload. But the quantized story is different. At INT4 quantization, the full 975B parameter set compresses to roughly 488 GB — still beyond a single consumer GPU, but feasible on a multi-GPU rig with 4 to 8 high-VRAM cards. At INT8, you are looking at approximately 975 GB, which pushes into small-cluster territory. The real opportunity for homelab builders is Inkling-Small, which is designed for the subset of users who want the MoE routing efficiency without the full expert library, and for fine-tuned derivatives that prune the expert pool down to a deployable size.
For anyone planning a multi-GPU inference rig with RTX 4090-class cards, Inkling is the model that justifies the investment. A 4x RTX 4090 setup with 96 GB of total VRAM cannot run the full Inkling at usable quantization, but it can run expert-pruned derivatives, Inkling-Small, and the broader family of 30B-to-70B open MoE models that are now the practical sweet spot for local deployment. The hardware you buy for Inkling-class models is the same hardware that runs every other open-weights release in 2026.
The Open-Weights Flywheel: Why This Keeps Accelerating
Inkling did not appear in a vacuum. It is the latest data point in a trend that has been compounding for 18 months. DeepSeek's V4 Flash, Mistral's Large 3, Nvidia's Nemotron 3 Ultra, and now Inkling have established a rhythm: every few months, a new open-weights model lands that either matches or exceeds the previous frontier, usually with a more efficient architecture and a more permissive license. The flywheel is real. Each release gives the community a new baseline to fine-tune from, which produces better task-specific models, which drives more adoption, which incentivizes the next release.
The strategic implication for infrastructure builders is that the open-weights frontier is no longer a consolation prize for organizations that cannot afford frontier API access. In several categories — coding, knowledge work, agentic tool use — the best open models are within single-digit percentage points of the best closed models, and they offer advantages that closed APIs structurally cannot: data sovereignty, no per-token costs, customizable behavior, and the ability to run offline. If your use case involves sensitive data, high-volume inference, or specialized workflows, the calculus has shifted. Local deployment is no longer the budget option. In 2026, it is frequently the better option.
The Security Flip Side: LegacyHive and the Patch Gap
The same week Inkling shipped, the security world dealt with a reminder that local infrastructure cuts both ways. On July 15, researcher Nightmare-Eclipse published LegacyHive, a proof-of-concept Windows zero-day that works on all supported Windows desktop and server versions — including systems fully patched with the July 2026 Patch Tuesday updates. Microsoft's July release was already the largest on record, with 622 vulnerabilities disclosed, including 416 affecting Windows itself. LegacyHive landed the day after and demonstrated that even fully patched systems have exploitable surface area.
For local AI inference builders, the connection is direct. If you are running an Inkling-class model on a local server, that server is now a high-value target. It has significant compute, network connectivity, and increasingly, access to sensitive data flowing through inference workloads. A compromised host means a compromised model — and a compromised model can be silently modified to exfiltrate data, inject biased outputs, or serve as a lateral movement platform into the rest of your network. The security baseline for AI infrastructure is not optional. It starts with patching, but it cannot end there.
The practical steps are the same ones we have been recommending all year. Segment your inference network from your general-purpose infrastructure. Use a dedicated UPS and power infrastructure not just for uptime but because clean power reduces the failure modes you have to reason about. Treat your model weights as artifacts with integrity checks — hash your downloaded weights and verify them on load, because supply-chain attacks on open-weights repositories are an obvious next step for sophisticated actors. And run your inference stack on a hardened, minimal OS, not a general-purpose desktop that browses the web and opens email attachments.
What to Build Now
If you are making hardware decisions in July 2026, the open-weights trajectory tells you to invest in memory capacity over raw compute. The bottleneck for MoE models is not FLOPS — it is VRAM. A rig with four 24 GB GPUs will outperform a rig with two faster GPUs for MoE inference, because the expert weights need somewhere to live. Plan for 96 GB of VRAM as the minimum viable configuration for serious open-weights work, and 192 GB as the target for Inkling-class models at aggressive quantization.
On the software side, learn vLLM or SGLang if you have not already. These are the runtimes that the open-weights ecosystem is standardizing on, and the model cards from Thinking Machines, DeepSeek, and Nvidia all assume one of them. Ollama is great for quick experiments and small models, but the frontier of open weights has moved past what it was designed for. The good news is that vLLM and SGLang are both production-hardened, well-documented, and run on the same hardware you already own.
Finally, budget for power and cooling. We covered the AI power wall in detail last week, and the same physics apply at the homelab scale. A four-GPU inference rig pulling 2,000 watts needs a dedicated circuit, real ventilation, and a UPS that can handle the inrush current. The most common failure mode for local AI infrastructure is not software — it is a tripped breaker at 2 AM that takes down a 12-hour fine-tuning run. A proper rack with managed power distribution is not overkill. It is the baseline.
The open-weights frontier is not slowing down. If anything, it is accelerating — Inkling is 975B this month, and the architecture that makes it deployable is the same architecture that will scale to the next release. The builders who win the local AI era will be the ones who sized their hardware for memory, hardened their infrastructure against the LegacyHive-class threats that are now routine, and invested in the power and cooling that keeps the whole stack running. The models are here. The question is whether your infrastructure is ready for them.
Get weekly AI & infrastructure guides
Join the GeniusTechLab newsletter for new open-weights model analysis, local inference hardware guides, and security recommendations — one email a week, no spam.
Subscribe to the newsletter →