The new Nemotron 3.5 Lightning is not just another open model from NVIDIA: it is a 30B mixture-of-experts design built for always-on agents, claiming up to 4x faster token generation and 30% faster task completion than comparable open models.
What Was Released
On August 11, NVIDIA announced Nemotron 3.5 Lightning as the newest member of its Nemotron 3 line. The model is a 30B total / 3B active-parameter hybrid Mamba-Transformer MoE, sized for single-node deployment while supporting long-context use and structured tool calling. The GitHub cookbook maintained by NVIDIA-NeMo describes it as designed for “fast reasoning, coding, and agentic workflows,” and the release day materials emphasize the same use cases: agents that stay resident, respond quickly, and call tools.
The open-weights license is the strategic part. Because the weights are open, developers can fine-tune the model against their own examples to match specific tasks, interests, and workflows. NVIDIA’s blog lists three concrete fine-tuning directions: writing in a preferred style with established tones and terminology, learning a specialty such as photography, gaming, or 3D design, and coding a certain way, meaning preferred conventions, frameworks, and testing approaches. The blog positions these as the difference between a generic model and a personalized local assistant, one that manages email and calendars, runs a smart-home routine, or works alongside developers on a local codebase.
The Numbers That Matter
The performance claims are the headline. NVIDIA says Nemotron 3.5 Lightning delivers up to 4x faster token generation and 30% faster time to completion compared to open models in its class. Those are the same class of claims NVIDIA makes for most of its model launches, and the caveat applies here too: “up to” and “in its class” leave room for interpretation, and real throughput depends on hardware and quantization choice.
The efficiency angle is where the model is genuinely interesting. With only 3B active parameters out of 30B total, the MoE architecture means a given token only routes through a fraction of the network. For local deployment, that is the difference between a model that feels responsive on a consumer GPU and one that does not. The hybrid Mamba-Transformer design also matters for always-on agents: hybrid architectures tend to use less memory for long-context processing than pure transformer stacks, which is relevant when an agent holds a long conversation or a large file in context for hours.
NVIDIA collaborated with vLLM, Ollama, llama.cpp, and LM Studio to provide local deployment options, offering both NVFP4 and GGUF formats. Unsloth provides day-one support with optimized and quantized models via Unsloth Studio. That list matters because it means the model ships with the mainstream local tooling already working, not waiting for the community to bolt on support.
Where It Runs
The model runs locally on NVIDIA RTX PCs, DGX Spark, and OEM GB10 systems, and on NVIDIA Jetson for edge deployments. It scales up to RTX PRO workstations, DGX Station, and GB300 deskside systems, and from there to data centers and cloud environments. NVIDIA Blackwell systems are available from Acer, ASUS, Dell, Exxact, GIGABYTE, HP, Lenovo, MSI, and Supermicro, so the hardware spread is wide. Nemotron 3.5 Lightning is also available through OpenRouter and on build.nvidia.com as an NVIDIA NIM microservice.
The other release-day component is NeMo Switchyard, an open-source routing library NVIDIA describes as automatically directing each step of an agent workflow to the best-fit model based on accuracy, speed, and cost. The pitch is that instead of running every step of a workflow on one frontier model, developers route each step across a system of models. NVIDIA’s internal benchmarks claim that Switchyard maintained frontier-level task completion while cutting benchmark completion cost to roughly one-third of Opus 4.8 alone. The library is on GitHub.
That kind of routing is the pragmatic answer to the cost problem in 2026: frontier models are expensive, and most steps in an agent workflow do not need frontier intelligence. Routing lets a cheap local model handle the routine steps and escalates only the hard ones. It also gives developers flexibility to work across models and providers, which is increasingly the standard architecture pattern for production agents.
Why This Matters
The release is part of a larger push. NVIDIA is spending August celebrating the local AI ecosystem, with this blog series, partner announcements, and model releases framed around “local AI.” Nemotron 3.5 Lightning is explicitly aimed at the group of developers who want agent capability without a per-token cloud bill.
There is a tension worth naming: NVIDIA sells the GPUs, the software stack, and now the models that run on them, so its incentives point toward “run everything locally on NVIDIA hardware.” But the open-weights choice cuts against lock-in in a real way. Anyone can download the weights, fine-tune them, and run them on non-NVIDIA hardware through vLLM or llama.cpp. The Mamba-Transformer hybrid and NVFP4/GGUF support also make the model portable across frameworks, which is more than some open releases offer.
For developers, the practical takeaway is the deployment path. If you are building a local agent today, the mainstream stack, vLLM, Ollama, llama.cpp, LM Studio, Unsloth, is already supported on day one, and the model is sized to run on a single node. For teams already running local models, our look at MiniMax H3 on Apple Silicon with MLX covers the hardware question from the Mac side, which is a useful comparison if you are weighing NVIDIA versus Apple for local inference.
The Honest Caveats
The fine-tuning promise has a limit. A 3B-active model that learns your email style is useful, but it is not a frontier model. NVIDIA’s own framing, “fast, specialized agentic tasks,” concedes the point: this is for tasks where a focused, well-tuned small model beats a generic giant, not for open-ended reasoning where frontier capability is the requirement.
The performance claims should be treated as vendor benchmarks until third parties reproduce them. NVIDIA says 4x faster token generation, but the comparison set, hardware configuration, and context length are not fully specified in the announcement, and the GitHub cookbook’s own numbers will be the more reliable source once the community runs them. The NeMo Switchyard cost claim, “one-third of Opus 4.8,” is also an internal benchmark with unspecified workload details.
The Mamba-Transformer hybrid is newer than the transformer-only designs most developers are used to, which means the ecosystem knowledge is thinner. Tooling support is solid at launch, but if you hit an edge case in quantization or fine-tuning, there is less accumulated community experience than you would find for a mature architecture.
Who Should Use It
Developers running agents on local hardware, especially those already in the NVIDIA ecosystem with RTX or DGX machines, are the natural audience. The always-on agent use case, an assistant that manages email, runs routines, or works on a codebase for hours, is where the combination of open weights, fine-tunability, and low active-parameter count pays off.
Teams that want the control of open weights without the per-token cloud bill, and developers who want to fine-tune a model to their own workflows rather than prompt a generic one, should also look at this. The NeMo Switchyard angle is relevant for teams with mixed local and cloud deployments, since routing across models is becoming the standard cost-control pattern.
The Bottom Line
Nemotron 3.5 Lightning is a well-positioned open MoE for the local agent crowd: 3B active parameters, hybrid architecture, day-one support in the mainstream local tooling, and open weights that make fine-tuning a real option. The performance numbers come from NVIDIA’s own benchmarks, so treat them as directional until the community measures them. If you are building agents that need to stay resident on local hardware, this is worth testing this week, and the cookbook notebooks make that test easy.