Nvidia has released Nemotron 3.5 Lightning, an open-weight reasoning model designed to make AI agent workloads faster and cheaper to run rather than maximize performance on every intelligence benchmark.

The model combines 31.6 billion total parameters with a sparse architecture that activates only 3.6 billion parameters for each token. According to benchmarking reported by The Decoder, it reaches nearly 670 tokens per second in pre-release testing with Nvidia’s final NVFP4 weights, making throughput the central feature of the new Nemotron 3.5 release.

That positioning matters as developers move from occasional chatbot requests toward agent systems that generate long sequences of tool calls, code, and intermediate reasoning. For those workloads, latency, hardware utilization, and serving cost can matter as much as a model’s top benchmark score.

A smaller active footprint with a one-million-token context

Nemotron 3.5 Lightning succeeds Nemotron 3 Nano 30B A3B and retains the earlier model’s hybrid Mamba-Transformer design. Its total parameter count is substantially larger than the number used during any individual calculation, a structure intended to provide more capacity without imposing the full compute cost of a dense 31.6-billion-parameter model.

Nvidia is offering the reasoning model in BF16 and NVFP4 formats. The latter uses lower-precision weights to reduce the memory and computation required for inference. The Decoder reported that NVFP4 achieved the same score as the BF16 version on the Artificial Analysis Intelligence Index, with only a small quality difference according to that evaluation.

The model is text-only and supports a context window of up to one million tokens. That capacity could be useful for coding agents, document-heavy workflows, and applications that need to maintain extensive task history, although context length alone does not establish how efficiently or reliably a system will handle very long inputs.

Weights are available under Nvidia’s permissive OpenMDW-1.1 license. The Decoder also listed serverless access through providers including DeepInfra, Fireworks, FriendliAI, CoreWeave, GMI Cloud, Nebius, and Crusoe. Availability across multiple inference platforms lowers the barrier for teams that want to test the model without immediately operating their own Nvidia infrastructure.

Speed is the product claim; intelligence is competitive, not dominant

Artificial Analysis gave Nemotron 3.5 Lightning an Intelligence Index score of 24, as reported by The Decoder. That matched OpenAI’s gpt-oss-120b and trailed Nvidia’s Nemotron 3 Super, which scored 26. The result represented a nine-point improvement over Nemotron 3 Nano’s reported score of 15.

The comparison also shows the limits of Nvidia’s approach. The Decoder reported higher scores for Qwen3.6 35B A3B and Meta’s Muse Glimmer, at 32 and 35 respectively. Proprietary systems remained ahead on the combined speed-and-intelligence tradeoff in the cited analysis: Google’s Gemini 3.5 Flash-Lite reportedly scored 37, while GPT-5.6 Luna reached 52.

Lightning’s strongest reported results appeared in agent-oriented tests. On GDPval-AA v2, Artificial Analysis recorded an Elo rating of 824, compared with 800 for gpt-oss-120b and 698 for Nemotron 3 Super. On Terminal-Bench v2.1, the model’s score rose from 7 percent for its predecessor to 24.3 percent, close to gpt-oss-120b’s reported 26.2 percent.

These figures come from an independent benchmarking platform but are presented through The Decoder’s coverage, not as a complete set of independently reproduced results in the supplied evidence. They should therefore be treated as benchmark snapshots rather than proof that Lightning will outperform larger models across production workloads. Inference speed can vary significantly with hardware, batch size, quantization, serving software, prompt length, and output length.

The headline throughput figure also comes from pre-release testing using final NVFP4 weights. The Decoder said Lightning completed an Intelligence Index task in about half a minute, compared with approximately 3.5 minutes for Qwen3.6 35B A3B and 5.8 minutes for Gemma 4 31B in the cited comparison. Those timings help clarify Nvidia’s target: repeated, latency-sensitive tasks where the cost of waiting or serving a larger model compounds over many interactions.

What the release means for AI builders

For product teams, Lightning offers a possible middle layer between small, inexpensive models and large systems reserved for difficult reasoning. A company could use it for routine agent steps—classification, tool selection, code navigation, document retrieval, or structured execution—while escalating ambiguous or high-risk cases to a more capable model.

That architecture can reduce the number of requests sent to expensive proprietary APIs, but only if the smaller model performs reliably on the specific workflow. A benchmark advantage on Terminal-Bench does not automatically translate to dependable database changes, financial analysis, customer support actions, or production code. Teams will need to test failure recovery, tool-use accuracy, hallucination rates, and the frequency with which tasks require escalation.

The sparse design and NVFP4 option may also affect deployment economics. Activating 3.6 billion parameters per step can reduce compute demand relative to a dense model with a similar total size, while quantization can lower memory requirements. The practical benefit will depend on whether a team has compatible hardware and serving infrastructure, as well as how much overhead comes from long contexts, batching, and agent orchestration.

Nvidia’s broader strategy is visible in the release. The company has previously argued that models below 10 billion active parameters can handle many agent tasks at a fraction of the cost of much larger systems. Lightning turns that efficiency argument into a product with a 31.6-billion-parameter mixture-style design but a 3.6-billion-parameter active path.

The related release from Fastino Labs provides an early signal about possible downstream use. StreetInsider reported that the company released specialized open-weight finance and healthcare models post-trained on Nemotron 3.5 Lightning entirely using an agent. The supplied source does not provide technical details, evaluation results, or adoption figures for those models, so the announcement is best understood as an example of ecosystem activity rather than evidence of production-scale demand.

What to watch next

The most important follow-up will be independent testing of end-to-end agent reliability. Developers should look for evaluations that measure completed tasks, not only token throughput or benchmark scores, across coding, browsing, enterprise software, and long-running workflows.

Serving economics will be another key signal. Public deployments can show whether NVFP4 and sparse activation produce meaningful savings on real workloads, particularly when models must handle large contexts or many concurrent users. The performance gap between local deployment and hosted inference will also matter for enterprises weighing control against operational simplicity.

Nvidia’s post-training partnerships are worth monitoring as well. Artificial Analysis reported that CodeRabbit and Harvey worked with Nvidia on domain-specific post-training. More specialized variants could determine whether Lightning becomes a general-purpose agent model or a foundation for narrow, high-volume systems.

Finally, buyers should track licensing clarity, tooling support, and the model’s behavior under safety constraints. An open-weight model is easier to inspect and deploy than a closed API, but organizations remain responsible for monitoring, access controls, data handling, and failure containment.

Creati.ai perspective

Nemotron 3.5 Lightning is not Nvidia’s attempt to win the raw intelligence leaderboard. It is a bet that many useful AI agents need fast, repeatable execution more than they need the strongest possible answer on every task.

That makes the release strategically relevant even where benchmark leaders remain ahead. If the reported throughput holds across common deployment setups, Lightning could give builders a practical routing option for high-volume agent steps. Its success will ultimately be judged less by parameter counts than by whether teams can complete real workflows at lower latency and cost without adding unacceptable reliability or safety risks.