Alibaba has released the open weights for Qwen3.8-2.4T-A95B, a 2.4-trillion-parameter mixture-of-experts model, while NVIDIA has published a deployment path for running it across the company’s GB300 NVL72 rack-scale system. The announcement puts a very large open-weight model at the center of a hardware-and-software serving stack designed for long-context reasoning and agentic applications.

The model activates 95 billion parameters per token rather than using its full parameter count on every request. NVIDIA says its initial FP8 deployment can deliver more than 4,000 tokens per second per GPU and more than 350 tokens per second per user on GB300 NVL72, although those figures come from NVIDIA’s own infrastructure testing and should not be treated as independent benchmarks.

A 2.4T model built for long-running workflows

Qwen3.8-2.4T-A95B, also referred to in NVIDIA’s article as Qwen3.8-Max, is positioned for coding, large-scale document analysis and multi-step agentic workloads. Alibaba’s release makes the model’s weights available through Hugging Face and ModelScope, according to NVIDIA.

The architecture combines fine-grained mixture-of-experts routing with alternating full-attention and linear-attention layers. A learned router selects the experts needed for each token, allowing the model’s serving cost to depend more closely on its active parameters than on the entire 2.4T-parameter capacity.

The attention design addresses a specific problem in agentic systems: context can accumulate system instructions, tool results, retrieved documents, source code, logs and intermediate reasoning over many turns. NVIDIA says the model supports a context window of up to one million tokens and output of up to 128K tokens. Its linear-attention layers use a bounded recurrent state in place of a growing key-value cache, while full-attention layers retain broader token-to-token interaction where needed.

Those capabilities are relevant to product teams building agents that must maintain state across extended workflows. They also raise practical infrastructure questions, because a one-million-token context can shift the bottleneck from raw model computation to memory capacity, communication and cache management.

Why the GB300 system matters

NVIDIA’s GB300 NVL72 integrates 72 Blackwell Ultra GPUs into a single rack-scale platform. The system provides a 72-GPU NVLink domain with 130 TB/s of all-to-all communication, according to NVIDIA. That interconnect is important for a large MoE model because routing expert requests across conventional server networks can introduce communication overhead and uneven utilization.

Serving Qwen3.8-2.4T-A95B therefore depends on more than placing model files on a collection of accelerators. The deployment requires distributed execution, optimized kernels and an inference runtime that can coordinate expert traffic across the rack. NVIDIA’s blog presents the GB300 configuration as a way to make that coordination part of the system design rather than relying solely on off-the-shelf networking.

NVIDIA says the reported Day 0 results were achieved in FP8 without additional model tuning. The company expects further gains from NVFP4 precision, but it did not provide a future performance figure in the supplied announcement. FP8 and lower-precision execution can reduce memory and computation requirements, but production buyers will still need to assess output quality, numerical stability and the operational cost of the full rack-scale deployment.

Configurable reasoning changes the serving trade-off

A notable product feature is Qwen3.8-2.4T-A95B’s built-in reasoning controls. Developers can select low, high or xhigh reasoning modes on a per-request basis, according to NVIDIA. The controls are intended to let applications trade inference depth for response quality and latency.

That is a more operationally useful model than treating reasoning as a fixed global setting. A coding agent handling an ambiguous architecture problem could use a higher setting, while a document-processing pipeline might reduce reasoning depth to maximize throughput. In enterprise systems, the choice could also be tied to task priority, user tier or an application’s latency budget.

The controls do not eliminate the need for evaluation. Teams will need to measure whether higher reasoning settings improve task completion enough to justify extra compute, and whether lower settings preserve accuracy for repetitive workloads. The announcement provides no independent analysis of quality differences between the three modes.

Evidence, software choices and vendor claims

The primary evidence for this deployment comes from an NVIDIA Developer Blog post, making NVIDIA the source for the performance numbers, system description and serving recommendations. A separate NVIDIA Developer listing points to the same announcement but does not add independent reporting. The available source material does not include third-party benchmark results, customer deployments or verified production adoption.

NVIDIA lists SGLang, vLLM and NVIDIA Dynamo as open-source inference options for developers seeking control over serving performance. It also offers NVIDIA NIM, described in the post as a model-free inference container that can serve supported models. The intended workflow is to download the model weights, deploy them through the container and scale the service as required.

For customization, NVIDIA recommends NVIDIA NeMo AutoModel. The PyTorch-native fine-tuning library supports direct use of Hugging Face checkpoints, full supervised fine-tuning and LoRA-based adaptation, according to the company. This gives teams a possible path from the public checkpoint to domain-specific versions, but the size of the model means that training, checkpoint storage and evaluation remain substantial infrastructure projects.

The “over 4K tokens per second per GPU” and “over 350 tokens per second per user” figures are especially important to interpret carefully. NVIDIA identifies them as Day 0 results on its own hardware and software stack. They indicate the performance target of the deployment recipe, not a universal result that should transfer unchanged to other servers, precision settings, batch profiles or application workloads.

What the deployment means for builders and buyers

For AI builders, the announcement expands the open-weight design space toward models whose total capacity is measured in trillions of parameters while per-token activation remains lower. That may support specialized reasoning and agentic behavior without requiring a dense 2.4T model to execute every parameter on every token.

However, the hardware threshold is high. A 72-GPU rack-scale platform is not a typical development environment, and teams will need to consider utilization, scheduling, model replication, failure recovery and power costs alongside tokens-per-second figures. The model may be technically open, but operating it at the advertised scale is likely to be concentrated among well-funded enterprises, cloud providers and research organizations with access to advanced infrastructure.

The deployment also illustrates a growing split between model openness and operational accessibility. Open weights can make experimentation and fine-tuning possible, but long-context serving and large-scale MoE communication still depend on specialized systems. Buyers evaluating the model should compare end-to-end cost per completed task, not just raw generation speed. They should also test tool use, long-context retrieval, reliability over multi-step runs and behavior under different reasoning settings.

What to watch next

The clearest follow-up signal will be independent testing of Qwen3.8-2.4T-A95B across SGLang, vLLM and NVIDIA Dynamo, including latency, throughput, memory use and quality at each reasoning level. Comparisons with other open-weight models will show whether the architecture delivers practical advantages beyond its parameter count.

Developers should also watch for the promised NVFP4 optimizations, additional deployment recipes and evidence from real production workloads. Adoption of the model’s weights, fine-tuned checkpoints and long-context agent applications will be more meaningful than download activity alone. Finally, the cost and availability of GB300 NVL72 systems will determine whether this remains primarily a showcase for rack-scale infrastructure or becomes a broadly usable serving option.

Creati.ai perspective

NVIDIA’s announcement is significant less because it makes a 2.4T-parameter model universally accessible than because it demonstrates how open weights, MoE routing, configurable reasoning and rack-scale interconnects are being designed together. The deployment treats inference as a systems problem, with communication and memory becoming as important as accelerator arithmetic.

For product teams, the practical question is not whether the largest model can generate tokens quickly. It is whether its reasoning controls and long-context architecture improve a defined workflow enough to justify specialized infrastructure. Until independent benchmarks and production evidence emerge, Qwen3.8-2.4T-A95B is best viewed as a promising open model paired with a vendor-reported serving blueprint, rather than a proven default for enterprise deployment.