Categories
Misc

3 Ways to Boost MoE Inference on NVIDIA DGX Spark

Abstract green computing cube with layered panels, performance icons, and streams of data on a black backgroundGenerating tokens faster starts with identifying what limits inference performance. In the batch-size-1 evaluation of Ling-3.0-flash from Inclusion AI on NVIDIA…Abstract green computing cube with layered panels, performance icons, and streams of data on a black background

Generating tokens faster starts with identifying what limits inference performance. In the batch-size-1 evaluation of Ling-3.0-flash from Inclusion AI on NVIDIA DGX Spark, profiling showed that decoding was memory-bound: reading model weights drove execution time. Two operations stood out—the mixture-of-experts (MoE) computation and the output-layer projection that produces scores across the model’…

Source

Categories
Misc

Fine-Tuning NVIDIA Nemotron to Know Local Rules, Not Just Local Languages

As AI adoption expands globally, people across languages and cultures are using it in more aspects of their daily lives. General-purpose language models serve…

As AI adoption expands globally, people across languages and cultures are using it in more aspects of their daily lives. General-purpose language models serve as a powerful foundation, but they sometimes lack the local knowledge, cultural nuance, and precision a particular application needs. Post-training can adapt an open model to these needs. With suitable data and evaluation…

Source

Categories
Misc

NVIDIA RTX Spark Powers Neural Rendering and AI Characters in Games

NVIDIA RTX Spark is a new Windows PC platform for AI development, creation, and gaming in slim laptops and compact desktops. Powered by an NVIDIA Blackwell RTX…

NVIDIA RTX Spark is a new Windows PC platform for AI development, creation, and gaming in slim laptops and compact desktops. Powered by an NVIDIA Blackwell RTX GPU and NVIDIA Grace CPU, it combines up to 128 GB of unified memory with the full NVIDIA CUDA stack and RTX technologies including ray tracing and DLSS 5. This post covers the following technologies for game developers building on RTX…

Source

Categories
Misc

Can we trust LLMs to judge whether a research idea is novel?

Categories
Misc

Build specialized multimodal models with LFM3

Categories
Misc

NVIDIA and Meta to Expand Collaboration on Muse and Muse Family of Models Amid Robust Growth

NVIDIA and Meta today announced they will partner to expand their work on Muse and the Muse family of models to improve inference and training performance, reflecting popular demand for useful AI agents.

Categories
Misc

How NVIDIA Minerva Turns Telemetry Chaos Into Peak GPU Yield

AI Factories are some of the most technologically complex operational environments ever created. Hundreds of thousands of GPUs, NVIDIA Spectrum-X Ethernet…

AI Factories are some of the most technologically complex operational environments ever created. Hundreds of thousands of GPUs, NVIDIA Spectrum-X Ethernet scale-out, scale-across and scale-in fabrics, NVIDIA NVLink scale-up fabrics, distributed schedulers, and multi-week training runs operate in continuous lockstep.These layers function as one production system even when different teams own their…

Source

Categories
Misc

How NVIDIA Mission Control 2.5 Brings Autonomous Recovery to Kubernetes

Training large-scale AI models on thousands of GPUs is a high-stakes operation where monitoring and automation must be built-in to optimize cluster resiliency…

Training large-scale AI models on thousands of GPUs is a high-stakes operation where monitoring and automation must be built-in to optimize cluster resiliency and uptime at scale. This concept isn’t new to data center operators who have measured mean-time-to-repair (MTTR) and mean-time-between-failure (MTBF) long before AI factories existed. Now the stakes are higher. Modern AI factories run…

Source

Categories
Offsites

Sphere Intersection Puzzle Corrected

Categories
Misc

Scaling NVIDIA TensorRT Performance Optimization Beyond a Single Build Route with the Global Performance Tuner, Part 1

Deploying a trained neural network involves more than translating each operator into a GPU kernel. It is a compilation problem with many legitimate answers. For…

Deploying a trained neural network involves more than translating each operator into a GPU kernel. It is a compilation problem with many legitimate answers. For example, when NVIDIA TensorRT builds an engine, it makes interdependent decisions about graph-IR transformations, fusion, implementation selection, memory placement, and quantization. This process is called scheduling.

Source