Category: Misc

In the previous post, Profiling LLM Training Workflows on NVIDIA Grace Hopper, we explored the importance of profiling large language model (LLM) training…

In the previous post, Profiling LLM Training Workflows on NVIDIA Grace Hopper, we explored the importance of profiling large language model (LLM) training workflows and analyzed bottlenecks using NVIDIA Nsight Systems. We also discussed how the NVIDIA GH200 Grace Hopper Superchip enables efficient training processes. While profiling helps identify inefficiencies…

Source

The rapid advancements in AI have resulted in an era of exponential growth in model sizes, particularly in the domain of large language models (LLMs). These…

The rapid advancements in AI have resulted in an era of exponential growth in model sizes, particularly in the domain of large language models (LLMs). These models, with their transformative capabilities, are driving innovation across industries. However, the increasing complexity and computational demands of training such models necessitate a meticulous approach to optimization and profiling.

Source

NVIDIA DALI, a portable, open source software library for decoding and augmenting images, videos, and speech, recently introduced several features that improve…

NVIDIA DALI, a portable, open source software library for decoding and augmenting images, videos, and speech, recently introduced several features that improve performance and enable DALI with new use cases. These updates aim at simplifying the integration of DALI into existing PyTorch data processing logic, improving flexibility in building data processing pipelines by enabling CPU-to-GPU flows…

Source

Agents have been the primary drivers of applying large language models (LLMs) to solve complex problems. Since AutoGPT in 2023, various techniques have been…

Agents have been the primary drivers of applying large language models (LLMs) to solve complex problems. Since AutoGPT in 2023, various techniques have been developed to build reliable agents across industries. The discourse around agentic reasoning and AI reasoning models further adds a layer of nuance when designing these applications. The rapid pace of this development also makes it hard for…

Source

​​LLM Streaming sends a model’s response incrementally in real time, token by token, as it’s being generated. The output streaming capability has evolved…

​​LLM Streaming sends a model’s response incrementally in real time, token by token, as it’s being generated. The output streaming capability has evolved from a nice-to-have feature to an essential component of modern LLM applications. The traditional approach of waiting several seconds for full LLM responses creates delays, especially in complex applications with multiple model calls.

Source

Researchers, using AI to analyze routine brain scans, have discovered a promising new method to reliably identify a common but hard-to-detect precursor of many…

Researchers, using AI to analyze routine brain scans, have discovered a promising new method to reliably identify a common but hard-to-detect precursor of many strokes. In a study published in the journal Cerebrovascular Diseases, scientists from the Royal Melbourne Hospital described a new AI model that could one day prevent at-risk patients from becoming stroke victims.

Source

NVIDIA has achieved a world-record large language model (LLM) inference speed. A single NVIDIA DGX B200 node with eight NVIDIA Blackwell GPUs can achieve over…

NVIDIA has achieved a world-record large language model (LLM) inference speed. A single NVIDIA DGX B200 node with eight NVIDIA Blackwell GPUs can achieve over 1,000 tokens per second (TPS) per user on the 400-billion-parameter Llama 4 Maverick model, the largest and most powerful model available in the Llama 4 collection. This speed was independently measured by the AI benchmarking service…

Source