2026-09-15 17:00
Exploring model architectures: Dense vs. Mixture-of-Experts (MoE) 🤖 Dense models activate all parameters per token, ensuring consistency. In contrast, MoE models, like Nemotron 3.5 Lightning, only activate a subset for each token, enhancing capacity without using all parameters. ⚙️ The choice between these models impacts throughput, memory cost, and complexity, making it essential to consider deployment needs. Learn more about their performance and application scenarios! 📊 #MachineLearning...
Source: Nvidia Developer Blog
Elizabeth Goodman
2026-09-15 16:55
NVIDIA's Vera Rubin platform is designed for power-efficient AI at scale, focusing on performance per watt. ⚡ Key features include the NVL72 core, which optimizes both large and small batch processing. Innovations like NVIDIA DSX MaxLPS software help manage power across racks, increasing GPU provisioning by 40% and boosting throughput by 35%. The Groq 3 LPX accelerator enhances low-latency performance through a deterministic execution model, allowing precise scheduling of data movements and...
Source: Nvidia Developer Blog
Tanya Lenz
2026-09-15 16:55
🚀 For large-scale AI factories, maintaining continuous output is crucial for productivity. NVIDIA's NVLink 6 provides essential multi-layer resiliency, ensuring that workloads progress without interruption. This is vital as AI models grow and require robust network infrastructure to handle transient errors effectively. The Vera Rubin platform connects GPUs into a single compute unit, enhancing reliability and maximizing cluster productivity. Learn more about how NVLink 6 ensures optimal...
Source: Nvidia Developer Blog
Elizabeth Goodman
2026-09-15 15:00
🌐 Federated learning (FL) can start simply, but managing larger projects introduces complexity. Different environments like Docker, Kubernetes, and Slurm can hinder collaboration. 🔧 NVIDIA FLARE addresses this by separating federation services from job execution. This allows each site to use its preferred infrastructure while maintaining control over data and compute policies. 📦 With FLARE 2.8 supporting Docker and Kubernetes, and 2.9 adding Slurm, organizations can efficiently manage...
Source: Nvidia Developer Blog
Elizabeth Goodman
2026-09-14 16:39
🚀 Mixture of Experts (MoE) is transforming large-scale AI model training. Models like DeepSeek, Qwen, and Mixtral outperform dense models while reducing compute costs. 🔍 MoE uses conditional computation, activating only selected expert networks for efficiency. However, scaling MoE training poses challenges, such as token routing and expert dispatch. 📈 Recent advancements with JAX and NVIDIA Transformer Engine have improved MoE performance from 103 TFLOPS/GPU to 1,068 TFLOPS/GPU, highlighting...
Source: Nvidia Developer Blog
Tanya Lenz
2026-09-10 16:55
🚀 Deploying a large language model is just the beginning. To ensure production readiness, teams must optimize for many concurrent users while maintaining responsiveness. 🔧 NVIDIA NIM provides a deployable microservice, offering a validated configuration and benchmarking capabilities. This enhances performance and ensures applications meet latency targets. 📊 A case study on Nemotron 3 Ultra shows NIM can deliver up to 2.5x higher throughput for agentic workloads, proving its effectiveness in...
Source: Nvidia Developer Blog
Elizabeth Goodman
2026-09-10 15:00
Unlock the power of biomolecular structure prediction with NVIDIA's BioNeMo Inference Runtime (BioIR). 🚀 BioIR accelerates structure-prediction models on NVIDIA GPUs, maintaining the PyTorch workflow. It boosts throughput for large batches by utilizing Ray for multiple model replicas. Notably, BioIR contributed to the AlphaFold Database, generating high-confidence predictions for over 4,777 proteomes! 🔬 Explore its end-to-end capabilities and enhance your research efficiency....
Source: Nvidia Developer Blog
Elizabeth Goodman
2026-09-10 09:00
NVIDIA's supply chain is among the most complex globally, measured from "wafer-out" to "first token." The process involves two key intervals: time-to-rack and time-to-token. Time-to-rack spans from silicon fabrication to system assembly, while time-to-token covers operational setup, including power and software. With millions of parts and global suppliers, the supply chain faces challenges in component availability, impacting assembly timelines. Components arrive from various sources,...
Source: Nvidia Developer Blog
Elizabeth Goodman
2026-09-09 20:31
🚀 Explore the benefits of Encode-Prefill-Decode (EPD) disaggregation for multimodal models! This technique optimizes inference by separating vision encoding from prefill and decode stages, significantly enhancing performance. With NVIDIA Dynamo, users can achieve up to 5x faster time to first token and 7x quicker end-to-end responses. However, EPD disaggregation is not always the best choice. It's ideal for image-heavy prompts and quantized mixture-of-experts models. #AI #MachineLearning...
Source: Nvidia Developer Blog
Tanya Lenz
2026-09-09 20:24
🚀 Exciting updates in CUDA Toolkit 13.4! This release introduces support for Windows on Arm, expanding developer capabilities on this platform. Previously available only on Linux, CUDA applications can now run seamlessly on Windows. Key highlights include early access to the NVIDIA Rubin architecture, improved GPU management with Multi-Process Service V3, and advancements in CUDA Python functionality. These enhancements aim to optimize GPU resource management and improve performance for...
Source: Nvidia Developer Blog
Jonathan Bentz
2026-09-08 12:00
🚀 NVIDIA is advancing GPU programming with the introduction of CUDA Rust! In September 2026, the company announced its focus on native programming in Rust, expanding on established CUDA C++ and Python toolchains. This move aims to enhance the development of GPU kernels, allowing them to be written and compiled directly in Rust. CUDA Rust offers two programming models: SIMT and Tile. The Tile model is recommended for its flexibility, while SIMT provides control for specific thread management....
Source: Nvidia Developer Blog
Elizabeth Goodman
2026-09-04 18:04
🚀 Exciting advancements in AI are here! The article discusses the development of a memory-driven Chief of Staff agent using NVIDIA NemoClaw. This AI agent reconstructs context to enhance productivity in enterprise workflows. Key features include a self model that organizes relevant information, tracks obligations, and allows user corrections to build trust. The design emphasizes maintaining context, separating evidence from action, and prioritizing user intent. For more insights, check the...
Source: Nvidia Developer Blog
Tanya Lenz
2026-09-04 16:21
🚀 Running AI at the edge is becoming easier! Recent advancements have led to the development of compact open models that can perform multi-step reasoning without relying on data centers. This shift allows NVIDIA Jetson to run models like Nemotron 3.5 Lightning and Qwen3.8-27B locally, enabling applications in various fields. Developers can now optimize inference performance and validate configurations more effectively. Key topics include model selection, NVFP4 quantization, and usage of vLLM....
Source: Nvidia Developer Blog
Elizabeth Goodman
2026-09-03 22:36
Modern AI platforms are complex, often requiring users to navigate multiple clusters and services. A central identity gateway can streamline user identity across federated environments. Traditional single sign-on (SSO) methods fall short when users move between systems. A central gateway maintains user sessions while data-plane gateways validate identities for applications, enhancing security and user experience. This approach has shown a 55% reduction in login events at NVIDIA, supporting...
Source: Nvidia Developer Blog
Elizabeth Goodman
2026-09-03 16:00
Source: Nvidia Developer Blog
Tanya Lenz
2026-09-02 17:15
Source: Nvidia Developer Blog
Elizabeth Goodman
2026-09-02 16:04
🚀 In the latest installment of the AI model co-design series, the focus is on accelerating LLM inference using speculative decoding. This technique enhances efficiency by predicting multiple tokens in one go while ensuring accuracy. Five guidelines are provided for selecting draft length and mechanisms to optimize performance. For deeper insights into model design choices and their impact on throughput, check out previous posts in the series. #AI #MachineLearning #LLM #SpeculativeDecoding...
Source: Nvidia Developer Blog
Tanya Lenz
2026-09-01 17:00
AI is transforming cybersecurity with adaptive agentic systems that enhance coordination and efficiency. Security teams are integrating these systems to identify gaps in defenses, but many still rely on predefined workflows. Continuous offense-defense testing is essential for improving coverage and validating detections against real-world activities. NVIDIA and CrowdStrike have developed an advanced system using NVIDIA Nemotron models, achieving better accuracy at a lower cost. This approach...
Source: Nvidia Developer Blog
Michelle Horton
2026-09-01 15:00
Navigating the GPU landscape for AI inference can be challenging. This article discusses how organizations can effectively size their GPU resources while optimizing Total Cost of Ownership (TCO). Key considerations include understanding your use case, latency targets, and token patterns. The four primary workload categories are AI Chatbots, AI Agents, Content Generation, and Translation Apps. The piece emphasizes that choosing the right model and planning around real workload behavior can...
Source: Nvidia Developer Blog
Elizabeth Goodman
2026-08-31 16:30
🚀 Agentic AI is transforming research by enabling AI scientists to read literature, propose hypotheses, and prioritize experiments. NVIDIA's BioNeMo Agent Toolkit enhances scientific workflows by providing specialized tools for protein folding and molecular docking. This integration with Claude Science allows for efficient protein structure prediction using advanced models. To get started, ensure you have the necessary GPU setup and follow the guidelines for running BioNeMo NIM microservices....
Source: Nvidia Developer Blog
Michelle Horton
2026-08-31 16:00
🚗✨ Adapting perception stacks for different vehicle types is crucial in autonomous driving. The NVIDIA Omniverse NuRec enables developers to use existing real-world data to prepare software for vehicles that may not yet exist. By reconstructing real-world drives, NuRec allows for rendering new camera views tailored to specific vehicle configurations. This process helps identify potential weak spots and gaps in data collection. Learn how to follow four key steps for effective carline...
Source: Nvidia Developer Blog
Michelle Horton
2026-08-28 17:06
Unlock the potential of Open AI models with NVIDIA TensorRT Model Connect! 🚀 This tool simplifies deploying models from Hugging Face to native C++ applications in just two commands. It provides reference implementations, allowing users to inspect and modify as needed. Additionally, TensorRT Model Connect supports custom GPU kernels and adapts to the evolving open model ecosystem. #NVIDIA #TensorRT #AIModels #MachineLearning #CPlusPlus
Source: Nvidia Developer Blog
Tanya Lenz
2026-08-26 21:06
NVIDIA introduces NVLink Fusion to enhance AI infrastructure for large models and complex workloads. This technology supports the deployment of custom AI accelerators (XPUs) and CPUs, streamlining their integration into data centers. Key benefits include improved performance, reduced development complexity, and faster market readiness. NVHBM technology offers increased memory bandwidth, efficient area usage, and lower power consumption, addressing critical challenges in AI accelerator design....
Source: Nvidia Developer Blog
Farshad Ghodsian
2026-08-26 20:05
🚀 Navigation is key for robots to achieve autonomy, combining perception and motion effectively. Developing navigation for different robots or environments can be costly and complex. An agent-driven workflow simplifies this by allowing developers to define the robot and its goals while automating asset preparation and testing. The COMPASS framework enhances this process by enabling scalable cross-embodiment mobility, reusing navigation behaviors and training specialized policies without...
Source: Nvidia Developer Blog
Tanya Lenz
2026-08-26 17:07
🚀 Alibaba has released the model weights for Qwen3.8-Flash-Next, previewing the upcoming Qwen4 architecture. This multimodal mixture-of-experts model features a 125B-parameter main model and an additional 51B N-gram embeddings, with a native 262,144-token context window. NVIDIA supports this model through SGLang, vLLM, and TensorRT, making it suited for high-volume applications like agentic coding and document processing. Notably, Qwen3.8-Flash-Next employs innovative architectures to enhance...
Source: Nvidia Developer Blog
Michelle Horton
2026-08-25 20:57
🚀 NVIDIA Dynamo introduces Shadow Engine Recovery, a game-changer for LLM engine failures! Traditional recovery methods involve time-consuming cold restarts, which can take several minutes and disrupt service. Shadow engine recovery keeps a fully initialized backup engine ready to take over in seconds. This new feature minimizes service disruption significantly. Tests show recovery times reduced from 283 seconds to just 7.3 seconds! #NVIDIA #Dynamo #TechInnovation #AI #LLMRecovery
Source: Nvidia Developer Blog
Michelle Horton
2026-08-25 15:00
🚀 CUDA Python 1.0 is here, providing a stable platform for Python developers needing GPU access without deep knowledge of CUDA C++. Key features include: - **cuda.core** for Pythonic access to CUDA runtime. - **cuda.compute** for parallel algorithms in Python. - **cuda.bindings** for low-level CUDA C APIs. This release introduces semantic versioning, ensuring predictable updates and stable APIs. Developers can now collaborate more effectively across GPU libraries. #CUDAPython #GPU...
Source: Nvidia Developer Blog
Elizabeth Goodman
2026-08-24 15:08
The rise of generative AI is reshaping data center design, highlighting limitations in traditional Ethernet networks. As AI workloads require synchronized communication across numerous GPUs, traditional Ethernet struggles with performance bottlenecks. NVIDIA's Spectrum-X Ethernet offers a solution, designed specifically for high-demand AI applications, providing low latency and improved bandwidth utilization. The article outlines how Spectrum-X addresses the shortcomings of standard Ethernet,...
Source: Nvidia Developer Blog
Elizabeth Goodman
2026-08-24 15:00
NVIDIA's latest advancements in AI, particularly with Vera Rubin and Blackwell, are driving significant changes in agentic AI performance. AI agents now handle complex workflows, increasing prompt token usage drastically—up to 15 times more than ordinary chat. This shift necessitates new benchmarks to assess hardware efficiency in real-world scenarios. The SemiAnalysis AgentX benchmark evaluates AI infrastructure for agentic-coding inference, showing impressive results: Vera Rubin NVL72...
Source: Nvidia Developer Blog
Elizabeth Goodman
2026-08-24 15:00
🚀 Exciting advancements in AI technology! NVIDIA Groq 3 LPX serves as the interactive AI inference accelerator for the NVIDIA Vera Rubin platform. This combination enhances performance across various AI workloads, enabling high throughput and interactivity. Recent benchmarks show Groq 3 LPX achieves an impressive 3,431 output tokens/second, supporting multiagent systems with long context and high interactivity. This capability is crucial for multiturn inference in agentic sessions. #NVIDIA...
Source: Nvidia Developer Blog
Tanya Lenz
2026-08-24 15:00
🔍 AI factories are evolving to focus on maximizing performance per watt, rather than just the number of GPUs. NVIDIA's DSX MaxLPS suite aims to enhance AI output within a fixed power budget by optimizing power allocation, improving performance efficiency, and utilizing advanced cooling techniques. This approach addresses traditional data center challenges by reallocating unused power effectively, enhancing overall productivity. #AI #DataCenters #NVIDIA #Efficiency #TechInnovation
Source: Nvidia Developer Blog
Tanya Lenz
2026-08-24 15:00
🚀 NVIDIA has unveiled its Scale-In network infrastructure, powered by BlueField-4, to enhance Agentic AI factories. This new architecture is designed to optimize data movement, security, and operational efficiency. It focuses on dedicated DPU processing to handle multi-terabit bandwidth, ensuring smooth performance for diverse applications. Scale-In transforms traditional north-south network setups, creating a coordinated domain for AI factories, allowing for better tenant isolation and...
Source: Nvidia Developer Blog
Michelle Horton
2026-08-24 15:00
🚀 AI factories are complex systems where efficiency is key to converting power and capital into completed tasks. GPUs power the models, while CPUs manage orchestration and execution. However, agentic workloads present unique challenges due to their unpredictable nature. Telemetry data shows over 97% of sessions exhibit distinct trajectory profiles, complicating fleet management and design. #AIFactories #NVIDIA #AgenticAI #DataAnalysis #TechInnovation
Source: Nvidia Developer Blog
Michelle Horton
2026-08-21 16:21
Unlock the power of GPU-accelerated clustering with AdaptGrow! 🚀 This innovative matrix factorization algorithm transforms rolling correlation and tail-dependence matrices into effective clusters. It supports quant strategies in portfolio construction and risk management. AdaptGrow addresses challenges in instrument groupings, offering both hard labels and soft factor loadings. It efficiently scales from single GPUs to multi-node systems, ensuring accurate risk assessment. Learn more about...
Source: Nvidia Developer Blog
Elizabeth Goodman
2026-08-21 15:00
🔍 AI factories are evolving to focus on maximizing performance per watt, rather than just the number of GPUs. NVIDIA's DSX MaxLPS suite aims to enhance AI output within a fixed power budget by optimizing power allocation, improving performance efficiency, and utilizing advanced cooling techniques. This approach addresses traditional data center challenges by reallocating unused power effectively, enhancing overall productivity. #AI #DataCenters #NVIDIA #Efficiency #TechInnovation
Source: Nvidia Developer Blog
Tanya Lenz
2026-08-21 13:00
NVIDIA's AVO has achieved a significant milestone by reaching 100% on the ARC-AGI-3 benchmark. This development showcases a general-purpose architecture designed for long-horizon autonomous agents. AVO operates as a coding agent system, capable of inspecting and modifying code, running commands, and validating its outcomes. It emphasizes sustained operation over extended tasks, adapting seamlessly to different environments. This architecture was initially tested in GPU-kernel optimization,...
Source: Nvidia Developer Blog
Tanya Lenz
2026-08-21 13:00
As AI agents become more advanced, ensuring security and trust in their applications is crucial. NVIDIA's security teams emphasize the importance of the agent stack's layers, including models and secure runtimes. Recent incidents highlight the risks of agents exceeding their intended boundaries. Research shows that key security principles can be applied to manage these risks effectively. Understanding behavioral and infrastructure controls is essential for guiding AI agent actions safely. 🔐🤖🌐...
Source: Nvidia Developer Blog
Michelle Horton
2026-08-20 16:00
Recommender systems (RecSys) face challenges at scale, especially with user histories evolving rapidly. Traditional models struggle to manage vast data volumes, often leading to bottlenecks during training and inference. 📊 The long-tail problem highlights how few popular items dominate user interactions, leaving niche items underrepresented. New users or items also face the cold start problem, lacking interaction history for effective recommendations. 🔄 Generative recommenders (GRs) are...
Source: Nvidia Developer Blog
Elizabeth Goodman
2026-08-19 22:22
🚀 NVIDIA Holoscan is revolutionizing real-time AI applications from medical imaging to robotics. The platform pairs with HoloHub, which offers a variety of reference applications and components. This article outlines using an AI coding agent to build a real-time endoscopic tool segmentation app. Using the Holoscan CLI, developers can streamline operations while iterating through defined goals and constraints. The process is adaptable to various coding agents, enhancing flexibility in...
Source: Nvidia Developer Blog
Elizabeth Goodman
2026-08-19 17:50
🚀 Modern vision-language models (VLMs) support tasks like visual question answering and captioning. However, the challenge lies in managing distributed data across institutions. NVIDIA FLARE enables federated learning, allowing sites to train models without centralizing raw data. Key design decisions focus on efficient model updates and aggregation methods. FedUMM, a collaboration between William & Mary and NVIDIA, serves as a practical example of federating lightweight components over a...
Source: Nvidia Developer Blog
Tanya Lenz