Category: Misc

Organizations are rapidly becoming more advanced in the use of AI, and many are looking to leverage the latest technologies to maximize workload performance and…

Organizations are rapidly becoming more advanced in the use of AI, and many are looking to leverage the latest technologies to maximize workload performance and efficiency. One of the most prevalent trends today is the use of CPUs based on Arm architecture to build data center servers. 

To ensure that these new systems are enterprise-ready and optimally configured, NVIDIA has approved the first NVIDIA-Certified Systems with Arm CPUs and NVIDIA GPUs. This post presents the benefits of NVIDIA-Certified Arm systems, and what customers should expect to see in the near future.

Using Arm architecture for HPC

Arm-based systems are common for edge applications. They are already widely used by large-scale cloud service providers, and are starting to become more popular for data center applications. According to Gartner®, 12% of new servers for high-performance computing (HPC) will be Arm-based by 2025.¹ 

Systems based on Arm architecture have the ability to run many cores with high energy efficiency, along with high memory bandwidth and low latency. In fact, recent results for the MLPerf benchmarks show Arm systems delivering almost the same performance for inference as x86-based systems, with one test showing the Arm-based server outperforming a similar x86 system.

The certification by NVIDIA of Arm-based systems is the culmination of a process that started in 2019, when NVIDIA ported the CUDA-X libraries to Arm. This paved the way for NVIDIA partners to start building energy-efficient, AI-enabled systems. NVIDIA also partnered with GIGABYTE in 2021 to develop and offer the Arm HPC Developer Kit. 

Now, NVIDIA Certification will help businesses choose the best enterprise-grade systems.

NVIDIA-Certified Arm systems

NVIDIA-Certified Systems offer NVIDIA GPUs and NVIDIA high-speed, secure network adapters from leading NVIDIA partners in configurations validated for optimum performance, manageability, and scale. Announced at the beginning of 2021, the program gives customers and partners confidence to choose enterprise-grade hardware solutions to power their accelerated computing workloads—from the desktop to the data center and edge.

More than 200 certified systems are now available—covering data center, desktop, and edge—from over 30 partners. NVIDIA-Certified Systems have excellent performance on a range of modern accelerated computing workloads, including AI and data science, 3D computing and visualization, and HPC. 

The certification also validates key enterprise capabilities, including management, security, and scalability. This ensures that certified systems can take advantage of powerful software including: 

• NVIDIA AI Enterprise for best-in-class production AI solutions 
• NVIDIA Fleet Command for secure edge management
• NVIDIA Omniverse Enterprise for real-time design collaboration and simulation  

GIGABYTE: The first Arm-ready certified system

The first NVIDIA-Certified Arm system is the GIGABYTE G242-P33, which features the Neoverse-based Ampere Altra processor and up to four NVIDIA A100 Tensor Core GPUs. GIGABYTE has been part of the NVIDIA-Certified Systems program since its inception, and now offers more than 15 NVIDIA-Certified Systems. 

“Qualifying Arm-based servers for NVIDIA accelerators continues to be one of GIGABYTE’s top priorities, and with NVIDIA-Certified Systems we will take the performance validation a step further to not only support the new NVIDIA H100 but also to include NVIDIA BlueField-2 DPU and InfiniBand products,” said Etay Lee, CEO of GIGABYTE. 

“Customers want an Arm-ready solution that comes with a wealth of NVIDIA resources and support to achieve faster insights,” Lee added. “That is what our Ampere Altra servers have delivered, starting with our server for the NVIDIA Arm HPC Developer Kit.”

As the Arm architecture becomes more adopted in data centers, it will be important to choose systems that are optimally configured. This is particularly the case for Arm systems equipped with GPUs and high-speed networking, since this architecture is new to many enterprises. 

Customers might not have expertise to design such a system properly, but NVIDIA-Certified Systems provide them with an easy way to make the best choices. To find Arm-based certified systems, see the Qualified Systems Catalog. The catalog will grow as more systems are certified.

¹Gartner, “Forecast Analysis: Arm-Based Servers, Worldwide,” G00755363, November 2021.

GARTNER is a registered trademark and service mark of Gartner, Inc. and/or its affiliates in the U.S. and internationally and is used herein with permission. All rights reserved.

Modern data centers can run thousands of services and applications. When an issue occurs, as a network administrator, you are guilty by default. You have to…

Modern data centers can run thousands of services and applications. When an issue occurs, as a network administrator, you are guilty by default. You have to prove your innocence on a daily basis, as it is easy to blame the network. It is an unfair world.

Correlating application performance issues to the network is hard to do. You can start by checking basic connectivity using simple pings or traceroutes, check your SNMP-based monitoring tools, sniffers, or even reading device counters to look for drops. In the meantime, users suffer from application slowness, poor performance, or even unavailability.

Unfortunately, all these classic network troubleshooting methods are time-consuming and don’t guarantee success, as it is sometimes nearly impossible to pinpoint problems using them.

NetQ to the rescue

To facilitate network troubleshooting, NVIDIA developed NetQ—a scalable, modern network operations toolset that provides network visibility in real time.

The NetQ team recently introduced the unique flow analysis tool to provide further visibility enhancements. Flow analysis allows network administrators to instantly correlate service traffic flows to the paths taken in the fabric, dramatically reducing the mean time to innocence (MTTI) or even ensuring there is no network issue.

Flow analysis enables you to discover and visualize all paths that a specific application’s traffic flow takes between endpoints in the fabric. It monitors the fabric-wide latency and buffer utilization statistics. With EVPN and multi-tenancy becoming the standard solution in most modern data centers, the flow analysis tool was designed to sample TCP or UDP data on overlay and underlay networks within different VRFs.

Flow analysis becomes even more powerful when used with What Just Happened (WJH) ASIC telemetry. While flows are being analyzed, flow-related WJH events from all switches in traffic paths are presented to help you discover if there were drops that caused the service issue. These two features working together maximize the probability of pinpointing the actual problem affecting an application.   

By the numbers

Flow analysis is supported on NVIDIA Spectrum 2 and later switches running Cumulus Linux 5.0 or later. It can also provide partial-path discovery for brownfield deployments with unsupported switches or switches running older versions of Cumulus Linux or SONiC.

Flow analysis samples traffic based on the packet’s four or five tuples, including VXLAN inner and outer headers. Its sampling lifetime is limited to 10, 15, 20, or 30 minutes. You can decide whether to run it on creation or schedule it for a later time.

The sample rate granularity is also configurable to low (1 per 10000), medium (1 per 1000), high (1 per 100), or all packets (1 per 1). The higher the sampling rate, the more accurate your analyzed data. A higher sampling rate results in higher CPU utilization, so I recommend setting lower sampling rates for heavy traffic flows.

Try it yourself in NVIDIA Air

NVIDIA Air is a tool for creating data center digital twins. With Air, you can build your own Cumulus Linux virtual data center, test it, validate it with NetQ, explore features, and learn some best practices. It is entirely free to use!

Try out flow analysis by spinning up the prebuilt NVIDIA Air Infrastructure Simulation Platform demo in the Air Marketplace. Follow the guided tour and see the significant benefits that flow analysis with NetQ can bring to your organization.

For more information, see the following resources:

• NVIDIA NetQ User Guide
• NVIDIA Air User Guide
• NVIDIA Cumulus Linux User Guide
• Analyzing Fabric-wide Network Latency With NetQ 4.1.0
• Automate Network Monitoring and Reduce Downtime with the Latest Release of NVIDIA NetQ
• Close Knowledge Gaps and Elevate Training with Digital Twin NVIDIA Air
• Troubleshooting Networks with NetQ

NVIDIA artist Sabour Amirazodi demonstrates his video editing workflows featuring AI this week in a special edition of In the NVIDIA Studio.

The post Video Virtuoso Sabour Amirazodi Shares AI-Powered Editing Tips This Week ‘In the NVIDIA Studio’ appeared first on NVIDIA Blog.

Join us on October 4 to explore new features in JetPack 5.0.2. Learn how to develop for any Jetson Orin module using emulation support on the Jetson AGX Orin…

Join us on October 4 to explore new features in JetPack 5.0.2. Learn how to develop for any Jetson Orin module using emulation support on the Jetson AGX Orin Developer Kit.

Build better GPU-accelerated Speech AI applications with the latest NVIDIA Riva updates, including enterprise support.

Build better GPU-accelerated Speech AI applications with the latest NVIDIA Riva updates, including enterprise support.

Learn the basics of physics-informed deep learning and how to use NVIDIA Modulus, the physics machine learning platform, in this self-paced online course.

Learn the basics of physics-informed deep learning and how to use NVIDIA Modulus, the physics machine learning platform, in this self-paced online course.

The massive virtual worlds created by growing numbers of companies and creators could be more easily populated with a diverse array of 3D buildings, vehicles, characters and more — thanks to a new AI model from NVIDIA Research. Trained using only 2D images, NVIDIA GET3D generates 3D shapes with high-fidelity textures and complex geometric details. Read article >

The post World-Class: NVIDIA Research Builds AI Model to Populate Virtual Worlds With 3D Objects, Characters appeared first on NVIDIA Blog.

Autonomous vehicle sensors require the same rigorous testing and validation as the car itself, and one simulation platform is up to the task. Global tier-1 supplier Continental and software-defined lidar maker AEye announced this week at NVIDIA GTC that they will migrate their intelligent lidar sensor model into NVIDIA DRIVE Sim. The companies are the Read article >

The post Continental and AEye Join NVIDIA DRIVE Sim Sensor Ecosystem, Providing Rich Capabilities for AV Development appeared first on NVIDIA Blog.

When it rains, it pours. And this GFN Thursday brings a downpour of news for GeForce NOW members. The Logitech G CLOUD is the latest gaming handheld device to support GeForce NOW, giving members a brand new way to keep the gaming going. But that’s not all: Portal with RTX joins GeForce NOW in November, Read article >

The post Go Hands On: Logitech G CLOUD Launches With Support for GeForce NOW appeared first on NVIDIA Blog.

When examining an intricate speech AI robotic system, it’s easy for developers to feel intimidated by its complexity. Arthur C. Clarke claimed, “Any…

When examining an intricate speech AI robotic system, it’s easy for developers to feel intimidated by its complexity. Arthur C. Clarke claimed, “Any sufficiently advanced technology is indistinguishable from magic.”

From accepting natural-language commands to safely interacting in real-time with its environment and the humans around it, today’s speech AI robotics systems can perform tasks to a level previously unachievable by machines.

Take Spot, a speech AI-enabled robot that can fetch drinks on its own, for example. To easily add speech AI skills, such as automatic speech recognition (ASR) or text-to-speech (TTS), many developers leverage simpler low-code building blocks when building complex robot systems.

For developers creating robotic applications with speech AI skills, this post breaks down the low-code building blocks provided by the NVIDIA Riva SDK.

By following along with the provided code examples, you learn how speech AI technology makes it possible for intelligent robots to take food orders, relay those orders to a restaurant employee, and finally navigate back home when prompted.

Design an AI robotic system using building blocks

Complex systems consist of several building blocks. Each building block is much simpler to understand on its own.

When you understand the function of each component, the end product becomes less daunting. If you’re using low-code building blocks, you can now focus on domain-specific customizations requiring more effort.

Our latest project uses “Spot,” a four-legged robot, and an NVIDIA Jetson Orin, which is connected to Spot through an Ethernet cable. This project is a prime example of using AI building blocks to form a complex speech AI robot system.

Our goal was to build a robot that could fetch us snacks on its own from a local restaurant, with as little intervention from us as possible. We also set out to write as little code as possible by using what we could from open-source libraries and tools. Almost all the software used in this project was freely available.

To achieve this goal, an AI system must be able to interact with humans vocally, perceive its environment (in our case, with an embedded camera), and navigate through the surroundings safely. Figure 2 shows how interaction, platform, and navigation represent our Spot robot’s three fundamental operation components, and how those components are further subdivided into low-code building blocks.

This post focuses solely on the human interaction blocks from the Riva SDK.

Add speech recognition and speech synthesis skills using Riva

We have so many interactions with people every day that it is easy to overlook how complex those interactions actually are. Speaking comes naturally to humans but is not nearly so simple for an intelligent machine to understand and talk.

Riva is a fully customizable, GPU-accelerated speech AI SDK that handles ASR and TTS skills, and is deployable on-premises, in all clouds, at the edge, and on embedded devices. It facilitates human-machine speech interactions.

Riva runs entirely locally on the Spot robot. Therefore, processing is secure and does not require internet access. It is also completely configurable with a simple parameter file, so no extra coding is needed.

Riva code examples for each speech AI task

Riva provides ready-to-use Python scripts and command-line tools for real-time transformation of audio data captured by a microphone into text (ASR, speech recognition, or speech-to-text) and for converting text into an audio output (TTS, or speech synthesis).

Adapting these scripts for compatibility with Open Robotics (ROS) requires only minor changes. This helps simplify the robotic system development process.

ASR customizations

The Riva OOTB Python client ASR script is named transcribe_mic.py. By default, it prints ASR output to the terminal. By modifying it, the ASR output is routed to a ROS topic and can be read by anything in the ROS network. The critical additions to the script’s main() function are shown in the following code example:

   inter_pub = rospy.Publisher('intermediate', String, queue_size=10)
   final_pub = rospy.Publisher('final', String, queue_size=10)
   rospy.init_node('riva_asr', anonymous=True)

The following code example includes more critical additions to main:

       for response in responses:
           if not response.results:
               continue
           partial_transcript = ""
           for result in response.results:
               if not result.alternatives:
                   continue
               transcript = result.alternatives[0].transcript
               if result.is_final:
                   for i, alternative in enumerate(result.alternatives):
                       final_pub.publish(alternative.transcript)
              else:
                  partial_transcript += transcript
           if partial_transcript:
               inter_pub.publish(partial_transcript)

TTS customizations

Riva also provides the talk.py script for TTS. By default, you enter text in a terminal or Python interpreter, from which Riva generates audio output. For Spot to speak, the input text talk.py script is modified so that the text comes from a ROS callback rather than a human’s keystrokes. The key changes to the OOTB script include this function for extracting the text:

def callback(msg):
   global TTS
   TTS = msg.data

They also include these additions to the main() function:

   rospy.init_node('riva_tts', anonymous=True)
   rospy.Subscriber("speak", String, callback)

These altered conditional statements in the main() function are also key:

       while not rospy.is_shutdown():
           if TTS != None:
               text = TTS

Voice interaction script

Simple scripts like voice_control.py consist primarily of the callback and talker functions. They tell Spot what words to listen for and how to respond. 

def callback(msg):
   global pub, order
   rospy.loginfo(msg.data)
   if "hey spot" in msg.data.lower() and "fetch me" in msg.data.lower():
       order_start = msg.data.index("fetch me")
       order = msg.data[order_start + 9:]
       pub.publish("Fetching " + order)

def talker():
   global pub
   rospy.init_node("spot_voice_control", anonymous=True)
   pub = rospy.Publisher("speak", String, queue_size=10)
   rospy.Subscriber("final", String, callback)
   rospy.spin()

In other words, if the text contains “Hey Spot, … fetch me…” Spot saves the rest of the sentence as an order. After the ASR transcript indicates that the sentence is finished, Spot activates the TTS client and recites the word “Fetching” plus the contents of the order. Other scripts then engage a ROS action server instructing Spot to navigate to the restaurant, while taking care to avoid cars and other obstacles.

When Spot reaches the restaurant, it waits for a person to take its order by saying “Hello Spot.” If the ASR analysis script detects this sequence, Spot recites the order and ends it with “please.” The restaurant employee places the ordered food and any change in the appropriate container on Spot’s back. Spot returns home after Riva ASR recognizes that the restaurant staffer has said, “Go home, Spot.”

The technology behind a speech AI SDK like Riva for building and deploying fully customizable real-time speech AI applications deployable on-premises, in all clouds, at the edge, and embedded, brings AI robotics into the real world.

When a robot seamlessly interacts with people, it opens up a world of new areas where robots can help without needing a technical person on a computer to do the translation.

Deploy your own speech AI robot with a low-code solution

Teams such as NVIDIA, Open Robotics, and the robotics community, in general, have done a fantastic job in solving speech AI and robotics problems and making that technology available and accessible for everyday robotics users.

Anyone eager to get into the industry or to improve the technology they already have can look to these groups for inspiration and examples of cutting-edge technology. These technologies are usable through free SDKs (Riva, ROS, NVIDIA DeepStream, NVIDIA CUDA) and capable hardware (robots, NVIDIA Jetson Orin, sensors).

I am thrilled to see this level of community support from technology leaders and invite you to build your own speech AI robot. Robots are awesome!

For more information, see the following related resources:

• Explore how to get started with integrating and deploying Riva ASR and TTS models in production with high-performance inference and minimal effort with the free ebook, Building Speech AI Applications.
• Explore the NVIDIA Riva Quick Start Guide for Embedded Platforms for scripts and utilities for getting started with Riva Speech Skills on Embedded platforms.
• Get started with downloading Riva, try guided Riva labs on ready-to-run infrastructure in LaunchPad, or get support for large-scale Riva deployments with NVIDIA AI Enterprise software.