Categories
Misc

Develop for All Six NVIDIA Jetson Orin Modules with the Power of One Developer Kit

With the Jetson Orin Nano announcement this week at GTC, the entire Jetson Orin module lineup is now revealed. With up to 40 TOPS of AI performance, Orin Nano…

With the Jetson Orin Nano announcement this week at GTC, the entire Jetson Orin module lineup is now revealed. With up to 40 TOPS of AI performance, Orin Nano modules set the new standard for entry-level AI, just as Jetson AGX Orin is already redefining robotics and other autonomous edge use cases with 275 TOPS of server class compute.

All Jetson Orin modules and the Jetson AGX Orin Developer Kit are based on a single SoC architecture with an NVIDIA Ampere Architecture GPU, a high-performance CPU, and the latest accelerators. This shared architecture means you can develop software for one Jetson Orin module and then easily deploy it to any of the others.

You can begin development today for any Jetson Orin module using the Jetson AGX Orin Developer Kit. The developer kit’s ability to natively emulate performance for any of the modules lets you start now and shorten your time to market. The developer kit can accurately emulate the performance of any Jetson Orin modules by configuring the hardware features and clocks to match that of the target module.

Development teams benefit from the simplicity of needing only one type of developer kit, irrespective of which modules are targeted for production. This also simplifies CI/CD infrastructure. Whether you are developing for robotics, video analytics, or any other use case, the capability of this one developer kit brings many benefits.

Transform the Jetson AGX Orin Developer Kit into any Jetson Orin module

With one step, you can transform a Jetson AGX Orin Developer Kit into any one of the Jetson Orin modules. We provide flashing configuration files for this process.

Emulating Jetson Orin module on the Jetson AGX Orin Developer Kit, follows the same steps as mentioned in to flash a Jetson AGX Orin Developer Kit using the flashing utilities. After placing your developer kit in Force Recovery Mode, the flash.sh command-line tool is used to flash it with a new image. For example, the following command flashes the developer kit with its default configuration:

$ sudo ./flash.sh jetson-agx-orin-devkit mmcblk0p1

The exact command that you use should be modified with the name of the flash configuration appropriate for your targeted Jetson Orin module being emulated. For example, to emulate a Jetson Orin NX 16GB module, use the following command:

$ sudo ./flash.sh jetson-agx-orin-devkit-as-nx-16gb mmcblk0p1

Table 1 lists the Jetson Orin modules and the flash.sh command appropriate for each.

Jetson Orin module to be emulated Flashing command
Jetson AGX Orin 64GB sudo ./flash.sh jetson-agx-orin-devkit mmcblk0p1
Jetson AGX Orin 32GB sudo ./flash.sh jetson-agx-orin-devkit-as-jao-32gb mmcblk0p
Jetson Orin NX 16GB sudo ./flash.sh jetson-agx-orin-devkit-as-onx16gb mmcblk0p1
Jetson Orin NX 8GB sudo ./flash.sh jetson-agx-orin-devkit-as-onx8gb mmcblk0p1
Jetson Orin Nano 8GB* sudo ./flash.sh jetson-agx-orin-devkit-as-nano8gb mmcblk0p1
Jetson Orin Nano 4GB sudo ./flash.sh jetson-agx-orin-devkit-as-nano4gb mmcblk0p1
Table 1. Flash.sh commands for Jetson Orin modules

Flash configurations for Jetson Orin Nano modules are not yet included in NVIDIA JetPack, as of version 5.0.2. Use these new configurations after downloading them and applying an overlay patch on top of NVIDIA JetPack 5.0.2 per the instructions found inside the downloaded file.

For more information about the flashing configurations useful for emulation, see Emulation Flash Configurations.

After flashing is complete, complete the initial bootup and configuration. Then you can install the rest of the NVIDIA JetPack components using SDK Manager or simply by using a package manager on the running developer kit:

sudo apt update
sudo apt install nvidia-jetpack

Now you have the developer kit running and NVIDIA JetPack installed. Your Jetson AGX Orin Developer Kit now emulates the performance and power of the specified Jetson Orin module.

Accurately emulate any Jetson Orin module

This native emulation is accurate because it configures the developer kit to match the clock frequencies, number of GPU and CPU cores, and hardware accelerators available with the target module.

For example, when emulating the Jetson Orin NX 16GB module:

  • The developer kit GPU is configured with 1024 CUDA cores and 32 Tensor Cores with a max frequency of 918 MHz.
  • The CPU complex is configured with eight Arm Cortex-A78AE cores running at 2 GHz.
  • The DRAM is configured to 16 GB with a bandwidth of 102 GB/s.
  • The system offers the same power profiles supported by the Jetson Orin NX 16GB module.
Screenshot shows that the available power modes on a developer kit flashed to emulate Jetson Orin NX 16GB matches the power modes available on Jetson Orin NX 16GB.
Figure 1. Available power modes

Open the Jetson Power graphical user interface from the top menu on the desktop and you see that the system has been configured accurately as per the target module being emulated. Max clocks can be configured by running the following command, and the Jetson Power graphical user interface will show the change.

sudo jetson_clocks

Figure 2 shows the Jetson Power graphical user interface after configuring max clocks when the Jetson AGX Orin Developer Kit is flashed to an emulated Jetson AGX Orin 64GB module compared to when flashed to emulate a Jetson Orin NX 16GB module.

Screenshot shows two sets of various system configurations adapted to the target modules.
Figure 2. Jetson Power graphical user interface shown on a developer kit flashed to emulate Jetson AGX Orin 64GB (left) and Jetson Orin NX 16GB with MAXN power mode selected (right)

By running various samples provided with NVIDIA JetPack, you can see that the performance is adjusted to match that of the module being emulated. For example, the benchmarking sample packaged with the VPI library can be used to show CPU, GPU, and PVA performance for Jetson AGX Orin 64GB, Jetson Orin NX 16GB, and Jetson Orin Nano 8GB modules after configuring the Jetson AGX Orin Developer Kit to emulate the respective module.

To run the VPI benchmarking sample, use the following commands:

cd /opt/nvidia/vpi2/samples/05-benchmark
sudo cmake .
sudo make
sudo ./vpi_sample_05_benchmark 

The VPI benchmarking sample outputs the latency in milliseconds for the Gaussian algorithm. Table 2 shows the results for each of the targeted modules.

Algorithm: 5X5
Gaussian Filter Input Image Size: 1920 X 1080
Input Format: U16
Emulated as Jetson AGX Orin 64GB Emulated as Jetson Orin NX 16GB Emulated as Jetson Orin Nano 8GB
Running on CPU 0.331 0.492 0.838
Running on GPU 0.065 0.143 0.210
Running on PVA 1.169 1.888
Table 2. Latency in milliseconds for targeted modules

Similarly, you can run multimedia samples for encode and decode.

For decode, run the following commands:

cd /usr/src/jetson_multimedia_api/samples/00_video_decode
sudo make
sudo ./video_decode H264 --disable-rendering --stats --max-perf 

For encode, run the following commands:

cd /usr/src/jetson_multimedia_api/samples/01_video_encode
sudo make
sudo ./video_encode input.yuv 1920 1080 H264 out.h264 -fps 30 1 -ifi 1000 -idri 1000 --max-perf --stats

Table 3 reports the FPS numbers after running these encode and decode samples using H.264 1080P 30FPS video streams.

Encode/Decoder Sample Emulated as Jetson AGX Orin 64GB Emulated as Jetson Orin NX 16GB Emulated as Jetson Orin Nano 8GB
Encode 178 142 110*
Decode 400 374 231
Table 3. FPS numbers after running encode and decode samples

*Jetson Orin Nano does not include a NVEncoder. For Table 3, encoding for Jetson Orin Nano was done on CPU using ffmpeg. 110 FPS is achieved when using four CPU cores. When using two CPU cores, FPS of 73 was achieved, and when using a single CPU core, FPS of 33 was achieved.

To demonstrate the accuracy of emulation, we ran some AI model benchmarks on the Jetson AGX Orin Developer Kit emulated as Jetson AGX Orin 32GB. We then compared it with results obtained by running the same benchmarks on the real Jetson AGX Orin 32GB module. As you can see from the results, the difference between emulated and real performance is insignificant.

Although Jetson AGX Orin Developer Kit includes a 32GB module, it provides the same level of performance and comes with 275 TOPS matching the Jetson AGX Orin 64GB. There is no special flashing configuration required for emulation of Jetson AGX Orin 64 GB but you must use the appropriate flashing configuration to emulate Jetson AGX Orin 32GB on Jetson AGX Orin Developer Kit.

Model Jetson AGX Orin 32GB Emulated Jetson AGX Orin 32GB Real
PeopleNet (V2.5) 327 320
Action Recognition 2D 1161 1151
Action Recognition 3D 70 71
LPR Net 2776 2724
Dashcam Net 1321 1303
BodyPose Net 359 363
Table 4. Performance comparison between real and emulated Jetson AGX Orin modules

Do end-to-end development for any Jetson Orin module

You can work with the entire Jetson software stack while emulating a Jetson Orin module. Frameworks such as NVIDIA DeepStream, NVIDIA Isaac, and NVIDIA Riva work in emulation mode, and tools like TAO Toolkit perform as expected with pretrained models from NGC. The software stack is agnostic of the emulation and the performance accurately matches that of the target being emulated.

Diagram shows the stack architecture with the NVIDIA JetPack SDK, frameworks such as NVIDIA DeepStream, NVIDIA Isaac, and NVIDIA Riva, and NVIDIA tools like Train-Adapt-Optimize and pretrained models.
Figure 4. NVIDIA Jetson software stack

If you are developing a robotics use case or developing a vision AI pipeline, you can do end-to-end development today for any Jetson Orin module using the Jetson AGX Orin Developer Kit and emulation mode.

Develop robotics applications with NVIDIA Isaac ROS for any Jetson Orin module. Just use the right flashing configuration to flash and start your ROS development. Figure 5 shows running Isaac ROS Stereo Disparity DNN on the Jetson AGX Orin Developer Kit emulated as Jetson Orin Nano 8GB.

Stereo disparity is running on a warehouse scene where a front loader is traveling across the screen.
Figure 5. NVIDIA Isaac ROS Stereo Disparity DNN running on Jetson AGX Orin Developer Kit emulated as Jetson Orin Nano 8GB

Develop vision AI pipelines using DeepStream on the Jetson AGX Orin Developer Kit for any Jetson Orin module. It just works!

Figure 6 shows an IVA pipeline running people detection using DeepStream on the Jetson AGX Orin Developer Kit emulated as Jetson Orin Nano 8GB with four streams of H.265 1080P 30FPS.

People are detected with a blue bounding box and cars with a red bounding box.
Figure 6. DeepStream vision pipeline running people and car detection running on Jetson AGX Orin Developer Kit emulated as Jetson Orin Nano 8GB

Get to market faster with the Jetson AGX Orin Developer Kit 

With the emulation support, you can get to production faster by starting and finishing your application development on the Jetson AGX Orin Developer Kit. Buy the kit and start your development. We will also cover emulation in detail in the upcoming NVIDIA JetPack 5.0.2 webinar. Register for the webinar today!

Leave a Reply

Your email address will not be published. Required fields are marked *