I’m learning tensorflow, so sorry if my question is too stupid.
I’ve just bought a RTX 3070, upgrading from my old GTX 970, both for gaming and for using with tensorflow.
If I use the two GPUs, my motherboard will “split” the speed of the PCIe3 slots from 16x to 8x. So my question is: is it worth it to keep the GTX 970 along with the RTX 3070 to send a part of the tensor calculation to the 970 using Strategy? Can the 970 give me more speed for training NNs, or will the gains be negligible? Do you guys think I should keep just the RTX 3070 on the motherboard?
My PSU is powerful enough for both (1000 watts), so the question here is just about the usefulness of the GTX 970 as a second gpu for NN training in tensorflow, since it has some CUDA cores that (I suppose) could potentially be used alongside the 3070.
What do you guys think? Have any of you tried something like that?
Is there such a thing a default null where a bunch of random real world images are used to make no choice? If we had two label classes class-1 & class-2 our model will always choose one or the other but we need a third class to say it is neither class one or class two. So there should be a default class that cross references & removes any data in this default class that correlates to either of our classes?
If we transform the image data… We input the raw, then transform it by splitting channels to R, G, B then input the seperate color channels then we transform it again to greyscale & input that so we do multiple rounds of transform for each single input data point…. Then to get a classification / prediction we all predictions from each transform and compare the results of each? Wouldn’t this allow us to achieve more accurate results…
Clara Holoscan SDK 0.2 offers real-time AI inference capabilities and fast I/O for high-performance streaming applications in medical devices.
Advances in edge computing, video cameras, real-time processing, and AI have helped transform medical devices over the years. NVIDIA developed the NVIDIA Clara Holoscan platform to support the development of software-defined AI medical devices. The platform consists of NVIDIA Clara Developer Kits, the NVIDIA Clara Holoscan SDK, and NVIDIA Clara Holoscan MGX for production-ready deployment. Â
The latest release of the NVIDIA Clara Holoscan SDK 0.2 offers real-time AI inference capabilities and fast I/O for high-performance streaming applications in medical devices. This includes endoscopy, ultrasound, surgical robots, microscopy, and genomics sequencing instruments.Â
The release also consists of:
Core backend on NVIDIA Graph eXecution Framework (GXF) vs. GStreamer.
A sample endoscopy AI application.
A customizable AI pipeline to add your own model.
Support for both the Clara AGX Developer Kit with the Jetson AGX Xavier and NVIDIA RTX 6000 and the Clara Holoscan Development Kit with the Jetson AGX Orin and NVIDIA RTX A6000.
Support on the NVIDIA JetPack 5.0 SDK, which includes Ubuntu 20.04.
Graph eXecution Framework processes streaming data
The most significant change in the Clara Holoscan SDK 0.2 is the shift of the core backend from GStreamer to the NVIDIA GXF. GXF is a framework supporting component-based programming for streaming data processing pipelines. It is built for very efficient data ingestion, data transfer, and AI/ML workloads.Â
With GXF, developers can create reusable components and combine them in graphs to build applications for different products quickly. GXF supports the processing of video and AU streams as well as user-defined streaming data types used in medical devices such as raw ultrasound, radiology imaging scanners, and microscopes.
A recent test using the NVIDIA Latency Display Analysis Tool on a 1080p video stream showed that GXF offers a significant speedup compared to previous solutions. In the test, GXF reduced the overhead in an AI Inferencing application by nearly 3x compared to a similar GStreamer-based pipeline in the Clara Holoscan SDK 0.1.Â
Figure 1. GXF in Clara Holoscan SDK 0.2 compared to GStreamer in the previous SDK 0.1
Additionally, GXF supports user-customizable components to support generic data processing pipelines. GXF handles the critical parts of building a high-performance application due to two important components.Â
First is a scheduler that determines when components execute. The scheduler supports single or multithreaded execution, with conditional execution, asynchronous scheduling, and other custom tools.Â
Second, GXF has a memory allocator that provides a system with an upfront allocation of a large contiguous memory pool and reuses regions as needed. To ensure zero-copy data exchange between components, memory can be pinned to the device.
Figure 2. An example of a generic pipeline where a developer can customize the workflow including data processing, model inference, storage, and visualization
Endoscopy AI sample application on Clara Holoscan
Digital endoscopy has evolved as a key technology for medical screenings and minimally invasive surgeries. Using real-time AI platforms to process and analyze the video signal produced by the endoscopic camera has been growing.Â
This technology is helping with anomaly detection and measurements, image enhancements, alerts, and analytics. The Clara Holoscan SDK 0.2 includes a sample AI-enabled endoscopy application showcasing the end-to-end functionality of GXF and support for devices that interface with AJA with an HDMI input.Â
The endoscopy AI sample application has a deep learning model to perform object detection and tool tracking in real time on an endoscopy video stream.Â
The application uses several NVIDIA features to minimize the overall latency, including:
GPUDirect RDMA video data transfer to eliminate the overhead of copying to or from system memory.
NVIDIA Performance Primitive Library for CUDA-accelerated 2D image transformations before AI inference.
TensorRT runtime for optimized AI Inference and speed-up.
CUDA and OpenGL interoperability, which provides efficient resource sharing on the GPU for visualization.
To learn more about the endoscopy AI sample application, its hardware and software reference architecture on Clara Holoscan, as well as the path to production, download the Clara Holoscan Endoscopy Whitepaper.
Figure 3: An endoscopy image from a gallbladder surgery showing AI-powered frame-by-frame tool identification and tracking. Image courtesy of Research Group Camma, IHU Strasbourg and the University of Strasbourg
Bring your own model AI application
Developers can bring their own AI model into the Clara Holoscan reference pipeline to create their own streaming workflow quickly. Swapping out of one model for another is accomplished by updating one configuration file and exporting data to the GXF native data format. Models saved in portable ONNX, as well as the NVIDIA performance-optimized TRT format, can be run on GXF’s built-in inference engines.Â
Support for the Clara Developer Kit
The Clara Holoscan SDK 0.2 is supported on the Clara AGX and the new Clara Holoscan Developer Kit. The next generation Clara Holoscan Development Kit is built with a high-performance NVIDIA Orin module, a powerful RTX A6000 GPU, and the connectivity performance of the ConnectX SmartNIC.Â
This kit is the ideal solution for developing the next generation of software-defined medical devices. Orin is geared for autonomous machines with high-speed interface support for multiple sensors and 8X the performance of the last generation for multiple concurrent AI inference pipelines.  Â
Updated JetPack 5.0HP1 with Ubuntu 20.04
The NVIDIA JetPack SDK contains the base OS for the Clara Holoscan SDK. For version 0.2, the JetPack SDK is being upgraded from version 4.5 to version 5.0HP1. This upgrades the OS to L4T rel-34, to be on par with Ubuntu 20.04 with LTS Kernel 5.10.
Get started with the Clara Holoscan SDK
The Clara Holoscan SDK 0.2 and source code are now accessible on GitHub with an Apache 2.0 license.Â
This release of Isaac Sim adds more tools for AI-based robotics including Isaac Gym support for RL, Isaac Cortex for cobot programming, and much more.
Today, NVIDIA announced the availability of the 2022.1 release of NVIDIA Isaac Sim. As a robotics simulation and synthetic data generation (SDG) tool, this NVIDIA Omniverse application accelerates the development, testing, and training of AI in robotics.
With Isaac Sim, developers can generate production quality datasets to train AI-perception models. Developers will also be able to simulate robotic navigation and manipulation, as well as build a test environment to validate robotics applications continually.
The latest version advances the age of AI robots with new tools like NVIDIA Isaac Cortex, a decision framework for training collaborative robots (cobots), and Isaac Gym, a GPU-accelerated reinforcement learning (RL) framework. NVIDIA Isaac Replicator, a set of synthetic data generation tools, APIs, and workflows, has also been updated with new capabilities to generate industrial environments for SDG procedurally.
Figure 1. Stacking blocks example from Isaac CortexÂ
NVIDIA Isaac Sim 2022.1 release highlights
Isaac Cortex: Program cobot tasks as easily as programming game AI. Leverage this decision framework for cobots to develop task-aware and adaptive skills. Using its belief representation of the world, analogous to the robot’s brain, real or simulated data can be used as inputs and the resulting actuations will be generated.
Isaac GYM: Train robots in minutes instead of weeks. Train complex robotic skills using RL. The Isaac GYM is a GPU-accelerated tool that keeps the entire RL training workflow on the GPU, which is critical to reduce training time.
Omnigraph: Simplify application development and debugging with visual programming. Build robotic applications by visually connecting compute nodes together in this Omniverse visual programming and scripting environment. Robotic applications tend to be very modular and lend themselves well to visual programming.
Isaac Sim/Gazebo Connector: Move between both simulators depending on tasks. ROS developers using Gazebo can import simulation assets into Isaac Sim for tasks like generating synthetic datasets or high-fidelity rendering. Additionally, multiple gazebo simulations can stay live synched by connecting to Omniverse’s nucleus server.
Additional Features:
Windows Support (limited)
New RobotsÂ
Quadrupeds:A1, GO1, Anymal
AMR: Obelix
New modular warehouse and conveyor assets
New ROS pipelines implemented in Omnigraph
Video 1. ShadowHands demo training in Isaac Gym
Training AI with synthetic dataÂ
Isaac Replicator is the synthetic data generation tool in Isaac Sim. Synthetic data is very useful in robotics to bootstrap training, address long-tail dataset challenges, and provide unavailable real-world data like speed and direction from synthetic video. Autonomous machines require synthetic data in training to ensure model robustness.
In the latest release, a new SDG feature called SceneBlox was added to generate scenes procedurally. SceneBlox can be used to create industrial environments like warehouses automatically. New examples were also added that demonstrate how to generate synthetic data and train a pose estimation model using Replicator.
Figure 2. Example of a warehouse procedurally generated using SceneBlox
#include "tensorflow/lite/interpreter.h" // getting a not found error
How can I add resolve this error? My assumption is that I’d need to add the tflite to my bash to make it available for all of my projects. How can I add tflite to the bash file?
For Seq2Seq deep learning architectures, viz., LSTM/GRU and multivariate, multistep time series forecasting, its important to convert the data to a 3D dimension: (batch_size, look_back, number_features). Here _look_back_ decides the number of past data points/samples to consider using _number_features_ from your training dataset. Similarly, _look_ahead_ needs to be defined which defines the number of steps in future, you want your model to forecast for.
I have a written a function to help achieve this:
def split_series_multivariate(data, n_past, n_future): ''' Create training and testing splits required by Seq2Seq architecture(s) for multivariate, multistep and multivariate output time-series modeling. ''' X, y = list(), list() for window_start in range(len(data)): past_end = window_start + n_past future_end = past_end + n_future if future_end > len(data): break # slice past and future parts of window- past, future = data[window_start: past_end, :], data[past_end: future_end, :] # past, future = data[window_start: past_end, :], data[past_end: future_end, 4] X.append(past) y.append(future) return np.array(X), np.array(y)
But, _look_back_ and _look_ahead_ are hyper-parameters which need to be tuned for a given dataset.
# Define hyper-parameters for Seq2Seq modeling: # look-back window size- n_past = 30 # number of future steps to predict for- n_future = 10 # number of features used n_features = 8
What is the _best practice_ for choosing/finding _look_back_ and _look_ahead_ hyper-parameters?
Some NVIDIA Jetson modules have limited storage space, which imposes a challenge in packing applications and libraries. Here are ways to cut down on disk usage.
NVIDIA Jetson provides flexible storage options/configurations for development, but some of the Jetson modules are equipped with a limited eMMC flash memory storage size for more cost-conscious, large-scale product deployment.
It may initially seem impossible to fit your applications and necessary libraries in the limited storage space, especially with the full set of NVIIA JetPack, BSP, and all the development software that NVIDIA has prepackaged for Jetson.
Table 1. Disk usage in the original configurations
However, you can cut down on disk usage by removing unnecessary packages, libraries, and other assets. Table 2 shows how you can reclaim more than 8 GB of storage space on some of the latest NVIDIA JetPack versions.
Table 2. Disk usage in an optimized deployment configuration
In this post, I present simplified steps to minimize the disk usage on your Jetson device, while sharing tips on ways to analyze disk usage, actual commands, and the example outputs on different versions of JetPack. I also show how to check if an AI application is still working functionally under the slimmed-down configuration.
Identifying what takes up space
The jetson-min-disk documentation shows how to analyze the current disk usage, identify what files and directories take up the space, and clarify the package dependencies. It also shows example command outputs on NVIDIA JetPack 4.6.x and NVIDIA JetPack 5.0.x, so that you can assess how much you may be able to cut down for your application.
Minimized configurations
Figure 1 shows an overview of the minimal configurations. The jetson-min-disk documentation introduces multiple configurations ([A] to [D]) for different development and productization needs.
Figure 1. Different minimal configurations
You can take the following actions to regain disk space.
Remove the desktop user interface.
Remove the documentation and samples package.
Remove dev packages.
Remove the desktop graphical user interface
You can remove ubuntu-desktop if you know your system does not require a graphical user interface on the NVIDIA Jetson native display output through HDMI, DP/eDP, or LVDS. For more information, see Removing GUI.
If you have installed the full set of JetPack components (libraries and SDKs) either with the sudo apt install nvidia-jetpack command or by using SDK Manager to install all, you may have packages that you do not need for your application.
Documentation and samples packages are some of the safest to remove, so you can start by uninstalling them. For more information, see Removing docs/sample.
When you are done with building your applications, you do not need dev packages that provide header files and static libraries. You can remove them after checking how much disk space each package takes up. For more information, see Removing dev packages.
If you are using a host x86-64 Linux host machine to flash your Jetson, you can create a minimal configuration RootFS and flash that image onto Jetson.
For more information about building the minimal L4T RooFS image, see Option: Minimal L4T.
Verification
The guide introduces ways to use the NVIDIA DeepStream reference app as a typical AI application to verify the minimally configured Jetson environment. For more information, see Verification.
git clone https://github.com/NVIDIA-AI-IOT/jetson-min-disk/
cd jetson-min-disk
cd test-docker
./docker-run-deepstream-app-overlay.sh
Figure 2. DeepStream reference app with overlay output config in action
Conclusion
In this post, I demonstrated ways to work within NVIDIA Jetson storage space limitations while identifying and keeping the essential runtime libraries for AI applications.
The documentation cross-referenced in this post provides commands and tips for different NVIDIA JetPack versions. They can be great tools if you are interested in optimizing the storage usage, especially on NVIDIA Jetson production modules.
Clara Holoscan SDK 0.2 offers real-time AI inference capabilities and fast I/O for high-performance streaming applications in medical devices.
This release of Isaac Sim adds more tools for AI-based robotics including Isaac Gym support for RL, Isaac Cortex for cobot programming, and much more.
Some NVIDIA Jetson modules have limited storage space, which imposes a challenge in packing applications and libraries. Here are ways to cut down on disk usage.
