Boost productivity and model training with new pretrained models and features such as ONNX model weights import, REST APIs, and TensorBoard visualization.
Today, NVIDIA announced the general availability of the latest version of the TAO Toolkit. As a low-code version of the NVIDIA Train, Adapt and Optimize (TAO) framework, the toolkit simplifies and accelerates the creation of AI models for speech and vision AI applications.
With TAO, developers can use the power of transfer learning to create production-ready models customized and optimized for many use-cases. These include detecting defects, translating languages, or managing traffic—without the need for massive amounts of data.
This version boosts developer productivity with new pretrained vision and speech models. It also includes key new features such as ONNX model weights import, REST APIs, and TensorBoard integration.
Deploy TAO Toolkit as-a-Service with REST APIs:Build a new AI service or integrate into an existing one with REST APIs. You can manage and orchestrate the TAO Toolkit service on Kubernetes. With TAO Toolkit as-a-service IT managers can deliver scalable services using industry-standard APIs.
Bring your own model weights: Fine-tune and optimize your non-TAO models with TAO. Import pretrained weights from ONNX and take advantage of TAO features like pruning and quantization on your own model. This is supported for image classification and segmentation tasks.
Visualize with TensorBoard: Understand your model training performance by visualizing scalars such as training and validation loss, model weights, and predicted images in TensorBoard. Compare results between experiments by changing hyperparameters and choose the one that best fits your needs.
Pretrained models: Pretrained models speed up the customization process for you to fine-tune through the power of transfer learning, with less data.
Some of the new pretrained models in this latest version can:
Apply data gathered from LIDAR sensors for robotics and automotive applications.
Classify human actions based on human poses that can be used in public safety, retail, and worker safety use cases.
Estimate keypoints on humans, animals, and objects to help portray actions or simply define the object shape.
Create custom voices with just 30 minutes of recorded data to power smart devices, game characters, and quick service restaurants.
Enterprise support for TAO Toolkit is available with NVIDIA AI Enterprise, an end-to-end software suite for AI development and deployment. This new release of TAO Toolkit will be included in the next quarterly update to NVIDIA AI Enterprise.
Watch demo videos of new features including Rest APIs, TensorBoard, and pretrained models.
Solutions using TAO Toolkit
Develop and deploy AI-powered Robots with NVIDIA Isaac Sim and NVIDIA TAO.
Read the partner blog from LexSet on how to use their platform to generate synthetic data and fine-tune object detection and segmentation models with the TAO Toolkit.
Computer vision specialist Landing AI has a unique calling card: Its co-founder and CEO is a tech rock star. At Google Brain, Andrew Ng became famous for showing how deep learning could recognize cats in a sea of images with uncanny speed and accuracy. Later, he founded Coursera, where his machine learning courses have attracted Read article >
Join NVIDIA at Automate 2022, June 6-9, to learn about AI platforms for manufacturing, robotics, and logistics that improve efficiency, scalability, and production across industries.
Clara Parabricks now includes rapid variant annotation tools, support for tumor-only variant calling in clinical settings, and additional support on ampere GPUs.
Bioinformaticians are constantly looking for new tools that simplify and enhance genomic analysis pipelines. With over 60 tools, NVIDIA Clara Parabricks powers accurate and accelerated genomic analysis for germline and somatic workflows in research and clinical settings.
The rapid increase in sequencing data demands faster variant call format (VCF) reading and writing speeds. Clara Parabricks 3.8 expands rapid variant annotation, post-variant calling, support of custom database annotation with snpswift, and variant consequence calling with bcftools. On Clara Parabricks, snpswift provides fast and accurate VCF database annotation and leverages a wide range of databases.
Advances in sequencing technologies are amplifying the role of genomics in clinical oncology. NVIDIA Clara Parabricks now provides tumor-only calling with somatic callers LoFreq and Mutect2 for clinical cancer workflows. Tumor-normal calling is available on Parabricks with LoFreq, Mutect2, Strelka2, SomaticSniper, Muse.
Genomic scientists can further accelerate genomic analysis workflows by running Clara Parabricks with a wider array of GPU architectures, including A6000, A100, A10, A30, A40, V100, and T4. This also supports customers using next-generation sequencing instruments powered by specific NVIDIA GPUs for basecalling that want to use the same GPUs for secondary analysis with Clara Parabricks.
Expanded rapid variant annotation
In the January release of Clara Parabricks 3.7, a new variant annotation tool was added that helps provide functional information of a variant for downstream genomic analysis. This is important, as correct variant annotation can assist with final conclusions of genomic studies and clinical diagnosis.
Parabricks’ variant annotation tool, snpswift, provides fast and accurate VCF database annotation that delivers results in shorter runtimes than other community variant annotation solutions such as vcfanno. Snpswift brings more functionality and acceleration while retaining the essential functionality of accurate allele-based database annotation of VCF files. The new snpswift tool also supports annotating a VCF file with gene name data from ENSEMBL, helping to make sense of coding variants.
Supported databases include dbSNP, gnomAD, COSMIC, ClinVar, and 1000 Genomes. Snpswift can annotate these jointly to provide important information for filtering VCF variants and interpreting their significance. Additionally, snpswift is able to annotate VCFs with information from an ENSEMBL GTF to add detailed information and leverage other widely used databases in the field.
Figure 1. Clara Parabricks’ variant annotation tool snpswift provides faster, more accurate VCF database annotation than other community variant annotation tools such as vcfanno
In addition to these widely used databases, many research institutions and companies have their own rich internal datasets, which can provide valuable domain-specific information for each variant.
Snpswift in Clara Parabricks 3.8 now annotates VCFs with multiple custom TSV databases. Snpswift is also able to annotate a 6 million variant HG002 VCF with a custom TSV containing 500K variants in less than 30 seconds. Users are able to annotate VCFs with VCF, GTF, and TSV databases jointly—all with one command and in one run.
Finally, Clara Parabricks 3.8 has included consequence prediction. Predicting the functional outcome of a variant is a vital step in annotation for categorization and prioritization of genetic variants. Parabricks 3.8 offers a bcftoolscsq command that wraps the well known and extremely fast bcftools csq tool, providing haplotype-aware consequence predictions. This leads to phasing of variants in a VCF file to avoid common errors when variants affect the same codon.
Figure 2. Image taken from this Consequence Calling GitHub link. Performance comparison of BCFtools/csq with three popular consequence callers using a single-sample VCF with 4.5M sites
Clara Parabricks has demonstrated 60x acceleration for state-of-the-art bioinformatics tools compared to CPU-based environments. End-to-end analysis of whole-genome workflows runs in 22 minutes and exome workflows in just 4 minutes. Large-scale sequencing projects and other whole-genome studies are able to analyze over 60 genomes a day on a single DGX server while reducing associated costs and generating more useful insights than ever before.
To get started on NVIDIA Clara Parabricks for your germline, cancer, and RNA-Seq analysis workflows, try a free 90-day trial. You can access Clara Parabricks on premise or in the cloud with AWS Marketplace.
An overview of how to annotate variants with Parabricks 3.8 and run consequence prediction using example data is available here as a GitHub Gist.
I have adapted this autoencoder code from one of the tutorials and is as below. I am training the network on mnist images.
I found while experimenting with the network that model.fit() fires encoder-decoder network twice; even when the number of training sample is just 1 and number of epochs selected is also 1 with batch_size is None
import numpy as np import tensorflow as tf import tensorflow.keras as k import matplotlib.pyplot as plt from tensorflow.keras.layers import Dense, Conv2D, MaxPooling2D, UpSampling2D # seed values np.random.seed(111) tf.random.set_seed(111)
# Encoder Network class Encoder(k.layers.Layer): def __init__(self): super(Encoder, self).__init__() self.conv1 = Conv2D(filters=32, kernel_size=3, strides=1, activation = 'relu', padding='same') self.conv2 = Conv2D(filters=32, kernel_size=3, strides=1, activation='relu', padding='same') self.conv3 = Conv2D(filters=16, kernel_size=3, strides=1, activation='relu', padding='same') self.pool = MaxPooling2D(padding='same') def call(self, input_features): x = self.conv1(input_features) x = self.pool(x) x = self.conv2(x) x = self.pool(x) x = self.conv3(x) x = self.pool(x) return x # Decoder Network class Decoder(k.layers.Layer): def __init__(self): super(Decoder, self).__init__() self.conv1 = Conv2D(filters=16, kernel_size=3, strides=1, activation='relu', padding='same') self.conv2 = Conv2D(filters=32, kernel_size=3, strides=1, activation='relu', padding='same') self.conv3 = Conv2D(filters=32, kernel_size=3, strides=1, activation='relu', padding='valid') self.conv4 = Conv2D(filters = 1, kernel_size=3, strides=1, activation='softmax', padding='same') self.upsample = UpSampling2D(size=(2,2)) def call(self, encoded_features): x = self.conv1(encoded_features) x = self.upsample(x) x = self.conv2(x) x = self.upsample(x) x = self.conv3(x) x = self.upsample(x) x = self.conv4(x) return x # Autoencoder Network class Autoencoder(k.Model): def __init__(self): super(Autoencoder, self).__init__() self.encoder = Encoder() self.decoder = Decoder() def call(self, input_features): print("Autoencoder call") encode = self.encoder(input_features) decode = self.decoder(encode) return decode
Train the model
model = Autoencoder() model.compile(loss='binary_crossentropy', optimizer='adam') sample = np.expand_dims(x_train[1], axis=0) sample_noisy = np.expand_dims(x_train_noisy[1], axis=0) print("shape of sample: {}".format(sample.shape)) print("shape of sample_noisy: {}n".format(sample_noisy.shape)) loss = model.fit(x=sample_noisy, y=sample, epochs=1)
I am training the model on only one sample for only 1 iteration. However, the print statements shows that my autoencoder.call() function is getting called twice.
I have trained a model on my computer using the YoloV5 algorithm and I was trying to run it on my Raspberry Pi. It was a classification model and I converted the model from .pt to .tflite. Unfortunately, when I run it, it tells me the index is out of range. Did I convert the file wrong? My labels file has the right amount of classes…
I load a really big image. I then downscale the image, and feed it to keras, who’s gonna perform filter = model.predict(image) on it. I then wanna take the results of model.predict(image) and be able to use it as a filter, i could apply to the original image
I want to do this since i have plenty of power for a 4k or even 6k image, but larger than that, and the model starts to struggle. But applying a 6k filter on a 8k, 10k or even 12k image doesn’t really affect the results at all (tested with good ol’ photoshop) So performing model.predict(image) on a lower res version, would save RAM, computational power, and a lot of time 🙂
Libraries and drivers are not one and the same. This blog explains which is the best for your need to clear up any confusion.
The NVIDIA DOCA Software framework includes everything needed to program the NVIDIA BlueField data processing unit (DPU) and provides a consistent experience regardless of the development environment. NVIDIA offers the following resources:
Developer Program
SDK manager support
A compilation of tools:
Compilers
Benchmarks
API reference and programmer’s guides
Reference applications
Use cases
NVIDIA delivers the stack by offering a DOCA SDK for developers and DOCA runtime software for out-of-the-box deployment.
DOCA drivers or DOCA libraries?
The DOCA drivers and DOCA libraries are critical pieces for developers, IT security and operations teams, and IT administrators. They are used to develop and deploy software-defined and hardware-accelerated applications for DPUs. However, I sometimes receive questions about the correct one to use.
To ensure that there is no confusion and to determine which might be best for your development needs, I’ve written this post to discuss when to use which.
DOCA drivers
DOCA libraries
Hardware-accelerated
Yes
Yes
Code management
Fine-grained control
Implicit initialization and unified APIs
Coding complexity
High complexity
Simplified, with programming guides
License
Mostly open source
DOCA
Multi-generation compatibility
Limited
Supported
Per-use case logic
Developers’ responsibility
Built-in
Reference applications
Partially available
Available for every library
Performance
Optimized
Maximized
Scale
Component dependent
Maximized
Table 1. DOCA drivers vs. DOCA libraries
Table 1 compares drivers and libraries and emphasizes the pros and cons of each. Essentially, DOCA drivers provide more room for customization, while DOCA libraries are architected to provide the best per-use case performance and scale with lower coding complexity.
DOCA libraries
First, DOCA libraries are higher-level abstraction APIs tuned for specific use cases. Libraries can be used to achieve outstanding performance with quicker development times and time-to-market. They also include a variety of guides and sample applications that provide a shorter learning curve than DOCA drivers when used for development.
NVIDIA libraries have been pre-accelerated. They enable you to build various applications quickly, with significant performance gains, as the logic has been created and tuned for designated use cases. They also ensure multi-generation compatibility, which can’t be guaranteed when using DOCA drivers.
The libraries aim to address a specific use case, such as a firewall, gateway, or storage controller. They use PMD and DPDK and contain additional functionality and logic that doesn’t exist within DPDK or at the driver level.
For example, if you use RegEx to identify complex string patterns for deep packet inspection (DPI), the DOCA DPI library includes preprocessing (packet header parsing) and post-processing routines to make it easier to use the RegEx accelerator to provide actions on network packets. The DPDK RegEx API does not include any of this. The DOCA DPI library API is abstracted and easier to develop packet inspection routines with, as there is no need to understand the logic.
DOCA libraries enable you to choose the preferred APIs with built-in hardware acceleration. The current revision of DOCA 1.3 includes over 120 DOCA APIs:
These services are available through the NGC Catalog and are deployable on BlueField DPUs in minutes.
The libraries’ value is delivered through a runtime environment, DOCA services, and an expansive set of documentation. The typical library user is not expected to develop applications but rather to leverage existing applications and services from NVIDIA or third parties.
DOCA services are containerized drivers and libraries made up of multiple items that can run as a service to provide specific functionality. Each service offers different capabilities, such as the DOCA telemetry API, which can be pulled in minutes from the NGC catalog. It provides a fast and convenient way to collect user-defined data and transfer it to DOCA telemetry service (DTS).
In addition, the API offers several built-in outputs for user convenience, including saving data directly to storage, NetFlow, Fluent Bit forwarding, and Prometheus endpoint.
Each of these libraries share objects and are not tied in any way except that they each use the PMD driver. Similarly, each has a common infrastructure, and each has its own documentation and programmer’s guide.
DOCA drivers and DOCA SDK
Although libraries eliminate low-level programming, they may not support all features and functionality that you are looking for, so NVIDIA offers DOCA drivers. DOCA drivers are open source-based and provide more flexibility if you’re developing yourown solutions or must create a unique solution.
NVIDIA drivers are designed for developers and are delivered through the DOCA SDK. The SDK includes all the components required to create and build applications, including reference application sources, development tools, documentation, and the NVIDIA SDK manager. The SDK manager enables the quick deployment of the development environment and can also flash and install an image to a local DPU.
The developer container enables the development of DOCA-accelerated applications anywhere. You don’t have to do this on the Arm processors on the DPU. On a host with the physical DPU, you can do this in a developer container, which emulates the Arm processor. NVIDIA provides detailed documentation, examples, and API compatibility.
The DOCA SDK is the most efficient way for you to leverage the DOCA libraries and drivers and create unique and personalized software to meet your application development needs.
The DOCA runtime is also available for you to verify and test your applications.
DOCA Runtime
If you’re unready or unable to port your application to the Arm architecture, NVIDIA provides the DOCA runtime for x86. In this case, a gRPC client runs on the DPU and establishes a communications channel with the x86 runtime. The application can access DPU runtime components, and you don’t have to compile any Arm code.
DOCA simplifies the programming and application development for BlueField DPUs and removes obstacles by providing a higher level of abstraction. By providing runtime binaries and high-level APIs, the DOCA framework enables you to focus on application code rather than learning.
There are two development routes you can choose: through libraries and services or through an SDK and drivers. Currently, the DOCA software stack includes over 120 DOCA APIs that are being used by more than 2500 DOCA developers worldwide. They are available through the NGC Catalog.
If you are new to DOCA, NVIDIA offers a complimentary, self-paced course, Introduction to DOCA for DPUs. It covers the essentials of the DOCA platform.
I hope I’ve cleared up any confusion and I encourage you to start your development journey by joining the DOCA developer program today.
For more information, see the following resources:
Boost productivity and model training with new pretrained models and features such as ONNX model weights import, REST APIs, and TensorBoard visualization.
Join NVIDIA at Automate 2022, June 6-9, to learn about AI platforms for manufacturing, robotics, and logistics that improve efficiency, scalability, and production across industries. 
Clara Parabricks now includes rapid variant annotation tools, support for tumor-only variant calling in clinical settings, and additional support on ampere GPUs.
Libraries and drivers are not one and the same. This blog explains which is the best for your need to clear up any confusion.