DataBloomSkycatch, a San Francisco-based startup, has been helping companies mine both data and minerals for nearly a decade. The software-maker is now digging into the creation of digital twins, with an initial focus on the mining and construction industry, using the NVIDIA Omniverse platform for connecting and building custom 3D pipelines. SkyVerse, which is a Read article >
The post Unearthing Data: Vision AI Startup Digs Into Digital Twins for Mining and Construction appeared first on NVIDIA Blog.
The numbers are in, and they paint a picture of data centers going a deeper shade of green, thanks to energy-efficient networks accelerated with data processing units (DPUs). A suite of tests run with help from Ericsson, RedHat and VMware show power reductions up to 24% on servers using NVIDIA BlueField-2 DPUs. In one case, Read article >
The post Take the Green Train: NVIDIA BlueField DPUs Drive Data Center Efficiency appeared first on NVIDIA Blog.
It’s a brand new month, which means this GFN Thursday is all about the new games streaming from the cloud. In November, 26 titles will join the GeForce NOW library. Kick off with 11 additions this week, like Total War: THREE KINGDOMS and new content updates for Genshin Impact and Apex Legends. Plus, leading 5G Read article >
The post Check Out 26 New Games Streaming on GeForce NOW in November appeared first on NVIDIA Blog.
CFO Commentary to Be Provided in Writing Ahead of CallSANTA CLARA, Calif., Nov. 02, 2022 (GLOBE NEWSWIRE) — NVIDIA will host a conference call on Wednesday, November 16, at 2 p.m. PT (5 p.m. …
Today, NVIDIA announced the long-term support (LTS) release of NVIDIA DOCA 1.5. NVIDIA DOCA is the open cloud SDK and acceleration framework for NVIDIA…
Today, NVIDIA announced the long-term support (LTS) release of NVIDIA DOCA 1.5.
NVIDIA DOCA is the open cloud SDK and acceleration framework for NVIDIA BlueField DPUs. It unlocks data center innovation by enabling you to rapidly create applications and services for BlueField DPUs by using industry-standard APIs.
The new NVIDIA DOCA 1.5 release includes several important platform updates, making this an LTS release due to the stability and robustness of the code base. In addition, NVIDIA DOCA now supports NVIDIA ConnectX SmartNICs to simplify the transition from NIC/SmartNIC to the NVIDIA BlueField DPU.
New NVIDIA DOCA 1.5 features focus on adding advanced programmability, security, and functionality to support new storage use cases.
Highlights of the NVIDIA DOCA 1.5 release include the following:
The NVIDIA commitment to forward and backward compatibility ensures that applications developed with NVIDIA DOCA will run seamlessly on future versions of the BlueField DPU. They can take advantage of future hardware upgrades for higher performance and increased scale.
For more information, see the full release notes.
The adoption of NVIDIA DOCA has been driven by the delivery of substantial performance gains and through the option of a dual development approach through either NVIDIA DOCA drivers or libraries.
This release adds live migration support for VirtIO-blk and support for transitional virtio-blk emulation devices, with the ability to support a mix of both Virtio0.95 and Virtio1.0 devices simultaneously.
Other additions include AppShield for ransomware inspection and a regular expression (regex) library with reference applications to enable the security matching of repeated code and text patterns.
A TPM firmware trusted application is designed to support the deployment of sensitive applications on the Arm TrustZone. This adds an additional level of security that enables the use of hardware keys to authenticate and encrypt data on the DPU Arm cores.
NVIDIA DOCA now exposes collected telemetry data for logging and metrics. BlueField can be used to sample data on demand and log metrics for later querying by implementing Prometheus, a free software application for event monitoring.
Precision timing is in the heart of the data center. NVIDIA DOCA FireFly is a timing service for the data center that supports all timing needs in one place. With Nanosecond-level clock synchronization, we can enable a new spectrum of timing and delay-critical applications.
Improving the accuracy of data center timing represents an order of magnitude improvement as accuracy changes from milliseconds to nanoseconds with FireFly.
With a synchronized data center, you can accelerate globally synchronized data centers, AI, high-performance computing, professional media production, telco virtual network functions, and precise monitoring. All the servers in the data center can be harmonized to provide something that is bigger than a compute node.
Storage data compression/decompression is a CPU-intensive operation. The NVIDIA DOCA Compression library implements storage data compression and decompression onto the BlueField DPU. This offloads storage operations from the CPU to free up cycles for other compute functions and lowering server TCO.
With added support for NVIDIA ConnectX SmartNICs, the open NVIDIA DOCA software unlocks the benefits of the most comprehensive software APIs, libraries, and services. Developers and IT leaders can foster data center innovation on the most widely deployed, high-performance SmartNICs.
This introduces a broad range of networking, storage, and security capabilities and enhancements to deliver breakthrough performance for software partners, server and storage vendors, end users, and global system integrators. NVIDIA DOCA support for ConnectX SmartNICs helps to speed and simplify the transition from ConnectX SmartNICs to the BlueField DPU.
NVIDIA DOCA is built on open APIs such as DPDK for networking, OFED for RDMA, and SPDK for storage. It’s fully compatible with all major OS and hypervisors. Applications written with NVIDIA DOCA run on BlueField-2 and future versions of the BlueField DPU.
The recent hackathon in China focused on BlueField DPU innovations that use the NVIDIA DOCA software framework to streamline the development process. Participants continue to find new ways to use the DPU for offloading, accelerating, and isolating a broad range of services. There were 13 teams competing over 24 hours, with four winners announced:
Congratulations to the winners and thanks to all the teams that participated in making this NVIDIA DOCA Hackathon such a success!
For more information about how leading companies are using NVIDIA software-defined, hardware-accelerated data center solutions to change the world, see the Modernize Your Data Center with Accelerated Networking free ebook.
More resources:
A SmartNIC is a programmable accelerator that makes data center networking, security and storage efficient and flexible.
A SmartNIC is a programmable accelerator that makes data center networking, security and storage efficient and flexible.
Learn how to upgrade your Jetson devices with the latest CUDA version at this webinar on November 9.
Learn how to upgrade your Jetson devices with the latest CUDA version at this webinar on November 9.
Spearheading research in very high-speed silicon nanophotonics/plasmonics, the European plaCMOS project has reached a successful conclusion. The 51-month…
Spearheading research in very high-speed silicon nanophotonics/plasmonics, the European plaCMOS project has reached a successful conclusion. The 51-month project explored ferroelectric materials to improve performance and reliability. The team achieved world-leading advancements related to key components used in optical links: modulators, photodiodes, and optical switches.
Modulators using barium-titanate integrated on silicon were demonstrated, and monolithically integrated modulators with BiCMOS drivers were tested up to 187 GBaud. Germanium photodiode designs were generated achieving 3 dB bandwidths up to 265 GHz. Ferroelectric nonvolatile optical BTO switches were demonstrated with 100 states in a closed loop control scheme.
These groundbreaking results have been published in the article, A Ferroelectric Multilevel Nonvolatile Photonic Phase Shifter in the journal Nature Photonics. High-profile articles have also appeared in Nature Electronics, Nature Materials, and IEEE/OSA journals.
The consortium brought together eight partners from industry and academia, all renowned experts in their fields: NVIDIA Mellanox, MICRAM Microelectronic GmbH (now Keysight Technologies), ETH Zurich, IHP Leibniz-institut für innovative mikroelektronik, Aristotle University of Thessaloniki, IBM Research GmbH, Universität des Saarlandes, and Lumiphase AG.
Funding was provided by the European Commission’s Horizon 2020 program for research and innovation and the project was coordinated by Elad Mentovich, Head of the Advanced Development Group at NVIDIA Mellanox.
The innovative technologies developed in plaCMOS provide the foundation for the evolution of optical interconnects in data center networks for the second half of the decade. The team has furthered numerous research fields, including materials engineering and nanofabrication, plasmonic-photonic devices, high-speed analog electronics, and transceiver design.
Research on the leading-edge technologies established in plaCMOS continues in the spin-off projects, NEBULA and plasmoniAC. These new projects aim to extend the plaCMOS material platform and investigate new applications of the technology in co-packaged optics, inter-data center coherent links, and optical neuromorphic computing.
For more information, see the articles listed below.
NVIDIA announces new SDKs available in the NGC catalog, a hub of GPU-optimized deep learning, machine learning, and HPC applications. With highly performant…
NVIDIA announces new SDKs available in the NGC catalog, a hub of GPU-optimized deep learning, machine learning, and HPC applications. With highly performant software containers, pretrained models, industry-specific SDKs, and Jupyter notebooks available, AI developers and data scientists can simplify and reduce complexities in their end-to-end workflows.
This post provides an overview of new and updated services in the NGC catalog, along with the latest advanced SDKs to help you streamline workflows and build solutions faster.
Recent advances in large language models (LLMs) have fueled state-of-the-art performance for NLP applications, such as virtual scribes in healthcare, interactive virtual assistants, and many more.
NVIDIA NeMo Megatron, an end-to-end framework for training and deploying LLMs with up to trillions of parameters, is now available in open beta from the NGC catalog. It consists of an end-to-end workflow for automated distributed data processing; training large-scale customized GPT-3, T5, and multilingual T5 (mT5) models; and deploying models for inference at scale.
NeMo Megatron can be deployed on several cloud platforms, including Microsoft Azure, Amazon Web Services, and Oracle Cloud Infrastructure. It can also be accessed through NVIDIA DGX SuperPODs and NVIDIA DGX Foundry.
Request NeMo Megatron in open beta.
The NVIDIA NeMo LLM service provides the fastest path to customize foundation LLMs and deploy them at scale, using the NVIDIA-managed cloud API or through private and public clouds.
NVIDIA and community-built foundation models can be customized using prompt learning capabilities, which are compute-efficient techniques that embed context in user queries to enable greater accuracy in specific use cases. These techniques require just a few hundred samples to achieve high accuracy in building applications. These applications can range from text summarization and paraphrasing to story generation.
This service also provides access to the Megatron 530B model, one of the world’s largest LLMs with 530 billion parameters. Additional model checkpoints include 3B T5 and NVIDIA-trained 5B and 20B GPT-3.
Apply now for NeMo LLM early access.
The NVIDIA BioNeMo service is a unified cloud environment for end-to-end, AI-based drug discovery workflows, without the need for IT infrastructure.
Today, the BioNeMo service includes two protein models, with models for DNA, RNA, generative chemistry, and other biology and chemistry models coming soon.
ESM-1 is a protein LLM, which was trained on 52 million protein sequences, and can be used to help drug discovery researchers understand protein properties, such as cellular location or solubility, and secondary structures, such as alpha helix or beta sheet.
The second protein model in the BioNeMo service is OpenFold, a PyTorch-based NVIDIA-optimized reproduction of AlphaFold2 that quickly predicts the 3D structure of a protein from its primary amino acid sequence.
With the BioNeMo service, chemists, biologists, and AI drug discovery researchers can generate novel therapeutics and understand the properties and function of proteins and DNA. Ultimately, they can combine many AI models in a connected, large-scale, in silico AI workflow that requires supercomputing scale over multiple GPUs.
BioNeMo will enable end-to-end modular drug discovery to accelerate research and better understand proteins, DNA, and chemicals.
Apply now for BioNeMo early access.
A digital twin is a virtual representation—a true-to-reality simulation of physics and materials—of a real-world physical asset or system, which is continuously updated. Digital twins aren’t just for inanimate objects and people. They can replicate a fulfillment center process to test out human-robot interactions before activating certain robot functions in live environments and the applications are as wide as the imagination.
NVIDIA Omniverse Replicator is a highly extensible framework built on the NVIDIA Omniverse platform that enables physically accurate 3D synthetic data generation to accelerate the training and accuracy of perception networks.
Technical artists, software developers, and ML engineers can now easily build custom, physically accurate, synthetic data generation pipelines in the cloud or on-premises with the Omniverse Replicator container available from the NGC catalog.
Download the Omniverse Replicator container for self-service cloud deployment.
NVIDIA Modulus is a neural network AI framework that enables you to create customizable training pipelines for digital twins, climate models, and physics-based modeling and simulation.
Modulus is integrated with NVIDIA Omniverse so that you can visualize the outputs of Modulus-trained models. This interface enables interactive exploration of design variables and parameters for inferring new system behavior and visualizing it in near real time.
The latest release (v22.09), includes key enhancements to increase composition flexibility for neural operator architectures, features to improve training convergence and performance, and most importantly, significant improvements to the user experience and documentation.
Download the latest version of Modulus.
The most popular deep learning frameworks for training and inference are updated monthly. Pull the latest version (v22.09):
We are constantly adding state-of-the-art models for a variety of speech and vision models. The following pretrained models are new on NGC:
Explore more pretrained models for common AI tasks on the NGC Models page.
One of the key contributors in originating flash floods is the blockage of cross-drainage hydraulic structures, such as culverts, by unwanted, flood-borne…
One of the key contributors in originating flash floods is the blockage of cross-drainage hydraulic structures, such as culverts, by unwanted, flood-borne debris being transported.
The accumulation and interaction of debris with culverts often result in reduced hydraulic capacity, diversion of upstream flows, and structural failure. For example, the Newcastle, Australia floods in 2007, Wollongong, Australia floods in 1998 and Pentre, United Kingdom floods in 2021, are just a few instances where blockages were reported as a primary reason for cross-drainage hydraulic structure failure.
In this post, we describe our technique for building a diverse visual dataset for computer vision model training, including examples of synthetic images. We break down each component of our solution and provide insights on future research directions.
Non-linear debris accumulation, the unavailability of real-time data, and complex hydrodynamics suggested the invalidity of a conventional numerical modeling-based approach to address the problem. In this context, post-flood visual information was used to develop the blockage policies involving several assumptions, which many argue are not a true representative of blockage.
This suggests the need for better understanding and exploring the blockage issue from a technology perspective to aid flood management officials and policymakers.
To help address the blockage problem, StopBlock was initiated as a part of SMART Stormwater Management. Overall, this project involved collaboration between city councils in the Illawarra (Wollongong, Shellharbour, and Kiama) and Shoalhaven regions, Lendlease, and the University of Wollongong’s SMART Infrastructure Facility.
StopBlock aims to assess and monitor the visual blockage at culverts in real time using the latest technologies:
In addition, we build and deployed an artificial intelligence of things (AIoT) solution using NVIDIA edge computing, the latest computer vision detection and classification models, a CCTV camera, and a 4G module. The solution detected the visual blockage status (blocked, partially blocked, or clear) at three culvert sites within the Illawarra region.
Training computer vision CNN models requires numerous images related to the intended task. The problem of culvert blockage detection has not been addressed from this perspective before. No database of image data and datasets exists for this purpose.
We developed a new training database consisting of diverse image data related to culvert blockage. These images showed varying culvert types, debris types, camera angles, scaling, and lighting conditions.
Limited data from real culvert blockage was available through the city council records. We adopted the idea of using the combination of real, lab-simulated, and synthetic visual data.
We collected real images of culverts (blocked and clear) from multiple sources:
The collected images represent great diversity in terms of culvert types, debris types, illumination conditions, camera viewpoints, scale, resolution, and even backgrounds. The images of culvert openings and blockages (ICOB) dataset consisted of 929 images in total.
We collected simulated images from scaled laboratory experiments to optimize the existing visual dataset, as not enough real images were available.
A thorough hydraulics laboratory investigation was performed where a series of experiments used scaled physical models of culverts. Blockage scenarios used scaled debris (urban and vegetative) under various flooding conditions.
The images represented diversity in terms of culvert types (single circular, double circular, single box, or double box), blockage types (urban, vegetative, or mixed), simulated lighting conditions, camera viewpoints (two cameras), and flooding conditions (inlet discharge levels). However, the dataset was limited in terms of reflections, clear water, identical background, and identical scaling.
In total, we collected 1,630 images from these experiments to establish the VHD dataset.
We generated synthetic images of culverts (SIC) using a three-dimensional computer application based on the Unity gaming engine with the goal of enhancing the datasets for training.
The application is specifically designed to simulate culvert blockage scenarios and can generate virtually countless instances of blocked culverts with any possible blockage situation that you can think of. You can also alter culvert types, water levels, debris types, camera viewpoints, time of the day, and scaling.
The app design enables you to select scene features from dropdown menus and drag debris objects from a library to place anywhere in the scene with any possible orientation. You can write code using parameters to recreate multiple scenarios and batch capture the images with corresponding labels, to aid the training process.
Some highlighted limitations included unrealistic effects and animations and a single natural background. Figure 3 shows samples from the SIC dataset.
We developed an AIoT solution using edge computing hardware, computer vision models, and sensors for the real-time visual blockage monitoring at culverts:
More specifically, in terms of software, a two-stage detection-classification pipeline is adopted (Figure 4).
In the first stage, a computer vision object detection model (YOLOv4) is used to detect the culvert openings. The detected openings are cropped from the original image and are processed for the classification stage. If no culvert opening is detected, an alert is issued to suggest that the culvert might be submerged.
At the second stage, a CNN classification model such as ResNet-50) is used to classify the cropped culvert openings into one of three blockage classes (blocked, partially blocked, or clear). The blockage-related information is then transmitted to a web dashboard for flood management officials to facilitate the decision-making process.
We trained the YOLOv4 and ResNet-50 models used for detection and classification, respectively, using the NVIDIA TAO platform powered by Python, TensorFlow, and Keras. We used a Linux machine equipped with the NVIDIA A100 GPU for training the models using images from the ICOB, VHD, and SIC datasets.
Here’s the four-stage approach adopted for development:
Relating to software performance, the culvert opening detection model achieved the validation mAP of 0.90 while the blockage classification model achieved a validation accuracy of 0.88.
We developed an end-to-end video analytics pipeline on the NVIDIA DeepStream 6 SDK, using the trained computer vision models to make the inference on the NVIDIA TX2-powered edge computer. Using these detection and classification models, the DeepStream pipeline achieved the FPS of 24.8 for NVIDIA TX2 hardware.
We built the smart device for culvert blockage monitoring using a CCTV camera, NVIDIA TX2 edge computer, and 4G dongle (Figure 5). We optimized the developed hardware for power consumption and computational time for real-time utility. Powered by a solar panel, the hardware consumes only 9.1W average power. The AIoT solution is also configured to transmit the blockage metadata every hour to the web dashboard.
The solution is configured to consider the privacy issues and avoid storing any images on board or in the cloud. Instead, it only processes the images and transmits the blockage metadata. Figure 5 shows the installation of the AIoT hardware at one of the remote sites to monitor the culvert visual blockage.
The potential of computer vision can be further explored to establish a better understanding of visual blockage by extracting blockage-related information:
In the context of flood management decision making, knowing the blockage status of a given culvert is not always enough to make a maintenance-related decision. Going one step further and estimating the percentage visual blockage at a given culvert assists flood management officials in prioritizing the culverts with high visual blockage.
A segmentation-classification pipeline to segment the visible openings from image and classifying the segmented masks into one of four percentage visual blockage classes can be one of the potential solutions. Figure 6 shows the conceptual block diagram for the percentage visual blockage estimation.
The type of flood-borne debris interacting and accumulating at the culvert can result in distinct flooding impacts. Usually, vegetative debris is considered less concerning because of its porous nature in comparison to compact, urban debris.
Automatic detection of debris type is another crucial aspect to be explored.
A CNN classification model may be used to facilitate the manual culvert inspections as a simplistic solution while keeping the flood management official in the loop. Given the complexity of the problem and preliminary analysis, it is not possible to only use a CNN classification model to automate the process. However, a partially automated framework can be developed to facilitate the process.
Figure 7 shows the concept of such a framework based on the classification probability of the trained model. If the classification probability for a given image is less than a given threshold, it can be flagged to flood management officials for cross-validation.
We provided an edge-computing solution for the visual blockage detection at the culverts to assist the timely maintenance and to avoid the blockage-related flooding events.
A classification-detection computer vision model is developed and deployed using the NVIDIA edge-computing hardware to retrieve the blockage status of a culvert as “clear,” “blocked,” or “partially blocked.” To facilitate the training of computer vision models for this unique problem domain, we used simulated and artificially generated images related to culvert visual blockage.
There is a tremendous scope of extending the provided solution in multiple ways to achieve further improved and additional visual blockage information. Estimation of percentage visual blockage, detection of flood-borne debris, and developing a partially automated visual blockage classification framework are a few potential enhancements that can be made within the existing solution.