Month: August 2021

Categories
Misc

Basic assistance with matching an output layer’s shape with another output layer…

I am trying to concatenate two layers for a U-Net during the deconvolution process, but my initial input shape has an odd number of layers for depth (its 3D). So after doing a few max pools, the depth becomes even – when I start doing Conv3DTranspose, the depth only ever doubles.

So I arrive at a point where the output shape of my Conv3DTranspose is (Height, Width, 76) and the item im trying to concat it with is (Height, Width, 77)

What can I do to match the shapes? I tried following along with what the documentation says for predicting the new_depth, yet somehow I can’t seem to make it work!

submitted by /u/little_seed
[visit reddit] [comments]

Categories
Misc

Parkinson’s Disease Dataset

Parkinson's Disease Dataset

I’m following this tutorial (https://youtu.be/XN16tmsxVRM), and I’m wondering if someone could help me with a larger dataset similar to this one:

https://preview.redd.it/tnddiusxnig71.png?width=1102&format=png&auto=webp&s=c41ce65846d2cd8b6f6f384a0d89fa35825b2788

submitted by /u/AdPsychological4804
[visit reddit] [comments]

Categories
Offsites

The C4_200M Synthetic Dataset for Grammatical Error Correction

Posted by Felix Stahlberg and Shankar Kumar, Research Scientists, Google Research

Grammatical error correction (GEC) attempts to model grammar and other types of writing errors in order to provide grammar and spelling suggestions, improving the quality of written output in documents, emails, blog posts and even informal chats. Over the past 15 years, there has been a substantial improvement in GEC quality, which can in large part be credited to recasting the problem as a “translation” task. When introduced in Google Docs, for example, this approach resulted in a significant increase in the number of accepted grammar correction suggestions.

One of the biggest challenges for GEC models, however, is data sparsity. Unlike other natural language processing (NLP) tasks, such as speech recognition and machine translation, there is very limited training data available for GEC, even for high-resource languages like English. A common remedy for this is to generate synthetic data using a range of techniques, from heuristic-based random word- or character-level corruptions to model-based approaches. However, such methods tend to be simplistic and do not reflect the true distribution of error types from actual users.

In “Synthetic Data Generation for Grammatical Error Correction with Tagged Corruption Models”, presented at the EACL 16th Workshop on Innovative Use of NLP for Building Educational Applications, we introduce tagged corruption models. Inspired by the popular back-translation data synthesis technique for machine translation, this approach enables the precise control of synthetic data generation, ensuring diverse outputs that are more consistent with the distribution of errors seen in practice. We used tagged corruption models to generate a new 200M sentence dataset, which we have released in order to provide researchers with realistic pre-training data for GEC. By integrating this new dataset into our training pipeline, we were able to significantly improve on GEC baselines.

Tagged Corruption Models
The idea behind applying a conventional corruption model to GEC is to begin with a grammatically correct sentence and then to “corrupt” it by adding errors. A corruption model can be easily trained by switching the source and target sentences in existing GEC datasets, a method that previous studies have shown that can be very effective for generating improved GEC datasets.

A conventional corruption model generates an ungrammatical sentence (red) given a clean input sentence (green).

The tagged corruption model that we propose builds on this idea by taking a clean sentence as input along with an error type tag that describes the kind of error one wishes to reproduce. It then generates an ungrammatical version of the input sentence that contains the given error type. Choosing different error types for different sentences increases the diversity of corruptions compared to a conventional corruption model.

Tagged corruption models generate corruptions (red) for the clean input sentence (green) depending on the error type tag. A determiner error may lead to dropping the “a”, whereas a noun-inflection error may produce the incorrect plural “sheeps”.

To use this model for data generation we first randomly selected 200M clean sentences from the C4 corpus, and assigned an error type tag to each sentence such that their relative frequencies matched the error type tag distribution of the small development set BEA-dev. Since BEA-dev is a carefully curated set that covers a wide range of different English proficiency levels, we expect its tag distribution to be representative for writing errors found in the wild. We then used a tagged corruption model to synthesize the source sentence.

Synthetic data generation with tagged corruption models. The clean C4 sentences (green) are paired with the corrupted sentences (red) in the synthetic GEC training corpus. The corrupted sentences are generated using a tagged corruption model by following the error type frequencies in the development set (bar chart).

Results
In our experiments, tagged corruption models outperformed untagged corruption models on two standard development sets (CoNLL-13 and BEA-dev) by more than three F0.5-points (a standard metric in GEC research that combines precision and recall with more weight on precision), advancing the state-of-the-art on the two widely used academic test sets, CoNLL-14 and BEA-test.

In addition, the use of tagged corruption models not only yields gains on standard GEC test sets, it is also able to adapt GEC systems to the proficiency levels of users. This could be useful, for example, because the error tag distribution for native English writers often differs significantly from the distributions for non-native English speakers. For example, native speakers tend to make more punctuation and spelling mistakes, whereas determiner errors (e.g., missing or superfluous articles, like “a”, “an” or “the”) are more common in text from non-native writers.

Conclusion
Neural sequence models are notoriously data-hungry, but the availability of annotated training data for grammatical error correction is rare. Our new C4_200M corpus is a synthetic dataset containing diverse grammatical errors, which yields state-of-the-art performance when used to pre-train GEC systems. By releasing the dataset we hope to provide GEC researchers with a valuable resource to train strong baseline systems.

Categories
Misc

Three’s Company: NVIDIA Studio 3D Showcase at SIGGRAPH Spotlights NVIDIA Omniverse Update, New NVIDIA RTX A2000 Desktop GPU, August Studio Driver

The future of 3D graphics is on display at the SIGGRAPH 2021 virtual conference, where NVIDIA Studio is leading the way, showcasing exclusive benefits that NVIDIA RTX technologies bring to creators working with 3D workflows. It starts with NVIDIA Omniverse, an immersive and connected shared virtual world where artists create one-of-a-kind digital scenes, perfect 3D Read article >

The post Three’s Company: NVIDIA Studio 3D Showcase at SIGGRAPH Spotlights NVIDIA Omniverse Update, New NVIDIA RTX A2000 Desktop GPU, August Studio Driver appeared first on The Official NVIDIA Blog.

Categories
Misc

GPU Operator 1.8 Adds Support for DGX, HGX, and Upgrades

In the last post, we looked at how the GPU Operator has evolved, adding a rich feature set to handle GPU discovery, support for the new Multi-Instance GPU (MIG) capability of the NVIDIA Ampere Architecture, vGPU, and certification for use with Red Hat OpenShift.  In this post, we look at the new features added in … Continued

In the last post, we looked at how the GPU Operator has evolved, adding a rich feature set to handle GPU discovery, support for the new Multi-Instance GPU (MIG) capability of the NVIDIA Ampere Architecture, vGPU, and certification for use with Red Hat OpenShift. 

In this post, we look at the new features added in the GPU Operator release 1.8, further simplifying GPU management for various deployment scenarios, inlcuding:

  • Added support for GPU Operator upgrades
  • Support for NVSwitch Systems, including DGX A100 and NVIDIA HGX A100 servers
  • Support for gathering GPU Operator State Metrics
  • GPU Operator and Network Operator Improve Multinode Training
  • Support for Red Hat OpenShift 4.8

NVIDIA Software Lifecycle Management 

Version 1.8 of GPU Operator provides an update mechanism for organizations to update their GPU Operator version without disrupting the workflow of the cluster the GPU Operator is running on. Previous releases of GPU Operator required users to uninstall the prior version before installing the new version, meaning no GPUs in the cluster were usable during the upgrade process. 

Starting with 1.8, upgrading versions doesn’t disrupt the workflow. The mechanism updates one node at a time in a rolling fashion, so the other nodes can continue to be used. The next node is updated only when the installation is complete and the previous node is back online. Users can be confident that their workflow will be better managed when updating GPU Operator. 

Now Supporting NVSwitch Systems

With 1.8, the GPU Operator automatically deploys the software required for initializing the fabric on NVIDIA NVSwitch systems, including the NVIDIA HGX A100 and DGX A100. Once initialized, all GPUs can communicate with one another at full NVLink bandwidth to create an end-end scalable computing platform. 

The GPU Operator is also certified for use with Red Hat OpenShift 4 on DGX A100 systems. 

Support for Gathering GPU Operator State Metrics

With 1.8, the GPU Operator now reports various metrics for users to monitor overall health of the GPU Operator and operator deployed resources under the gpu-operator-resources namespace. SRE teams and Cluster administrators can now configure necessary Prometheus resources to gather metrics and also to trigger alerts on certain failure conditions. 

For the OpenShift Container Platform, these resources are automatically created in this release. Monitoring solutions like Grafana can be used to build dashboards and visualize the operational status of GPU Operator and node components.

Better Together: NVIDIA Accelerated Compute and Networking

Recently, NVIDIA announced the 1.0 release of the NVIDIA Network Operator. An analog to the NVIDIA GPU Operator, the Network Operator simplifies scale-out network design for Kubernetes by automating aspects of network deployment and configuration that would otherwise require manual work. It loads the required drivers, libraries, device plug-ins, and CNIs on any cluster node with an NVIDIA network interface.

When they are deployed together, the NVIDIA GPU and Network operators enable GPUDirect RDMA, a fast data path between NVIDIA GPUs on different nodes. This is a critical technology enabler for data-intensive workloads like AI Multi-Node Training.

Learn more about the latest NVIDIA Network Operator release.

Added Support for Red Hat OpenShift

We continue our line of support for Red Hat OpenShift.

  • GPU Operator 1.8 and 1.7 support both Red Hat OpenShift 4.8 and 4.7
  • GPU Operator 1.6 supports Red Hat OpenShift 4.7
  • GPU Operator 1.5 supports Red Hat OpenShift 4.6
  • GPU Operator 1.4 and 1.3 support Red Hat OpenShift 4.5 and 4.4, respectively

Summary

The following resources are available for using NVIDIA GPU Operator: 

The NVIDIA GPU Operator is a key component to many edge computing solutions. Learn more about NVIDIA solutions for edge computing.

Categories
Misc

NVIDIA Opens Omniverse Platform to Developers Worldwide

NVIDIA Developer Program provides access to integrated technologies for simulation and real time rendering with Omniverse.

NVIDIA Developer Program is now bringing NVIDIA Omniverse to over 2.5 million developers around the world. At SIGGRAPH, we’re introducing exclusive events, sessions, and other resources to unveil Omniverse as our newest platform for developers.

NVIDIA is delivering a suite of Omniverse apps and tools to enhance developer pipelines. Developers can plug into any layer of the platform stack — whether at the top level, utilizing prebuilt Omniverse apps; or the platform component level, so they can easily build custom extensions and tools to boost workflows. 

With the NVIDIA Developer Program, Omniverse users have access to a wide range of technical resources, tutorials and more. From online training to interacting with the community, these developer resources are free, providing users with the tools and knowledge to familiarize themselves with the Omniverse platform.

Learn about all the different ways developers can engage with us at SIGGRAPH and dive into the Omniverse platform. 

Get Access to the New NVIDIA Omniverse User Group

Join NVIDIA Omniverse for our inaugural User Group, an exclusive event hosted by Omniverse’s senior leadership on August 12, 5:00 – 6:30 p.m. PDT. The event is open to all developers, researchers, creators, students, hobbyists, and other professionals using the platform, will provide attendees with a chance to:

  • Hear the vision and future of Omniverse from Rev Lebaredian, VP of Omniverse & Simulation Technology, and Richard Kerris, VP of Omniverse Developer Platform.
  • Learn how you can build on and extend the Omniverse ecosystem.
  • See the unveiling of the Create With Marbles: Marvelous Machines contest winners.
  • Attend “Meet the Expert” breakout sessions and speak with engineering leaders about specific platform applications and features.

Register now to join this exclusive event. 

Get Started With the Omniverse Developer Resource Center

The new NVIDIA Omniverse Resource Center has all the information needed to help Omniverse users get started and familiarize themselves with the platform. From talks and sessions to technical documentations, the Resource Center organizes all assets that are available for certain topics, making it easier for users to find overviews of different features and capabilities in Omniverse, including:

  • Learning USD: Get an introduction to USD, from its creation to current implementation in Omniverse. 
  • Using Connectors: Learn how to connect your application to the Omniverse platform and submit your Connector for exposure to Omniverse users.
  • Building apps and extensions: Use Omniverse Kit to build entirely new applications or add new functionality to the Omniverse ecosystem by creating extensions.
  • Building microservices: Enable new capabilities to be exposed as microservices to any connected clients and applications.

For more resources, check out our developer tutorials. 

New Graphics and Simulation Courses by NVIDIA Deep Learning Institute (DLI)

The NVIDIA Deep Learning Institute (DLI) offers resources for diverse learning needs—from learning materials to self-paced and live training taught by NVIDIA-certified instructors, to teaching kits for educators—giving individuals, teams, and institutions what they need to advance their knowledge in AI, graphics, and simulation.

Attendees can learn more about these free trainings and resources, including:

  • Universal Scene Description (USD) for collaborative 3D workflows. This free, self-paced course provides an introduction to USD and important concepts like layer composition, references, and variants and includes hands-on exercises with live scripted examples. 
  • The new “Masterclass by the Masters” video series for artists to see how others are using and building on Omniverse. The first masterclass collection is “Using Omniverse for Artistic 3D Workflows,” which highlights how different creators are using the platform for multi-app design workflows.
  • The Graphics & Omniverse Teaching Kit, which includes downloadable teaching materials and online courses that provide the foundation for understanding and building hands-on expertise in graphics and Omniverse. Educators can request early access.

Learn more about NVIDIA DLI.

Engage With the Community

NVIDIA hosts deep dives with developers on Omniverse applications such as Omniverse Create, Issac Sim Machinima, and more. We routinely invite product managers and community experts to present the latest features and most popular workflows. Doing this live on Twitch gives you a chance to ask questions in an interactive environment. Follow our Twitch channel for the schedule of upcoming streams and look at past events in our Community Stream playlist.

For additional support, check out the Omniverse forums and join our Discord server to chat with the community.

 

Categories
Misc

Introducing Omniverse USD Support for Blender

In the SIGGRAPH Special Address, NVIDIA revealed that the upcoming release of Blender 3.0 includes USD support. 

In the SIGGRAPH Special Address, NVIDIA revealed that the upcoming release of Blender 3.0 includes USD support. 

The USD support in the new release was developed by NVIDIA in close collaboration with Blender Foundation to bring the open standard to Blender artists. In addition, NVIDIA announced a Blender 3.0 alpha USD branch with additional features permitting integration with Omniverse. 

Blender is OSS for 3D design and animation that provides a suite of tools supporting the entire 3D pipeline, from modeling and sculpting to animation, simulation, and rendering. Thousands of programmers from around the world contribute to the OSS. Today, Blender is an important part of the 3D ecosystem across disciplines and industries, downloaded over 14 million times in 2020. 

NVIDIA Omniverse is a real-time simulation and collaboration platform for 3D production pipelines. The Blender 3.0 alpha USD branch includes additional USD import and export options, as well as Omniverse universal material mapper support, allowing lossless materials exchange between Blender and Omniverse. It will be available for download from the Omniverse Launcher, as well as Blender’s website and GitHub on August 16.  

Materials in Omniverse are physically based, represented using Material Definition Language (MDL). With the Blender 3.0 alpha USD branch, users can export and import Omniverse preset MDLs for an enhanced 3D workflow.

All source code is available for the Blender 3.0 alpha USD branch, so organizations, developers, and studios can customize, build upon, and learn from the newest additions.

Download the NVIDIA Omniverse open beta today and get started with Blender 3.0 USD.

For additional support, check out the developer forum and join the Omniverse Discord server to chat with the community.

 

Categories
Misc

NVIDIA Brings Millions More Into the Metaverse With Expanded Omniverse Platform

NVIDIA today announced a major expansion of NVIDIA Omniverse™ — the world’s first simulation and collaboration platform that is delivering the foundation of the metaverse — through new integrations with Blender and Adobe that will open it to millions more users.

Categories
Misc

From Our Kitchen to Yours: NVIDIA Omniverse Changes the Way Industries Collaborate

Talk about a magic trick. One moment, NVIDIA CEO Jensen Huang was holding forth from behind his sturdy kitchen counter. The next, the kitchen and everything in it slid away, leaving Huang alone with the audience and NVIDIA’s DGX Station A100, a glimpse at an alternate digital reality. For most, the metaverse is something seen Read article >

The post From Our Kitchen to Yours: NVIDIA Omniverse Changes the Way Industries Collaborate appeared first on The Official NVIDIA Blog.

Categories
Misc

A Code for the Code: Simulations Obey Laws of Physics with USD

Life in the metaverse is getting more real.  Starting today, developers can create and share realistic simulations in a standard way. Apple, NVIDIA and Pixar Animation Studios have defined a common approach for expressing physically accurate models in Universal Scene Description (USD), the common language of virtual 3D worlds.  Pixar released USD and described it in 2016 at SIGGRAPH. It was originally designed so artists could Read article >

The post A Code for the Code: Simulations Obey Laws of Physics with USD appeared first on The Official NVIDIA Blog.