DataBloomHow long should it take to train VGG16 on cifar10? I tried using Apple M1 with Tensorflow-Metal plugging, but it crashes. Then tried in google colab, but it also crashes with error “Your session crashed after using all available RAM.”. Is this normal?? I’ve heard this was supposed to take minutes to train
submitted by /u/berimbolo21
[visit reddit] [comments]
SANTA CLARA, Calif., May 19, 2022 — NVIDIA today announced it will hold its 2022 Annual Meeting of Stockholders online on Thursday, June 2, at 11 a.m. PT. The meeting will take place virtually…
SANTA CLARA, Calif., May 19, 2022 — NVIDIA will present at the following event for the financial community:
BofA 2022 Securities Global Technology Conference
Tuesday, June 7, at 8 a.m. Pacific…
I am training a VGG16 net with cifar10 dataset on my M1 mac, and I get this error shortly after training starts. I have tensorflow metal, and training works for smaller datasets like MNIST, so what is going on here?
Error: command buffer exited with error status.
The Metal Performance Shaders operations encoded on it may not have completed.
Error:
(null)
Ignored (for causing prior/excessive GPU errors) (00000004:kIOGPUCommandBufferCallbackErrorSubmissionsIgnored)
<AGXG13GFamilyCommandBuffer: 0x1631088b0>
label = <none>
device = <AGXG13GDevice: 0x11f98be00>
name = Apple M1
commandQueue = <AGXG13GFamilyCommandQueue: 0x1613a4800>
label = <none>
device = <AGXG13GDevice: 0x11f98be00>
name = Apple M1
retainedReferences = 1
submitted by /u/berimbolo21
[visit reddit] [comments]
Hello!
I am currently working on a speech enhancement software. I created a model with a keras functional layer and put it in a RNN wrapper. Currently, it works on full soundfiles and everything is fine.
The states that i need the model/RNN layer to save are complex vectors:
[tf.zeros(shape=(batch_size, self.K), dtype=tf.complex64),
tf.zeros(shape=(batch_size, self.K), dtype=tf.complex64),
tf.zeros(shape=(batch_size, self.K), dtype=tf.complex64),
tf.ones(shape=(batch_size, self.K), dtype=tf.complex64)]
However, since I want to make this a real-time application I need the model to save some states that are important. Thats why I tried to make the RNN layer of my model stateful. But now I get some weir Error:
Traceback (most recent call last):
File “my python script”, line 104, in <module>
model_tmp.build_model()
File “my python script”, line 107, in build_model
pred_out = layer(inp)
File “…libsite-packageskeraslayersrecurrent.py“, line 679, in __call__
return super(RNN, self).__call__(inputs, **kwargs)
File “…libsite-packageskerasutilstraceback_utils.py”, line 67, in error_handler
raise e.with_traceback(filtered_tb) from None
File “…AppDataRoamingPythonPython39site-packagestensorflowpythonframeworkops.py“, line 1662, in convert_to_tensor
raise ValueError(
ValueError: Tensor conversion requested dtype float32 for Tensor with dtype complex64: <tf.Tensor: shape=(1, 1024), dtype=complex64, numpy=
array([[0.+0.j, 0.+0.j, 0.+0.j, …, 0.+0.j, 0.+0.j, 0.+0.j]],
dtype=complex64)>
So I debugged and “convert_to_tensor” is called with following:
value: tf.Tensor([[0.+0.j 0.+0.j 0.+0.j … 0.+0.j 0.+0.j 0.+0.j]], shape=(1, 1024), dtype=complex64)
dtype: <dtype: ‘float32’>
name: ‘initial_value’
I just dont know why it works when the RNN layer is not stateful and suddenly not. I hope you can somehow help me.
submitted by /u/QuantumProst
[visit reddit] [comments]
On June 2, learn how to leverage Replicator Composer—a tool in NVIDIA Isaac Sim—to generate synthetic data, and train and test computer vision models.
Hello
I’m pretty new to Tensorflow world and have implemented some models using existing layers like convolution, dense and LSTM. Now I wanted to implement Elman Neural Network (wiki page) in tensorflow 2, and because it seems there isn’t any existing implementation in Tensorflow, so I decided to implement myself. Although there are couple of implementations in the web for Elman neural network but non of them have implemented via subclassing Tensorflow’s Model class. I thought it would be lot easier to implement via subclassing Model class. Following is my implementation. The problem is when I call .fit on a train dataset batch_size variable in call method is None, and I couldn’t determine how to solve this problem. Any Ideas?
class ElmanNeuralNetwork(tf.keras.Model): def __init__(self, input_dim, hidden_units, output_dim, n_classes): super(ElmanNeuralNetwork, self).__init__() self.hidden_units = hidden_units self.U_h = self.add_weight(shape=(self.hidden_units, self.hidden_units), initializer='random_normal', trainable=True) self.W_h = self.add_weight(shape=(input_dim, self.hidden_units), initializer='random_normal', trainable=True) self.b_h = self.add_weight(shape=(self.hidden_units,), initializer='random_normal', trainable=True) self.W_y = self.add_weight(shape=(self.hidden_units, output_dim), initializer='random_normal', trainable=True) self.b_y = self.add_weight(shape=(output_dim,), initializer='random_normal', trainable=True) self.softmax_layer = tf.keras.layers.Dense(n_classes, activation='softmax') def call(self, x): batch_size, n_step, n_feature = x.shape h = tf.zeros((batch_size,self.hidden_units)) for _ in range(n_step): h = tf.keras.activations.tanh(tf.matmul(self.W_h, x) + tf.matmul(self.U_h, h) + self.b_h) y = tf.keras.activations.tanh(tf.matmul(self.W_y, h) + self.b_y) preds = self.softmax_layer(y) return preds
submitted by /u/engmrgh
[visit reddit] [comments]
Fortnite on GeForce NOW with touch controls on mobile is now available to all members, streaming through the Safari web browser on iOS and the GeForce NOW Android app. The full launch — including the removal of the waitlist — follows a successful beta period in which more than 500,000 participants streamed over 4 million Read article >
The post ‘Fortnite’ Arrives This GFN Thursday With GeForce Performance You Can Touch appeared first on NVIDIA Blog.
Edge AI and high-performance computing are modernizing the electric grid, from power generation to transmission and distribution to the grid-edge.
Utilities are challenged to integrate distributed clean energy resources—such as wind farms, rooftop solar, home batteries, and electric vehicles—onto legacy electric grid infrastructure. Existing systems were built to manage a one-way flow of power from a small number of industrial-scale generation plants, often run using coal, natural gas, or nuclear.
Energy output from these sites was predictable and controllable to align power generation with electricity demand. Scheduling outages for maintenance was a routine task that limited service disruptions for customers counting on a reliable grid to keep the lights on.
However, the global shift to accelerate the energy transition and achieve net-zero emissions has exposed the flaws of existing grid infrastructure. Output is difficult to predict and control because there are numerous energy resources that each generate small amounts of power that can either be consumed or redirected back to the grid. These resources are distributed in different locations and generate varying amounts of power, making them difficult to manage.
To manage distributed energy resources, a new approach to power grid management is needed that’s based on edge AI and high-performance computing (HPC). This combination creates an accelerated computing platform that enhances grid resiliency through demand forecasting, optimal power flow, grid simulation, and predictive maintenance of existing infrastructure to transform legacy grids into software-defined smart grids.
The first step to digitizing grids starts with power generation. Globally, industrial plants generate 24,000 terawatt-hours (TWh) of electricity per year, with nearly 38% produced by nuclear, hydro, wind, solar, and renewables.
Heavy machinery—such as generators, turbines, pumps, and reactors—requires routine maintenance, but unplanned downtime can prevent effective service delivery, impact revenue, and lead to expensive repairs. Manually intensive inspections reduce productivity, create unnecessary risks to worker health and safety, and could damage industrial equipment.
To minimize maintenance disruptions, energy companies are building industrial digital twins of real-world power generation sites. Siemens Energy, a global energy technology provider, created a virtual replica of their heat recovery steam generator (HRSG) using NVIDIA Modulus (a physics-machine learning framework) and NVIDIA Omniverse (a 3D design collaboration and physically accurate simulation platform). HRSGs use heat exhaust from gas turbines to create steam used to drive steam turbines, which improves plant efficiency and output while reducing operational costs.
The company developed a new workflow that feeds temperature, pressure, and velocity data to Modulus and simulates steam and water flow in real-time using Omniverse. By accurately predicting corrosion with a digital twin, Siemens Energy estimates that utilities could save $1.7 billion annually by automating equipment inspections and cutting downtime by 10% per year.
Power is also generated in offshore platforms, such as wind farms, where higher wind speeds drive turbines and generate more electricity per capacity installed than onshore sites. Traditional methods to simulate turbulent wind flow, wake effects, and other scenarios are computationally intensive and costly.
Siemens Gamesa, a leader in renewable energy, built a digital twin to model their offshore wind farms, which generate over 100 gigawatts of energy each year. Using the NVIDIA digital twin platform for scientific computing, the company was able to simulate wake effects up to 4,000x faster with high accuracy and fidelity. Faster computational fluid dynamics simulations shorten the time to design an ideal wind farm, which leads to increased electricity output and lower operating costs.
Energy companies are exploring processes to sustainably power industrial plants, replacing fossil fuels with “green hydrogen”. A new industry consortium is developing Gigastack, the world’s first industrial-scale green hydrogen project, to accelerate the country’s net-zero emissions goals and create a framework for successful operations around the world.
The project, now in the second of three phases, aims to use renewable electricity produced by offshore wind to split water into oxygen and renewable hydrogen, to fuel the Phillips 66 Humber Refinery in the United Kingdom. Worley, the project’s lead engineering firm, leveraged Aspen Technology’s Optiplant software to remotely collaborate on a physically accurate 3D conceptual layout of the Humber Refinery with designers and engineers around the world. The digital twin layers multiple dimensions to automate plant processes, such as scheduling, cost estimation, emissions, maintenance, and worker safety.
After it’s generated, electricity must move through transmission and distribution lines to reach end customers, whether they’re residential, commercial, or industrial. A significant portion of the energy is lost in this process, which reduces the available energy supply. In addition, extreme weather events, such as snowstorms, wildfires, and floods, threaten the reliability of the grid.
In the event of widespread power outages due to large storms or other causes, utilities route trucks to service downed lines in the field to restore power quickly and efficiently. However, routing vehicles is an exponentially complex problem.
NVIDIA cuOpt helps utilities dynamically reoptimize truck rolls as new challenges arise or power grid assets go offline. The route optimization software suite uses GPU-accelerated logistics solvers relying on heuristics and optimizations to deliver dynamic rerouting simulations and subsecond solver-response time.
Accelerated computing enables utilities to optimize power flow, predict grid anomalies, prevent unplanned blackouts, and automate maintenance. Electric Power Research Institute (EPRI), an independent, nonprofit energy research and development organization, is building a grid simulator with NVIDIA to schedule outages of power systems and minimize downtime.
Using AI and HPC, utilities can model the electric grid as a connected graph with transformers and substations as nodes and transmission lines as edges. These models are trained on historical data to simulate specific grid outages, and address challenges to accommodate variable renewable energy, distributed energy resources, and shifting flow patterns.
This method of contingency planning extends beyond N-1 (single asset failure) to multipoint failures that enhance resiliency at the regional–, micro–, and nano-grid levels. The tool simulates AC power flow (ACPF) as steady-state branch currents and node voltages of a transmission grid circuit, given input profiles for loads and generators.
As clean energy resources expand, the grid simulator can guide operators to redispatch or reconfigure network topology by solving millions of simultaneous ACPF problems in seconds.
With utilities’ limited visibility into their infrastructure, vegetation near grids poses a wildfire threat to neighboring communities and wildlife habitats. Managing vegetation requires real-time data analysis to proactively monitor transmission lines and transformers.
HEAVY.AI, an advanced data analytics company, analyzes terabytes of geospatial data from satellite, LIDAR, weather, and vegetation health. A strike tree model enables risk-based vegetation management, while a web-based dashboard provides insights for predictive maintenance and wildfire mitigation.
At electric substations, safety and security are critical to automation and maintenance of reliable grid operations. Startups such as IronYun are using computer vision to monitor security access or equipment health. Using existing IP cameras and intelligent video analytics, the company’s platform can help ensure protocols for worker health and safety are followed and prevent onsite breaches.
At the edge of the grid, energy reaches its final destination in homes, office buildings, factories, stores, and more.
Utilidata is working with NVIDIA to develop a smart grid chip that can be embedded in smart meters. The chip can enhance grid resiliency, integrate distributed energy resources (DERs) among solar, storage, and EVs, and accelerate the transition to a decarbonized grid. The chip is designed to be a platform for innovation, with a set of core grid operations services developed by Utilidata that will enable applications developed by third parties.
Figure 3 shows the platform tools and Utilidata’s core services for advanced metering infrastructure, which come embedded on the smart grid chip, and examples of applications that could be developed.
Utilities can also automate and simplify the inspection of distribution poles, power lines, and other pole-mounted equipment. Using computer vision and smart camera systems, Noteworthy AI can detect cracks, corrosion, breakages, high-risk vegetation, and other potential issues through inference at the grid edge. The camera system, built on the NVIDIA Jetson edge AI platform, is mounted to utility trucks and collects 60-megabyte images of grid infrastructure during field servicing. Running multiple AI models at the edge on Jetson, with only a subset of images sent to the cloud for analysis, reduces the cost of compute for Noteworthy AI.
Smart homes are also automating tasks using smart devices connected to the Internet of Things (IoT). Anuranet launched the Bullfrog Energy Ecosystem, which includes an NVIDIA Jetson-powered smart meter, smart breaker, and home control hub. The platform uses computer vision and conversational AI to drive personalized interactions with homeowners.
Anuranet’s platform automates controls for neighborhood-level microgrids, bringing together individual smart homes into energy-efficient communities.
Utilities are at the forefront of the clean energy transition, but must modernize transmission and distribution grid infrastructure to manage renewable energy resources at scale. Accelerated computing can power this digital transformation and create software-defined smart grids that deliver reliable energy and accelerate sustainability initiatives.
To learn more, read the Top AI Use Cases for Utilities industry brief.
You can also explore NVIDIA solutions transforming the power and utilities industry.
Sign up for Edge AI News to stay up to date with the latest trends, customers use cases, and technical walkthroughs.
In general, the information on internet about tensorflow object detection api tells we only need to divide the dataset into train and test. But I find other information about machine learning that also talks about a validation dataset. Is the validation dataset important in object detection and in this framework?
Thank you
submitted by /u/Emergency_Egg_9497
[visit reddit] [comments]