Categories
Misc

How to have better results at forecasting?

I am a beginner in machine learning and I would like to forecast some pollution data.

I am using a dataset with values for pm2.5, pm10 and pm1 as features and I am predicting the values for the pm2.5. I built an LSTM network but the predicted values are quite from the real values.

What I used:

win_length=2 batch_size=32 num_features=3 train_generator=TimeseriesGenerator(x_train,y_train,length=win_length,sampling_rate=1,batch_size=batch_size) test_generator=TimeseriesGenerator(x_test,y_test,length=win_length,sampling_rate=1,batch_size=batch_size) 

`

The used model is LSTM:

model=tf.keras.Sequential() model.add(tf.keras.layers.LSTM(200,input_shape=(win_length,num_features),return_sequences=True)) model.add(tf.keras.layers.LeakyReLU(alpha=0.5)) model.add(tf.keras.layers.LSTM(128,return_sequences=True)) model.add(tf.keras.layers.LeakyReLU(alpha=0.5)) model.add(tf.keras.layers.Dropout(0.3)) model.add(tf.keras.layers.LSTM(64,return_sequences=False)) model.add(tf.keras.layers.Dropout(0.3)) model.add(tf.keras.layers.Dense(1)) 

This is the data snippet and how the predicted values look compared to the original pm2.5 values: prediction snippet

How to increase the accuracy of the forecast? I am also attaching the jupyter notebook, which contains all the analysis: https://github.com/creativitylab/dataset/blob/main/pollution%20data.ipynb

submitted by /u/MobileInformal460
[visit reddit] [comments]

Categories
Misc

Expanding the NVIDIA DOCA Community for Developers in China

The developer community for NVIDIA DOCA continues to gain momentum around the world.

On January 13, NVIDIA hosted an online workshop to engage with the NVIDIA DOCA developer community in China. The core team at NVIDIA and leading partner representatives joined the workshop to discuss the application scenarios of NVIDIA BlueField DPUs and the NVIDIA DOCA software framework for cloud, data center, and edge. The workshop focused on the requirements for DOCA developers in key industries, such as consumer Internet, cybersecurity, and higher education, and designed a plan to expand the DOCA developer community in China.

Since the June 2021 launch of the DOCA community in China, nearly 1,000 developers have registered for the DOCA Early Access Program, accounting for almost half of global registrations. BlueField DPUs and DOCA show great potential for adoption in China, with developer numbers continuing to grow.

In the second half of 2021, NVIDIA has held three online bootcamps to introduce the application and future technology evolution of the BlueField DPU and DOCA in modern data centers. They also explored the software stack, development environment, developer resources, developer guides, and reference applications of the DOCA Software Development Kit (SDK). More than 3,500 developers participated in the bootcamps. At the latest bootcamp, the newest services and applications released with DOCA 1.2 attracted much attention. Technical information and success stories posted on social media and in knowledge communities have many developers and industry professionals excited about future advancements.

NVIDIA is working with leading global platform providers and partners, such as Juniper Networks, Excelero, VMware, and Palo Alto Networks, to integrate and extend solutions based on BlueField DPU and the DOCA software framework. Through the workshop, NVIDIA will help enable developers in China to develop applications in scenarios such as Zero Trust Security, Morpheus AI Security, edge network service platforms, and high-speed distributed storage. A rich developer program in China will be launched in 2022.

DOCA Developer Bootcamp and Virtual Hackathon

Following the hackathons in Europe and North America, NVIDIA intends to host the first spring DOCA Developer Hackathon in China in the second quarter of 2022. Before the hackathon, NVIDIA will host an online bootcamp to teach contestants about BlueField DPU and DOCA programming skills.

NVIDIA will invite teams of developers from partners, customers, and academia to learn, collaborate, and accelerate their software designs under the guidance of NVIDIA expert mentors. The aim is to foster innovative, breakthrough software projects based on BlueField DPU and DOCA 1.2 in high-performance networking, virtualization, cybersecurity, distributed storage, accelerated AI, edge computing, and video streaming processing. The program will empower the developer community in China to create revolutionary data center infrastructure applications and services. After evaluation, NVIDIA will reward outstanding innovation teams.

DPU and DOCA Excellence Center

Leadtek (Shanghai) Information Technology Co., Ltd. and Shanghai Zentek Intelligent Technology Co., Ltd. are highly familiar with NVIDIA products and solutions, including deep learning applications in cloud, data center, and edge scenarios. They have partnered with the NVIDIA Deep Learning Institute to train partners and customers. 

As the first group of members of the NVIDIA authorized DPU and DOCA Excellence Center, the two partners have set up their own Excellence Centers and begun their pilots. During the pilots, each partner will independently build and operate a virtual development platform based on the BlueField-2 DPU, establish a third-party DPU development environment, provide an online practice development environment for DOCA developers in China, and contribute to the DPU and DOCA ecosystem with NVIDIA.

The implementation of the DOCA developer program in China will help grow the global DOCA developer community, facilitate talent development, and enhance the capabilities of developers. It will also boost the performance advantages of solutions based on BlueField DPU and the DOCA SDK, and accelerate time-to-market, creating greater value for customers and partners.

Apply now to join the DOCA developer community and get early access to the DOCA software framework.

Categories
Misc

From Imagination to Animation, How an Omniverse Creator Makes Films Virtually

Growing up in the Philippines, award-winning filmmaker Jae Solina says he turned to movies for a reminder that the world was much larger than himself and his homeland.

The post From Imagination to Animation, How an Omniverse Creator Makes Films Virtually appeared first on The Official NVIDIA Blog.

Categories
Misc

Single or multiple output for model

I want to predict the genre(s) of the given text. The dataset I am planning on using is this kaggle dataset. While I know how to predict a single genre, I am not sure how to work with a possibility of more than 1 genre if needed.

submitted by /u/Electric_Dragon1703
[visit reddit] [comments]

Categories
Misc

Global AI Weather Forecaster Makes Predictions in Seconds

Graphic of a hurricane approaching the southwest US from a global view.Using convolutional neural networks researchers create an algorithm that can quickly calculate global forecasts 4 to 6 weeks into the future.Graphic of a hurricane approaching the southwest US from a global view.

New weather-forecasting research using AI is fast-tracking global weather predictions. The study, recently published in the Journal of Advances in Modeling Earth Systems, could help identify potential extreme weather 2–6 weeks into the future. Accurate predictions of extreme weather with a longer lead time give communities and critical sectors such as public health, water management, energy, and agriculture more time to prepare for and mitigate potential disasters.

Climate change is amplifying the intensity and frequency of extreme weather events, with 2021 shattering storm, heatwave, flood, and drought records across the globe. According to a recent NOAA report, last year the US experienced 20 separate climate-induced weather disasters, each totaling over $1 billion in damage. 

Short-term and seasonal weather forecasting can play a large role in decreasing the socioeconomic and human costs of extreme weather. In 2019, meteorologists warned local and national leaders in the Philippines of a torrential rainstorm looming about 3 weeks out. The forecast gave communities time to weatherize structures and evacuate before the Category 4 Typhoon hit, saving lives, and reducing overall damage to the region. 

Current weather forecasting relies on supercomputers processing large amounts of global data such as temperature, pressure, humidity, and wind speed. These systems require massive computational resources and take time to process. 

Also, according to the authors, the ability to accurately predict forecasts further out, from several weeks to months, decreases significantly. 

Looking to improve current weather forecasting the researchers aimed to create a computationally efficient model, capable of accurately predicting upcoming weather called the Deep Learning Weather Prediction (DLWP). Originally introduced in a paper published in 2020, the DLWP relies on an AI algorithm that learns and recognizes patterns in historical weather data based on global grids.

The current work refines the DLWP by training a deep convolutional neural network on two additional data points—temperature at the atmospheric boundary layer and total column water vapor. They also improved the grid resolution at the equator to approximately 1.4°. 

Running on a single cuDNN-accelerated TensorFlow deep learning framework on an NVIDIA V100 GPU, the model runs 320 ensemble 6-week forecasts in just 3 minutes. The algorithm can process a 1-week forecast in 1/10th of a second. 

The DLWP is able to produce realistic forecasting of weather events such as Hurricane Irma, a Category 4 storm that hit Florida and the Caribbean in 2017. While the speedy DLWP model matches the performance of current state-of-the-art weather forecasters 4 to 6 weeks into the future, it has limitations predicting precipitation and is less accurate in shorter lead times of 2–3 weeks. 

According to the study, the DLWP may also prove a valuable tool for supplementing spring and summer forecasts in the tropics, a region that challenges current weather models.

The open-source code is available on GitHub.


Read the study in Journal of Advances in Modeling Earth Systems. >>

Read more. >>

Categories
Misc

How Retailers Meet Tough Challenges Using NVIDIA AI

At the National Retail Federation’s annual trade show, conversations tend to touch on recurring themes: “Will we be able to stock must-have products for next Christmas?,” “What incentives can I offer to loyal workers?” and “What happens to my margins if Susie Consumer purchases three of the same dresses online and returns two?” The $26 Read article >

The post How Retailers Meet Tough Challenges Using NVIDIA AI  appeared first on The Official NVIDIA Blog.

Categories
Misc

AI Startup to Take a Bite Out of Fast-Food Labor Crunch

Addressing a growing labor crisis among quick-service restaurants, startup Vistry is harnessing AI to automate the process of taking orders. The company will share its story at the NRF Big Show, the annual industry gathering of the National Retail Federation in New York, starting Jan. 16. “They’re closing restaurants because there is not enough labor,” Read article >

The post AI Startup to Take a Bite Out of Fast-Food Labor Crunch appeared first on The Official NVIDIA Blog.

Categories
Misc

GFN Thursday: ‘Fortnite’ Comes to iOS Safari and Android Through NVIDIA GeForce NOW via Closed Beta

Starting next week, Fortnite on GeForce NOW will launch in a limited-time closed beta for mobile, all streamed through the Safari web browser on iOS and the GeForce NOW Android app. The beta is open for registration for all GeForce NOW members, and will help test our server capacity, graphics delivery and new touch controls Read article >

The post GFN Thursday: ‘Fortnite’ Comes to iOS Safari and Android Through NVIDIA GeForce NOW via Closed Beta appeared first on The Official NVIDIA Blog.

Categories
Misc

World Record-Setting DNA Sequencing Technique Helps Clinicians Rapidly Diagnose Critical Care Patients

Cutting down the time needed to sequence and analyze a patient’s whole genome from days to hours isn’t just about clinical efficiency — it can save lives. By accelerating every step of this process — from collecting a blood sample to sequencing the whole genome to identifying variants linked to diseases — a research team Read article >

The post World Record-Setting DNA Sequencing Technique Helps Clinicians Rapidly Diagnose Critical Care Patients appeared first on The Official NVIDIA Blog.

Categories
Misc

Slice of 20 elements from rank 1 tensor then reshaping throws "Input to reshape is tensor with 10272 values, but requested shape requires multiple of 20"

I posted this question to stack exchange here:

https://stackoverflow.com/questions/70686521/slice-of-20-elements-of-rank1-tensor-then-reshaping-throws-input-to-reshape-is

My input tensor “`Data = Input(shape=(856,))“` is a vector of float32 values concatenated from many different devices. I am trying to apply different TensorFlow functions to different subslices of each input chunk. Some of these functions include a 1D Convolution which requires a reshape.

slice = Data[:20]

reshape = tf.reshape(slice, (-1, 20, 1))

Doing this crashes after trying to fit my model. It throws the following errors:

tensorflow.python.framework.errors_impl.InvalidArgumentError: Input to reshape is a tensor with 10272 values, but the requested shape requires a multiple of 20

[[node model/tf.reshape_1/Reshape

(defined at /home/.local/lib/python3.8/site-packages/keras/layers/core/tf_op_layer.py:261)

]] [Op:__inference_train_function_1858]

Errors may have originated from an input operation.

Input Source operations connected to node model/tf.reshape_1/Reshape:

In[0] model/tf.__operators__.getitem_1/strided_slice:

In[1] model/tf.reshape_1/Reshape/shape:

I am not sure how slicing 20 elements from a tensor of 856 could result in a tensor of 10272 values.

I have also tried using the “`tf.slice“` function a couple of different ways; both fail. Referencing the docs: https://www.tensorflow.org/guide/tensor_slicing

slice = tf.slice(Data, begin=[0], size=[20])

And fails, stating:

Shape must be rank 1 but is rank 2 for ‘{{node tf.slice/Slice}} = Slice[Index=DT_INT32, T=DT_FLOAT](Placeholder, tf.slice/Slice/begin, tf.slice/Slice/size)’ with input shapes: [?,856], [1], [1].

For reference, here is what some of the values look like in the input data

array([-9.55784683e+01, -1.70557899e+01, 2.95967350e+01, 7.81378937e+00,

9.02729130e+00, 5.49621725e+00, 4.19811630e+00, 5.84186697e+00,

4.90438080e+00, 3.73845983e+00, 5.12300587e+00, 2.61530232e+00,

2.67061424e+00, 3.91038632e+00, 2.31110978e+00, 4.20644665e+00,

4.50000000e+00, 9.87345278e-01, 1.59740388e+00, 6.30727148e+00,

submitted by /u/EightEqualsEqualsDe
[visit reddit] [comments]