Categories
Misc

Using TF timeseries_dataset_from_array with more samples

Using TF timeseries_dataset_from_array with more samples

I have to handle a huge amount of samples, where each sample contains unique time series. The goal is to feed this data into the Tensorflow LSTM model and predict some features. I have created the tf timeseries_dataset_from_arraygenerator function to feed the data to the TF model, but I haven’t figured out how to create a generator function when I have multiple samples. If I use the usual pipeline, tf timeseries_dataset_from_arrayoverlap the time series of two individual samples.

Does anyone have an idea how to effectively pass a time series of multiple samples to the TF model?

E.g. the Human Activity Recognition Dataset is one such dataset where each person has a separate long, time series, and each user’s time series can be further parsed with the SLIDING/ROLLING WINDOS-like timeseries_dataset_from_arrayfunction.

Here is a simpler example:

I want to use timeseries_dataset_from_arrayto generate samples for the TF model. Example: sample 1 where column 0 has 0, sample 2 starts where column 0 has 100. Here is a simpler example:

https://preview.redd.it/0hzmsm9tt6481.png?width=786&format=png&auto=webp&s=d82f1fdbcb9dff8337de274bf6325ebac52c7e81

I want to get 3D data (samples, timesteps, features) without overlap.For example (6,2,7) Like this:

https://preview.redd.it/t9cfot0vt6481.png?width=865&format=png&auto=webp&s=0524e0776a3f1938c6178ee5b16dda80885ed44d

Here is the sample code:

from tensorflow.keras.preprocessing import timeseries_dataset_from_array import numpy as np x = np.array([[0,1,2,3,4,5,6], [0,11,12,13,14,15,16], [0,21,22,23,24,25,26], [0,31,32,33,34,35,36], [0,41,42,43,44,45,46] ]) xx = np.concatenate((x, x+100), axis=0)#.reshape(2,5,6) sequence_length=2 stride=1 rate=1 input_dataset = timeseries_dataset_from_array(xx, None, sequence_length, sequence_stride=stride, sampling_rate=rate) x_test = np.concatenate([x for x in input_dataset], axis=0) 

submitted by /u/korosig
[visit reddit] [comments]

Leave a Reply

Your email address will not be published.