Categories
Misc

How do I initialize data for the numpy array used by model.fit()? I am trying to adapt the classification tutorial for my own data;

https://www.tensorflow.org/tutorials/keras/classification

I am trying to adapt this to take in data that is divided into 1152 16bit little endian signed data;
The file shape is a 1D array.
The data is aligned (zero padded).

bash du -b *.raw 41575680 labels.raw 41575680 input.raw

“`python

!/usr/bin/env python3

import tensorflow as tf import numpy as np

labels = “/home/test/trainData/labels.raw” inputs = “/home/test/trainData/input.raw”

label = np.fromfile(labels, dtype=”int16″) #expected output inputa = np.fromfile(inputs, dtype=”int16″) label = label / 65535.0 inputa = inputa / 65535.0

model = tf.keras.Sequential([ tf.keras.layers.Flatten(input_shape=(1152, )), tf.keras.layers.Dense(288, activation=’relu’), # 1/4th of 1152 tf.keras.layers.Dense(1152) ])

model.compile(optimizer=’adam’, loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),#change later metrics=[‘accuracy’]) model.fit(inputa, label, epochs=10) “`

Input and output shape: (20787840,) #samples, s/1152=18045, file/2/1152=18045 (20787840,) #samples, s/1152=18045, file/2/1152=18045

Traceback: “`python Traceback (most recent call last): File “/home/test/./test.py”, line 22, in <module> model.fit(inputa, label, epochs=10) File “/usr/lib/python3.10/site-packages/keras/utils/traceback_utils.py”, line 67, in error_handler raise e.with_traceback(filtered_tb) from None File “/usr/lib/python3.10/site-packages/tensorflow/python/framework/func_graph.py”, line 1147, in autograph_handler raise e.ag_error_metadata.to_exception(e) ValueError: in user code:

File "/usr/lib/python3.10/site-packages/keras/engine/training.py", line 1021, in train_function * return step_function(self, iterator) File "/usr/lib/python3.10/site-packages/keras/engine/training.py", line 1010, in step_function ** outputs = model.distribute_strategy.run(run_step, args=(data,)) File "/usr/lib/python3.10/site-packages/keras/engine/training.py", line 1000, in run_step ** outputs = model.train_step(data) File "/usr/lib/python3.10/site-packages/keras/engine/training.py", line 859, in train_step y_pred = self(x, training=True) File "/usr/lib/python3.10/site-packages/keras/utils/traceback_utils.py", line 67, in error_handler raise e.with_traceback(filtered_tb) from None File "/usr/lib/python3.10/site-packages/keras/engine/input_spec.py", line 248, in assert_input_compatibility raise ValueError( ValueError: Exception encountered when calling layer "sequential" (type Sequential). Input 0 of layer "dense" is incompatible with the layer: expected axis -1 of input shape to have value 1152, but received input with shape (32, 1) Call arguments received: • inputs=tf.Tensor(shape=(32,), dtype=float32) • training=True • mask=None 

“`

What’s the issue here? The data is 1152 samples/2304 bytes aligned.
How does it enter a blackhole and turn into (32,1)

” Input 0 of layer “dense” is incompatible with the layer: expected min_ndim=2, found ndim=1. Full shape received: (32,)” When I use InputLayer instead of Flatten.

submitted by /u/Remarkable-Guess-224
[visit reddit] [comments]

Leave a Reply

Your email address will not be published. Required fields are marked *