DataBloom

https://www.tensorflow.org/tutorials/keras/classification

I am trying to adapt this to take in data that is divided into 1152 16bit little endian signed data;
The file shape is a 1D array.
The data is aligned (zero padded).

bash du -b *.raw 41575680 labels.raw 41575680 input.raw

“`python

!/usr/bin/env python3

import tensorflow as tf import numpy as np

labels = “/home/test/trainData/labels.raw” inputs = “/home/test/trainData/input.raw”

label = np.fromfile(labels, dtype=”int16″) #expected output inputa = np.fromfile(inputs, dtype=”int16″) label = label / 65535.0 inputa = inputa / 65535.0

model = tf.keras.Sequential([ tf.keras.layers.Flatten(input_shape=(1152, )), tf.keras.layers.Dense(288, activation=’relu’), # 1/4th of 1152 tf.keras.layers.Dense(1152) ])

model.compile(optimizer=’adam’, loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),#change later metrics=[‘accuracy’]) model.fit(inputa, label, epochs=10) “`

Input and output shape: (20787840,) #samples, s/1152=18045, file/2/1152=18045 (20787840,) #samples, s/1152=18045, file/2/1152=18045

Traceback: “`python Traceback (most recent call last): File “/home/test/./test.py”, line 22, in <module> model.fit(inputa, label, epochs=10) File “/usr/lib/python3.10/site-packages/keras/utils/traceback_utils.py”, line 67, in error_handler raise e.with_traceback(filtered_tb) from None File “/usr/lib/python3.10/site-packages/tensorflow/python/framework/func_graph.py”, line 1147, in autograph_handler raise e.ag_error_metadata.to_exception(e) ValueError: in user code:

File "/usr/lib/python3.10/site-packages/keras/engine/training.py", line 1021, in train_function * return step_function(self, iterator) File "/usr/lib/python3.10/site-packages/keras/engine/training.py", line 1010, in step_function ** outputs = model.distribute_strategy.run(run_step, args=(data,)) File "/usr/lib/python3.10/site-packages/keras/engine/training.py", line 1000, in run_step ** outputs = model.train_step(data) File "/usr/lib/python3.10/site-packages/keras/engine/training.py", line 859, in train_step y_pred = self(x, training=True) File "/usr/lib/python3.10/site-packages/keras/utils/traceback_utils.py", line 67, in error_handler raise e.with_traceback(filtered_tb) from None File "/usr/lib/python3.10/site-packages/keras/engine/input_spec.py", line 248, in assert_input_compatibility raise ValueError( ValueError: Exception encountered when calling layer "sequential" (type Sequential). Input 0 of layer "dense" is incompatible with the layer: expected axis -1 of input shape to have value 1152, but received input with shape (32, 1) Call arguments received: • inputs=tf.Tensor(shape=(32,), dtype=float32) • training=True • mask=None 

“`

What’s the issue here? The data is 1152 samples/2304 bytes aligned.
How does it enter a blackhole and turn into (32,1)

” Input 0 of layer “dense” is incompatible with the layer: expected min_ndim=2, found ndim=1. Full shape received: (32,)” When I use InputLayer instead of Flatten.

Hey all,

I’ve been trying to get tensorflow to work with my rtx 3080 on anaconda without much success.

Im on windows and currently have toolkit 11.0 runtime libraries and cudnn 8.11 installed, which I use to remove stars in my astronomy images. I’m not sure how to get these to work in anaconda or how to get the other versions to work.

I think it’s fair to say Tensorflow with GPU support is a pain to install.

What is currently the recommended / easiest way to install the latest tensorflow for Windows 10?

I successfully installed tensorflow-gpu from Anaconda but the tensorflow version is limited to 2.3.

Additional questions:

1. What should we use to install the latest versions? (pip / anaconda)

2. Does anyone have a good step-by-step guide ? I found the tensorflow documentation to be horrible and most of the online guides are outdated

3. Do we actually need the tensorflow-gpu package from Anaconda? I thought tensorflow-gpu was part of tensorflow since tfv2.

Hey I have spent the last days trying and failing to install an array of differing installation procedures in order to try to get Tensorflow to work on my Mac M1 chip. Does anyone have a confirmed working method as of 02/2022.

Thank you !

Much appreciated for any tips/ links.

Hello.

I am starting to learn Tensorflow in Python/Jupyter, and I thought I’d create a small ML project for fun that can perform certain actions based on sound events in the room. I’m looking for source code examples in python for real-time sound classification. Most examples I found on google will perform audio classification on existing wav files stored on the hard disk, but I am actually looking for something that can do live audio classification from a microphone. Preferably with minimal latency.

I’d like to see source code for something like this: https://www.youtube.com/watch?v=f6ypnGXMado

Thanks in advance.

Tensorflow MaxPooling2D layer takes as input a tensor with the following structure (Batch, Width, Length, Channels), Caffe’s input instead is structured as (Batch, Channels, Width, Length).

I’m working with both frameworks but I would like Caffe to mimics Tensorflow behavior, how can I modify a tensor that produces a specific output on Tensorflow in a way that, when fed to Caffe’s MaxPool, provides the same output?

Hey, so as to my understanding of how IamgeDataGenerator works , is that it choose a random operation and apply it to the image. With this also applies to rescale, std_normalistion and zac_whiteing, with those not be applied to all the images?

I am a novice to Tensorflow but have significant experience in the scikit-learn environment. I am trying to shift some codes to the more flexible TF/Keras routines in Python. One modification I would like to do is to build a simple neural network that trains on an earth mover’s distance/Wasserstein distance between arrays representing densities, rather than on direct root-mean-square differences in the densities.

I have been able to modify a loss function, but do not know how to implement the EMD in TF. I use pyEMD to calculate the EMD between two of the arrays in Python, but is doesn’t translate to tensors in TF. It seems like I may have to “unstack” the tensors, evaluate by each row, then “restack” as a tensor. It doesn’t seem reasonable to do it this way, but I am willing to make it work. At this point, I am focused on making it work rather than be as-fast-as-possible.

Does anyone have advice on either a TF-friendly EMD approach or how I may use pyEMD in a custom loss function? I appreciate any and all suggestions.