DataBloom

I want to set up distributed RL. Multiple worker and 1 learner. I have 1 GPU and 1 CPU with multiple cores.

So GPU:0 and CPU:0.

Now if i start the programm normal via python programm.py

It detects the GPU and CPU and lists it once i call tf.config.list_physical_device()

However if i start it as MPI application via:

mpiexec -np 4 python programm.py

Every process just lists CPU:0 and no gpu is detected. How can i make at least one process see the gpu?

I use tf2.7 and mpi4py

Here is the full code. This currently works just fine on my M1 MacBook running Monterey and Tensorflow-Metal. However, when I export the dataset and code to my laptop with an RTX 3060 Laptop GPU with Pop_OS! that is when I start getting the [UNK] characters generated and “NaN” loss. I’m unsure of what steps to take to make this better. Any advice would be appreciated.

import os, sys, time import numpy as np import tensorflow as tf from tensorflow.keras.models import Sequential from tensorflow.keras.callbacks import ModelCheckpoint from tensorflow.keras.optimizers import Adam from tensorflow.keras.preprocessing.sequence import pad_sequences from tensorflow.keras.preprocessing.text import Tokenizer from tensorflow.keras.layers import StringLookup from tensorflow.keras.layers import Embedding from tensorflow.keras.layers import Bidirectional from tensorflow.keras.layers import SimpleRNN from tensorflow.keras.layers import Dense BATCH_SIZE = 128 BUFFER_SIZE = 10_000 EMBEDDING_DIMENSION = 128 RNN_UNITS = 1024 CHECKPOINT_DIR = './training_checkpoints' CHECKPOINT_PREFIX = os.path.join(CHECKPOINT_DIR, "ckpt_{epoch}") EPOCHS = 16 def text_from_ids(ids): return tf.strings.reduce_join(chars_from_ids(ids), axis=1) def split_input_target(sequence): input_text = sequence[:-1] target_text = sequence[1:] return input_text, target_text def generate_text(model, seed_text, next_words, max_sequence_len): for _ in range(next_words): token_list = Tokenizer().texts_to_sequences([seed_text])[0] token_list = pad_sequences([token_list], maxlen=max_sequence_len-1, padding='pre') predicted = model.predict(token_list, verbose=0) output_word = "" for word,index in Tokenizer().word_index.items(): if index == predicted: output_word = word break seed_text += " "+output_word return seed_text.title() def generate_char(inputs): input_ids = tf.convert_to_tensor(ids_from_chars(inputs)) predicted_logits = model(inputs=np.array([input_ids])) predicted_logits = predicted_logits[:, -1, :] # print(predicted_logits) predicted_logits = predicted_logits/1.0 # print(predicted_logits) predicted_ids = tf.random.categorical(predicted_logits, num_samples=1) predicted_ids = tf.squeeze(predicted_ids, axis=-1) return chars_from_ids(predicted_ids) text = open("./data.txt", "rb").read().decode(encoding="UTF-8") vocab = sorted(set(text)) vocab_size = len(vocab) print(f"Text Length: {len(text)}") print(f"Text Vocab: {vocab}") print(f"Text Vocab Size: {vocab_size}") ids_from_chars = StringLookup(vocabulary=list(vocab), mask_token=None, name='lookup') chars_from_ids = StringLookup(vocabulary=ids_from_chars.get_vocabulary(), invert=True, mask_token=None) all_ids = ids_from_chars(tf.strings.unicode_split(text, "UTF-8")) ids_dataset = tf.data.Dataset.from_tensor_slices(all_ids) sequence_length = 100 examples_per_epoch = len(text)//(sequence_length+1) sequences = ids_dataset.batch(sequence_length+1, drop_remainder=True) dataset = sequences.map(split_input_target) dataset = ( dataset.shuffle(BUFFER_SIZE) .batch(BATCH_SIZE, drop_remainder=True) .prefetch(tf.data.experimental.AUTOTUNE) ) model = Sequential() model.add(Embedding(vocab_size, EMBEDDING_DIMENSION, batch_input_shape=[BATCH_SIZE, None])) model.add(SimpleRNN(RNN_UNITS, return_sequences=True)) model.add(Dense(vocab_size,)) checkpoint_callback = ModelCheckpoint( filepath=CHECKPOINT_PREFIX, save_weights_only=False, save_best_only=False, verbose=1 ) loss = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True) model.compile(loss = loss, optimizer='adam', run_eagerly=True) model.summary() model.fit(dataset, batch_size=BATCH_SIZE, epochs=EPOCHS, callbacks=[checkpoint_callback]) model.save("./model/") model = tf.keras.models.load_model("./model/") next_char = tf.constant(["After "]) result = [] for n in range(256): next_char = generate_char(next_char) result.append(next_char) print(tf.strings.join(result)[0].numpy().decode("utf-8")) 

For the record, I’m absolutely expecting to have made some obvious error here, but I can’t seem to find it myself.

For context, I have implemented a very simple version of a game similar to doodle jump, and among other things I have attempted to train a FFNN to predict the correct / best inputs based on the state of the game. I have generated ~15k examples through manual play, each frame having a 25% of being recorded for both state and inputs, and for a slight bit of dataset balancing, half of all frames without any input are discarded.

I’m using a sequential model with 3 dense layers, using 15, 5 and 3 units respectively. I’ve specified a sigmoid activation for the input and hidden layers, and the input shape for the input layer.

For each example, the input consists of 15 scalar values (7 pairs of values representing the distance of a platform to the player sprite in horizontal and vertical direction respectively, plus 1 value for the currently remaining timer. (I’m using a 20 second timer to make comparisons reasonable.)), while the labels consist of 3 integers, each either 1 or 0, representing whether the key associated with that position has been pressed on that frame or not. (Left, Right, Up in that order.) The model compilation specifies the use of the Adam optimzer and MeanSquaredError loss.

What I’m specifically hoping to predict is a set of 3 values, which I can check against a set threshold to determine wheter the associated key should be pressed on that frame.

When training the model, however, I’m seeing no relevant drop in loss over 100 epochs, (most recently it went from 0.1923 to 0.1901), and indeed the trained models behaviour will consistently see it pressin right and jump, with the value for the left key often being negative, which on the one hand seems to indicate extreme underfitting (Since it’s predicting negative values when all example labels were positive), but on the other the sheer regularity with which this occurs might point to an error in my methodology.

I realise that any answer to this will be speculative at best, and that’s absolutely fine. I’ve tried everything I can think of (Varying number and size of the layers, varying the optimizer and/or loss function, varying the threshold, deleting and rebuilding the dataset…), so any ideas wouldbe very welcome.

As the title is self-descriptive, I’m getting a dramatic accuracy difference for the same model when I deploy it on MATLAB compared to TensorFlow & Keras. My model is actually a basic one, which uses transfer technique to fine-tune a pre-trained model, namely, ResNet50, by excluding the top and adding a Dense layer to utilize it for another classification task. I’ve added the sample code that shows the architecture of the model below. The model was trained under the same hyper-parameters on both platforms. The accuracy values I get on TensorFlow and MATLAB are 0.7455, and 0.424, respectively, on the CIFAR-10. What could be the reason behind this great accuracy difference? Could you please help me?

​

Model Architecture:

base_model = ResNet50(include_top=False, weights=’imagenet’, input_shape=(75, 75, 3),pooling=’max’, classes=10)

base_model.trainable = False

model = Sequential()

model.add(base_model)

model.add(Dense(10, activation=’softmax’))

Browsing through freely available sources I find both statements: DQN is good / is not good for stochastic environments.

As far as I understand it, the Q-Network predicts the expected return of an action in a state, which can then be used to decide e.g. greedily; and training makes that prediction better. If the environment is stochastic, repeated learning should nudge the prediction to the distribution center as the loss minimum.

So it should work, but might need a lot of time to get there (law of great numbers), especially since the game is being played by 2 agents suffering from the same problem, and being part of the “environment” stochastic behaviour for the opponent!

Maybe there is another technique in Deep Learning / Reinforcement Learning much better suited for such a strongly stochastic environment? Any advices?

Packt has Published “The TensorFlow Workshop ”

Grab your digital copy now if you feel you are interested.

As part of our marketing activities, we are offering free digital copies of the book in return for unbiased feedback in the form of a reader review.

Get started with TensorFlow fundamentals to build and train deep learning models with real-world data, practical exercises, and challenging activities.

Here is what you will learn from the book:

1. Get to grips with TensorFlow’s mathematical operations
2. Pre-process a wide variety of tabular, sequential, and image data
3. Understand the purpose and usage of different deep learning layers
4. Perform hyperparameter-tuning to prevent overfitting of training data
5. Use pre-trained models to speed up the development of learning models
6. Generate new data based on existing patterns using generative models

Key Features

• Understand the fundamentals of tensors, neural networks, and deep learning
• Discover how to implement and fine-tune deep learning models for real-world datasets
• Build your experience and confidence with hands-on exercises and activities

Please comment below or DM me for more details