DataBloom

Can I make a model that takes different number of inputs. The output becomes more accurate as inputs increase. So more the inputs more the accurate output will be. But I kinda want it to slide between accuracy. Like if i give 5 inputs a less accurate output will be, if i want more accurate result I want to give more inputs. Can I do something like this?

Hello everyone, I am looking for advice on a specific task: optimizing the result of a binary classifier.

I have real-world DBs of health data from two centers regarding ER patients with a specific pathology and I am trying to see if i can distinguish between high grade/emergency cases and low grade/less urgent cases. It’s not an easy task, even for medical professionals. I have a number of parameters, both boolean (medical history, symptoms) and continuous (labs) that have been shown to be somewhat statistically relevant in distinguishing the two, as well as labels for each. Positivity is about 5% of the total, which is about 1000+200 cases. The data is incomplete, though, as not all clues are available for every patient. I am planning to use the larger DB for train/test/initial validation and the second, smaller DB as my real-world validation.

I set up a Sequential model using some online tutorials, though it is unclear to me what guides the choice of layer numbers/type/activation/etc. My current setup, after some fiddling, is the following (abridged, obviously):
epochs= 100
TEST_SIZE = 0.25
x_train, x_test, y_train, y_test = train_test_split( X, Y, test_size=TEST_SIZE)
# define the model
model = tf.keras.models.Sequential()
model.add(tf.keras.layers.Dense(10, input_dim=x_train.shape[1], activation=’relu’, kernel_initializer=’he_normal’))
model.add(tf.keras.layers.Dense(1, activation=’sigmoid’))

# compile the keras model
model.compile(optimizer=’adam’,
loss=tf.keras.losses.BinaryCrossentropy(),
metrics=[tf.keras.metrics.RecallAtPrecision(0.95), tf.keras.metrics.Accuracy(), tf.keras.metrics.Recall(), tf.keras.metrics.Precision()])

# Fit model on training data
history = model.fit(x_train, y_train, validation_split=0.33, batch_size=16, epochs=EPOCHS)

Now, this is not a time sensitive operation, so I would like to get the best result possible. I would also like not to miss positive cases if I can help it, within reason. I suppose I could add in a “maybe” category if it increases accuracy significantly on the other two categories.
This is what I am at right now (it varies every run, sometimes significantly):

Original Set
RecallAtPrecision: 0.00
Accuracy: 0.24
recall – specificity: 60.00
precision – sensibility: 50.00

Validation
RecallAtPrecision: 8.00
Accuracy: 0.00
recall – specificity: 52.00
precision – sensibility: 76.47

Now, to the issues:

1. Why accuracy is so low, while sensibility/specificity are not that bad?
2. What can i do to make things better? More or less layers? More or less nodes? More epochs? Different setup altogether? Would it make sense to automate the search for the best parameters somehow?
3. Any general advice on the matter? Any accessible resources that I could look up? (please note that I am a heath professional, not a software developer. I only learned python and a few rudiments of ML in the last few months to see if I can automate stuff like this).

Thank you in advance for your kind responses. I am counting on you to illuminate my way forward

I am trying to carry out K-Means algorithm in a federated environment using the in-built algorithm function provided by TFF.

I am passing a list of

<PrefetchDataset element_spec=OrderedDict( [('x', TensorSpec(shape=(7811, 3), dtype=tf.float64, name=None))] )> 

of a couple of clients in the 2nd argument of next function for tff.learning.algorithms.build_fed_kmeans

The build_fed_kmeans function has been called like so:

iterative_process = tff.learning.algorithms.build_fed_kmeans( 10, data_shape = (7811,3), random_seed = (2,5) ) 

Error

TypeNotAssignableError: Type `<x=float64[7811,3]>*` is not assignable to type `float32[7811,3]*` 

How can I go correcting this error?

I am trying to load TFRecords dataset from minIO S3 bucket and read them and use them for training on the go as done in this link: https://www.google.com/amp/s/blog.min.io/hyper-scale-machine-learning-with-minio-and-tensorflow/amp/

But when I do model.fit(), It gives following error: next(iter(training_mapped_ds)).

Traceback (most recent call last): File “/home/vishal/miniotf2.py”, line 281, in <module> batch =next(iter(mapped_ds)) File “/home/vishal/anaconda3/lib/python3.9/site-packages/tensorflow/python/data/ops/iterator_ops.py”, line 766, in __next_ return self._next_internal() File “/home/vishal/anaconda3/lib/python3.9/site-packages/tensorflow/python/data/ops/iterator_ops.py”, line 749, in _next_internal ret = gen_dataset_ops.iterator_get_next( File “/home/vishal/anaconda3/lib/python3.9/site-packages/tensorflow/python/ops/gen_dataset_ops.py”, line 3017, in iterator_get_next _ops.raise_from_not_ok_status(e, name) File “/home/vishal/anaconda3/lib/python3.9/site-packages/tensorflow/python/framework/ops.py”, line 7164, in raise_from_not_ok_status raise core._status_to_exception(e) from None # pylint: disable=protected-access tensorflow.python.framework.errors_impl.UnknownError: : curlCode: 35, SSL connect error [Op:IteratorGetNext]

recently became obsessed with QR codes in shapes other than square, but all of the documentation out there only explain how to code QR code scanners using libraries (Pyzbar for example) rather than coding the thing from scratch. Now, given that I’m not particularly interested in square QR codes, one would think that coding and creating my own QR codes is the thing to do here. My information, howsoever, are hardly sufficient for me to say anything for sure, or if this whole thing is even possible. (because if it is, why is the hexagon QR code for example still not a thing??)

I guess what I’m trying to say is I would appreciate any information you guys would have on said topic ^-^

DANKE IM VORAUS!

Implemented Conditional-VAE on MNIST dataset using TensorFlow-2.8 and tf.GradientTape() API. You can refer to the full code here.

For generating synthetic data using trained network, there seems to be two ways:

1. Use learned latent space: z = mu + (eps log_var)* to generate (theoretically, infinite amounts of) data. Here, we are learning ‘mu’ and ‘log_var’ vectors using the given data, and, ‘eps’ is sampled from multivariate, standard, Gaussian distribution to add stochasticity.
2. Use multivariate, standard, Gaussian distribution = N(0, 1) as z which is then passed through VAE’s decoder.

What is the “the standard way” to generate data? (from the two options above), or, how can we find that. Neither the original Auto-Encoding Variational Bayes paper nor the β-VAE paper seem to specify the best way to generate images. The latter does say: “The most informative latent units zm of β-VAE have the highest KL divergence from the unit Gaussian prior”, confirming at least that the posterior distribution is not N(0,I) and the difference matters – reference.

I was monitoring my system RAM using free -m in the linux terminal after every cell execution. One single round of training in Federated Learning using the following code snippet used 4 GB of my RAM space (I have 8 in total!)

federated_train_data = get_new_federated_data() # Tensorflow Datasets of 4 clients state, metrics = iterative_process.next(state, federated_train_data) print('round 1, metrics={}'.format(metrics)) 

Is there a way to reduce/optimize ram space allocation? I don’t have CUDA set up. If that is the solution, I would really appreciate if someone could guide me with the steps to install the same on Debian since I have setup my project in a conda virtual environment and don’t have much knowledge about this OS.

GPU: Nvidia GTX 750 Ti

CPU: i5 4460

RAM: 8 GB

Been struggling to get the tensorflow object detection api installed on windows. I’m willing to pay anyone who is able to help me successfully install it. Assuming I don’t figure it out on my own.