Categories
Misc

How to load a huge amount of images with tf.data

I’m newbie, I’ve working with tensorflow dataset so it’s my first time loading huge external data but there’s a problem, my data set it’s to big for my memory capacity.

I tried wit .flow_from_directory() but it seems to organize the data with classes and classes are the folders inside. This is not the case of my dataset, it’s a train folder -> a lot of folders with random names and inside there is the images, so .flow_from_directory() reads that random name as the label or the class. Is there a way to change that?

I’ve read the tf.data documentation but honestly, I don’t know how to solve my problem yet. I want to load all the data at the same time but it’s too big, so I need help. Please don’t only send me to read the documentation again :(.

submitted by /u/Current_Falcon_3187
[visit reddit] [comments]

Categories
Misc

[Discussion] Guys, I’m not a robot and I’d like to share a great TFX resource with you <3

Hi guys, my friend recently created a great online course describing how to apply BERT to sentiment analysis using TFX and Vertex AI. We want to reach as many people as possible because we spent several weeks working on it. The course is free, and we’ve also included the entire pipeline and helper library codebase, ready for use as a template in your project.

I’ve tried to share the link with you several times because I think it’s the perfect group (after all, we are showing the possibilities that Tensorflow Extended offers.

Unfortunately, my post gets deleted every time. So, I’m trying again, without the marketing bullshit.

BERT SENTIMENT ANALYSIS ON VERTEX AI USING TFX

Please check out this course and let me know what you think. I hope it will be helpful to some of you. And you think something is missing. Feel free to discuss. We will gladly add the missing parts!

Btw. Any ideas on how to add links to avoid being considered spam?

submitted by /u/Novel_Cryptographer6
[visit reddit] [comments]

Categories
Misc

Fantastical 3D Creatures Roar to Life ‘In the NVIDIA Studio’ With Artist Massimo Righi

The year of the tiger comes into focus this week In the NVIDIA Studio, which welcomes 3D creature artist Massimo Righi. An award-winning 3D artist with two decades of experience in the film industry, Righi has received multiple artist-of-the-month accolades and features in top creative publications.

The post Fantastical 3D Creatures Roar to Life ‘In the NVIDIA Studio’ With Artist Massimo Righi appeared first on NVIDIA Blog.

Categories
Misc

question about reading tfrecord

i would like to read a “.tfrecord” dataset in my python code and print some of the rows with all their individual values, whats the easiest way to do it? i have the tfrecord file and metadata.json

submitted by /u/samas69420
[visit reddit] [comments]

Categories
Misc

custom op in tensorflow

Can I write custom.op file in c, and compile it directly?

submitted by /u/Section_Disastrous
[visit reddit] [comments]

Categories
Misc

HPC Researchers Seed the Future of In-Network Computing With NVIDIA BlueField DPUs

Across Europe and the U.S., HPC developers are supercharging supercomputers with the power of Arm cores and accelerators inside NVIDIA BlueField-2 DPUs. At Los Alamos National Laboratory (LANL) that work is one part of a broad, multiyear collaboration with NVIDIA that targets 30x speedups in computational multi-physics applications. LANL researchers foresee significant performance gains using Read article >

The post HPC Researchers Seed the Future of In-Network Computing With NVIDIA BlueField DPUs appeared first on NVIDIA Blog.

Categories
Misc

Scientists Building Digital Twins in NVIDIA Omniverse to Accelerate Clean Energy Research

As global climate change accelerates, finding and securing clean energy is a crucial challenge for many researchers, organizations and governments. The U.K.’s Atomic Energy Authority (UKAEA), through an evaluation project at the University of Manchester, has been testing the NVIDIA Omniverse simulation platform to accelerate the design and development of a full-scale fusion powerplant that Read article >

The post Scientists Building Digital Twins in NVIDIA Omniverse to Accelerate Clean Energy Research appeared first on NVIDIA Blog.

Categories
Misc

The Road to the Hybrid Quantum-HPC Data Center Starts Here

It’s time to start building tomorrow’s hybrid quantum computers. The motivation is compelling, the path is clear and key components for the job are available today. Quantum computing has the potential to bust through some of today’s toughest challenges, advancing everything from drug discovery to weather forecasting. In short, quantum computing will play a huge Read article >

The post The Road to the Hybrid Quantum-HPC Data Center Starts Here appeared first on NVIDIA Blog.

Categories
Misc

The Man With 100,000 Brains: AI’s Big Donation to Science

Jorge Cardoso wears many hats, and that’s appropriate given he has so many brains. A hundred thousand of them to be exact. Cardoso is a teacher, a CTO, an entrepreneur, a founding member of the MONAI open source consortium and a researcher in AI for medical imaging. In that last role, Cardoso and his team Read article >

The post The Man With 100,000 Brains: AI’s Big Donation to Science appeared first on NVIDIA Blog.

Categories
Misc

NVIDIA Accelerates AI, Digital Twins, Quantum Computing and Edge HPC at ISC 2022

Researchers grappling with today’s grand challenges are getting traction with accelerated computing, as showcased at ISC, Europe’s annual gathering of supercomputing experts. Some are building digital twins to simulate new energy sources. Some use AI+HPC to peer deep into the human brain. Others are taking HPC to the edge with highly sensitive instruments or accelerating Read article >

The post NVIDIA Accelerates AI, Digital Twins, Quantum Computing and Edge HPC at ISC 2022 appeared first on NVIDIA Blog.