Tensorflow pratice vs Mathematics

Hello there,

i often see the following code for regression problems(we have here a linear regression)

import tensorflow.compat.v1 as tf
import numpy as np
import matplotlib.pyplot as plt
learning_rate = 0.01
training_epochs = 100
x_train = np.linspace(-1, 1, 101)
y_train = 2 * x_train + np.random.randn(*x_train.shape) * 0.33
X = tf.placeholder(tf.float32)
Y = tf.placeholder(tf.float32)
def model(X, w):
return tf.multiply(X, w)
w = tf.Variable(0.0, name=”weights”)
y_model = model(X, w)
cost = tf.square(Y-y_model)
train_op = tf.train.GradientDescentOptimizer(learning_rate).minimize(cost)
sess = tf.Session()
init = tf.global_variables_initializer()
sess.run(init)
for epoch in range(training_epochs):
for (x, y) in zip(x_train, y_train):
sess.run(train_op, feed_dict={X: x, Y: y})
w_val = sess.run(w)
sess.close()
plt.scatter(x_train, y_train)
y_learned = x_train*w_val
plt.plot(x_train, y_learned, ‘r’)
plt.show()

But isnt that wrong? My problems are these lines:

for epoch in range(training_epochs):for (x, y) in zip(x_train, y_train):sess.run(train_op, feed_dict={X: x, Y: y})

Why is it a problem? Because if you look how we do it in pure mathematics it doesnt fit. We have the MSE function in math and we do gradient descent over the hole function. But here it seems that they are doing gradient descent just over the parts of MES function in line

“for (x, y) in zip(x_train, y_train):sess.run(train_op, feed_dict={X: x, Y: y})”

What do i mean with that? MSE=g1(x)+g2(x)+…+gn(x) and it seems like they do graph descent on g1(x) then on g2(x) and so on.How does TensorFlow exactly calculus in the back?
My problem is that through feed_dict={X: x, Y: y} only just one function will be called. Lets say x=1 and y=2 Tensorflow will go to X and Y then it will go to def model and only will call one part of the function of MSE lets say g1(x) but you need to go over all MSE with graph descent?