Categories

# Problem in tf.keras custom metric

I tried to create my own custom metric, by subclassing the tf.keras.metrics.Metric class, by doing something similar to what is done at this Keras link.

Premise:
Before feeding data to the Neural Network, I pre-process them, by diving them for a scalar value equal to scalar=1200.0.
Thus, if I use the Mean Squared Error (MSE) metric already included in Keras, it calculates the “mse” value based on normalized data (instead of the original, de-normalized, data).

My aim is to define a “custom MSE”, in particular, an MSE calculated on de-normalized data.This is what I tried:

```class custom_MSE(tf.keras.metrics.Metric): def __init__(self, name='custom_MSE', **kwargs): scalar_value = float(kwargs.pop('scalar')) super(custom_MSE, self).__init__(name=name, **kwargs) # super().__init__(name=name, **kwargs) self.mse_value = self.add_weight(name='mse_denormalized', initializer='zeros') self.scalar = scalar_value def update_state(self, y_true, y_pred, sample_weight=None) # SHAPES: y_true: (1000, 599); y_pred: (1000, 599, 1) y_true = tf.expand_dims(y_true, -1) # (1000, 599, 1) # de-normalization y_true_denorm = tf.multiply(y_true, self.scalar) # (1000, 599, 1) y_pred_denorma = tf.multiply(y_pred, self.scalar) # (1000, 599, 1) # MSE calculation squared_diff = tf.square(y_true_denorm - y_pred_denorm) # (1000, 599, 1) values = tf.reduce_mean(squared_diff) # (1000, 599, 1) self.mse_value.assign_add(tf.reduce_sum(values)) # (1000, 599, 1) def result(self): return self.mse_value def reset_states(self): self.mse_value.assign(0)
```

If I compile the model by using the “mse” metric which comes with Keras, by compiling the model as:

```model.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=0.1, beta_1=0.9, beta_2=0.99), loss="mse", metrics="mse")
```

I obtain “mse” values of the order of 10^-4.

Then, I tried to compile the model, by using my own “custom_mse” metric, as:

```model.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=0.1, beta_1=0.9, beta_2=0.99), loss="mse", metrics=custom_MSE(name="custom_MSE", maxmin="1200.0"))
```

Since custom_MSE = mse * (scalar)^2,
I expected to obtain “custom_mse” values of the order of 10^(-4) * (1200.0)^2 = 10^2.
Instead, I obtained “custom_mse” values insanely too big: of the order of 10^6.

Is there anyone who could see something wrong in the above tf.keras.metrics.Metric subclass?

submitted by /u/RainbowRedditForum