Why loss values don’t make sense for Dice, Focal, IOU for boundary detection Unet in Keras?

I am using Keras for boundary/contour detection using a Unet. When I use binary cross-entropy as the loss, the losses decrease over time as expected the predicted boundaries look reasonable

However, I have tried custom losses for Dice, Focal, IOU, with varying LRs, and none of them are working well. I either get NaNs or non-decreasing/barely-decreasing values for the losses. This is regardless of what I use for the LR, whether it be .01 to 1e-6, or whether I vary the ALPHA and GAMMA and other parameters. This doesn’t make sense since for my images, most of the pixels are the background, and the pixels corresponding to boundaries are the minority. For imbalanced datasets, IOU, Dice, and Focal should work better than binary Cross-Entropy

The code I used for the losses are from https://www.kaggle.com/bigironsphere/loss-function-library-keras-pytorch#Jaccard/Intersection-over-Union-(IoU)-Loss

def DiceLoss(targets, inputs, smooth=1e-6): #flatten label and prediction tensors inputs = K.flatten(inputs) targets = K.flatten(targets) intersection = K.sum(K.dot(targets, inputs)) dice = (2*intersection + smooth) / (K.sum(targets) + K.sum(inputs) + smooth) return 1 - dice ALPHA = 0.8 GAMMA = 2 def FocalLoss(targets, inputs, alpha=ALPHA, gamma=GAMMA): inputs = K.flatten(inputs) targets = K.flatten(targets) BCE = K.binary_crossentropy(targets, inputs) BCE_EXP = K.exp(-BCE) focal_loss = K.mean(alpha * K.pow((1-BCE_EXP), gamma) * BCE) return focal_loss def IoULoss(targets, inputs, smooth=1e-6): #flatten label and prediction tensors inputs = K.flatten(inputs) targets = K.flatten(targets) intersection = K.sum(K.dot(targets, inputs)) total = K.sum(targets) + K.sum(inputs) union = total - intersection IoU = (intersection + smooth) / (union + smooth) return 1 - IoU 

Even if I try different code for the losses, such as the code below

smooth = 1. def dice_coef(y_true, y_pred): y_true_f = K.flatten(y_true) y_pred_f = K.flatten(y_pred) intersection = K.sum(y_true_f * y_pred_f) return (2. * intersection + smooth) / (K.sum(y_true_f) + K.sum(y_pred_f) + smooth) def dice_coef_loss(y_true, y_pred): return -dice_coef(y_true, y_pred) 

the loss values still don’t improve. That is, it will show something like

loss: nan - dice_coef_loss: .9607 - val_loss: nan - val_dice_coef_loss: .9631 

and the values won’t change much for each epoch

can anyone help?