Linear Regressions from Image…Is it accurate?

I’m trying to create a model to predict latitude and longitude from an image. However, my loss does not decrease at all during training, and the predictions are very far off.

Is what I’m doing realistically possible? And if so, how can I make my model more accurate?

Thanks!