Weights become nan’s after 1 minibatch when learning rate is 0

My learning rate is set to 0. For my network, when i training a single minibatch, everything seems fine. As soon as I train the second minibatch, all the weights go to nan’s. Any ideas on why? I am not using batchnorm in my model.

submitted by /u/idg101
[visit reddit] [comments]

Leave a Reply

Your email address will not be published. Required fields are marked *