1 min readAug 28, 2017
That’s happen, but still it’s bad behavior. Changing optimizer should help with this, try to use Adam, AdaGrad, RMSProp. Also this can happen if you dont use zero-centering and normalization.
Machine Learning and Computer Vision Researcher. Founder LearnML.Today