Just for the hell of it, I added a convolution with ReLU on the generator side of the mnist GAN. It doesn’t work very well but… it is significantly worse than a conv with sigmoid, which I found interesting.
Is there a way to pause training and inspect weights, or to display them periodically - much as the samples are displayed?
I think I know what’s going on (not that it really matters) - what I’d like to know is how to validate such intuitions. Any ideas?