Control of Initializers

This paper by He et al demonstrates an improvement on the Xavier/Glorot initialiaser. The newer He initializer works better with deep ReLU networks, the Xavier was initially developed to work with the sigmoid activation.

Is there already support for alternative initializers? (I haven’t looked :wink:

(NB with TF2 I believe the default initializer is Glorot_Uniform)

1 Like

Hey @JulianSMoore,
There is no easy setting in the component which will change the initializer quite yet, but they are customizable from within the custom code :slight_smile:

Thanks @robertl. I was under the impression that custom code was temporarily unavailable (or was it just notebook export?) - can you clarify?

Custom code is available in all components except Input and Target components.
The notebook export is the one that is temporarily disabled, but should be enabled again sometime this month :slight_smile: