Hi,
This is a cross post of a stack-overflow query. Sample code is attached there.
Task
- I am trying to implement Weight Standardization in Tensorflow 2.4.
- The goal here is to standardize the weights to mean=0, variance=1, BEFORE using them for convolution.
Methods tried
- I have tried two methods
- Subclassing by inheriting tf.keras.layers.Conv3D
- tf.keras.layers.Wrapper by following the example of Weight Normalization from the tensorflow_addons package
Problem
- In both the cases when I pass an input through the layer, and then check for the trainable weights, I only see the bias, and the kernel disappears. So what have I gotten wrong here? Is it the build() method?
submitted by /u/prerakmody
[visit reddit] [comments]