Categories
Misc

Issue implementing Weight Standardization in Tensorflow 2.4

Hi,

This is a cross post of a stack-overflow query. Sample code is attached there.

Task

  • I am trying to implement Weight Standardization in Tensorflow 2.4.
  • The goal here is to standardize the weights to mean=0, variance=1, BEFORE using them for convolution.

Methods tried

  1. I have tried two methods

Problem

  • In both the cases when I pass an input through the layer, and then check for the trainable weights, I only see the bias, and the kernel disappears. So what have I gotten wrong here? Is it the build() method?

submitted by /u/prerakmody
[visit reddit] [comments]

Leave a Reply

Your email address will not be published.