Keras implementation of Normalizer-Free Networks and SGD – Adaptive Gradient Clipping

Paper: https://arxiv.org/abs/2102.06171.pdf Original code: https://github.com/deepmind/deepmind-research/tree/master/nfnets Do star this repository if it helps your work! Note: Huge Credit to this comment for the pytorch implementation this repository is based on. Note: See this comment for a generic implementation for any optimizer as a temporary reference for anyone who needs it. Install from PyPi: pip3 install nfnets-keras or install the latest code using: pip3 install git+https://github.com/ypeleg/nfnets-keras NFNetF Model Use any of the NFNetF models like any other keras Model! from nfnets_keras import […]

Read more