Keras implementation of Normalizer-Free Networks and SGD – Adaptive Gradient Clipping
Paper: https://arxiv.org/abs/2102.06171.pdf
Original code: https://github.com/deepmind/deepmind-research/tree/master/nfnets
Do star this repository if it helps your work!
Note: Huge Credit to this comment for the pytorch implementation this repository is based on. Note: See this comment for a generic implementation for any optimizer as a temporary reference for anyone who needs it.
Install from PyPi:
pip3 install nfnets-keras
or install the latest code using:
pip3 install git+https://github.com/ypeleg/nfnets-keras
NFNetF Model
Use any of the NFNetF models like any other keras Model!
from nfnets_keras import NFNetF3
model = NFNetF3(include_top = True, num_classes = 10)
model.compile('adam', 'categorical_crossentropy')
model.fit(X,