NFNets and Adaptive Gradient Clipping for SGD implemented in PyTorch
Do star this repository if it helps your work!
Note: See this comment for a generic implementation for any optimizer as a temporary reference for anyone who needs it.
Install from PyPi:
pip3 install nfnets-pytorch
or install the latest code using:
pip3 install git+https://github.com/vballoli/nfnets-pytorch
WSConvTranspose2d like any other
from torch import nn
from nfnets import WSConv2d
conv = nn.Conv2d(3,6,3)
w_conv = WSConv2d(