Implementation of OmniNet, Omnidirectional Representations from Transformers in Pytorch
Omninet – Pytorch Implementation of OmniNet, Omnidirectional Representations from Transformers, in Pytorch. The authors propose that we should be attending to all the tokens of the previous layers, leveraging recent efficient attention advances to achieve this goal. Install $ pip install omninet-pytorch Usage import torch from omninet_pytorch import Omninet omninet = Omninet( dim = 512, # model dimension depth = 6, # depth dim_head = 64, # dimension per head heads =
Read more