A Simple Introduction to Sequence to Sequence Models

Overview In this article, I would give you an overview of sequence to sequence models which became quite popular for different tasks like machine translation, video captioning, image captioning, question answering, etc. Prerequisites: The reader should already be familiar with neural networks and, in particular, recurrent neural networks (RNNs). In addition, knowledge of LSTM or GRU models is preferable. If you are not familiar with LSTM I would prefer you to read LSTM- Long Short-Term Memory.

A Detailed Study of Self Supervised Contrastive Loss and Supervised Contrastive Loss

Introduction Supervised Contrastive Learning paper claims a big deal about supervised learning and cross-entropy loss vs supervised contrastive loss for better image representation and classification tasks. Let’s go in-depth in this paper what is about. Claim actually close to 1% improvement on image net data set¹. Architecture wise, its a very simple network resnet 50 having a 128-dimensional head. If you want you can add a few more layers as well. Architecture and training process from the paper Codeself.encoder = […]