A Detailed Study of Self Supervised Contrastive Loss and Supervised Contrastive Loss

Introduction

Supervised Contrastive Learning paper claims a big deal about supervised learning and cross-entropy loss vs supervised contrastive loss for better image representation and classification tasks. Let’s go in-depth in this paper what is about.

Claim actually close to 1% improvement on image net data set¹.

Architecture wise, its a very simple network resnet 50 having a 128-dimensional head. If you want you can add a few more layers as well.

Codeself.encoder = resnet50()self.head = nn.Linear(2048, 128)def forward(self, x):
 feat = self.encoder(x)
 #normalizing the 128 vector is required
 feat = F.normalize(self.head(feat), dim=1)
 return feat

As shown in
Visit source site