Deep Learning, NLP, NMT, AI, ML
Welcome back! This week’s Cypher will be a bit shorter than usual, it was a slow week in NLP land. But first, I want to update you on the BlenderBot 2.0 situation. On last week’s Cypher, the last hurdle to overcome with the instantiation of blenderbot inference was the search server (which gives the bot the ability to comb the web to answer factoid type of questions). Well we finally have a search server repo to work with! Thank you […]
Read more
Machine Learning education content from aggregating 1,300 questions from an ML Course. Pretty cool site with very simple and intuitive answers to technical ML questions. If you are looking for more math heavy stuff go elsewhere. Here’s an example: What do dropout layers do? Dropout layers throw things away. Now you would be asking, why would I want my model to throw data away? It turns out that throwing things away when training a model can drastically improve a model’s […]
Read more
Nova melting hypothetical planet | Bonestell NATURAL LANGUAGE PROCESSING (NLP) NEWSLETTER o̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿unicode suckso̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿ Way back in February of 2020, someone Twitter posted they had FOIA’d the NSA aka National Security Agency. This actor, by the name ‘cupcake’ was able to retrieve a 400-page printout of their COMP 3321 training course (😂). It was OCR’d and uploaded to the cloud totaling 118MB of absolute FOIA madness of Python learning material courtesy of the Men in Black by the way of Fort […]
Read more
Hey Welcome Back! A flood of EMNLP 2021 papers came in this week so today’s newsletter should be loads of fun! 😋 But first, a meme search engine: An article on The Gradient had an interesting take on NLU. It describes how a NNs’ capacity for NLU inference is inherently bounded to the background knowledge it knows (which is usually highly limited relative to a human). Although I would add a bit more nuance to this by sharing that this […]
Read more
Welcome back! We have a long newsletter this week as many new NLP repos were published as tech nerds return from their Summer vacation. 😁 This week I’ll add close to 150 new NLP repos to the NLP Index. So stay tuned for this update, it will drop this week. just explore… Embeddinghub is a database built for machine learning embeddings. It is built with four goals in mind. Store embeddings durably and with high availability Allow for approximate nearest […]
Read moreFrom the intense shock of the COVID-19 pandemic to the effects of climate change, our global society has never faced greater risk. The Societal Resilience team at Microsoft Research was established in recognition of this risk and tasked with developing open technologies that enable a scalable response in times of crisis. And just as we think about scalability in a holistic way—scaling across different forms of common problems, for different partners, in different domains—we also take a multi-horizon view of what it means to respond to crisis. When an acute crisis strikes, it creates an urgency to help real people, right now. However, […]
Read more
Sample testings of trained agents (DQN on Breakout, A3C on Pong, DoubleDQN on CartPole, continuous A3C on InvertedPendulum(MuJoCo)): Sample on-line plotting while training an A3C agent on Pong (with 16 learner processes): Sample loggings while training a DQN agent on CartPole (we use WARNING as the logging level currently to get rid of the INFO printouts from visdom):
Read more
Convolutional Neural Networks have become largely popular in image tasks such as image classification recently largely due to to Krizhevsky, et al. in their famous paper ImageNet Classification with Deep Convolutional Neural Networks. Famous models such as AlexNet, VGG-16, ResNet-50, etc. have scored state of the art results on image classfication datasets such as ImageNet and CIFAR-10. We present an application of CNN’s to the task of classifying trees by images of their leaves; specifically all 185 types of trees […]
Read more
New: Please check out contrastive-unpaired-translation (CUT), our new unpaired image-to-image translation model that enables fast and memory-efficient training. We provide PyTorch implementations for both unpaired and paired image-to-image translation. The code was written by Jun-Yan Zhu and Taesung Park, and supported by Tongzhou Wang. This PyTorch implementation produces results comparable to or better than our original Torch software. If you would like to reproduce the same results as in the papers, check out the original CycleGAN Torch and pix2pix Torch […]
Read more
@inproceedings{mnih2016asynchronous, title={Asynchronous methods for deep reinforcement learning}, author={Mnih, Volodymyr and Badia, Adria Puigdomenech and Mirza, Mehdi and Graves, Alex and Lillicrap, Timothy P and Harley, Tim and Silver, David and Kavukcuoglu, Koray}, booktitle={International Conference on Machine Learning}, year={2016}} This repository contains an implementation of Adavantage async Actor-Critic (A3C) in PyTorch based on the original paper by the authors and the PyTorch implementation by Ilya Kostrikov. A3C is the state-of-art Deep Reinforcement Learning method.
Read more