Welcome back! Hope you had a great week. We have a new leader on the SuperGLUE benchmark with a new Ernie model from Baidu comprising of 10 billion parameters trained on on a 4TB corpus. FYI, human baseline was already beat by Microsoft’s DeBERTa model at the beginning of the year… time for a new SuperSuperGLUE benchmark??? Paper BTW, if you are still interested in GitHub’s CoPilot, I stumbled upon the Codex paper this week: Paper DeepMind’s Perceiver transformer allows […]

Sometimes… cool things happen. A new chatbot from Facebook AI was released this Friday with remarkable features. This chatbot, BlenderBot 2.0, is an improvement on their previous bot from last year. The bot has better long-term memory and can search the internet for information during conversation! This is a convenient improvement versus traditional bots since information is not statically “memorized” but instead has the option to be dynamic and “staying up to date” via the internet. 🤯 I’ve recently tested […]

Welcome back! This week’s Cypher will be a bit shorter than usual, it was a slow week in NLP land. But first, I want to update you on the BlenderBot 2.0 situation. On last week’s Cypher, the last hurdle to overcome with the instantiation of blenderbot inference was the search server (which gives the bot the ability to comb the web to answer factoid type of questions). Well we finally have a search server repo to work with! Thank you […]

Machine Learning education content from aggregating 1,300 questions from an ML Course. Pretty cool site with very simple and intuitive answers to technical ML questions. If you are looking for more math heavy stuff go elsewhere. Here’s an example: What do dropout layers do? Dropout layers throw things away. Now you would be asking, why would I want my model to throw data away? It turns out that throwing things away when training a model can drastically improve a model’s […]

Nova melting hypothetical planet | Bonestell NATURAL LANGUAGE PROCESSING (NLP) NEWSLETTER o̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿unicode suckso̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿ Way back in February of 2020, someone Twitter posted they had FOIA’d the NSA aka National Security Agency. This actor, by the name ‘cupcake’ was able to retrieve a 400-page printout of their COMP 3321 training course (😂). It was OCR’d and uploaded to the cloud totaling 118MB of absolute FOIA madness of Python learning material courtesy of the Men in Black by the way of Fort […]

Hey Welcome Back! A flood of EMNLP 2021 papers came in this week so today’s newsletter should be loads of fun! 😋 But first, a meme search engine: An article on The Gradient had an interesting take on NLU. It describes how a NNs’ capacity for NLU inference is inherently bounded to the background knowledge it knows (which is usually highly limited relative to a human). Although I would add a bit more nuance to this by sharing that this […]

Welcome back! We have a long newsletter this week as many new NLP repos were published as tech nerds return from their Summer vacation. 😁 This week I’ll add close to 150 new NLP repos to the NLP Index. So stay tuned for this update, it will drop this week. just explore… Embeddinghub is a database built for machine learning embeddings. It is built with four goals in mind. Store embeddings durably and with high availability Allow for approximate nearest […]