Highlights Tokenization is a key (and mandatory) aspect of working with text data We’ll discuss the various nuances of tokenization, including how to handle Out-of-Vocabulary words (OOV)   Introduction Language is a thing of beauty. But mastering a new language from scratch is quite a daunting prospect. If you’ve ever picked up a language that wasn’t your mother tongue, you’ll relate to this! There are so many layers to peel off and syntaxes to consider – it’s quite a challenge. […]