Notes on Andrej Karpathy’s makemore videos. Part 2.

Below are my notes on Andrej Karpathy’s video tutorial on introduction to language modeling. You can watch Andrej’s original presentation on youtube. In Part 1, we worked on a bigram model that takes into account only the local context of a word. This approach is

November 4, 2022 Natural Language Processing, Python

Similarity to Probability — Part I: Visual Word Embedding for OCR Post Correction

In this post, I will revisit in more detail our previous work that uses human-inspired likelihood revision or similarity to probability [Blok et al. 2003] to re-rank or score any word or text fragment based on the semantic relation to an external context. We will use the most popular Semantic Similarity pre-trained model (e.g., w2v, GloVe, fasttext, etc.) to compute these relations.

November 4, 2022 Python

Good old-fashioned code optimization never goes out of style

Sometimes, making your Python data processing software faster doesn’t require libraries like NumPy or Pandas, or specialized techniques like vectorization. In fact, if you’re doing string processing, libraries like Pandas won’t help. Pushing calculation down to a faster implementation is just one way to speed up software. Another way to get faster results is to remove code that is redundant, repetitive, superfluous, needless, or otherwise does unnecessary work. The fastest software, after all, is software that doesn’t run at all. […]