Step by step guide to extract insights from free text (unstructured data)

Text Mining is one of the most complex analysis in the industry of analytics. The reason for this is that, while doing text mining, we deal with unstructured data. We do not have clearly defined observation and variables (rows and columns). Hence, for doing any kind of analytics, you need to first convert this unstructured data into a structured dataset and then proceed with normal modelling framework. The additional step of converting an unstructured data into a structured format is […]

Read more

Build Text Categorization Model with Spark NLP

Overview Setting up John Snow labs Spark-NLP on AWS EMR and using the library to perform a simple text categorization of BBC articles. Introduction Natural Language Processing is one of the important processes for data science teams across the globe. With ever-growing data, most of the organizations have already moved to big data platforms like Apache Hadoop and cloud offerings like AWS, Azure, and GCP. These platforms are more than capable of handling    

Read more

Framework to build a niche dictionary for text mining

Having the right dictionary is at the heart of any text mining analysis. Dictionary for text mining can be compared to maps while travelling in a new city. The more precise and accurate maps you use, the faster you reach to the destination. On the other hand, a wrong or incomplete map can end up confusing the traveler. Use of dictionary helps us convert unstructured text into structured data. The more precise dictionary you have for the analysis, the more accurate […]

Read more

Knowledge Graph – A Powerful Data Science Technique to Mine Information from Text (with Python code)

Overview Knowledge graphs are one of the most fascinating concepts in data science Learn how to build a knowledge graph to mine information from Wikipedia pages You will be working hands-on in Python to build a knowledge graph using the popular spaCy library   Introduction Lionel Messi needs no introduction. Even folks who don’t follow football have heard about the brilliance of one of the greatest players to have graced the sport. Here’s his Wikipedia page: Quite a lot of […]

Read more

Quick Introduction to Bag-of-Words (BoW) and TF-IDF for Creating Features from Text

The Challenge of Making Machines Understand Text “Language is a wonderful medium of communication” You and I would have understood that sentence in a fraction of a second. But machines simply cannot process text data in raw form. They need us to break down the text into a numerical format that’s easily readable by the machine (the idea behind Natural Language Processing!). This is where the concepts of Bag-of-Words (BoW) and TF-IDF come into play. Both BoW and TF-IDF are […]

Read more

How to Use Texthero to Prepare a Text-based Dataset for Your NLP Project

Introduction Natural Language Processing (NLP) is one of the most important fields of study and research in today’s world. It has many applications in the business sector such as chatbots, sentiment analysis, and document classification. Preprocessing and representing text is one of the trickiest and most annoying parts of working on an NLP project. Text-based datasets can be incredibly thorny and difficult to preprocess. But fortunately, the latest Python package called Texthero can help you solve these challenges. What is […]

Read more