Category: Big data

Beginner, Big data, Business Analytics, Data Mining, NLP, Technique, Text, Unstructured Data Leave a comment

Step by step guide to extract insights from free text (unstructured data)

Text Mining is one of the most complex analysis in the industry of analytics. The reason for this is that, while doing text mining, we deal with unstructured data. We do not have clearly defined observation and variables (rows and columns). Hence, for doing any kind of analytics, you need to first convert this unstructured data into a structured dataset and then proceed with normal modelling framework. The additional step of converting an unstructured data into a structured format is […]

Read more
Big data, Business Analytics, Intermediate, NLP, Technique Leave a comment

Information Retrieval System explained in simple terms!

Introduction While searching for things over internet, I always wondered, what kind of algorithms might be running behind these search engines which provide us with the most relevant information? How do they decide which result to show for which set of search keywords. This might be a no brainer for a few people, but definitely an interesting problem for some of the best brains around the world. To find the answer, I read every guide, tutorial, learning material that came my way. Eventually, I learnt […]

Read more
Beginner, Big data, Data Science, NLP, Python, Technique, Text, Unstructured Data Leave a comment

Steps for effective text data cleaning (with case study using Python)

Introduction   The days when one would get data in tabulated spreadsheets are truly behind us. A moment of silence for the data residing in the spreadsheet pockets. Today, more than 80% of the data is unstructured – it is either present in data silos or scattered around the digital archives. Data is being produced as we speak – from every conversation we make in the social media to every content generated from news sources. In order to produce any […]

Read more
Big data, Business Analytics, Intermediate, NLP, Technique Leave a comment

Framework to build a niche dictionary for text mining

Having the right dictionary is at the heart of any text mining analysis. Dictionary for text mining can be compared to maps while travelling in a new city. The more precise and accurate maps you use, the faster you reach to the destination. On the other hand, a wrong or incomplete map can end up confusing the traveler. Use of dictionary helps us convert unstructured text into structured data. The more precise dictionary you have for the analysis, the more accurate […]

Read more
Big data, Business Analytics, Data Exploration, Intermediate, NLP, Project, R, Sports, Text, Unstructured Data Leave a comment

Who is the world cheering for? 2014 FIFA WC winner predicted using Twitter feed (in R)

Sports are filled with emotions! Cheering of audience, reactions to events on various media channels are some of the factors, which make a huge impact on the mind of the players. If people support you, your chances to win are greatly enhanced. Live example of this fact, are the statistics of Indian cricket team playing in India and abroad. The win rate of Indian cricket team in India is approximately twice the win rate abroad. Football is again a game driven largely by emotions. […]

Read more
Banking, Big data, Business Analytics, Data Visualization, Intermediate, NLP, R, Technique, Text, Unstructured Data Leave a comment

Build a word cloud using text mining tools of R

 This is how a word cloud of our entire website looks like! A word cloud is a graphical representation of frequently used words in a collection of text files. The height of each word in this picture is an indication of frequency of occurrence of the word in the entire text. By the end of this article, you will be able to make a word cloud using R on any given set of text files. Such diagrams are very useful when doing […]

Read more
Big data, Business Analytics, Intermediate, NLP, R, Technique, Text, Unstructured Data Leave a comment

Hacks to perform faster Text Mining in R

Introduction Data science demands versatility. Move away from your regular methods, challenge your ways of working, explore new ways of doing things more efficiently. On reminiscing about my old days, my initial years in data science, I had also got trapped by this devil of ‘complacency’. At one point, I was not challenging myself enough. I wasn’t  experimenting with the ways of doing work. I accepted the things as they were, until I realized ‘Complacency is a state of mind […]

Read more
Beginner, Big data, Data Engineering, Database, NoSQL Leave a comment

5 Popular NoSQL Databases Every Data Science Professional Should Know About

Overview NoSQL databases are ubiquitous in the industry – a data scientist is expected to be familiar with these databases Here, we will see what is a NoSQL database and why you should learn about it We will also look at the features of 5 different NoSQL databases   Introduction Here’s a piece of advice I wish someone had given me when I was starting out in data science – learn as much as you can about working with databases. […]

Read more