Framework to build a niche dictionary for text mining
Having the right dictionary is at the heart of any text mining analysis.
Dictionary for text mining can be compared to maps while travelling in a new city. The more precise and accurate maps you use, the faster you reach to the destination. On the other hand, a wrong or incomplete map can end up confusing the traveler.
Use of dictionary helps us convert unstructured text into structured data. The more precise dictionary you have for the analysis, the more accurate will be the analysis or prediction. Imagine, you are doing a sentiment analysis on twitter and you wish to find how positive or negative are the tweets for a subject. Many tweets contain the word “awsum” and few of the tweets contain the word “awful”. Hence, in totality, the sentiment is positive about the subject. But, if our dictionary does not contain the word “awsum”, the sentences with the word “awsum” will not be tagged. Hence, our sentiment analysis will tell that the sentiment is negative instead of positive. The inaccuracy in