Estimating the Rating of Reviewers Based on the Text
Table 1: Number of reviews of each movie
['Name', 'Reviews', '5 Star', '4 Star', '3 Star', '2 Star', '1 Star']
We used the reviews of eight films from Amazon . Around 65 percent of the reviews are 5 star and 20 percent of them are 1, 2 and 3 stars. To have more distinct classes, 4-star reviews were excluded from the dataset. For preprocessing, first, all the reviews were divided in two categories/classes as High (5Stars) and Low (1, 2, and 3 stars). We then removed the stop words and tokenized the sentences. The resulted dataset consisted of around 39,802 sentences and 307,138 words. Words in a collection of documents mostly follow the zip’s law, where the rare and common words are scattered on the end of two tails of the graph. To address this problem, we removed the words with less than 10 counts. The words with high counts will be downscaled in feature selection, using TF-IDF or information gain. More details can be found in the following section.