Words are not Equal: Graded Weighting Model for building Composite Document Vectors
Table 3: Accuracies on our newly released 700-Movie Review Dataset
['[BOLD] Method', '[BOLD] Feature Selection', '[BOLD] Accuracy']
Dimensionality Reduction is the process of reducing the number of random variables in such a way that the remaining variables effectively reproduce most of the variability of the dataset. The reason for using such techniques is because of the curse of dimensionality which is a phenomena that occurs in high-dimension but doesn’t occur in low-dimension. With ANOVA-F, we selected around 4k features but with PCA, this number was just 50. So, the low accuracy with PCA can be attributed to the fact that we may have lost some important features in low dimension. Also, PCA cannot work with size of dimension d >size of learning set. This sharp decrease in accuracy in both cases happens because ANOVA-F selects features with larger variance across group and thus reduces noise to a larger extent whereas PCA reduces angular variance which is not effective in this case due to the distribution of data points in high-dimensional space.