KnowBias: A Novel AI Method to Detect Polarity in Online Content
Table 1: TEPC test results across various encoder suites for four datasets obtained by 80-20 train-test split. Twitter Political is the 20% split from the TEPC training data collected up to November 2018. Twitter Concatenated is the Twitter Political set plus tweets taken from the general stream collected up to November 2018 (i.e., the 20% test split from the neutral detector’s training data), where political tweets are labeled as ”biased” and noise tweets are labeled as ”neutral.” Reddit Political is data from four highly polarized Reddit subreddits collected up to May 2018. Cross-domain Twitter-Reddit data is a combination of the Twitter Political and Reddit Political data. A DNN classifier sourced from Scikit-Learn [21] was used in all cases.
['[EMPTY]', 'Wiki Words ', 'Universal Sentence Encoder Small ', 'LUSE ', 'Max-Pool  Wiki Words', 'Norm1-128 ']
This was not entirely surprising. We did not anticipate that this method will perform well in domain adaptation-reliant test cases (i.e., long-form articles) as it relies on the assumption that test data is similar to the Twitter training data. Instead, this method serves only as a baseline to which we can compare an improved approach for domain adaptation scenarios. Our goal is to allow for polarity prediction on long-form articles. Given that degradation occurs when neutral sentences are augmented, it is clear that filtering out these neutral sentences can improve accuracy. This strategy can only be effective if we can detect neutral sentences easily. Surprisingly, testing on the Twitter However, wiki-word embeddings (WWE) are significantly faster in terms of runtime and memory use, since they are word-based and ignore punctuation, while being comparable in performance to other encoder suites. To validate prediction accuracy, a smaller study was run, using politically knowledgeable users to crowdsource annotations for a set of tweets. Respondents were asked to classify as ”strongly conservative,” ”slightly conservative,” ”neutral (centrist) or apolitical,” ” slightly liberal,” or ”strongly liberal” as well as to state their personal political affiliation on the same spectrum. 79 users responded to the surveys, with a nearly balanced set among different political affiliations. We did not expect our accuracy to substantially change on the Twitter data between the TEPC and two-step methods, and indeed this is the case with no improvement in accuracy.