Neural Embedding Allocation:Distributed Representations of Topic Models
Figure 2: The four topics that were most improved by NEA over the original LDA topic, in terms of the difference between per-topic coherence score, with LDA trained on the Reuters-150 corpus for K=7,000.
['LDA share pittsburgh aa aaa ab abandon abandoned abc abdul aberrational', 'NEA International common share pittsburgh general agreement tender market june dividend', 'LDA tonnes yr aa aaa ab abandon abandoned abc abdul aberrational', 'NEA announced tonnes addition asked accounts shares surplus secretary heavy held', 'LDA blah aa aaa ab abandon abandoned abc abdul aberrational abide', 'NEA blah company account advantage acquisitions loss proposed considered announced base', 'LDA dlrs aa aaa ab abandon abandoned abc abdul aberrational abide', 'NEA debt canadian today canada decline competitive conditions dlrs price week']
In this case, NEA generated slightly more meaningful topics than LDA. We also identified the most improved topics based on the difference between per-topic coherence scores. We observe that these LDA topics were uninterpretable, and likely had very few words assigned to them. NEA tends to improve the quality of these “bad” topics, e.g. by replacing stop words (or words at the top of the dictionary) with more semantically related ones. In particular, we found that NEA gave the most improvement for topics with few words assigned to them (see and when K becomes large, the majority of topics have few assigned words (see As a result, NEA improves the quality of most of the topics.