Neural Embedding Allocation:Distributed Representations of Topic Models
Figure 1: The worst four topics produced by LDA, in terms of per-topic coherence score, and their corresponding NEA topics, with LDA trained on the NIPS corpus for K=7,000.
['LDA corresponds change cut exact coincides duplicates volatility trapping reading ters', 'NEA parameters important neural change results report cut multiple experiments minimizing', 'LDA symbolics addressing choice perturbing radii centered damping merits vax unexplored', 'NEA values case increase systems rate point feedback input reduces stage', 'LDA ryan learning bit inhibited nice automatica tucson infinitely stacked exceeded', 'NEA learning methods text space combined averaging area apply recognition bit', 'LDA paths close path make numbering channels rep scalars anism viously', 'NEA total paths global path time fixed function yields close computation']
In this case, NEA generated slightly more meaningful topics than LDA. We also identified the most improved topics based on the difference between per-topic coherence scores. We observe that these LDA topics were uninterpretable, and likely had very few words assigned to them. NEA tends to improve the quality of these “bad” topics, e.g. by replacing stop words (or words at the top of the dictionary) with more semantically related ones. In particular, we found that NEA gave the most improvement for topics with few words assigned to them (see and when K becomes large, the majority of topics have few assigned words (see As a result, NEA improves the quality of most of the topics.