Contents
Table 4: Number of unique words appearing at least N times in hep-th titles. The 16,105 words with at least two appearances were utilized as a training set for Word2vecfor the purposes of the current section.
['Number of Unique Words Appearing at Least  [ITALIC] N Times  [ITALIC] N=1', 'Number of Unique Words Appearing at Least  [ITALIC] N Times  [ITALIC] N=2', 'Number of Unique Words Appearing at Least  [ITALIC] N Times  [ITALIC] N=3', 'Number of Unique Words Appearing at Least  [ITALIC] N Times  [ITALIC] N=4', 'Number of Unique Words Appearing at Least  [ITALIC] N Times  [ITALIC] N=5', 'Number of Unique Words Appearing at Least  [ITALIC] N Times  [ITALIC] N=6', 'Number of Unique Words Appearing at Least  [ITALIC] N Times  [ITALIC] N=7', 'Number of Unique Words Appearing at Least  [ITALIC] N Times  [ITALIC] N=8', 'Number of Unique Words Appearing at Least  [ITALIC] N Times  [ITALIC] N=9', 'Number of Unique Words Appearing at Least  [ITALIC] N Times  [ITALIC] N=10']
Having suitably prepared a clean dataset, we then trained Word2vec on the collection of titles in hep-th. Given the typically small size of titles in academic papers, we chose a context window of length 5. To minimize the tendency of the neural network to focus on outliers, such as words that very rarely appear, we dropped all words that appear less than twice in the data set for the purpose of training. Finally, we follow standard practice in the literature by setting N=400 neurons for the hidden layer, using the CBOW model.