Contents
Table 2: The fifteen most common words in hep-th titles, in raw and clean data. A graphical representation of this table, in terms of “word-clouds”, is shown in Fig. 3.
['Raw Data Rank', 'Raw Data Word', 'Raw Data Count', 'Raw Data Rank', 'Raw Data Word', 'Raw Data Count', 'Cleaned Data Rank', 'Cleaned Data Word', 'Cleaned Data Count', 'Cleaned Data Rank', 'Cleaned Data Word', 'Cleaned Data Count']
Having raw and cleaned data at hand, we can begin our analysis with a simple frequency analysis of mono-grams and certain n-grams. To understand the effect of our data cleaning process, we provide the counts for both the raw data, and the cleaned data. Note, for example, that the count for a word such as “theory” drops significantly after cleaning. In the clean data, the count on the word “theory” excludes all bi-grams involving this word that appear at least 50 times in hep-th titles, such as “gauge-theory”, which appears 2014 times.