Tailoring and Evaluating the Wikipedia for in-Domain Comparable Corpora Extraction
Table 4: Mean N and standard deviation σN of the number of articles per domain for the WikiTailor model (top) and the IR-based model (bottom). We show five systems with different values for the two free parameters in both cases (cf. Section 4 for a description). Left-most numbers indicate the ranking of the edition in number of articles (cf. Table 1).
['[EMPTY]', '100-IR10  [ITALIC] N', '100-IR10  [ITALIC] σN', '100-IR100  [ITALIC] N', '100-IR100  [ITALIC] σN', '100-IRall  [ITALIC] N', '100-IRall  [ITALIC] σN', '50-IR10  [ITALIC] N', '50-IR10  [ITALIC] σN', '50-IR100  [ITALIC] N', '50-IR100  [ITALIC] σN']
The size for every system and language is a direct consequence of the aforementioned. Except for Arabic and German, the larger the edition, the larger the extracted collection of in-domain articles, but for small vocabularies differences among languages are less extreme. The loss in number of articles in English for small vocabularies respect to * -WTall is remarkable (from 1 M in 50-WTall to 50 k in 50-WT100). This is not the case for German (5 k vs. 3 k) although its initial vocabulary for 50-WTall was even larger than the English one. In general, IR retrieves larger collections than WT, up to the point that for queries with 100 terms and without any threshold for the relevance score (IRall) the extracted corpus can be almost the full Wikipedia. Also notice that the number of extracted articles in the IR models is proportional to the size of the collection instead of to the number of categories, as it happens with WT and small vocabularies (numbers on the left of the language in the table rank the editions according to the number of articles, the actual order in the table is regarding the number of categories). As expected, queries with less elements ( 50-IR* vs. 100-IR*) retrieve smaller collections. Some exceptions appear for Basque and Greek. This occurs when one does not look at the collection with all the hits (IRall) but at those recovering a percentage of the maximum score. Since the maximum score changes when using 100 query terms and 50 query terms, the same can happen for the number of elements.