How Far is It From Samples to Test Time?
Table 4: Error reductions over random baselines on Standard (original) splits, if available, Bootstrap splits, Random Length splits resulting from a sentence length-based separation, Rare Words splits based on word frequency.
['[BOLD] Task', '[BOLD] Model', '[BOLD] Splits  [BOLD] Standard', '[BOLD] Splits  [BOLD] Bootstrap', '[BOLD] Splits  [BOLD] Random Length', '[BOLD] Splits  [BOLD] Rare Words']
While bootstrap resampling leads to slightly lower error reduction than cross-validation we decided to report the latter in the main part of this paper, because it is a more wide-spread way to randomly split datasets. Random Length results are comparable to standard splits results. The split based on word frequency (Rare Words) leads to considerable drop in both tasks. However, it is not as strong as the drop of the heuristic split (length threshold) in the main part of the paper.