1 Introduction
Table 11: Worst-case absolute percentage error to the oracle downstream instability when using embedding distance measures as the selection criteria for dimension and precision parameters. Smallest errors are bolded.
['[ITALIC] Downstream Task  [ITALIC] Embedding Algorithm', 'SST-2 CBOW', 'SST-2 GloVe', 'SST-2 MC', 'Subj CBOW', 'Subj GloVe', 'Subj MC', 'CoNLL-2003 CBOW', 'CoNLL-2003 GloVe', 'CoNLL-2003 MC']
We also evaluate the the worst-case performance of the embedding distance measures when used as a selection criterion for dimension-precision parameters. First, on the easier task of choosing the more stable dimension-precision pair out of two choices, we define the worst-case performance as the maximum increase in instability that may occur by using the embedding distance measure to choose the dimension-precision parameters (rather than the ground truth choice). On the more challenging task of choosing the most stable dimension-precision pair under a memory budget, we define the worst-case performance as the worst-case absolute percentage error to the oracle parameters under a given memory budget.