Enabling Language Models to Fill in the Blanks
Table 4: Mixture infilling PPL of all models (a mixture of all granularities).
['[EMPTY]', 'Sto', 'Abs', 'Lyr']
This reflects general infilling ability across a wide variety of granularities (and hence lengths). Unlike our other quantitative evaluations, there may be multiple variable-length spans missing from each example in this evaluation. Results are similar to that of sentence infilling. Namely, that ILM outperforms LM and LM-Rev and is similar to LM-All despite using much less memory.