Pushing the bounds of dropout
Table 1: PTB training XEs with various dropout rate multipliers between deterministic and GMC. Observe the monotonic improvement in training fit when reducing the dropout rate at evaluation only.
['×0.0', '×0.1', '×0.2', '×0.3', '×0.4', '×0.5', '×0.6', '×0.7', '×0.8', '×0.9', '×1.0']
improves monotonically when reducing λ. Results on other datasets and with other power mean models are very similar. We call the union of the reduced dropout rate subvariants of all power mean family models the extended dropout family paramaterised by α,λ.