Parallel Scheduled Sampling
Table 3: Empirical results on CIFAR-10. We use Frechet Inception Distance (FID) (Heusel et al., 2017) and Inception Score (IS) (Salimans et al., 2016) metrics to evaluate the quality of the samples from teacher-forcing and Parallel Scheduled Sampling. We provide upper-bound scores by computing the metric on the ground-truth data. Parallel Scheduled Sampling with a single pass and a mixing probability of 50% significantly decreases FID by 20% and increases IS by 13.8% compared to the baseline.
['Training', 'Mixing', 'Num', 'Warm-up', 'Schedule', 'FID (↓)', 'IS (↑)']
We train the 50 million parameter Image Transformer model for 200K steps in both the cases. We find that Parallel Scheduled Sampling with a single pass and a mixing probability of 50% significantly decreases FID by 20% and increases IS by 13.8% compared to the baseline. Similarly to our summarization experiment, it is runtime infeasible to apply Sequential Schedule Sampling here.