Alleviating Sequence Information Loss with Data Overlapping and Prime Batch Sizes
Table 6: Token order imbalance (TOI) comparison for the IEMOCAP dataset on a SER task using angry, happy, neutral and sad classes for 60 epochs using the Transformer model.
['Experiment', 'WA (60 epochs)', 'UA (60 epochs)']
The previously described Transformer model is used in these experiments. the standard deviation computed for different P-values of Alleviated TOI (P). We want to highlight the fact that the goal of these experiments is to show the direct contribution of the Alleviated TOI technique for a different model. For this reason we use a smaller version of the Transformer in order to reduce the computational cost. We believe that with a more expressive model and more repetitions, the proposed method may further improve the results. This is in accordance with our hypothesis that a higher TOI ratio (P−1)/P improves the results.