Abstract
Table 4.2: CPC Model Training Summaries
['CPC model ID', 'number of epoch', 'model size', 'dev NCE loss', 'dev accuracy']
CDCK2 and CDCK5 are trained for 60 iterations, and CDCK6 is trained for 30 iterations due to time limitation. CDCK5 has around 1.8 million less model parameters than CDCK2 and CDCK6 because its GRU hidden dimension is 40, which is significantly smaller. Expectedly, due to the larger model size, CDCK2 and CDCK6 has smaller NCE losses L and higher positive sample prediction accuracies than CDCK5. Furthermore, CDCK6 attains higher prediction accuracies with half the training iterations, which suggests that the shared encoder approach is more powerful than the single autoregressive model approach.