End-to-end speech-to-dialog-act recognition
Table 3: Comparison of segmentation error rates.
['Model', 'WER', 'SER', 'NSER']
As DA segmentation is performed on inputs which do not necessarily have the same length as the ground-truth, we define segment error rate (SER), which is calculated as the regularized sum of distances between each DA ending position in the ASR result and the closest position in the ground-truth transcript. Let G and P are sets of indices for a DA ending symbol in each sentence. The distance is calculated as follow. d(G,P)=12⎛⎝∑g∈Gminp∈P|g−p|+∑p∈Pming∈G|p−g|⎞⎠ ( 8) With this evaluation, d(G,P)=0 if and only if two texts have the same length and segments. We also assess the model in terms of the number of segments (NSER). The error rate is calculated by |N∗−N|N, where N and N∗ are numbers of DA segments of the ground-truth and the recognized segmentation, and does not consider the position of these segments. While NSER only evaluates the number of segments, SER takes into account the relative position of the DA ending labels. The unified model outperforms the baseline method by 4.22% (relatively 25%) in SER.