WiSeBE: Window-based Sentence Boundary Evaluation
Table 5: WiSeBE evaluation
['Transcript', 'System', '[ITALIC] F1 [ITALIC] mean', '[ITALIC] F1 [ITALIC] RW', '[ITALIC] RGAR', '[ITALIC] WiSeBE']
The Agreement Ratio (RGAR) between references oscillates between 0.525 for v8 and 0.767 for v5. The lower the RGAR, the bigger the penalization WiSeBE will give to the final score. A good example is S2 for transcript v4 where F1RW reaches a value of 0.800, but after considering RGAR the WiSeBE score falls to 0.462. While it is true that they show a linear correlation, WiSeBE may produce a different system ranking than F1mean given the integral multi-reference principle it follows. However, what we consider the most profitable about WiSeBE is the twofold inclusion of all available references it performs. First, the construction of RW to provide a more inclusive reference against to whom be evaluated and then, the computation of RGAR, which scales the result depending of the agreement between references.