Visualizing NLP annotations for Crowdsourcing
Table 2: For parsing, compare time with and without the visualization toolkit
['[EMPTY]', 'Participant', 'W1', 'W2', 'W3', 'W4', 'W5', 'W6']
We compare the time efficiency and accuracy to generate training data with and without the toolkit. For clustering, time efficiency is defined as the seconds worker used for each correct co-reference cluster. Accuracy is computed by purity. Let Nij be the number of mentions in cluster i that belong to entity j, and let Nj=∑iNij. Then the purity of a cluster is pi=maxjpij. The overall purity is ∑iNiNpi. Purity range between 0 (bad) and 1 (good). It is clear that using our toolkit dramatically improve the efficiency and accuracy. Purity ranges between 0 (bad) and 1 (good). It is clear that using our toolkit dramatically improve the efficiency and accuracy.