Fingerspelling recognition in the wild with iterative visual attention
Table 3: Results on ChicagoFSWild/test and ChicagoFSWild+/test. Black: trained on ChicagoFSWild/train; Green: trained on ChicagoFSWild/train + ChicagoFSWild+/train.
['[BOLD] Method', '[BOLD] ChicagoFSWild', '[BOLD] ChicagoFSWild+']
Results on test All of these use some form of attention (standard or our iterative approach) and a language model. Training on additional data Finally, we report the effect of extending the training data with the ChicagoFSWild+/train set, increasing the number of training sequences from 5,455 to 55,856. The crowdsourced annotations in ChicagoFSWild+ may be noisier, but they are much more plentiful. In addition, the crowdsourced training data includes two annotations of each sequence, which can be seen as a form of natural data augmentation. The hand detector could become less accurate due to possible domain discrepancy between ChicagoFSWild (on which it was trained) and ChicagoFSWild+. In contrast, our model replacing the off-the-shelf hand detector with an iterative attention-based “detector” is not influenced by such a discrepancy.