American Sign Language fingerspelling recognition from video: Methods for unrestricted recognition and signer-independence
Table 3.2: Letter error rates (%) for different settings of SCRF and DNN training in the signer-adapted case. Details are given in Section 3.5.1.
['Signer', 'SCRF + DNNs trained from scratch 1', 'SCRF + DNNs trained from scratch 2', 'SCRF + DNNs trained from scratch 3', 'SCRF + DNNs trained from scratch 4', 'SCRF + DNNs trained from scratch  [BOLD] Mean', 'Sig.-indep. SCRF, DNNs trained from scratch 1', 'Sig.-indep. SCRF, DNNs trained from scratch 2', 'Sig.-indep. SCRF, DNNs trained from scratch 3', 'Sig.-indep. SCRF, DNNs trained from scratch 4', 'Sig.-indep. SCRF, DNNs trained from scratch  [BOLD] Mean', 'Sig.-indep. model w/ fine-tuned DNN 1', 'Sig.-indep. model w/ fine-tuned DNN 2', 'Sig.-indep. model w/ fine-tuned DNN 3', 'Sig.-indep. model w/ fine-tuned DNN 4', 'Sig.-indep. model w/ fine-tuned DNN  [BOLD] Mean']
In this section we consider alternatives to the adaptation setting we have chosen – adapting the DNNs while using sequence models (HMMs/SCRFs) trained only on signer-independent data. We fix the model to a first-pass SCRF and the adaptation data to 20% of the test signer’s data annotated with ground-truth peak labels. In this setting, we consider two alternative ways of using the adaptation data: (1) using the adaptation data from the test signer to train both the DNNs and sequence model from scratch, ignoring the signer-independent training set; and (2) training the DNNs from scratch on the adaptation data, but using the SCRF trained on the training signers. We compare these options with our best results using the signer-independent SCRF and DNNs fine-tuned on the adaptation data. Frame error rates of DNNs trained from scratch using 20% of the test signer’s However, training the SCRF on the training signers but DNNs from scratch on the adaptation data (option (2) above) improves performance further. However, neither of these outperforms our previous best approach of signer-independent SCRFs plus DNNs fine-tuned on the adaptation data. It may be due to fact that small improvement of DNNs can effect letter recognition performance more.