Structural Embedding of Syntactic Trees for Machine Comprehension
Table 2: Performance comparison on the official blind test set. Ensemble models are trained over the five single runs with the identical network and hyper-parameters.
['[BOLD] Method', '[BOLD] Single  [BOLD] EM', '[BOLD] Single  [BOLD] F1', '[BOLD] Ensemble  [BOLD] EM', '[BOLD] Ensemble  [BOLD] F1']
Moreover, we found that both SECT-LSTM and SEDT-LSTM have better performance than their CNN counterparts, which suggests that LSTM can more effectively preserve the syntactic information. As a result, we conducted further analysis of only SECT-LSTM and SEDT-LSTM models in the subsequent subsections and drop the suffix “-LSTM” for abbreviation. We built an ensemble model from the 5 single models for the baseline method BiDAF and our proposed methods SEPOS, SECT-LSTM, and SEDT-LSTM. The ensemble model choose the answer with the highest sum of confidence scores among the 5 single models for each question.