A Critical Analysis of Biased Parsers in Unsupervised Parsing
Table 1: F1 scores using the same evaluation setup as Shen et al. (2019). Numbers in the model name give the layer s was extracted from. R/L indicates which variant of the parsing algorithm was used. Results with † are reproduced from Table 2 of Shen et al. (2019).
['ON-LSTM-1†', 'R', 'WSJ10 35.2', 'PTB23 20.0']
Results on the 7422 sentence WSJ10 set (consisting of sentences from the PTB of up to 10 words) and the PTB23 set (consisting of all sentences from §23, the traditional PTB test set for supervised parse evaluation) are shown in Tab. et al. We note three things. First, the F1 scores of the two models are comparable; however, the LSTM baseline is slightly worse on WSJ10. Second, whereas the ON-LSTM model requires that a single layer be selected to compute the parser’s scoring criterion (since each layer will have a different expected forget depth), the unordered nature of LSTM forget gates means our baseline can use gates from all layers jointly. Third, the L-variant of the parser is substantially worse.