Effective Use of Transformer Networks for Entity Tracking
Table 6: Model predictions from the document level entity first GPT model in 1049 cases of intermediate compositions. The model achieves only 51% accuracy in these cases.
['[EMPTY]', '0→0', '0→1', '1→0', '1→1']
Intermediate Compositions Formally, we pick the set of examples where the ground truth is a transition from 0→1 (not present to present) and the 1 is a “combined” case. The model achieves an accuracy of 51.1% on these bigrams, which is relatively low given the overall model performance. In the error cases, the model defaults to the 1→1 pattern indicative of the First Occ baseline.