Image-to-Markup Generation with Coarse-to-Fine Attention
Table 2: [Top] Main experimental results on the im2latex-100k dataset. Reports the BLEU score compared to the tokenized formulas (BLEU (tok)), BLEU score compared to the normalized formulas (BLEU (norm)), exact match accuracy, and exact match accuracy after deleting whitespace columns. All systems except Im2Tex-Tok are trained on normalized data. [Bottom] Results on the CROHME handwriting datasets. We list the best 4 systems in 2013 and 2014 competition: MyScript, U Valencia, TUAT, USP, and MyScript, UPV, U Nates, TUAT. All Im2Tex systems use out-of-domain synthetic data as well as the small given training set. *Note that the proprietary MyScript system uses a large corpus of private in-domain handwritten training data.
['[BOLD] Dataset', '[BOLD] Model', '[BOLD] Attention', '[BOLD] BLEU (tok)', '[BOLD] BLEU (norm)', '[BOLD] Match', '[BOLD] Match (-ws)']
The Infty system is able to do quite well in terms of text accuracy, but performs poorly on exact match image metrics. The poor results of the neural CTC system validate our expectation that the strict left-to-right order assumption is unsuitable in this case. Our reimplementation of image captioning Caption does better, pushing the number above 50%. Our standard attention system Im2Tex with RNN encoder increases this value above 75%, achieving high accuracy on this task. The LaTeX normalizer provides a few points of accuracy gain and achieves high normalized BLEU. This indicates that the decoder LM is able to learn well despite the ambiguities in real-world LaTeX.