LayoutLM: Pre-training of Text and Layout for Document Image Understanding
Table 3. Different initialization methods for BASE and LARGE (Text + Layout, MVLM)
['[BOLD] Initialization', '[BOLD] Model', '[BOLD] Precision', '[BOLD] Recall', '[BOLD] F1']
Furthermore, we compare different initialization methods for the LayoutLM model including from scratch, BERT and RoBERTa. For the LARGE setting, the LayoutLMLARGE model initialized with RoBERTaLARGE further improve 1.3 points over the BERTLARGE model. We will pre-train more models with RoBERTa as the initialization in the future, especially for the LARGE settings.