Graph-Aware Transformer: Is Attention All Graphs Need?
Table 2: MAE of 12 regression tasks in QM9 - the effect of pre-training
['[BOLD] Mode  [BOLD] Target', 'Multi-task DimeNet', 'Multi-task p-GRAT', 'Multi-task pp-GRAT', 'Single-task MPNN', 'Single-task DimeNet', 'Single-task p-GRAT', 'Single-task pp-GRAT']
For the single-task mode, pp-GRAT achieves new state-of-the-art performance on 4 out of 12 tasks. Also, in terms of the stdMAE, pp-GRAT obtains new state-of-the-art performance across 12 tasks, overall. These results are not achievable without the pre-trainings. In addition, the underlined numbers under p-GRAT model represent superior performance than DimeNet even though they are inferior than pp-GRAT. Lastly, by comparing the results of GRAT with p-GRAT and pp-GRAT, we can see the positive effect of the pre-trainings very clearly throughout all 12 tasks.