SAC: Accelerating and Structuring Self-Attention via Sparse Adaptive Connection
Table 3: Summary of results in terms of classification accuracies on transductive tasks (Cora, Citeseer and Pubmed) or micro-averaged F1 score on inductive tasks (PPI). In the first column, we report the kind of data available to each method during training (X: features, A adjacency matrix, Y: labels).
['[BOLD] Available data', '[BOLD] Method', '[BOLD] Cora', '[BOLD] Citeseer', '[BOLD] Pubmed', '[BOLD] PPI']
We note that SAC achieves significant performance boosts over existing methods across all four datasets, i.e., outperforms our implemented GAT +1.8, +1.1, +0.7 and +1.1 respectively on Cora, Citeseer, Pubmed and PPI. The explanation for SAC’s advantage is as follows: graph node representation learning concerns about both label propagation and relatedness between nearby nodes in the vector space , the latter of which is what GCN handles. As verified in many recent works Liu et al. The attention edge prediction stage in SAC fosters information exchange between nodes that are not directly linked in graph but similar in terms of label propagation. SAC actually offers the probability in bridging the aspects, leading to better performances.