AI2D-RST: A multimodal corpus of 1000 primary school science diagrams
Table 3: Class-wise marginal κ scores for connectivity
['Connection', '[ITALIC] κ', 'z-score', 'p-value', 'Frequency in the entire corpus']
The source and target of each connection were highlighted in the AI2D layout segmentation and presented to the annotators, who were then asked to place the connection into one of four categories: directed, undirected, bidirectional or no connection. Measuring inter-annotator agreement returned a marginal κ of 0.878 and uniform κ of 0.916.