Data Extraction from Charts via Single Deep Neural Network
Table 2: Accuracy of bar chart prediction in Simulated (Simul), FigureQA (FigQA), annotated (Annot) and Excel bar chart data sets
['Object', 'Simul', 'FigQA', 'Annot', 'Excel']
To further evaluate the performance of the bar chart data extraction model, we propose the following evaluation metric. We use a 4-tuple to represent the prediction of each bar in a bar chart: (x-tick label, value, lower y-tick label, upper y-tick label). In the 4-tuple, x-tick label is matched to the bar by the object matching branch (x-tick label can be either below the bar or in the legend), value is predicted using the top boundary location of the bar. The introduction of the lower and upper y-tick labels makes the evaluation more reliable since the predicted value can slightly differ from the ground truth. The lower y-tick label is the y-tick label immediately below the actual value of the bar (and similarly defined upper y-tick label). Besides the 4-tuple, each bar chart has its prediction of title, and x- and y-axis labels. , y-axis labels in the first 4 rows represent the percentage of correctly predicted chart instances in each test data set. In all elements consisting of text we count true positive only if all words match correctly. We define true positive charts (”ALL”) as predicted with correct title, x-, y-axis labels and all 4-tuples with less than 1% error in value. The following 11 rows are based on each tuple prediction. For example, the true positive instance for “tuple 10% err” means the tuple is predicted with a less than 10% error in value and the remaining 3 textual elements are completely correct. The error is defined as: Error=|Valuepred−ValueGT||ValueGT| (8) where ValueGT is the ground truth value of each bar.