AutoDiscern: Rating the Quality of Online Health Information with Hierarchical Encoder Attention-based Neural Networks
Table 3: Comparison of performance metrics for the HEA BioBERT architecture at 80% and 100% coverage. Coverage refers to the percent of articles the model makes a prediction for (as opposed to abstaining from making a prediction when the model has a confidence below the Threshold). The Precision, Recall, and Accuracy scores reflect the accuracy of the model on the resulting 80% of predicted articles.
['Question', 'Coverage', 'Threshold', 'Precision', 'Recall', 'Accuracy']
We explored the relationship between model’s coverage, accuracy, and confidence (prediction probability threshold) focusing on the BioBERT model (see The trend is that as the model’s coverage decreases, the higher is the confidence (i.e. outcome probability) and the accuracy of the model.