Dynamic Knowledge Graph Construction for Zero-shot Commonsense Question Answering
Table 7: Percentile thresholds κ for predicting an emotion for the ℂ𝕆𝕄𝔼𝕋 - CA and ℂ𝕆𝕄𝔼𝕋 - CGA models. Thresholds for ℂ𝕆𝕄𝔼𝕋 - CGA tend to be higher because the score is the sum of all graph level scores
['[BOLD] Dimension', '[BOLD] CA  [ITALIC] κ', '[BOLD] CGA  [ITALIC] κ']
where ϕnens is the score from the model for dimension n and κn is the threshold for that dimension. Normally, we could tune these decision thresholds on the validation set, but this would violate the zero-shot setting. Instead, decision thresholds are chosen by producing a cumulative distribution function over model scores for the validation examples. Then, we select the threshold as the score at the percentile of the positive label distribution (i.e., if the joy emotion is present for 20% of examples, we set the score at the 20th percentile of the CDF as the threshold).