Predicting Human Trustfulness from Facebook Language
Table 2: Comparing the language model performance on 3-question trust score vs. 10-question trust score. Pearson rdis is dissattenuated Pearson r and MSE is the mean squared error.
['train label', 'test label', '[BOLD] Pearson  [ITALIC] rdis', '[BOLD] MSE']
Comparing the second and third settings, we see the result of testing on 10-question trust score outperforms the 3-question trust score by 0.07 margin in dissattenuated Pearson r and MSE by a margin of 0.11. To further understand why 10-question trust seems to be easier to predict, we calculate the variance for both 3-question and 10-question trust, yielding σ2=0.85 and σ2=0.72 respectively. This suggests that 10-question trust has less noise than 3-question trust. Due to these results, in all of the following experiments we only train on 3-question trust labels and test on 10-question trust labels.