Automatically Identifying Complaints in Social Media
Table 8: Performance of models in Macro F1 on tweets from each domain.
['[BOLD] Domain', '[BOLD] In-Domain', '[BOLD] Pooling', '[BOLD] EasyAdapt']
Table 8 shows the model performance in macro-averaged F1 using the best performing feature set. Results show that, in all but one case, adding out-of-domain data helps predictive performance. The apparel domain is qualitatively very different from the others as a large number of complaints are about returns or the company not stocking items, hence leading to different features being important for prediction. Domain adaptation is beneficial the majority of domains, lowering performance on a single domain compared to data pooling. This highlights the differences in expressing complaints across domains. Overall, predictive performance is high across all domains, with the exception of transport.