Improving Segmentation for Technical Support Problems
Table 4: Results comparing the models using various pre-trained embeddings. The en data source is the downloaded pre-trained ELMo model. For simple concatenation, we present the results for the best model at each n combinations of data sources. For example, when concatenating any 2 datasources, the en + config combination gives the best performance.
['[BOLD] Model', '[BOLD] P', '[BOLD] R', '[BOLD] F1']
For the simple concatenation method, we present results for the best n-way combination of embeddings from different data sources, for each n (1, 2, 3, and 4). We find that combining embeddings from multiple language models trained on different data sources considerably outperforms using embeddings from a single pre-trained model (using both the naive concatenation and CDME). This is an artifact of the support problems containing large sections of non-natural language text. We also find that contextual weighting does better than a simple concatenation.