Syntactic and Semantic Features For Code-Switching Factored Language Models
TABLE X: Summary: PPL of different FLMs, compared to and interpolated with the baseline 3-gram model
['Model', 'PPL dev', 'PPL eval']
The main idea behind the backoff graph is to first drop either the oldest feature (C(-2)) or the open class word (OC(-1)) in order to continue with a model similar to the second best model (FLM Brown word clusters + POS). Afterwards, the results of all possible backoff paths are combined using their average. In the case of backoff to one or two factors including the penultimate Brown word cluster (C(-2)), general backoff is applied (indicated by dashed lines in the figure). A possible explanation is that the penultimate Brown word cluster may be necessary for the prediction of the next word in some but not in all cases. The backoff graph has been investigated experimentally and chosen because of superior results in terms of PPL on the dev set compared to other backoff strategies. In addition, it provides results and weights for interpolating the FLMs with the baseline 3-gram language model.