Linear Ensembles of Word Embedding Models
Table 1: Final errors and the number of iterations until convergence for both SOLS and SOPP. The first column shows the embedding size.
['[BOLD] Dim', '[BOLD] SOLS  [BOLD] Error', '[BOLD] SOLS #  [BOLD] Iter', '[BOLD] SOPP  [BOLD] Error', '[BOLD] SOPP #  [BOLD] Iter']
During training the iterative algorithm was run until the convergence threshold th=0.001 was reached. It can be seen that the convergence with SOPP took significantly fewer iterations than with SOLS. This difference is probably due to two aspects: 1) SOPP has the additional orthogonality constraint which reduces the space of feasible solutions; 2) although SOLS uses the exact analytical solutions for the least squares problem, the final solution for Y does not move directly to the direction pointed to by the analytical solutions due to the variance rescaling.