Distilled Wasserstein Learning forWord Embedding and Topic Modeling
Table 1: Admission classification accuracy (%) for various methods.
['Word Feature', 'Doc. Feature', 'Metric', 'Dim.', 'Mortality 1-NN', 'Mortality 5-NN', 'Adm. Type 1-NN', 'Adm. Type 5-NN']
Our DWL method is superior to its competitors on classification accuracy. Besides this encouraging result, we also observe two interesting and important phenomena. First, for our DWL method the model trained from scratch has comparable performance to that fine-tuned from Word2Vec’s embeddings, which means that our method is robust to initialization when exploring clustering structure of admissions. Second, compared with measuring Wasserstein distance between documents, representing the documents by the average pooling of embeddings and measuring their Euclidean distance obtains comparable results. Considering the fact that measuring Euclidean distance has much lower complexity than measuring Wasserstein distance, this phenomenon implies that although our DWL method is time-consuming in the training phase, the trained models can be easily deployed for large-scale data in the testing phase.