SBERT-WK: A Sentence Embedding Method by Dissecting BERT-based Word Models
TABLE VII: Inference time comparison of InferSent, BERT, XLNET, SBERT and SBERT-WK. Data are collected from 5 trails.
['Model', 'CPU (ms)', 'GPU (ms)']
We evaluate the inference speed against the STSB datasets. For fair comparison, the batch size is set to 1. Both results are reported. On the other hand, we report CPU results of SBERT-WK only. With CPU, the total inference time of SBERT-WK (QR) is 8.59 ms (overhead) plus 168.67ms (SBERT baseline). As compared with the baseline BERT model, the overhead is about 5%. SVD computation is slightly slower than QR factorization.