NLSC: Unrestricted Natural Language-based Service Composition through Sentence Embeddings
Table III: Scalability
['[BOLD] Services', '[BOLD] Wiki Model  [BOLD] Harm.', '[BOLD] Wiki Model  [BOLD] StdDev', '[BOLD] Wiki Model  [BOLD] TPS', '[BOLD] Wiki Model  [BOLD] Rate', '[BOLD] Twitter Model  [BOLD] Harm.', '[BOLD] Twitter Model  [BOLD] StdDev', '[BOLD] Twitter Model  [BOLD] TPS', '[BOLD] Twitter Model  [BOLD] Rate']
Generally speaking, the system’s performance is not affected when progressively scaled the system up to thousands of services, actually, one can observe an improvement rate that decreases inversely proportional the number of services. In particular, we observed that the time response per service decreased from 5 to 50 to 500 to 5,000 services, though not in the same proportions. For instance, for the Wiki model, when using only 5 services the composition took 0.022 ms per service, while taking 0.005 ms per service when using 50 services, which corresponds to a 77.21% of improvement. The reason why the performance of the system is not affected when increasing the number of services is that the composition model keeps some information in memory which can be reused while constructing the composite service, for instance, when resolving and mapping the service method “searchFlights”, arguments “fromOrigin”, “toDestination”, “departureDate”, “arrivalDate”, etc. are stored in the working memory. Later on, when resolving service method “bookFlight”, almost all the arguments can be easily retrieved from the working memory without requiring to perform data-type disambiguation and firing compositional rules. Likewise, service method “searchHotels” and “bookHotel” can retrieve information from the working memory such as “toDestination”, “departureDate”, and “arrivalDate”. Regarding the two pre-trained models (Wikipedia and Twitter) we observed that using Twitter took more time for the system to perform the semantic matching based on cosine similarity (mostly due to the considerable difference in size of both models), however, NLSC increased its accuracy in almost 28%, which leads to the conclusion that there should be a trade-off between the accuracy of the mappings between the user request and the service description, and the system’s response time (performance) which will increase when the model is bigger in size.