SOFTWARE ARCHITECTURE OF HIGH-PRODUCTION COMPLEXES FOR text of the news PROCESSING FOR SOLVING THE PROBLEMS OF DATA MINING
The article proposes the architecture of the software for processing collections of news text messages, as well as the corresponding composition and structure of the information system. The system is a few steps of obtaining and processing information, functioning on the basis of a hybrid computing cluster. Each stage of receiving, processing and storing information in a generalized information system is represented by a microservice as a separate program unit. At the same time, the possibility of using different technology stacks for each service is emphasized so that properly selected specialized solutions increase the efficiency and quality of the result, and the shortcomings of the classical micro-service architecture are offset by the internal heterogeneity of the micro-services expressed in the form of flexible modularization. The essence of the proposed approach is the application of the principle of pipeline concurrency based on a microservice architecture with dynamic service boundaries.