Algorithms of the automatic text analysis for the Russian language for solving applied problems using TAWT framework
The paper reviews the existing tools of linguistic text analysis. The authors identified the problem of selecting suitable tools, adapting them to work with texts in Russian, and integrating them with each other. This makes it difficult to use these tools both for research purposes and makes it almost impossible to use them in applied systems. The paper describes the new open-source Java framework TAWT, developed by the authors, that provides convenient ready-made software tools and data structures for the main stages of text analysis in the Russian language that meet the modern requirements for performance, reliability, project build engine, etc. The paper proposes an approach to the intellectualization of information systems and business processes using software tools for linguistic text analysis to implement algorithms for automating the processing of technical documentation, which is the scientific novelty of the work. The application of the developed TAWT framework allowed implementing algorithms for automating the analysis of the text in Russian in terms of the analysis of technical documentation: validating the structure of documents and the list of abbreviations used in the document, searching for similar documents, and obtaining their brief content. All this will simplify the technical documentation management, speed up the process of its preparation, and upgrade the quality of the created documents. The algorithms of automatic text analysis implemented by the TAWT framework have been successfully applied in application systems for searching friends by interests in social networks, identifying fraudulent messages, working with thematic synonyms, as well as for creating software tools for selecting keywords from texts in Russian and referencing them.