Text as Environment: A Deep Reinforcement Learning Text Readability Assessment Model
Table 1. Model architecture details.
['[BOLD] Property', '[BOLD] Value']
As stated before, the input of the model is a list of five vectors carrying the GloVe and language model values of the corresponding words in the focus window. Thus, a 2D vector is fed to the model per each window, which is a raw representation of perceived words. To elicit useful features, a set of convolutional layers map the 2D input vector into a 1D feature vector, which can be used in the following layers of the DRL model to learn the Q values for the actions in the textual environment. The implemented convolutional encoder consists of four 2D convolutional layers. The sequential nature of the input text vector representation creates a need for memorizing the previously observed words and, as intended, determining the Q values of each action by integrating the information perceived throughout the revealed text. Consequently, a recurrent (LSTM) layer is utilized to capture and learn the temporal information in the extracted feature vectors. Dueling Q networks calculate the Q values of each action by independently calculating the state value and action advantage values. The Q value can be calculated by adding up the state value and advantage value. The benefit of dueling models is the detachment of prediction of the current state value from the value of taking a specific action, which is much useful in the current study as the careful judgment between the selection of a readability level in the current state or moving further through text is crucial. Each value (state value or advantage value) is calculated using separate dense layers, and the model takes a two-stream architecture after the LSTM layer. The products of these two streams are added up to form the Q values, which indicates the value of each readability level or move action.