Extracting Action Sequences from Texts Based on Deep Reinforcement Learning
Table 1: Datasets used in our experiments
['[EMPTY]', '[BOLD] WHS', '[BOLD] CT', '[BOLD] WHG']
We conducted experiments on three datasets, i.e., “Microsoft Windows Help and Support” In our task, a single text with n optional items or n exclusive pairs can generate more than 2n potential label sequences (i.e. each item of them can be extracted or not be extracted). Especially, we observe that n is larger than 30 in some texts of our datasets, which means more than 1 billion sequences will be generated. We thus restrict n≤8 ( no more than 28 label sequences) to generate reasonable number of sequences. We set the input dimension to be (500×100) for action names and (100×150) for action arguments, the number of feature-maps to be 32. We used 0.25 dropout on the concatenated max pooling outputs and exploited a 256 dimensional fully-connected layer before the final two dimensional outputs. We set the replay memory Ω=100000, discount factor γ=0.9. We varied ρ from 0.05 to 0.95 with the interval of 0.05 and found the best value is 0.80 (that is why we set ρ=0.80 in the experiment). Before online training, we pre-train an initial model of EASDRL by combining all labeled texts of WHS, CT and WHG, with 30 labeled texts of WHG for testing. The accuracy of this initial model is low since it is domain-independent. We “invited” humans to provide feedbacks for these 80 texts (with an average of 5 texts for each human). When a human finishes the job assigned to him, we update our model (as well as the baseline model). We compare EASDRL to the best offline-trained baseline BLCC-2. We can see that EASDRL outperforms BLCC-2 significantly, which demonstrates the effectiveness of our reinforcement learning framework.