Creative Procedural-Knowledge Extraction From Web Design Tutorials
Table 5: BLEU scores for the usage summarization task evaluated in the validation and testing sets. The best performance in either set is bolded. The naming schema of different algorithms: [number of layers]-layer-[whether or not the attention mechanism Bahdanau et al. (2014) is applied].
['dropout', 'validation 0', 'validation 0.2', 'validation 0.5', 'testing 0', 'testing 0.2', 'testing 0.5']
An usage summarization module takes a raw sentence as the input and generates a command usage summary. A natural model that can accomplish this task is the sequence-to-sequence model Sutskever et al. Then a seperate RNN-based decoder takes the representation as the input and sequentially generates a list of words as the summary. We experiment the Neural Machine Translation (NMT) model Bahdanau et al. (2) different dropout rates, and (3) the number of recurrent layers. To evaluate summarization performances, sentences that contain at least one action (43,582 out of 94,022 sentences satisfy the requirement) are randomly divided into a training set (23,582 sentences), a validation set (10,000 sentences), and a testing set (10,000 sentences); and each sentence corresponds to one or more reference summaries. NMT models are trained on the training set (batch size: 128, iterations: 100,000) using the Adam optimizer Kingma and Ba Finally, models’ performances are reported using the testing BLEU scores, as shown in Table.