Query-Reduction Networksfor Question Answering
Figure 3: (top) bAbI QA dataset (Weston et al., 2016) visualization of update and reset gates in QRN ‘2r’ model (bottom two) bAbI dialog and DSTC2 dialog dataset (Bordes and Weston, 2016) visualization of update and reset gates in QRN ‘2r’ model. Note that the stories can have as many as 800+ sentences; we only show part of them here. More visualizations are shown in Figure 4 (bAbI QA) and Figure 5 (dialog datasets).
['Task 15: Deduction', 'Layer 1  [ITALIC] z1', 'Layer 1 → [ITALIC] r1', 'Layer 1 ← [ITALIC] r1', 'Layer 2  [ITALIC] z2']
In QA Task 2 example (top left), we observe high update gate values in the first layer on facts that state who has the apple, and in the second layer, the high update gate values are on those that inform where that person went to. We also observe that the forward reset gate at t=2 in the first layer (→r12) is low, which is signifying that apple no more belongs to Sandra. In dialog Task 3 (bottom left), the model is able to infer that three restaurants are already recommended so that it can recommend another one. In dialog Task 6 (bottom), the model focuses on the sentences containing Spanish, and does not concentrate much on other facts such as I don’t care.