Bidirectional Context-Aware Hierarchical Attention Network for Document Understanding
Table 3: Average runtime per iteration (in ms), with TensorFlow and a GeForce RTX 2080 Ti GPU, on Amazon. Gating runtimes are within brackets.
['[BOLD] CAHAN-SUM-Σ LR', '[BOLD] CAHAN-SUM-Σ BI', '[BOLD] CAHAN-RNN LR', '[BOLD] CAHAN-RNN BI', '[BOLD] HAN']
However, with CAHAN-SUM, the representations of the preceding/following sentences are now required before computing the current sentence representation. With CAHAN-RNN, one even has to wait until the level 2 RNN has processed the preceding/following sentence vectors before being able to encode the current sentence. Therefore, the sentence encoding process, which was parallelizable with basic HAN due to independence, has now become a sequential process. This is why in practice, we observe slightly greater runtime increases, in the range 5-22% (