Legal Question Answering using Ranking SVM and Deep Convolutional Neural Network
Table 7: Phase 2 results with different models
['[BOLD] Method', '[BOLD] Accuracy%']
In CNN models, we found that these models are sensitive to the initial value of parameters. Different values lead to large difference in results (± 5%). Therefore, each model was run n times (n=10) and we chose the best-optimized parameters against the validation set. Also, CNN with LSI produces a better result as opposed to CNN with TF-IDF. We suspect that this is because TF-IDF vector is large but quite sparse (most values are zero), therefore it increases the number of parameters in CNN and consequently makes the model to be overfitted easily.