BERT Goes to Law School: Quantifying the Competitive Advantage of Access to Large Legal Corpora in Contract Understanding
Table 2: Baseline vs Pre-trained vs fine-tuned BERT with BERT unfreezing
['Metric (weighted)', 'Baseline', 'BERT', 'FT BERT (SM)', 'FT BERT (LG)']
We use 4 metrics to compare performance across various experiments: Matthews Correlation Coefficient, as well as Precision, Recall and F1 score weighted by class size. It’s clear that using pre-trained BERT, we’re able to achieve a significant performance lift compared to the base line. It is also clear that fine-tuning BERT on a domain-specific corpus noticeably improves this lift, even when the corpus size is small and we train for a short time.