The Right Tool for the Job: Matching Model and Instance Complexities
Table 2: Fine-tuning times (in minutes) of our model compared to the most accurate baseline: the standard BERT-large model with a single output layer.
['[BOLD] Dataset', '[BOLD] Training Time Ours', '[BOLD] Training Time Standard']
Fine-tuning BERT-large with our approach has a similar cost to fine-tuning the standard BERT-large model, with a single output layer. Our model is not slower to fine-tune in four out of five cases, and is even slightly faster in three of them.