Medication Regimen Extraction From Medical Conversations
Table 2: ROUGE-1 scores of baselines and models for the MR extraction task on the 10% test dataset. PT: using pretrained encoder; B: baseline; MD: Multi-decoder
['Models', 'Dosage F1', 'Frequency F1', 'Frequency Recall', 'Frequency Precision']
Embedding: (trained and provided by the authors). Pertaining Encoder: We trained the PGNet as a summarization task using the medical summaries and used the trained model to initialize the encoders (and the embeddings) of the corresponding QA models. We use a vocab size of 4073 words, derived from the training dataset with a frequency threshold of 30 for the task. We trained the models using Adagrad optimizer with a learning rate of 0.015, normal gradient clipping set at 2 and trained for around 150000 iterations (stopped using validation dataset). On the summarization task PGNet obtained ROUGE-1 F1 scores of 41.42 with ELMo and 39.15 with BERT embeddings. We compare the effects of pretraining the model in Table: Embeddings: This could be because we concatenated the numbers with a hyphen, and because ELMo uses character-level tokens it can learn the tagging better than BERT. Similar observations are found in recent literature. On the other hand, on Frequency extraction, without pretraining, ELMo’s performance lags by a big margin of ∼8.5 ROUGE-1 F1 compared to BERT-based embeddings. Other Variations: Considering various models’ performance (without pretraining) and the resource constraint, we choose ELMo and BERT embeddings to analyze the effects of pretraining the encoder. Both in terms of performance and the training speed, there is no clear winner between shared and multi-decoder approaches. Medication tagging and data augmentation increase the best-performing model’s ROUGE-1 F1 score by ∼1.5 for the Dosage extraction task.