Medication Regimen Extraction From Medical Conversations
Table 4: ROUGE-1 scores of the best performing models on the ASR (Google: G-STT and IBM: IBM-STT) and human written transcripts (HW) on the Dtest dataset for the MR extraction task. PT: using pretrained encoder; HS: Human segmentation; AS: Auto segmentation; MD: Multi-decoder; P: Precision; R: Recall
['Transcripts', 'Models', 'Dosage F1', 'Frequency F1', 'Frequency R', 'Frequency P']
Since the number of recognized medications in IBM-STT is only 95 compared to 725 (human written), we mainly consider the models’ performance on G-STT ’s transcripts (343). This shows that the models are robust to ASR variations discussed above. The lower performance compared to human transcripts is mainly due to incorrect recognition of Dosage and other medications in the same segments (changing the meaning of the text). By comparing the performance of the model on the human written transcripts with human (defined) segmentation and the same with auto segmentation, we see a 10 point drop in Dosage and 6 point drop in Frequency extraction tasks. This points out the need for more sophisticated segmentation algorithms.