[
Table 3: Clinical Finding Scores. The precision scores for each of the labels are listed and aggregated into the overall precision scores. Recall scores are shown in the last two rows. Macro denotes averaging the numbers in the table directly and micro accounts for class prevalence.
['[EMPTY]', 'Label', 'Count', '1-NN', 'S&T', 'SA&T', 'TieNet', 'Ours (NLG)', 'Ours (CCR)', 'Ours (full)']
Note that the labeling process generates discrete binary label as opposed to predicting continuous probabilities, and as such we are unable to obtain discriminative metrics such as the Area Under the Receiver Operator Characteristic (AUROC) or the Area Under the Precision-Recall Curve (AUPRC). Precision-wise, Ours (CCR) achieves the highest overall scores including macro-average and micro-average. The runner-up is Ours (full) model, which additionally considers language fluency. Note that the macro- metrics can be quite noisy as the per-class metric can be dependent on just a few examples. Many entries in the table are zeros, as they never yield positive predictions and we regard them as zeros to penalize such behavior. Regarding the recall metric, we are able to see a substantial drop in Ours (CCR) and Ours (full) as a result of optimizing for accuracy. Accuracy is closely associated with precision but overpursuing it might harm in terms of recall. Rare combinations of tokens in the corpus can be easily neglected, resulting in predictions of mostly major classes. A formerly practicing clinician co-author reviewed a larger subset of our generated reports manually. They drew several conclusions. TieNet instead misses some positive findings in such cases. Second, there are significant issues in all generated reports, regardless of the source model, which include the description of supportive lines and tubes, as well as lung lesions. For example, TieNet is prone to generate nasogastric tube mentions while our model tends to mention tracheostomy or endotracheal tube, and yet both models have difficulty identifying some specific lines such as chest tube or PICC line. Similarly, both systems do not generate the sentence with positive lung parenchymal findings correctly.