Improving Mild Cognitive Impairment Prediction via Reinforcement Learning and Dialogue Simulation
Table 2. Performance of basline vs. RL on MCI prediction on 10 stratified shuffle splits
['[BOLD] Model', '[BOLD] Feature', '[BOLD] AUC', '[BOLD] Sen.', '[BOLD] Specificity', '[BOLD] F1-Score']
Using the original LIWC representation, we were able to recover close to the 72.5% AUC baseline from the original paper using SVM and LR classifiers. The encoder was pre-trained on the BookCorpus dataset, which is a large collection of novels pertaining to numerous literary genres. The advantage of pre-training on this dataset is that BookCorpus contains an abundant number of turn-based dialogues between various character types. These conversations capture a wide range of conversational response styles, idiosyncrasies and temperaments. For this reason, we choose this classifier for the RL portion of our pipeline. As a baseline reference, we also included performance using raw word count distributions for all models. Here, the turns notation RL(T =t) denote the number of questions the agent is allowed to ask before a prediction is made from the simulated user responses. It is important to note that turn 0 was set to greetings by default and was not counted toward the conversation.