Linking emotions to behaviors through deep transfer learning
Table 5: Behavior binary classification accuracy in percentage for reduced context-dependent behavior recognition from emotion-informed embeddings. Bold numbers represent the best performing system.
['[EMPTY]', '[BOLD] Average', 'Acceptance', 'Blame', 'Positivity', 'Negativity', 'Sadness']
For most behaviors, we observe a better classification as the receptive field size increases, especially in the range from 4 seconds to 32 seconds, demonstrating a need for longer observations for behaviors. Furthermore, the results suggest different behaviors require different observation window length to be quantified, which is also observed by sandeepJournal2019 using lexical analysis. By comparing results with different receptive window sizes, we can indirectly obtain the appropriate behavior analysis window size for each behavior code. This is in good agreement with the behavior descriptions. For example, behaviors of acceptance, positivity and blame often require relatively longer observations since they relate to understanding and respect for partner’s views, positive negotiation, and accusation respectively, which often require multiple turns in a dialog and context to be captured. On the other hand, sadness which can be expressed via emitting a long, deep, audible breath, and is also related to short-term expression of unhappy affect, can be captured with shorter windows. We use PUR to sense the relative improvement from E-BP based context-dependent and E-BP based reduced context-dependent models respectively, to the baseline B-BP based context-dependent model. The larger value of PUR suggests the clear improvement of behavior prediction.