Learning to Segment Actions from Observation and Narration
Table 4: Comparison between the semi-Markov and hidden semi-Markov models (SMM and HSMM) with the Markov and hidden Markov (MM and HMM) models, which ablate the semi-Markov’s duration model.
['[EMPTY]', 'All Label', 'Step Label', 'Step', 'Seq.']
We examine the effect of the (hidden) semi-Markov model’s Poisson duration model by comparing to a (hidden) Markov model (HMM in the unsupervised/weakly-supervised settings, or MM in the supervised setting). We use the model as described in Sec. We then train as described in Sec. While this does away with explicit modeling of duration, the transition distribution still allows the model to learn expected durations for each region type by implicitly parameterizing a geometric distribution over region length. We observe that results are overall very similar, with the exceptions that removing the duration model decreases performance substantially on all metrics in the discriminative supervised setting, and increases performance on step label accuracy and step recall in the constrained unsupervised setting (HSMM+Ord+Narr and HMM+Ord+Narr). This suggests that the HMM transition distribution is able to model region duration as well as the HSMM’s explicit duration model, or that duration overall plays a small role in modeling in most settings relative to the importance of the features.