Learning Articulated Motion Models from Visual and Lingual Signals
TABLE I: Overall Performance of Our Framework
['[EMPTY]', 'Object', '[ITALIC] N∗', '[ITALIC] Nv', '[ITALIC] Sv', 'Vision-Only  [ITALIC] Sh', 'Vision-Only  [ITALIC] Ss', 'Our Framework  [ITALIC] Sh', 'Our Framework  [ITALIC] Ss', 'Our Framework  [ITALIC] eparam']
The table indicates the ground-truth number of parts for each object (N∗), a list of the number of parts identified using visual trajectory clustering for each demonstration (Nv), and the fraction of videos for which the correct number of parts was identified (Sv). We then present the hard (Sh) and soft (Ss) model selection rates for our method as well as for the baseline. Our method bests the vision-only baseline in estimating the full kinematic graph for five of the eight objects, matching its performance on the remaining three objects. Specifically, our framework yields accurate estimates of the full kinematic graphs for six more demonstrations than the vision-only baseline, two more for single-part objects and four more for multi-part objects, corresponding to a 21% absolute improvement. Similarly, we are able to estimate a valid sub-graph of the ground-truth kinematic graph for all 28 demonstrations, whereas the vision-only baseline fails to estimate valid sub-graphs for ten of the videos (two for single-part and eight for multi-part objects), corresponding to a 36% absolute improvement. Consequently, neither method estimated the full kinematic graph for any video, however our framework was able to exploit lingual cues to yield accurate sub-graph estimates for each video. Similarly, clustering failed to identify the three parts that comprise the monitor in all but two videos, for which our method then estimated the correct kinematic graph (and an accurate sub-graph for the remaining four videos).