Probabilistic Semantic Retrieval for Surveillance Videos with Activity Graphs
TABLE I: Area-Under-Curve (AUC) of precision-recall curves on VIRAT dataset with human annotated bounding boxes for Bag-of-Words approach (BoW [6]), Manually Specified Graph Matching (MSGM [5]), and our proposed approach.
['Query', 'BoW ', 'MSGM ', 'Proposed']
On human annotated data, where we assume no uncertainty at the object level, we can see that both MSGM and the proposed method significantly outperform BoW. The queries all include some level of structural constraints between objects, for example, there is an underlying distance constraint for the people, car and object involved in object deposit. In a cluttered surveillance video where multiple activities occur at the same time, when an algorithm attempts to solve for a bipartite matching between people, car and objects, while ignoring the global spatial relationships between them, unrelated agents from different activities could be chosen, resulting in low detection accuracy for BoW. This shows that global structural relationships rather than isolated object-level descriptors are important.