Topic Identification for Speech without ASRThis work was partially supported by DARPA LORELEI Grant No̱ HR0011-15-2-0024, NSF Grant No̱ CRI-1513128, and IARPA Contract No̱ 2012-12050800010.
Table 2: Multi-label topic ID average precision on LORELEI languages, with the number of speech segments in parentheses.
['Dataset', 'Feature', 'Model', 'Overall', 'In-domain topics']
5.2.1 Experimental setup We further evaluate our topic ID performance on the speech corpora of three languages released by the DARPA LORELEI (Low Resource Languages for Emergent Incidents) Program. For each language there are a number of audio speech files, and each speech file is cut into segments of various lengths (up to 120 seconds). Each speech segment is seen as either in-domain or out-of-domain. In-domain data is defined as any speech segment relating to an incident or incidents, and in-domain data will fall into a set of domain-specific categories; these categories are known as situation types, or in-domain topics. There are 11 situation types: “Civil Unrest or Wide-spread Crime”, “Elections and Politics”, “Evacuation”, “Food Supply”, “Urgent Rescue”, “Utilities, Energy, or Sanitation”, “Infrastructure”, “Medical Assistance”, “Shelter”, “Terrorism or other Extreme Violence”, and “Water Supply”. We consider “Out-of-domain” as the 12th topic label, so each speech segment either corresponds to one or multiple in-domain topics, or is “Out-of-domain”. For each configuration, only a single 10-fold CV result is reported, since we observe less variance in results here than in Switchboard. We have 16.5 hours in-domain data and 8.5 hours out-of-domain data for Turkish, 2.9 and 13.2 hours for Uzbek, and 7.7 and 7.2 hours for Mandarin. Also we use mini-batch size 30 and fix the training epochs as 100. All CNNs use word2vec pre-training.