Exploratory Analysis of Highly Heterogeneous Document Collections
Table 2: Critical topics found and effective usage patterns.
['[BOLD] Topic', '[BOLD] Docs', '[BOLD] Facets Employed']
Identified Critical Topics. Of course, numerous non-critical topics within the document collection were also discovered (some of which were of a personal, non-work-related nature). As for the critical topics, there were 89 documents found pertaining to Technology-Z and 232 documents pertaining to Technology-Y (including duplicate files). Through a subsequent exhaustive manual review of the case, we estimate that no additional information on military critical technologies of interest was present on this case. Using our facet-based system, most documents for these two critical topics were identified in less than an hour (and in some cases only minutes). By contrast, domain experts informed us that cases of this size typically require hours or days of analysis to produce similar results, which is consistent with our experience during the manual review. Several facets were identified as highly effective in identifying these topics when used in combination with each other. We discuss these next. Effective Usage Patterns Discovered. The combination labeled as Method A indicates that the “Topic Cluster” facet populated by LDA was first explored and used to filter the search results. The “Top Discovered Keywords” facet (populated by KERA), then, was used to identify document sets related to the critical technology. The combination labeled as Method B indicates that the “Report Type Filter” was used to locate documents pertaining to Technical Information followed again by the “Top Discovered Keywords” facet. Finally, for Method C, the “Top Folders” facet (i.e., our Location Facet) was used to filter the search results, with the KERA-populated tag cloud being used to quickly assess documents within folders. Notice that multiple facet combinations often exist to locate the same set of critical documents. This highlights a significant advantage to our multi-faceted system: topics are more likely to be discovered by users when more paths lead towards them. As shown in the table, Method A and Method B were employed heavily to find both Technology-Y and Technology-Z. Method C was only used for Technology-Z, not Technology-Y. Since the files for Technology-Y were scattered across many directories, the “Top Folders” facet was not as useful. As can be seen, the “Top Discovered Keywords” facet populated by KERA played an indispensable role in all three methods, as it allowed for quick exploration and assessment of the document collection. It was particularly useful as a complement to the “Topic Cluster” facet, as we now explain.