Schema2QA: High-Quality and Low-Cost Q&A Agents for the Structured Web
Table 5. Size of training, dev, and test sets.
['# of properties', '[BOLD] Restaurants # of properties', '[BOLD] Restaurants 25', '[BOLD] People 13', '[BOLD] Movies 16', '[BOLD] Books 15', '[BOLD] Music 19']
We experiment on five Schema.org domains: restaurant, people, movies, books, and music. We scrape Schema.org metadata from Yelp, LinkedIn, IMDb, Goodreads, and Last.fm, for each domain, respectively. We include (1) all properties of these domains in Schema.org for which data is available, and (2) useful properties of types which no data is needed for training, such as enum and boolean type. We end up writing about 100 annotations per domain, many of which result from error analysis of the validation data. No real data are used in training. Furthermore, only less than 2% of the synthesized data are paraphrased by crowdsource workers, resulting in a low data acquisition cost. Realistic data, however, are used for validation and testing, as they have been shown to be significantly more challenging and meaningful than testing with paraphrases (