LOCuST is a large-scale text-to-SQL benchmark on real-world statistical data provided by Statistics Netherlands.
Our benchmark contains over 2,000 English and Dutch tables covering 22 statistical domains. In concrete, our datasets provide the following:
2,244 tables containing real-world statistics;
table schemas ranging from small (< 5 columns) to very large (> 1,200 columns);
a coverage of 22 statistical domains;
2,567 manually annotated complex questions and answers;
an extensive metadata knowledge graph accompanying the tables, containing labels, descriptions, units, and other associated metadata for covered concepts.
Our benchmark consists of three tasks: (1) table retrieval; (2) query generation, where tables are given; and (3) end-to-end QA,
which is a combination of table retrieval and query generation. All tasks are run on the same question-answer data.
