![]() Please find the evaluation code from here. We will use Test Suite Accuracy as our official evaluation metric for Spider, SParC, and CoSQL. Please check out a nice work from Google Research (including new Spider splits) for studying compositional generalization in semantic parsing! ![]() We open-sourced simple but SOTA/strong models for 21 tasks including text-to-SQL! Please check out our code in the UnifiedSKG repo!! Please check out our recent work UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Models. It injects GPT-3 Codex prompt API calls in programming languages! Please check out Binder demo, code, paper, and video on the Binder project site!! Please check out our recent work Binder: an easy but sota neural-symbolic built on GPT-3 Codex & SQL/Python interpreter. Please check out examples, data, and code on the DS-1000 project site!! Please check out our recent work DS-1000: A Natural and Reliable Benchmark for Data Science Code Generation. Related works: DS-1000, Binder, UnifiedSKG, multi-turn SParC and conversational CoSQL text-to-SQL tasks.ĭS-1000 Challenge ('22) Binder Framework (ICLR '23) UnifiedSKG Framework (EMNLP'22) SParC Challenge (ACL'19) CoSQL Challenge (EMNLP'19) Why we call it "Spider"? It is because our dataset is complex and cross-domain like a spider crawling across mutiple complex(with many foreign keys) nests(databases). To do well on it, systems must generalize well to not only new SQL queries but also new database schemas. In Spider 1.0, different complex SQL queries and databases appear in train and test sets. It consists of 10,181 questions and 5,693 unique complex SQL queries on 200 databases with multiple tables covering 138 different domains. The goal of the Spider challenge is to develop natural language interfaces to cross-domain databases. ![]() Spider is a large-scale complex and cross-domain semantic parsing and text-to-SQL dataset annotated by 11 Yale students.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |