Spider 2.0-AIFunc: Extending Real-World Text-to-SQL to AI-Native SQL Workflows

Jul 16, 2026

Upload by Riley

Major cloud data platforms expose large language model capabilities as native SQL functions, letting analysts perform classification, filtering, sentiment analysis, extraction, similarity search, and aggregation inside standard SQL queries. Existing text-to-SQL benchmarks evaluate only conventional SQL and do not test the ability to generate AI-native SQL. Spider 2.0-AIFunc introduces 465 verified instances over 125 real-world databases, covering six Snowflake AI function types, built via an agent pipeline with stability-checked execution and evaluation across ten state-of-the-art models.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages23

Reading Time58 min

Abstract

1 Introduction

Motivation for AI-native SQL workflows

Example: sentiment and complaint categorization with AI functions

arXiv :2607 .06229v 1 [ cs .CL] 7 Jul 2026 Spider 2.0-AIFunc: Extending Real-World Text-to-SQL to AINative SQL Workflows Tianyang Liu1 Canwen Xu2, Fangyu Lei3, Nikki Lijing Kuang2, Jixuan Chen1, Tao Yu3, Julian McAuley1, Zhewei Yao2, Yuxiong He2 1UC San Diego 2Snowflake AI Research 3University of Hong Kong {til040,[jmcauley}@ucsd.edu {canwen.xu](jmcauley}@ucsd.edu {canwen.xu) ,zhewei.yao,[yuxiong.he}@snowflake.com](yuxiong.he}@snowflake.com) Abstract Major cloud data platforms now expose large language model capabilities as native SQL functions, enabling analysts to perform classification, filtering, sentiment analysis, extraction, similarity search, and aggregation within ordinary SQL queries. Yet existing text-to-SQL benchmarks evaluate only conventional SQL and provide no signal on whether models can generate such AI-native SQL. We introduce Spider 2.0-AIFunc, a benchmark of 465 verified instances across 125 real-world databases covering six types of AI functions on the Snowflake platform. Starting from an existing enterprise text-to-SQL benchmark, we construct Spider 2.0-AIFunc through an agent-based pipeline that rewrites source tasks into AI-native form, simultaneously transforming target queries and refining natural language instructions to make the intended AI-native solution explicit and reduce ambiguity. All instances pass a multi-round repeated execution protocol across temporally separated windows to confirm result stability before release. Evaluating ten state-of-the-art language models, we find that the strongest proprietary models reach 67–70% execution accuracy while the best open-source model achieves 58.1%, a gap driven primarily by errors in predicate specification, schema grounding, and AI function parameterization. Agent frameworks designed for traditional text-to-SQL challenges, such as schema retrieval and relevant table selection, do not transfer effectively to AI-native SQL: a minimal agent setup consistently matches or outperforms more elaborate alternatives, suggesting that the strategies these frameworks employ are less critical in this setting. 1 Introduction Much of enterprise data analysis begins with questions that are easy to ask in natural language but awkward to express in traditional SQL: Which support tickets describe billing issues? Which survey responses mention churn risk? Which product reviews are negative, and what are customers complaining about? Consider the last case. A data analyst has a table of product reviews stored in a cloud data warehouse and wants to analyze their sentiment and categorize the negative ones by complaint type. Until recently, answering this kind of question meant extracting the text from the database, running sentiment analysis and classification models in a separate environment, and loading the results back. Today, on platforms like Snowflake, this can be done directly in SQL using AI functions such as AI_SENTIMENT and AI_CLASSIFY: SELECT review_id, AI_SENTIMENT(review_text) AS sentiment, AI_CLASSIFY ( review_text,[ ' shipping ' , ' quality ' , ' service ' , ' pricing ' ] ):labels AS complaint_type FROM product_reviews; ∗ Work done during Tianyang Liu’s internship at Snowflake. Figure 1: An example of transforming a traditional SQL task into an AI-native SQL instance in SPIDER 2.0-AIFUNC. The original instruction and gold SQL are rewritten to incorporate a Snowflake Cortex AI function (AI_SIMILARITY) . Highlighted spans indicate the injected AI requirement and the corresponding SQL transformation. Such functions can be combined with standard SQL aggregations, filters, and joins, enabling analysts to express semantic operations over unstructured text within ordinary SQL queries (Snowflake, 2026) . Snowflake is not alone in this direction. BigQuery (Google Cloud, 2026), Databricks (Databricks, 2026), and other major cloud platforms have all introduced SQL-callable AI functions, enabling a broad spectrum of semantic operations, fr

FAQs

What problem does Spider 2.0-AIFunc address in existing text-to-SQL work?

Existing benchmarks target conventional SQL only and provide no signal on whether models can generate AI-native SQL that includes AI function calls.

How is Spider 2.0-AIFunc constructed from prior benchmarks?

It rewrites Spider2-Snow instances into AI-native SQL by injecting Snowflake Cortex AI functions, transforming the target SQL and refining natural-language instructions, including handling multiple gold SQLs when present.

Why do agent frameworks for traditional text-to-SQL transfer poorly to AI-native SQL?

The document reports that minimal agent setups match or outperform more elaborate alternatives, indicating that prior strategies like schema retrieval and table selection are less critical for AI-native SQL generation.

Jul 16, 2026

Upload by Riley