Query-Centric Optimization of AI Workflows via Approximate Query Processing and Proxy Models

Jul 16, 2026

Upload by Logic

Many modern AI workflows, from LLM post-training pipelines to agentic reasoning, can be written as declarative queries where an expensive predicate is evaluated by a large model or reward function. A query-centric formulation enables classical database methods—approximate query processing and proxy-model filtering—to cut the number of costly invocations without changing the underlying models. Strategy AQP progressively samples, maintains running aggregates with confidence bounds, and stops early under an error threshold. Strategy PM trains a lightweight CPU decision tree to pre-filter with high-confidence predictions, routing only uncertain cases to the expensive oracle model. Evaluations on TPC-DS and real LLM pipelines show large reductions in oracle calls with bounded accuracy loss.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages12

Reading Time30 min

Abstract

Introduction

Example 1: LLM Post-Training as a Query

Example 2: Agentic Reasoning as Declarative Filtering

Query-Centric Optimization of AI Workflows via Approximate Query Processing and Proxy Models Huayi Wang Georgia Institute of Technology Atlanta, USA [hwang762@gatech.edu](hwang762@gatech.edu) Jun Xu Georgia Institute of Technology Atlanta, USA [jx@cc.gatech.edu](jx@cc.gatech.edu) Gromit Yeuk-Yin Chan Adobe Research San Jose, USA[ychan@adobe.com](ychan@adobe.com) arXiv :2607 .00254v1 [ cs .DB] 30 Jun 2026 ABSTRACT Many modern AI workflows—ranging from LLM post-training pipelines to agentic reasoning tasks—can be expressed as declarative queries whose expensive predicate is evaluated by a large model or reward function. We propose a query-centric formulation of these workflows and show that classical database techniques, namely approximate query processing (AQP) and proxy-model (PM) based filtering, can substantially reduce the number of expensive model invocations without requiring changes to the underlying models or pipelines. Our first strategy treats the workflow as an online aggregation problem: it progressively samples records, maintains a running aggregate estimate with a confidence interval, and terminates early once the interval stabilizes, accepting the estimate when it falls within a user-specified error bound. Our second strategy trains a lightweight, CPU-resident decision tree on a small set of oracle-labeled examples and uses it to pre-filter records whose outcome can be predicted with high confidence, routing only uncertain records to the expensive model. We evaluate both strategies on TPC-DS aggregate queries and on real LLM post-training pipelines including math reasoning, general instruction following, and code generation. On TPC-DS, Strategy AQP keeps aggregate error under 10% while reaching its adaptive stopping point at 10–15% of oracle calls under balanced distributions—an 85–90% reduction—and Strategy PM reduces oracle calls by 60–70% on natural-label workloads. On LLM pipelines, Strategy AQP reaches its adaptive stopping point at 20–50% of oracle calls with less than 5% accuracy loss on the structured math and code tasks (open-ended instruction following, scored by a reward model, shows a larger but bounded reduction), and Strategy PM reduces reward-model scoring time by up to 19× on structured tasks with less than 10% accuracy loss. PVLDB Reference Format: Huayi Wang, Jun Xu, and Gromit Yeuk-Yin Chan. Query-Centric Optimization of AI Workflows. PVLDB, 14(1): XXX-XXX, 2020 . doi:XX.XX/XXX.XX PVLDB Artifact Availability: The source code, data, and/or other artifacts have been made available at [https://github.com/Why1221/Query-Centric-AI-optimization](https://github.com/Why1221/Query-Centric-AI-optimization). This work is licensed under the Creative Commons BY-NC-ND 4.0 International License. Visit [https://creativecommons.org/licenses/by-nc-nd/4.0/ to view a copy of](https://creativecommons.org/licenses/by-nc-nd/4.0/ to view a copy of)[ ](https://creativecommons.org/licenses/by-nc-nd/4.0/ to view a copy of)[this license. For any use beyond those covered by this license](this license. For any use beyond those covered by this license), [obtain permission by](obtain permission by)[emailing info@vldb.org. Copyright](emailing info@vldb.org. Copyright) is held by the owner/author(s). Publication rights licensed to the VLDB Endowment. Proceedings of the VLDB Endowment, Vol. 14, No. 1 ISSN 2150-8097 . doi:XX.XX/XXX.XX 1 INTRODUCTION The rapid growth of expensive Large Language Model (LLM) computation inAI workflows [4, 9]—from training foundation models to invoking them hundreds of times in agentic task pipelines [3, 20]—has created new opportunities to optimize latency and cost. More importantly, many of the current AI-centric workflows can be expressed as database queries. Consider the following two examples: Example 1.1. A recurring pattern in modern LLM post-training pipelines — explicitly documented in both the Llama 2 and Llama 3 technical reports [7, 26] — is what ca

FAQs

What is the core idea of query-centric optimization for AI workflows?

Transform workflows into declarative queries whose expensive predicates are evaluated by a large model or reward function, then optimize those predicates using database techniques without modifying the models or pipelines.

How does Strategy AQP reduce expensive model invocations?

It treats the workflow as an online aggregation task, progressively samples records, keeps a running aggregate estimate with a confidence interval, and terminates early when the interval stabilizes within a user-specified error bound.

How does Strategy PM differ from AQP and what does it use to filter records?

Strategy PM trains a lightweight CPU-resident decision tree on oracle-labeled examples to pre-filter records with high-confidence predicted outcomes, sending only uncertain records to the expensive reward model for evaluation.

Jul 16, 2026

Upload by Logic