Semantic Data Processing with Large Language Models: A Survey

1. Introduction

1.1. Emergence of LLM-Powered Semantic Data Processing

The landscape of modern data analytics is being reshaped by three concurrent changes. First, traditional data analytics has long been centered on curated relational tables, while a growing amount of valuable information now resides in unstructured or semi-structured data, such as call transcripts, support tickets, and contracts [1,2]. Second, modern analytical workloads increasingly operate over heterogeneous data lakes, where raw data from diverse sources are stored in their original formats and must be modeled, organized, and discovered before analysis [3]. Third, users increasingly express their analytical needs in natural language [4,5]. Together, these shifts reshape the requirements of modern data analytics: systems must be able to interpret natural-language intent, identify relevant data sources, and apply semantic reasoning over heterogeneous content.

Example 1 (Healthcare Analytics). Consider a healthcare scenario [6] with two unstructured document collections: a Disease collection, where each document describes symptoms, causes, and diagnoses of a disease, and a Drug collection, where each document describes indications, mechanisms, and treatment usage of a drug. A user may ask: “I have had persistent fever, dry cough, and shortness of breath for several days. What disease might I have, and what drugs are usually used for treatment? Please organize the possible treatments by disease.” Answering this seemingly simple query requires discovering the relevant collections, matching symptoms to disease descriptions, linking candidate diseases to drug indications, and summarizing the recommended treatments by disease. This example illustrates the central problem studied in this survey: how to analyze, plan, optimize, and execute semantic queries over heterogeneous data.

Traditional data-processing pipelines provide limited support for such workloads. They typically rely on task-specific components, manually engineered rules, or separate pipelines for data discovery [7], query formulation [8], and information extraction [9]. Recent advances in large language models (LLMs) provide a more general semantic reasoning primitive that can be embedded into data systems as parsers, planners, and operators. This has motivated a growing trend toward LLM-powered Semantic Data Processing systems (LLM-SDPs), which process analytical tasks over heterogeneous data sources by constructing and executing plans composed of relational and LLM-powered semantic operators such as semantic filtering, projection, join, and aggregation. Representative production platforms [10,11,12,13,14,15,16] and research prototypes [17,18,19,20,21,22,23] have begun to instantiate this paradigm. It is also attracting broad discussion [24,25,26,27,28,29,30,31,32].

1.2. Distinctive Characteristics, Challenges, and the Need

Compared with traditional database systems [33,34], such as PostgreSQL [?] and MySQL [?], LLM-SDPs differ in three dimensions: (1) Data Setting: From Relational Tables to Data Lakes. The corpus is no longer limited to carefully curated relational tables, but often takes the form of a heterogeneous data lake that combines structured tables and unstructured document collections without a uniform predefined schema. (2) Operator Space: From Relational Operators to Semantic Operators. The operator space is no longer restricted to deterministic relational operators, but expands to a hybrid space of relational operators and natural-language-specified LLM-powered semantic operators. (3) Query Interface: From SQL to Natural Language. User intent is no longer expressed only through precisely defined SQL semantics, but increasingly ranges from natural-language operator predicates to fully open-ended natural-language questions.

These differences make LLM-SDPs difficult to build and optimize, giving rise to five challenges.

C1: From user intent to executable plans. Natural-language analytical requests often describe desired outcomes only. Users may not specify which data sources are relevant, how the query should be grounded, or which operators should be composed. LLM-SDPs therefore need to bridge the gap between ambiguous user intent and a valid logical plan over heterogeneous data.

C2: From fixed operators to flexible semantic implementations. In traditional systems, operators such as projection and join have well-defined semantics. In LLM-SDPs, the same semantic operation may be implemented in many ways, such as prompting an LLM or synthesizing code. This flexibility creates optimization opportunities, but also greatly enlarges the system design space.

C3: From cheap relational processing to costly semantic reasoning. Semantic operators often require model calls over many records, documents, or record pairs, making them substantially more expensive than relational operators. Their cost is not only measured by execution time, but also by monetary cost. This makes cost control a central concern when processing large data lakes.

C4: From deterministic outputs to uncertain answer quality. The quality of LLM-powered operators’ outputs depends on the input, prompt, model choice, and available context. The system cannot assume every operator invocation is reliable. It must reason about the trade-off between cost and quality, deciding when cheaper execution is sufficient and when an expensive one is needed.

C5: From static execution to adaptive processing. Even after a plan is generated and optimized, execution remains fragile. Individual model calls may return wrong outputs, inconsistent labels, or low-confidence decisions, and such errors can propagate across multi-step plans. LLM-SDPs need runtime mechanisms for validation, adaptation, recovery, and failure handling.

Existing work has begun to address these challenges. However, these efforts remain fragmented along three dimensions: (1) existing systems target different interfaces and workloads, ranging from SQL-oriented extensions [35,36,37,38] to pipeline-style and agentic frameworks [39,40,41,42]; (2) existing studies focus on different layers of the processing stack, from individual semantic operators [36,43,44,45] to end-to-end planning and execution [19,46,47]; and (3) similar design ideas, such as operator fusion [48,49,50] and model cascading [40,51,52], are often introduced under different terminology and rarely synthesized into shared abstractions. This fragmentation makes it difficult to compare existing techniques, understand their assumptions, and identify reusable system principles. Therefore, a systematic survey is needed to organize this emerging literature under a unified query-processing view.

1.3. Our Scope and Contributions

To address this need, we conduct the first survey to review LLM-powered semantic data processing from the perspective of query processing. We focus on the full lifecycle that transforms a user specification into an executable semantic query plan. Accordingly, we organize the literature around query analysis, semantic operator design, logical and physical optimization, query execution, and benchmark evaluation. This perspective brings together fragmented work on semantic operators, planning, optimization, and execution under a common database-inspired framework.

Several adjacent areas are discussed only when they contribute techniques to the LLM-SDPs stack: (1) TableQA and DocumentQA are treated as workloads that LLM-SDPs may support, rather than as the central abstraction of this survey. (2) Retrieval-augmented generation (RAG) is viewed as a possible physical implementation strategy for semantic operators, rather than as a complete query-processing framework. (3) Text-to-SQL contributes techniques for schema grounding and executable query generation, but LLM-SDPs are closer to text-to-plan systems over heterogeneous data sources. (4) General LLM agents contribute ideas such as planning and tool use, but their objective differs from LLM-SDPs, which emphasize explicit plans, operators, optimization opportunities, and evaluation criteria.

We make four main contributions:

• (1) We establish a unified abstraction for LLM-powered semantic data processing. We characterize LLM-SDPs through semantic operators and logical/physical query plans, and formulate an end-to-end processing model that connects natural-language intent, heterogeneous data sources, optimization, and execution. This abstraction also delineates LLM-SDPs from related paradigms including RAG, Text-to-SQL, QA systems, and general-purpose data agents.
• (2) We develop a comprehensive taxonomy of the semantic query-processing stack. We systematically organize existing work across query analysis, semantic operator design, logical optimization, physical optimization, query execution, and benchmarking. Within each stage, we derive design spaces that expose the major architectural choices made by existing systems.
• (3) We synthesize reusable system principles across fragmented literature. We identify recurring mechanisms and analyze when and why they work. This synthesis reveals common trade-offs and connects emerging LLM-SDPs techniques with established database principles.
• (4) We highlight open problems in reliable query planning, unified semantic-operator optimization and estimation, semantics- and uncertainty-aware query optimization, adaptive and failure-aware execution, lifecycle-aware benchmarking, and emerging workloads such as streaming analytics and Deep Research.

Difference from Previous Surveys.Existing surveys primarily focus on specific tasks, such as TableQA [53], DocumentQA [54], RAG [55,56], and Text-to-SQL [57,58], or study the area from the perspectives of data agents [59,60,61,62] or DBMS–LLM integration [63]. LROBench [64] is the closest to our scope, but focuses primarily on LLM-enhanced relational operators. In contrast, our survey centers on theend-to-end query-processing stack, connecting data discovery, plan generation, semantic operators, optimization, execution, and evaluation under a unified framework.

Organization.Section 2 introduces the processing model of LLM-SDPs and related areas. Section 3, Section 4, Section 5, Section 6 and Section 7 cover the main query-processing stack, including query analysis, semantic operator design, logical and physical optimization, and query execution. Section 8 reviews existing benchmarks and evaluation practices. Finally, Section 9 discusses open challenges and future directions.

2. Background and Overview

This section introduces the basic terminology and processing model used throughout the survey. We first give the definitions of the basic notions (Section 2.1). We then abstract a general framework for LLM-SDPs (Section 2.2), and use the healthcare query in Example 1 as a running example to illustrate how each component operates under this framework (Section 2.3). Figure 1 presents an overview of this framework: Figure 1(a) shows the generic processing pipeline of LLM-SDPs, while Figure 1(b) instantiates the pipeline using the healthcare example.

2.1. Operators and Plans

Operators are the basic building units of query plans in LLM-SDPs. Similar to traditional database systems, LLM-SDPs maintain an operator library that specifies the operations available for constructing analytical plans. The key difference is that this library typically contains both relational operators and semantic operators: (1)relational operatorsoperate over structured data with deterministic semantics defined by relational algebra, such as selection, projection, and join. (2) In contrast,semantic operatorsuse LLMs to operate over unstructured data, e.g., extraction targets, summarization goals specified in natural language. They introduce model-dependent behavior, non-determinism, and trade-offs among answer quality, latency, and monetary cost, as their outputs and costs depend on model choice, prompts, and input context.

Based on these operators, a query plan is constructed as the central abstraction connecting user intent, operator design, optimization, and execution in LLM-SDPs. Similar to classical database systems, we distinguish between logical plans and physical plans: (1)logical plansspecify what operations should be performed and how their inputs and outputs are connected. A logical plan is commonly represented as a directed acyclic graph, where nodes are operators and edges describe data dependencies, and is independent of any specific execution strategy. (2)physical plansinstantiate logical plans with concrete execution choices. For example, a physical plan may specify which model is used for a semantic operation. Different physical plans may implement the same logical plan, but they may differ substantially in answer quality, latency, and monetary cost.

2.2. Processing Architecture

As illustrated in Figure 1(a), LLM-SDPs accept a user query $\mathit{q}$ in natural language, SQL with semantic extensions, or a code-like declarative program and operate over a heterogeneous data lake $\mathcal{D}=\{d_1,d_2,\dots ,d_m\}$, where each $d_i$ denotes a logical data source, such as a relational table or a document collection. The goal of the system is to transform $\mathit{q}$ into an executable plan over $\mathcal{D}$ and return a result $\mathcal{R}$. At a high level, this process can be written as

$$(\mathit{q},\mathcal{D})\stackrel{{\mathcal{Q}}_A}{\to }{\mathcal{P}}_L\stackrel{{\mathcal{Q}}_O}{\to }{\mathcal{P}}_P\stackrel{{\mathcal{Q}}_E}{\to }\mathcal{R},$$

(1)

where ${\mathcal{Q}}_A$ denotes query analysis, which transforms the user input and data context into a logical plan ${\mathcal{P}}_L$, ${\mathcal{Q}}_O$ denotes query optimization, which refines ${\mathcal{P}}_L$ and instantiates it into a physical plan ${\mathcal{P}}_P$, and ${\mathcal{Q}}_E$ denotes query execution, which runs ${\mathcal{P}}_P$ and produces the final result $\mathcal{R}$.

The processing of a semantic query follows a lifecycle broadly aligned with traditional query processing, but each stage must be extended to handle natural-language specifications, heterogeneous data sources, and LLM-based semantic operators. With these concepts in place, the rest of the survey follows this lifecycle:

• Query analysisconverts the user input into an initial logical plan. In LLM-SDPs, this stage often needs to infer underspecified elements, such as relevant data sources, schema bindings, semantic predicates, and operator compositions.
• Operator designdefines the basic semantic primitives of each semantic operator and studies how these primitives can be implemented. These operator-level abstractions provide the building blocks and cost-quality trade-offs that later optimization stages exploit.
• Query optimizationrefines the initial plan before execution. The logical plans produced by query analysis are often intent-correct but not necessarily execution-efficient, because semantic operators may incur high latency, monetary cost, and uncertain output quality. Logical optimization rewrites the structure of the plan, such as decomposing, fusing, or reordering semantic operators. Physical optimization instantiates logical operators with concrete implementations, such as model choices, retrieval strategies, and cascade designs. Together, these optimization stages address the central trade-off among answer quality, latency, and monetary cost.
• Query executionruns the selected physical plan over the underlying data sources, invoking relational operators and LLM-based semantic operators to produce the final result. Because semantic operators can be expensive and uncertain, execution may also adapt at runtime based on intermediate results, observed costs, or quality signals.

2.3. Illustrative Example

We now revisit the healthcare query from [ex:overallexample]Example 1, as illustrated in Figure 1(b). The user asks which disease is likely given symptoms such as persistent fever, dry cough, and shortness of breath, and which drugs are commonly used for treatment. Under the LLM-SDPs architecture, the system may first identify theDiseaseandDrugcollections in the data lake as relevant sources. It then matches the symptom description against disease documents to find candidate diseases, links candidate diseases to drug documents through drug indications, extracts candidate disease names, and finally summarizes the recommended drugs by disease.

This example also illustrates why each stage in the LLM-SDPs architecture is necessary, motivating the later discussions: (1) Query analysis is needed because the user only provides a natural-language request: the system must understand the intended task, identify the relevantDiseaseandDrugcollections, and construct an initial logical plan. (2) Logical optimization is needed because the plan may admit multiple equivalent structures. For example, semantic projection and semantic join may be reordered depending on whether linking first can reduce the cost of subsequent extraction. (3) Physical optimization is needed because each logical operator can be implemented in different physical plans with different trade-offs in quality, latency, and cost. For example, symptom matching may use an LLM, a retrieval model, a classifier, or a hybrid strategy. (4) Query execution is needed to run the selected physical plan under practical constraints. For instance, symptom matching over many disease documents may be executed in batches.

3. Query Analysis

Query analysis transforms a user query into a semantically valid logical plan for subsequent optimization and execution. In traditional settings, this process is typically grounded in a closed-world assumption, where the data source comprises predefined relational tables and is explicitly referenced in the SQL via theFROMclause, while the query task (processing logic) is expressed in relational algebra. In contrast, LLM-SDPs may take natural-language task descriptions, where users specify the desired outcome but leave data sources, schema bindings, and operator sequences implicit. Query analysis must therefore infer these missing components and translate the task into a well-formed logical plan.

To characterize such uncertainty, we classify query analysis in LLM-SDPs along two orthogonal dimensions: (1) whether the data source is explicitly specified and (2) whether the processing logic is explicitly specified. The former concerns whether the target tables, documents, images, or other data collections can be directly identified from the input, while the latter concerns whether the operator-level computation is provided by the user. This taxonomy is summarized in Table 1.

Fully-specified Queries.When both data sources and processing logic are explicit, query analysis largely resembles that in traditional data-processing engines with Spark-like or SQL-like interfaces. Consequently, query analysis is relatively straightforward: it mainly parses the input specification, validates references to data sources and operators, and directly translates the specification into an internal logical plan. This design shifts logical-plan formulation to users, requiring greater user expertise but offering more direct control and more predictable system behavior.

In the following, we focus on under-specified queries, where the absence of explicit data binding or operator composition makes query analysis a central challenge in LLM-SDPs.

3.1. Under-Specified Queries

Under-specified queries arise when users describe analytical goals without explicitly specifying the data sources, schema bindings, or operator sequences needed to answer them. In LLM-SDPs, such queries are common and require the system to infer the missing data context and processing logic before optimization and execution.

3.1.1. Processing Model

At an abstract level, under-specified query analysis recovers two types of missing bindings. Data binding grounds the query to relevant data sources and schema elements, such as tables, documents, columns, or relations. Logic binding determines the operators and parameters needed to implement the requested computation.

Accordingly, as shown in Figure 2, we model under-specified query analysis as two logical stages: data discovery, which grounds the query to relevant data context, and plan generation, which synthesizes a logical plan over the grounded context.

3.1.2. Design Space

Based on the processing model, the design space of under-specified query analysis can be organized around the above two stages.

Data Discovery.Data discovery has been extensively studied in traditional settings [7,77] and natural language settings [78,79,80]. However, our focus here is not on these areas themselves. Instead, we examine how existing LLM-SDPs incorporate data discovery as an initial step of query analysis, typically through the combination of data source profiling and data source retrieval.

Data Source Profiling.Data source profiling summarizes heterogeneous data sources, such as structured tables and unstructured documents, into representations that can be consumed by LLMs, retrievers, or downstream planners. As illustrated in Figure 2, existing systems mainly rely on three types of profiling information: schema extraction [71,72,75], which captures structural information such as table names, column names, field descriptions, or document structures; content summaries [46,75,76], which describe representative records or values to support semantic matching; and metadata or statistics [71,72], which provide auxiliary information such as data types, value distributions, or provenance.

Data Source Retrieval.Given the built profiles, data source retrieval identifies the subset of data sources likely to be useful for the user query. As shown in Figure 2, existing systems mainly adopt two retrieval strategies: LLM-based retrieval [71,75], which prompts an LLM to judge the relevance between the query and candidate data profiles based on schema semantics, content descriptions, and task requirements, and embedding-based retrieval [46,73,76], which embeds queries and data profiles into vectors first and then retrieves relevant sources based on similarity. These strategies are often complementary: embedding-based retrieval can first narrow the search space, after which LLM-based retrieval performs more precise relevance judgment. Symphony [76] further combines retrieval with rule-based filtering or human-in-the-loop refinement to improve robustness.

Plan Generation.Given the intent of the user and the retrieved context, plan generation synthesizes a logical plan by selecting needed operators, binding them to relevant data sources, and composing them. Existing approaches can be broadly grouped into three paradigms: end-to-end generation, multi-stage generation, and iterative generation, as illustrated in Figure 2(b).

End-to-end Generation.End-to-end generation produces the plan by using an LLM directly. Existing systems enrich the one-shot prompt with operator specifications [19], schema descriptions and few-shot examples [47], or multiple candidate generations [46] inspired by Best-of-N [81] for plan aggregation. This paradigm keeps the pipeline simple. However, since the complete plan is generated in one step, errors in data grounding, operator selection, schema binding, or operator composition are difficult to isolate.

Multi-stage Generation.Multi-stage generation decomposes plan construction into explicit steps before producing the final logical plan. These methods first construct intermediate representations, such as stepwise textual plans followed by operator mapping [73], workflows that combine knowledge retrieval with task decomposition [75], or attribute-level grounding and DAG-style decomposition [72]. Exposing these intermediate states makes plan generation more inspectable and enables validation or repair of source bindings and operator choices. However, it also introduces additional orchestration overhead and may propagate early-stage errors to later stages.

Iterative Generation.Iterative generation builds the logical plan step by step rather than producing the entire plan at once. At each step, the system decides the next operator, subtask, or sub-plan based on the partial plan constructed so far and the unresolved part of the query. Existing systems instantiate this idea in different ways, such as retrieving an operator and rewriting the remaining query [70], decomposing the task into subtasks [76], or generating reasoning steps aligned with available operators [71]. This paradigm is useful for complex queries whose required processing logic is difficult to determine in a single generation step. However, it requires the system to track the partial plan, decide when generation should stop, and prevent errors made in earlier steps.

4. Operator Design

Semantic operators are the key primitives in LLM-SDPs. Figure 3 summarizes the evolution of representative studies on semantic operators. They extend query plans beyond deterministic relational operations and enable semantic processing over unstructured or heterogeneous data. Because both optimization and execution are largely operator-centric, understanding the semantics and implementation choices of semantic operators is essential for interpreting later optimization techniques. In this section, we focus on the design of the main semantic operators. For each operator, we introduce its definition, basic implementation, and design space, highlighting the trade-offs.

4.1. Semantic Projection

Semantic Projection generalizes the classical relational projection operator to unstructured and semi-structured data. In relational algebra, projection selects existing attributes from a relation according to a predefined schema. In contrast, Semantic Projection derives new structured attributes from textual objects according to a natural-language instruction.

4.1.1. Definition

Formally, given a text object $\mathcal{T}$ and a natural language instruction ${\mathcal{I}}_{nl}$ that specifies the desired attribute, Semantic Projection is defined as: ${\pi }_{\mathrm{sem}}(\mathcal{T},{\mathcal{I}}_{nl})⟶\mathcal{A}$, where $\mathcal{A}$ is the extracted attribute value. For example, given person profiles, users may extract a structured attribute such as age under an instruction like “Extract the person’s age from the text.”

4.1.2. Naive Implementation

Semantic Projection shares similarities with generative information extraction [96] and retrieval-augmented generation (RAG) [55,56,97]. The naive implementation performs semantic projection in a tuple-at-a-time manner using an LLM. For each input object ${\mathcal{T}}_i$, the system constructs a prompt by combining ${\mathcal{T}}_i$ with the instruction ${\mathcal{I}}_{nl}$, and asks the LLM to return the target attribute ${\mathcal{A}}_i$.

However, it suffers from three major bottlenecks. (1) First, long input objects cause context and token-cost issues, as projection tasks often require only a small portion of the input text. (2) Second, irrelevant context may distract the model from the evidence needed for the target attribute, reducing extraction accuracy. (3) Third, repeated LLM invocations limit scalability, since applying large models to every tuple and every attribute becomes expensive over large datasets.

4.1.3. Design Space

To overcome these bottlenecks, many methods have been proposed and can be modeled using the two-stage processing model as shown in Figure 4: (1) localize relevant evidence from the input object, and (2) derive the target attribute from the localized evidence.

S1: Evidence Localization.Evidence localization aims to reduce the amount of input to the projection executor while preserving the evidence needed for deriving the target attribute. Existing systems mainly follow two strategies: structure-aware localization and chunk-based localization:

Structure-aware Localization. Structure-aware localization exploits explicit or implicit document structures to identify regions likely to contain the target evidence. Such structures may include section headings and paragraph hierarchies, such as scientific papers. Common implementations include hierarchy-guided traversal [35] and execution-guided region selection [39]. Structure-aware localization can substantially reduce input size when structural signals are reliable, but it is less effective for weakly structured or highly heterogeneous text.

Chunk-based Localization. Chunk-based localization segments the input object into semantically coherent chunks and retrieves relevant chunks indexed using embeddings. This strategy is suitable for long or weakly structured text where reliable document structures are unavailable.

Natural language descriptions of attributes may lack sufficient information to retrieve the relevant chunks. For the example in Figure 4, when extracting theageattribute, simply embedding the word age may fail to retrieve a chunk such as “Wardell Stephen Curry (born March 14, 1988) is ...” because the chunk does not mention the attribute even though it contains the evidence. Existing studies enhance chunk retrieval in two ways. The first perspective [42] enriches each chunk embedding by capturing not only the chunk’s local text but also contextual or structural information, such as document structure and neighboring chunks. The second perspective [69,87] enriches the embeddings used for retrieval, replacing raw attribute names with more informative text constructed from attribute descriptions, validation examples, LLM-generated explanations, or representative evidence chunks.

S2: Attribute Derivation. After relevant evidence has been localized, existing systems explore alternative implementations for the derivation function:

Program-based Derivation. Program-based derivation replaces repeated LLM inference with executable programs synthesized or configured for the projection task [39,50,82,83]. This strategy is suitable when the target attribute follows relatively stable lexical, syntactic, or formatting patterns, e.g., dates, identifiers, and prices. Instead of invoking an LLM for every tuple, an LLM can be used offline or on a small set of examples to synthesize candidate extraction functions, which are then executed cheaply over the full dataset.

Model-based Derivation. Model-based derivation uses learned models to derive attributes. This strategy is suitable for repeated projection workloads where the target attributes are stable and sufficient supervision is available. Compared with synthesized programs, learned models can capture softer semantic patterns and support greater linguistic variation. Common implementations include smaller general-purpose fine-tuned models [84]. The main limitation is that such models require adaptation and may perform poorly when instructions, domains, or data distributions change.

Dynamic Selection. Dynamic selection chooses among multiple derivation executors, such as OpenIE [98], pre-trained language models [99], code generation, and LLMs. The selection process usually considers input characteristics, task complexity, confidence estimates, quality requirements, or budget constraints. Common forms include budget-aware selection [87] and model cascade execution [51,100], which will be further discussed in Section 4.2.

4.2. Semantic Filter

Semantic Filter evaluates a natural-language predicate over textual objects and returns a Boolean decision indicating whether each object satisfies the predicate.

4.2.1. Definition

Formally, given a text object $\mathcal{T}$ and a natural-language predicate ${\mathcal{P}}_{nl}$, Semantic Filter is defined as: ${\sigma }_{\mathrm{sem}}(\mathcal{T},{\mathcal{P}}_{nl})⟶\{True,False\}$, where the output indicates whether $\mathcal{T}$ satisfies ${\mathcal{P}}_{nl}$. For example, given research papers, users may filter documents under an instruction like “The paper was published after 2020.”

4.2.2. Naive Implementation

Semantic Filter is related to text binary classification. The naive implementation evaluates Semantic Filter in a tuple-at-a-time manner using an LLM. For each input object ${\mathcal{T}}_i$, the system constructs a prompt by combining ${\mathcal{T}}_i$ with the natural-language predicate ${\mathcal{P}}_{nl}$, and asks the LLM to return a Boolean answer.

It suffers from three major issues: (1) Semantic predicates vary in complexity. Some predicates can be reduced to structured conditions while others may require multi-step reasoning. (2) Long textual objects often contain only a small amount of evidence relevant to the predicate. Feeding the entire object to the LLM wastes tokens, increases latency, and may distract the model. (3) Repeated LLM-based classification over large datasets incurs substantial monetary and latency costs.

4.2.3. Design Space

To solve the issues above, many studies have been proposed and can be summarized under the processing model shown in Figure 5: (1) transform the natural-language predicate into a more executable form, (2) localize sufficient evidence from the input object, and (3) make the final Boolean decision efficiently and reliably.

S1: Predicate Transformation.Predicate transformation aims to reduce the semantic complexity of the original natural-language predicate before evidence localization and decision making. Existing systems mainly follow two strategies: predicate reduction and predicate decomposition.

Predicate Reduction. Predicate reduction maps a natural-language predicate into a structured, symbolic, or programmatic condition [38,41,71] and is often realized later as program-based classification in the decision-making stage. This strategy is effective when the predicate refers to explicit attributes, metadata, or stable lexical patterns. For instance, the predicate “The paper was published after 2020” can be reduced to a structured condition such as $\mathtt{year}>2020$ if the publication year is available as metadata or can be extracted beforehand. In such cases, Semantic Filter can avoid expensive LLM-based evaluation and instead use relational filtering or index lookup.

Predicate Decomposition. Predicate decomposition transforms a complex natural-language predicate into a composition of simpler sub-predicates [52]. These sub-predicates may be organized in logical forms such as conjunctive normal form or disjunctive normal form, enabling short-circuit evaluation and early pruning. For example, the judicial predicate “Does this opinion overturn a lower court decision?” can be decomposed into a surrogate form such as “$(\mathit{mentionanylowercourt}\wedge \mathit{opinionoverturnsalowercourtdecision})\vee \mathit{saysremandingthecase}$”.

S2: Evidence Localization. Evidence localization for Semantic Filter follows the same general strategies as Semantic Projection, including structure-aware localization and chunk-based localization, as discussed in Section 4.1.3.

S3: Decision Making. We categorize decision-making strategies into five categories:

Program-based Classification. Program-based classification evaluates the transformed predicate using deterministic or executable logic. Typical implementations include SQL predicates [38,41,71] or structured index lookup [84] when the predicate has been reduced to a structured condition [38,41,71]. Similarly, if latent attributes have been extracted and indexed offline, online filtering can be implemented as index lookup [84]. Program-based classification is highly efficient. However, it may fail for implicit semantic predicates, predicates requiring global context, or cases where the mapping from natural language to executable logic is uncertain.

Embedding-based Classification. Embedding-based classification maps the predicate and its localized evidence into a shared vector space and approximates the Boolean decision using nearest-neighbor search [19,47,68,70] or range search [69]. A document is classified as satisfying the predicate if it is in the top-k nearest neighbors of the predicate embedding or if its distance to the predicate embedding is below a predefined threshold.

This strategy is particularly useful for topical predicates. However, embedding similarity is not always equivalent to predicate satisfaction. A document may be semantically close to the predicate yet fail to satisfy it. Compared with program-based classification, embedding-based classification can capture semantic similarity. It is also scalable as embeddings can be searched efficiently.

Model-based Classification. Model-based classification trains lightweight models to approximate the Semantic Filter decision function. These models may be trained from sampled data labeled through online learning [36,44,48] or offline learning [36]. Compared with embedding-based classification, model-based classification can learn a more flexible decision boundary. One practical issue is label imbalance, which is discussed in Google [36]. Another key challenge is how to balance classification accuracy and training budget in online learning. UQE [48] continuously retrains a cheap proxy model on observed labels and uses the proxy to guide subsequent row selection, thereby balancing recall and budget. ScaleLLM [44] trains multiple lightweight classifiers and selects the best-performing one for large-scale inference. It further establishes budget-accuracy projections across various training scales, offering users quantitative guidance for customized budget allocation. Google [36] automatically explores imbalance-aware training configurations, and adaptively selects the cheapest proxy model that satisfies a predefined accuracy constraint.

Model Ensemble. Model ensemble combines multiple LLMs’ decisions to improve robustness or accuracy [101]. The ensemble considers multiple LLMs’ outputs and aggregates them through a specific rule. This strategy is useful when individual models are unstable. However, ensemble methods may introduce additional cost and require historical data to learn the ensemble rule.

Model Cascade. Model cascade is the most important decision-making strategy for large-scale data. It organizes multiple decision executors into a staged execution pipeline, where cheap executors first handle easy cases and uncertain cases are escalated to more expensive and accurate executors. Given localized evidence $\mathcal{E}$ and a transformed predicate ${\mathcal{P}}^{\prime }$, a general cascade can be written as: $g_1(\mathcal{E},{\mathcal{P}}^{\prime })\to g_2(\mathcal{E},{\mathcal{P}}^{\prime })\to \dots \to g_K(\mathcal{E},{\mathcal{P}}^{\prime }),$ where $g_1,\dots ,g_K$ can be programs, embedding classifiers, learned models, cheap LLMs or strong LLMs. Each stage typically produces both a prediction $\widehat{y}$ and a confidence or uncertainty signal. If the signal indicates that the current prediction is reliable, the system accepts the result; otherwise, it goes to the next stage. Compared with invoking an oracle LLM for every object, cascades exploit the fact that many objects are easy to classify and do not require the strongest model. We summarize the taxonomy of model cascade strategies in Figure 6, where we classify existing methods by cascade depth and proxy type.

(1) LLM Proxy. It uses a cheaper LLM as the first-stage classifier and escalates uncertain cases to an oracle LLM, which is first introduced by SUPG [102]. LOTUS [40] adapts it to semantic operators, and BARGAIN [85] further reduces oracle calls under quality constraints. SUPG-IT & GAMCAL [91] extends this line of work to streaming semantic SQL settings.

(2) Embedding-based Proxy. It uses similarity retrieval as the first-stage proxy and then relies on LLM verification for final decisions [19,47,68,70]. Unlike embedding-based classification, embedding-based cascades treat retrieval as a high-recall prefilter.

(3) Learned-model Proxy. It typically trains a lightweight model using sampled data labeled by an oracle LLM, and uses it as the first-stage proxy while deferring uncertain cases to the oracle LLM, such as ScaleDoc [43]. More broadly, the proxy does not have to be an explicit ML classifier. CSV [92] builds a cluster-based proxy by partitioning tuples in the embedding space, labeling a small number of samples in each cluster, and inferring the remaining labels through cluster-level voting. HoldUp [93] further constructs a graph from pairwise record comparisons and derives labels through global clustering and matching, thereby exploiting correlations among records rather than classifying each row independently. Compared with LLM proxies, learned-model proxies are faster and cheaper, while compared with embedding-based proxies, they capture more predicate-specific decision boundaries. However, a key challenge of learned-model proxies is how to maintain the proxy under workload shifts like predicate or data distribution shifts.

(4) LLM Proxy Sequence. It is the same as LLM Proxy, but extends it to a sequence of multiple LLMs with a finer-grained cost-quality trade-off. FrugalGPT [100] trains a regression model to estimate output quality and dynamically determines the escalation. SMART [103] profiles candidate LLMs on sampled data and then allocates the remaining workload across one or more LLMs.

(5) Heterogeneous Proxy Sequence. Different stages may use different types of proxies, such as programmatic filters, embedding retrieval, and so on. STRETTO [51] jointly optimizes categorical choices among different proxy types and continuous parameters such as thresholds. MOAR [50] introduces hybrid pre-filter cascades composed of synthesized code filters and cheap LLM-based filters, ordered by increasing cost under quality constraints.

4.3. Semantic Join

4.3.1. Definition

Formally, given two collections $\mathcal{A}=\{a_1,a_2,\dots ,a_m\}$ and $\mathcal{B}=\{b_1,b_2,\dots ,b_n\}$, and a natural-language join predicate ${\mathcal{P}}_{nl}$, Semantic Join is defined as: ${\bowtie }_{\mathrm{sem}}(\mathcal{A},\mathcal{B},{\mathcal{P}}_{nl})⟶\mathcal{M},$ where $\mathcal{M}\subseteq \mathcal{A}\times \mathcal{B}$ denotes the set of record pairs $(a_i,b_j)$ that satisfy ${\mathcal{P}}_{nl}$. For example, given product reviews and category labels, users may join the two relations under an instruction such as “Does this review belong to this product category?”

4.3.2. Naive Implementation

The naive implementation evaluates Semantic Join in a pair-at-a-time manner using an LLM. For each record pair $(a_i,b_j)\in \mathcal{A}\times \mathcal{B}$, the system constructs a prompt by combining the record pair with the natural-language join predicate ${\mathcal{P}}_{nl}$, and asks the LLM to determine whether the pair satisfies the predicate. The naive implementation is prohibitively expensive at scale as it incurs quadratic evaluation complexity.

4.3.3. Design Space

The processing model is organized into three stages: (1) reduce or restructure the Cartesian search space, (2) transform the natural-language join predicate into more executable matching criteria, and (3) evaluate record pairs efficiently and reliably.

S1: Search Space Reduction. Search space reduction reduces or restructures the Cartesian pair space before expensive semantic pair evaluation. Existing methods mainly rely on operator-level [67] or problem-level [66] rewriting: PLOP [67] rewrites semantic join as a relational join followed by a semantic filter to enable filter-ordering optimization, while Cortex AISQL [66] rewrites certain joins with a small label-side domain into multi-label classification and reduces complexity from quadratic to linear. These rewritings reduce the number of pairwise semantic evaluations required by the original Cartesian formulation.

S2: Predicate Transformation. Predicate transformation rewrites a natural-language join predicate into more explicit and executable matching criteria, such as latent attributes, feature-level comparisons, or structured logical forms. LOTUS [40] extracts such intermediate features through semantic projection and then compares the derived attributes, while ELEET [84] extends this idea to multi-modal joins by using relational-side tabular context to guide text-side extraction. FDJ [88] further represents complex join predicates as multiple feature-level comparisons, making implicit matching criteria explicit and reducing full-predicate LLM evaluation.

S3: Pair Evaluation. Pair evaluation determines whether each considered record pair satisfies the original or transformed join predicate. Existing systems differ in the granularity of evaluation and in the executor used to make the matching decision. We categorize these methods into LLM-based pairwise evaluation, block-based evaluation, similarity-based evaluation, and cascaded evaluation.

Block-based Evaluation. Block-based evaluation reduces per-pair overhead by increasing the granularity of model invocation. Inspired by block nested-loop join, BlockJoin [89] partitions the two input collections into dynamically adjusted blocks and evaluates multiple candidate pairs within a single LLM call. This idea is later followed by OmniTQA [72] and SEMA-SQL [71].

Similarity-based Evaluation. When the join predicate behaves like a soft equality relation, semantic join can be approximated by embedding similarity. Records from the two collections are embedded into a shared vector space, and pairs with sufficiently high similarity are treated as matches, as adopted by AnDB [68]. However, embeddings may capture topical relatedness but fail to capture the exact join semantics, thereby introducing both false positives and false negatives.

Cascaded Evaluation. To mitigate this issue, some systems [40,85,94] introduce cascaded evaluation, in which pairs with sufficiently high or low similarity are directly accepted or rejected, while only those falling into an intermediate similarity band are forwarded to an LLM for secondary verification. This design helps recover accuracy lost due to embedding errors.

4.4. Semantic Groupby

4.4.1. Definition

Given a collection $\mathcal{D}=\{d_1,d_2,\dots ,d_n\}$ and a natural-language grouping instruction ${\mathcal{I}}_{nl}$, Semantic Groupby is defined as ${\mu }_{\mathrm{sem}}(\mathcal{D},{\mathcal{I}}_{nl})⟶\mathcal{G},$ where $\mathcal{G}=\{(l_1,g_1),(l_2,g_2),\dots ,(l_k,g_k)\}$, $l_i$ denotes the semantic label of group $g_i$, and $\{g_1,g_2,\dots ,g_k\}$ forms a partition of $\mathcal{D}$. For example, given sports news, users may group documents under an instruction such as “Group news by the sport they describe.”

4.4.2. Processing Model

As an operator, the output of Semantic Groupby must contain both a partition of input records and interpretable labels for the resulting groups.

The processing of Semantic Groupby is centered on two coupled operations: grouping and labeling. Grouping determines which records should belong to the same semantic group under the natural-language instruction, while labeling assigns an interpretable description to each group. Unlike previous operators, these two operations do not form a fixed tuple-wise pipeline. Labels may be constructed first to guide record assignment, or groups may be discovered first and then labeled.

4.4.3. Design Space

Based on the above processing model, the design space of Semantic Groupby can be organized around the construction order between grouping and labeling. Existing methods mainly follow two paradigms: label-first grouping and group-first labeling.

Label-first Grouping. Label-first grouping first derives a label space and then assigns records to the corresponding labels. This paradigm is suitable when the grouping instruction provides, or allows the system to infer, a meaningful set of group labels before all records are partitioned. Existing methods mainly instantiate this paradigm through projection-based label extraction and taxonomy-guided label discovery.

Projection-based Label Extraction. When the grouping dimension is discrete and enumerable, semantic groupby can be reduced to semantic projection followed by standard relational groupby [68,70]. For example, for “Group news by the sport they describe.”, the system may first extract the sport described by each news article and then group records by the extracted sport attribute.

Taxonomy-guided Label Discovery. When the grouping criterion refers to an open-ended semantic space, some studies first construct a taxonomy or label set from representative records and then assign remaining records to the discovered groups [19,48]. This design has also been extended to streaming settings [49], where discovered groups are incrementally maintained and refined as the data distribution evolves.

Group-first Labeling. Group-first labeling first partitions records into semantic clusters and then generates labels for the resulting groups. This paradigm is suitable when the label space is difficult to enumerate in advance, or when the grouping structure is better discovered from the data distribution. Existing methods mainly differ in whether they cluster original representations directly or first derive instruction-aware representations.

Direct Embedding-based Clustering. When the original record representation is sufficiently aligned with the intended grouping semantics, some studies directly embed records, cluster them in the embedding space, and generate labels from representative elements in each cluster [19,68]. Streaming variants further adapt this idea through incremental clustering and periodic relabeling [49].

Summary-guided Clustering. When raw representations do not adequately capture the grouping instruction, some studies first derive a more suitable semantic representation, such as topic keywords or summaries, and then cluster records in the derived space [40,84].

4.5. Semantic Aggregation

4.5.1. Definition

Formally, given a collection $\mathcal{D}=\{d_1,d_2,\dots ,d_m\}$ and a natural-language aggregation instruction ${\mathcal{I}}_{nl}$, Semantic Aggregation is defined as ${\gamma }_{\mathrm{sem}}(\mathcal{D},{\mathcal{I}}_{nl})⟶\mathcal{R},$ where $\mathcal{R}$ denotes the aggregated result, which may be a scalar value, a structured object, or a textual summary. For example, given sports match reports, users may aggregate documents under an instruction such as “Tally the number of rule violations mentioned across these reports.”

4.5.2. Naive Implementation

The naive implementation directly prompts an LLM with the entire input collection $\mathcal{D}$ and the aggregation instruction ${\mathcal{I}}_{nl}$, and asks the model to produce the result $\mathcal{R}$.

This naive implementation is limited by one-shot long-context processing: large collections may exceed the model context window, and even feasible inputs may suffer from degradation when evidence is distributed across many records.

4.5.3. Design Space

The design space of semantic aggregation can be organized around one key question: can the aggregation implied by the natural-language instruction be reduced to a standard relational aggregation over extracted attributes? Existing methods therefore mainly follow two directions: point-wise aggregation and list-wise aggregation.

Point-wise Aggregation. Point-wise aggregation applies when the natural-language instruction implies a standard aggregation function over tuple-level latent attributes. In this case, semantic aggregation can be reduced to semantic projection followed by relational aggregation. For example, for “Tally the number of rule violations mentioned across these reports.”, the system may first extract the number of rule violations from each report and then compute the final tally using a standard aggregation function. This reduction is adopted by systems such as Unify [70].

List-wise Aggregation. List-wise aggregation applies when the aggregation result cannot be computed from independently extracted tuple-level values and instead requires joint reasoning over a set of records. This setting is common for summarization-style tasks, where the result must synthesize information distributed across multiple documents. Since large collections cannot be processed in one LLM call, existing systems decompose the input into manageable blocks and aggregate them through iterative or hierarchical reduction.

Iterative Aggregation. Iterative aggregation [42,50] follows a fold-style reduction strategy, where batches of input records are sequentially incorporated into an evolving aggregate state. This design is useful when aggregation is order-sensitive or when the system needs to maintain an evolving result over time, such as in streaming settings [49] where the aggregation state is incrementally updated as new tuples arrive.

Hierarchical Aggregation. Hierarchical aggregation [38,40,66,72] partitions the input collection into multiple blocks, computes partial aggregates independently, and recursively combines the intermediate results into a final output. Compared with iterative aggregation, this strategy exposes more parallelism and typically provides better scalability for large collections.

4.6. Semantic Orderby

4.6.1. Definition.

Formally, given a collection $\mathcal{D}=\{d_1,d_2,\dots ,d_m\}$ and a natural-language ranking instruction ${\mathcal{I}}_{nl}$, Semantic Orderby is defined as ${\tau }_{\mathrm{sem}}(\mathcal{D},{\mathcal{I}}_{nl})⟶\langle d_{\left(1\right)},d_{\left(2\right)},\dots ,d_{\left(m\right)}\rangle ,$ where $\langle d_{\left(1\right)},d_{\left(2\right)},\dots ,d_{\left(m\right)}\rangle$ denotes an ordered permutation of $\mathcal{D}$ such that, for any $i<j$, record $d_{\left(i\right)}$ is ranked ahead of $d_{\left(j\right)}$ under ${\mathcal{I}}_{nl}$. For example, given customer support calls, users may order records under an instruction such as “Rank these calls by the quality of support provided by the agent.”

4.6.2. Design Space

The central design question is how to derive ordering signals from a natural-language ranking instruction. Existing methods can be broadly categorized into three directions: (1) Point-wise Ordering [105,106], where the ranking criterion is converted into an independent score or sorting key for each tuple; (2) Pair-wise Ordering [107,108,109], where ordering is derived from relative preferences between tuple pairs; and (3) List-wise Ordering [110,111,112,113,114,115], where multiple tuples are ranked jointly in a single model call.

Point-wise Ordering. Point-wise ordering computes an independent score or sorting key for each record and then applies standard ordering over the derived values. This strategy is suitable when the ranking criterion can be reliably quantified for individual records. Existing systems instantiate this paradigm through semantic-projection-derived sorting keys [46,70], proxy scoring models that approximate LLM-derived ranking signals [36], and streaming or batched variants for top-k maintenance and efficient scoring [45,49].

Pair-wise Ordering. Pair-wise ordering derives the final ranking from relative preferences between pairs of records. This strategy is suitable when reliable point-wise scoring is difficult, or when the ranking criterion is subjective and easier to judge through comparison. Existing systems typically implement this paradigm by replacing the comparator in classical sorting algorithms with LLM-based pairwise comparison [38,40,41,45].

List-wise Ordering. List-wise ordering ranks multiple records jointly in a single model call. Instead of decomposing ordering into independent scores or pairwise comparisons, this strategy directly asks the model to reason over a list of candidates and produce a partial or complete ranking. Existing studies explore list-wise ordering as an external ordering access path [45] or develop specialized listwise rankers and recall-aware optimization strategies [95].

5. Logical Optimization

As introduced in Section 2.2, logical optimization focuses on rewriting the structure of semantic query plans. Unlike classical relational optimization, logical optimization in LLM-SDPs is not always limited to strictly equivalence-preserving transformations, because semantic operators are opaque and non-deterministic. Changing the granularity, combination, or placement of semantic operators may affect both execution efficiency and answer quality. In this section, we review key logical optimization techniques illustrated in Figure 7.

5.1. Operator Decomposition

Operator decomposition rewrites a semantic operator into a sequence of simpler logical operators. In this subsection, we focus on plan-level decomposition, where the logical structure of the query plan is changed by making intermediate semantic results explicit. This plan-level rewriting is motivated by two issues in LLM-powered semantic data processing: (1) long documents or large input groups may exceed the model context window, and even feasible long-context inputs can cause omissions or degraded outputs; and (2) a semantic operator may implicitly combine multiple reasoning steps, making it difficult for an LLM to execute reliably as a single step.

Data-oriented Decomposition.Data-oriented decomposition splits large inputs into smaller semantic processing units, such as document chunks, local contexts, or sampled subsets. This pattern is useful for long-document or large-group processing, where smaller units reduce context pressure [42,50]. In practice, such decomposition is often combined with later aggregation or reduction to recover a global result from partial outputs.

Task-oriented Decomposition.Task-oriented decomposition splits a complex semantic instruction into multiple simpler semantic subtasks. Instead of asking one operator to perform all reasoning at once, the plan may separate evidence discovery, extraction, verification, filtering, or aggregation into different logical steps [42,50]. This pattern is suitable when the original instruction contains multiple implicit objectives or when intermediate results are useful for downstream operators.

Decomposition is not universally beneficial. Therefore, systems must decide whether decompositions are worth applying. Existing studies approach this decision through accuracy-oriented rewrite selection with sampled validation [42] and multi-objective rewrite search that compares decomposed plans under both quality and cost [50].

5.2. Operator Fusion

In contrast to operator decomposition, operator fusion rewrites multiple related semantic operators into a single operator. Such fusion is motivated by the observation that adjacent semantic operators often require similar semantic understanding or share the same input. Executing them independently may require repeated LLM invocations and redundant reasoning. By evaluating related semantic tasks jointly, fusion can reduce such redundancy and simplify the logical plan.

Projection Fusion.Projection fusion [19,37,38,48,49,50,116] merges multiple semantic projections over the same input into a single multi-attribute projection. Instead of invoking separate operators to extract different latent attributes, this pattern has been widely adopted for extracting multiple semantic attributes from the same record.

Filtering Fusion.Filtering fusion [37,38,41,50] merges multiple semantic predicates into a single fused filtering operator. This rewrite can reduce repeated evaluation of the same input context and allows related predicates to be assessed jointly. However, fusion may increase the inference cost and eliminate opportunities for short-circuit pruning when individual predicates are highly selective [37]. To address this issue, SEMA [38] further uses sampling-based validation and predicate-correlation analysis to determine whether fusion is beneficial.

Projection–Groupby Fusion.Some studies [48,49,50] fuse semantic projection with downstream semantic groupby or aggregation when the extracted attributes are fed into the subsequent operator. Rather than materializing intermediate semantic attributes and processing them separately, the fused operator jointly performs attribute extraction and downstream groupby or aggregation, thereby reducing intermediate processing overhead.

Filtering–Projection Fusion.Filtering–projection [38,49,50] fusion combines attribute extraction and predicate evaluation into a single semantic operator. Instead of first materializing a latent attribute and then applying a semantic filter, the fused operator jointly produces the extracted value and the filtering decision. CPs [49] points out that the benefit of this rewrite may diminish when highly selective filtering could otherwise eliminate many tuples before projection.

5.3. Operator Reordering

Operator reordering is a fundamental technique in logical optimization that improves execution efficiency by adjusting the order of operators. The key intuition is to minimize intermediate result sizes and defer expensive operations whenever possible. Due to the high cost of semantic operators, reordering strategies must account for both operator cost and cardinality reduction effects.

5.3.1. Projection Pull-Up

In traditional databases, projection is typically pushed down to reduce tuple width early. However, due to the high cost of semantic projection, existing studies [35,39,47,51,67,69,71,75] instead adopt a lazy projection strategy, which postpones semantic extraction until the projected attributes are required by downstream operators. This allows preceding operators to reduce the cardinality of data requiring projection, thereby lowering overall cost.

5.3.2. Filter Reordering

Filter reordering plays a more central role, as filters directly affect intermediate cardinality. Semantic filters are often applied early to reduce downstream cost [35,39,41,48]. Beyond this general principle, recent work explores this problem from two different perspectives:

Filter Pull-up. Similar to UDF optimization [117,118,119,120,121], the high cost of semantic filtering and the inherent selectivity ofJOINoperators together create opportunities for cross-operator reordering. Existing studies mainly explore three forms of filter pull-up:

(1) Heuristic-based pull-up delays semantic operators based on the general assumption that semantic operators are much more expensive than relational operators. For example, STRETTO [51] applies relational filters and joins first and pulls up as many semantic operators as possible.

(2) Cost-based pull-up determines the placement of semantic filters by jointly considering semantic-operator cost and cardinality changes. For instance, SEMA-SQL [71] uses dynamic programming to defer semantic operators when doing so reduces total cost, while OmniTQA [72] delays semantic operators behind relational operators by default but pushes them down when downstreamJOINorUNIONoperators may significantly inflate intermediate results. Similar cost-aware placement is also considered in iPDB [37] and Cortex AISQL [66].

(3) Join-aware pull-up expands the placement space by transforming hybrid or join-dependent semantic operations into semantic filter placement problems. PLOP [67] decomposes semantic joins into relational joins and semantic filters, and then optimizes the placement of semantic filters and projections. QUEST [69] further converts certain join patterns intoIN-filter evaluation by first extracting join attributes from one side of the join.

Intra-filter Reordering. When multiple filters form a chain, their execution order can substantially affect performance. Existing studies can be roughly grouped into two strategies:

(1) Heuristic-based reordering orders filters using simple rules. Two common heuristics are widely adopted. The first is Relational Filters First [38,47,48,66,67,71,72,75], as they are much cheaper than semantic filters. The second is Low-selectivity Filters First [38,39,41,50,66], as they more aggressively reduce intermediate cardinality. A key challenge in this setting is estimating filter selectivity. To support this heuristic, existing studies either assume fixed selectivity values [41] or estimate selectivity through sampling-based execution [38,39,66].

(2) Cost-based reordering jointly considers per-tuple execution cost and filtering effectiveness. Existing systems instantiate this idea through analytical priority metrics based on cost and selectivity [35], dynamic-programming-based ordering using sampled selectivity estimates [51], document-specific ordering that accounts for token-dependent predicate cost [69], or deferred plan enumeration where alternative filter orders are compared during physical optimization [70].

6. Physical Optimization

Physical optimization instantiates a logical plan into an executable physical plan. Given an initial logical plan, together with possible plan alternatives produced by enumeration or rewriting, the optimizer must determine what objective to optimize, how to evaluate candidate plans, how to construct the physical plan space, and how to search this space. Accordingly, as summarized in Figure 8, we organize this section around four aspects: optimization objectives, plan evaluation, plan space construction, and plan space search.

6.1. Optimization Objectives

Physical optimization needs to specify what makes one plan preferable to another. In traditional databases, this objective is usually dominated by execution cost or latency. In LLM-SDPs, however, different physical plans also vary in monetary cost and answer quality. As shown in Figure 8, existing systems mainly adopt three objective formulations:

Single-objective Optimization.Some systems reduce plan selection to a primary objective. For example, latency-oriented optimization selects plans with lower execution time [37,70]. Other systems optimize a scalar cost model that incorporates multiple factors into one objective [68], or focus primarily on answer quality [42]. Single-objective optimization is simple, while it hides trade-offs among latency, monetary cost, and quality.

Constrained Optimization.Constrained optimization optimizes one metric while enforcing constraints on another. In LLM-SDPs, a common formulation is to minimize monetary cost [41,75] or latency [51] subject to answer-quality constraints. This view is adopted by systems that treat quality as an explicit condition rather than merely an observed outcome. Such formulations are suitable when users provide minimum quality requirements or budget constraints, but they require the optimizer to estimate whether a candidate plan can satisfy the constraint.

Multi-objective Optimization.Multi-objective optimization maintains a set of non-dominated plans over multiple metrics, such as latency, monetary cost, and answer quality. Rather than reducing all metrics to a single objective, these systems approximate a Pareto frontier and select a plan according to user preferences [39,49,50,122]. This formulation better exposes the cost-quality-latency trade-off, but it also increases the complexity of plan evaluation and search. Overall, the objective formulation determines whether the optimizer seeks the fastest plan, the cheapest plan, the highest-quality plan, a constraint-satisfying plan, or a Pareto-optimal set of alternatives.

6.2. Plan Evaluation

After defining the optimization objective, the optimizer must estimate the behavior of candidate plans. Plan evaluation in LLM-SDPs may estimate latency, monetary cost, selectivity, and answer quality simultaneously. Since semantic operators are input-dependent and LLM outputs are uncertain, existing systems commonly rely on sampled execution to approximate full-plan behavior.

6.2.1. Data Sampling

Data sampling determines which subset of data is used to estimate candidate plan behavior. The key challenge is to obtain representative samples under a limited evaluation budget. Existing systems mainly differ in how the sample set is constructed.

Basic Sampling.Basic sampling constructs the sample set without using semantic signals or learned proxies. Existing systems instantiate this strategy through truncation sampling [39], which uses the first N records as samples, or uniform random sampling [40,41,42,46], which draws records randomly from the dataset. These methods are broadly applicable and easy to implement, but may fail to cover rare yet semantically important cases.

Importance Sampling.Importance sampling biases the sample toward tuples that are more influential for estimation. Existing systems instantiate this idea through semantic-distance-based scores [70,75], proxy-model estimates [40,66,91,102], or streaming and batch-local importance weights [66,91]. Compared with uniform sampling, importance sampling is better suited to workloads where rare or high-impact tuples dominate estimation quality, but its effectiveness depends on the quality of the importance function.

Stratified Sampling.Stratified sampling partitions the dataset into multiple strata and samples from each stratum. In semantic workloads, strata can be constructed from embedding-based clusters or other semantic partitions, as in UQE [48]. This strategy improves coverage of heterogeneous subpopulations, but requires an additional partitioning step and depends on the quality of the stratum construction.

6.2.2. Plan Evaluation Paradigms

Given sampled data, systems still differ in how sampled execution is used to estimate candidate plans. As summarized in the first block of the table in Figure 8, the main distinction lies in the unit of evaluation: the operator level or the plan level.

Operator-level Evaluation.Operator-level evaluation estimates the behavior of individual semantic operators and composes these operator-level estimates into plan-level estimates. This paradigm is commonly based on Operator Independence, a strong assumption introduced by Palimpzest [39]: the cost, latency, selectivity, and quality of a semantic operator are independent of the physical implementations of other operators in the same plan. Under this assumption, Palimpzest profiles operators through sentinel plans [39], and related operator-level estimation is adopted or extended in Abacus [122], Nirvana [41], and CPs [49]. This paradigm is scalable because operator profiles can be reused across candidate plans, but it may miss cross-operator effects and error propagation.

Plan-level Evaluation.Plan-level evaluation estimates the behavior of candidate plans directly through sampled execution. In its basic form, each candidate plan is treated as a black box, and its observed runtime, monetary cost, and output quality are used as the plan-level estimate [42,50,70]. More recent studies further extend this paradigm by explicitly accounting for cross-operator interactions, error propagation, and end-to-end quality effects [51].

6.3. Plan Space Construction

Before selecting a physical plan, the optimizer must define the candidate plan space. As summarized in the middle block of the table in Figure 8, this space is mainly constructed from two sources: plan-level candidate generation and operator-level configurations.

Plan-level Candidate Generation.At the plan level, candidate physical plans may come from alternative logical plan structures.

(1) Planning-time Candidate Generation. One source comes from generating alternative plans during the initial planning stage. For example, Unify [70] employs a depth-first search (DFS) algorithm to explore the planning space and generate a predefined number of plans before optimization.

(2) Rewrite-based Candidate Generation. Another source arises when logical optimization produces multiple alternative candidate plans and defers the final choice to physical optimization. These alternatives may be generated through rewrite directives [42,50] or operator reordering [70].

Operator-level Configurations.Another major source of diversity lies in the configuration of individual operators, including implementation strategies, parameter settings, and LLM selection.

(1) Implementation Strategies. The same logical operator may admit multiple physical implementations with different computational characteristics, as discussed in Section 4. These implementation strategies are considered in many studies [39,49,50,51,68,75,122].

(2) Parameter Settings. Beyond choosing an implementation strategy, each physical implementation may further expose tunable parameters that affect its behavior. These parameters may include confidence thresholds [122], execution batch size [49], and other operator-specific settings [51].

(3) LLM Choice. For LLM-based operator implementations, LLM model selection is a major physical configuration dimension. Different LLMs may differ substantially in latency, monetary cost, and output quality. Although model choice can be viewed as a special case of parameter setting, it is often discussed separately because it is tightly coupled with the cost-quality trade-off. As a result, LLM choice is explicitly considered by many studies [39,41,50,51,75,122] when constructing the physical plan space.

6.4. Plan Space Search

Once the candidate plan space is constructed, the optimizer must decide how to explore it. Since multiple factors influence the execution behavior of semantic operators, exhaustive enumeration [41,42,70] may be feasible for small plan spaces but becomes expensive when the search space grows combinatorially. As summarized in the last block of Figure 8, recent studies use different strategies ranging from exhaustive to prior-guided and gradient-based approaches to balance search completeness and evaluation cost.

Dynamic Programming Search.Abacus [122] adopts Pareto-Cascades to maintain Pareto-optimal subplans under constrained optimization objectives. It avoids repeatedly evaluating equivalent subplans, but requires the optimization problem to admit an appropriate subplan decomposition.

Bandit-based Search.MOAR [50] adapts a bandit-style search to select complete pipelines for further rewriting and evaluation. This strategy supports global search without assuming optimal substructure, but its effectiveness depends on the utility signal used to guide exploration.

Bayesian Optimization Search.CPs [49] adopts multi-objective Bayesian optimization to identify accuracy–throughput trade-offs under a limited probing budget. This strategy reduces costly plan evaluations, but its effectiveness depends on the surrogate model and acquisition strategy.

Gradient-based Search.STRETTO [51] relaxes discrete choices and continuous parameters into an optimization problem solved through gradient-based methods. This strategy can scale to large configuration spaces, but requires a suitable continuous relaxation of the physical choices.

7. Query Execution

After physical optimization, the selected physical plan contains the operator implementations, model choices, and configuration parameters required for execution. This section focuses on system-level execution techniques that improve efficiency, context reuse, and robustness at runtime.

7.1. General Execution Optimizations

Many systems adopt general execution strategies to improve overall runtime efficiency, which do not change the logical meaning of the plan. Instead, they optimize the runtime schedule, invocation granularity, or shared execution state. Existing systems mainly exploit three types of execution-time opportunities: parallel execution, batch querying, and cross-operator runtime sharing.

Parallel Execution.Although physical plans are often represented as linear pipelines, in some cases they may form directed acyclic graphs (DAGs). In such scenarios, many studies [38,46,70] can schedule operators according to the dependency structure of the DAG. Specifically, the execution engine first performs a topological ordering of the operators. Operators that share the same topological level and do not depend on one another can be executed in parallel. This scheduling strategy enables the system to exploit parallelism in the plan, reducing latency.

Batch Querying.Batch querying amortizes LLM invocation overhead by grouping multiple records that require the same semantic task into a single LLM call. The grouped records can share the same prompt prefix, including the system instruction, task description, and output schema. This idea is related to general batch prompting techniques [123,124] and has been adopted by many LLM-SDPs [21,37,38,39,41,49,72]. By reducing repeated prompt construction and invocation overhead, batch querying can improve throughput and reduce monetary cost. Its effectiveness depends on two execution-level design choices.

(1) Batch Size Selection. Batch size selection determines how many records are included in each LLM call. Larger batches may improve throughput, but can also affect output quality [41,116]. Existing systems therefore treat batch size as an execution parameter and select it through sampling-based validation [38] or lightweight performance models [49].

(2) Batch Organization. In addition to batch size, the arrangement of records within a batch can also affect LLM execution efficiency. Batch organization determines how records or fields are arranged within and across batches. Beyond simply grouping records by task, QuestCache [86] reorders rows or fields to increase prompt-prefix reuse or KV-cache reuse.

Cross-operator Runtime Sharing.Some execution optimizations exploit runtime interactions among operators to reduce redundant work without changing the logical plan. Unlike logical operator fusion, cross-operator runtime sharing preserves the original operator structure, but allows the execution engine to coordinate related operators during execution. Existing systems mainly instantiate this idea through two patterns. First, shared execution state allows related operators that inspect overlapping data to share common intermediate runtime information. For example,GROUPBYandWHEREcan share the same samples in aggregation queries, thereby avoiding duplicated sampling [48]. Second, early termination uses downstream constraints, such asLIMIT, to stop execution once sufficient results have been obtained [47,48].

7.2. Context Management

During query execution, a large amount of historical execution context is continuously generated, including operator outputs, execution errors, and plan structures. Effectively storing and utilizing this historical information allows systems to build system context that can guide future decisions and improve overall system performance. We distinguish between context cache design, which determines what execution artifacts are stored and reused, and context management strategies, which determine how historical context is compressed, retained, and promoted over time.

7.2.1. Context Cache Design

To effectively leverage the historical execution context, caching mechanisms are introduced to record and reuse artifacts generated during execution, including intermediate results, observed failures, and successful plans. These types of information naturally correspond to three categories of caches: result caching, error caching, and plan caching.

Result Cache.General semantic caching studies [125,126,127,128] have shown that reusing semantically similar query results can substantially reduce LLM cost. However, these methods are mainly designed for query-level reuse. SEED [83] maintains reusable outputs indexed by task-input keys, while iPDB [37] and SEMA-SQL [71] explicitly exploit operator-level caching to avoid repeated LLM invocations. PLOP [67] further shows that caching can affect cardinality estimation for semantic filters and thereby influence operator ordering and plan selection.

Error Cache.To improve robustness and fault tolerance, several studies [70,75] record historical execution failures and reuse them to guide future planning or execution decisions. This is useful because LLM-SDPs may repeatedly encounter similar data errors, semantic errors, or grammar errors across related tasks [75].

Plan Cache.Successful execution plans often exhibit structural similarities across different queries, and such patterns can be summarized into reusable templates. To exploit this observation, some studies [75,129] cache historical general good cases. These cached plans are then used as planning hints to guide future workflow generation and optimization.

7.2.2. Context Management Strategies.

Since cached and historical execution context cannot grow indefinitely, execution engines need strategies for controlling context size while preserving useful information. Existing systems mainly adopt two strategies. (1) Summarization-based compression [46] uses LLMs to condense historical execution context into shorter representations, retaining information that is relevant to the current execution state. (2) Multi-level memory [75] organizes execution context into temporary, short-term, and long-term memory, and promotes frequently accessed or high-value information over time.

7.3. Failure Handling

Due to the inherent uncertainty of LLM outputs and the complexity of multi-step workflows, failures are common during execution [130]. As a result, effective failure handling mechanisms that retry, switch, or adjust the plan are crucial for maintaining system robustness.

Plan Retry.When the execution result of an operator does not meet the expected requirements, a simple yet effective strategy is to retry it. When a validation check after an operator fails, some studies [37,42,50] will retry the operation while incorporating context from prior failures, which helps improve the probability of success in subsequent attempts.

Plan Switching.Another strategy is to switch to an alternative execution plan. This strategy is useful when the system has already generated multiple executable plans, such as alternatives from planning [46] or rewrite-based optimization [50]. Once the current plan fails validation or execution, the engine can discard it and continue with another candidate.

Plan Adjustment.Plan adjustment repairs the current execution plan without abandoning the entire plan. The system modifies failed or unresolved parts of the plan through feedback-based plan repair [73], incremental replanning over remaining subqueries [70], and runtime alignment for incompatible intermediate representations [46].

8. Benchmark Evaluation

As summarized in Table 2, many representative benchmarks have been proposed so far, and they target different parts of the LLM-SDPs design space. In this section, we organize them according to their primary evaluation scope and then summarize the metrics used to assess answer quality, efficiency, and system-level behavior.

8.1. Benchmark Scope

As summarized in Table 2, existing benchmarks can be grouped by their primary evaluation focus into operator-centric, pipeline-centric, and agent-centric benchmarks. These categories are not strict boundaries, but indicate what each benchmark mainly evaluates.

Operator-centric Benchmarks.Operator-centric benchmarks focus on LLM-enhanced semantic operators. SemBench [135] narrows the scope to semantic query processing engines and emphasizes task-specific evaluation across different query semantics. LROBench [64] further strengthens this operator-centric view by treating operators as first-class benchmark units, enabling more fine-grained analysis of operator behavior and multi-operator compositions.

Pipeline-centric Benchmarks.Pipeline-centric benchmarks evaluate end-to-end query pipelines. TAGBench [136] moves beyond conventional Text2SQL and retrieval-style QA by studying database questions that require semantic reasoning. SWAN [137] focuses more specifically on hybrid querying, where LLM-generated values are integrated into relational execution. HyQBench [138] extends hybrid-query evaluation to large-scale, compositional workloads, covering both flat and nested queries with fine-grained annotations of their reasoning processes. UDABench [6] further shifts the focus toward unstructured and multimodal data analysis, highlighting semantic workloads over complex documents and the role of logical and physical optimization.

Agent-centric Benchmarks.Agent-centric benchmarks broaden the scope to general data-analysis agents. FDABench [139] evaluates heterogeneous data-analysis tasks across multiple system paradigms, while DABench [143] emphasizes enterprise-style agent workflows involving multi-database integration, messy joins, text transformation, and domain knowledge. AgenticDataBench [144] further evaluates end-to-end data-science agents over realistic workflows, using fine-grained data-science skills to characterize task coverage and diagnose agent behavior. These benchmarks are relevant to LLM-SDPs when semantic data processing appears as a subtask, but their primary focus is broader agentic data analysis rather than semantic query processing alone.

8.2. Evaluation Metrics

Existing benchmarks evaluate LLM-SDPs from three perspectives:

Answer Quality.Answer-quality metrics measure whether the final output is acceptable. Common metrics include exact match [136,139,144], execution accuracy [137,138], success rate [138,139,143,144], precision, recall, and F1 [6,135,138], ROUGE for generation tasks [139], factuality of generated answers [137] and task-specific scoring functions [144]. SemBench [135] further adopts task-specific metrics, such as relative error for aggregation and Spearman correlation for ranking.

Efficiency.Efficiency metrics capture the runtime and resource overhead. Existing benchmarks commonly report latency or execution time [6,135,136,139], token consumption or token cost [6,137,138,139,144], monetary cost or cost-related metrics [64,135], and model invocation counts [139].

Intermediate and System-Level Behavior.Some benchmarks also evaluate intermediate or system-level behavior, such as tool recall [139], factuality of intermediate generated data [137], operator-level accuracy [64,138], failure modes [144], and execution behavior in agentic workflows [143,144]. Compared with answer quality and efficiency, these metrics remain less standardized.

9. Open Challenges and Research Opportunities

The preceding sections systematically reviewed and organized existing techniques across the semantic query-processing lifecycle. We now follow the same processing stages to discuss the key open challenges and research opportunities.

9.1. Query Analysis: Toward Data-Aware and Verifiable Planning

Reliable query analysis requires both sufficient knowledge of the underlying data lake and robust mechanisms for grounding and plan generation.

Planning-oriented Data-Lake Metadata. Reliable planning depends critically on how the underlying data lake is represented. Recent work such as AutoDDG [145] and Metadata Reasoner [146] shows that richer metadata can substantially improve dataset discovery and source selection. However, they focus on tabular data and retrieval-oriented tasks. For LLM-SDPs, a broader challenge is to construct metadata for heterogeneous data lakes: metadata that describes not only what a source contains, but also how it can participate in a query plan, e.g., cross-source relationships, joinability, and supported semantic operations. An important question is what metadata should be materialized offline, acquired on demand, or selected for a particular query, and how its utility should be evaluated by downstream plan quality rather than retrieval accuracy alone.

Uncertainty-aware Grounding and Plan Verification.Source discovery and schema grounding are inherently uncertain. Existing systems improve grounding through metadata reasoning and data-aware linking [146,147], but typically resolve such ambiguity into selected sources or bindings before downstream planning. Future systems could instead preserve alternative bindings together with associated evidence or confidence for subsequent validation and optimization. Meanwhile, existing studies have explored multi-stage compilation and plan validation [42,147,148]. A broader direction is to develop a systematicgenerate–validate–repairprocess that checks source validity, operator compatibility, grounding, and plan dependencies before execution.

9.2. Semantic Operators: Toward Unified Optimization and Reliable Estimation

Semantic operators admit diverse implementation choices, yet systematically optimizing these choices requires reliable estimates of their behavior.

Unified Operator Optimization.Most existing systems optimize only a subset of the available implementation choices for each semantic operator. OrderbyLLM [45] and ListK [95], for example, provide more unified optimization frameworks for semantic ordering. A key direction is to develop similar frameworks for other semantic operators, systematically integrating alternative implementations, model choices, batching, cascading, and other operator-specific optimizations under explicit cost–quality objectives.

Operator Behavior Estimation.Unified optimization further requires reliable estimates of operator behavior. Existing systems typically rely on sampled execution to estimate selectivity, latency, monetary cost, and output quality [39,122]. A key challenge is to develop reusable estimators for semantic operators, including cardinality/selectivity, cost, and quality estimation, that generalize across data, instructions, models, and physical implementations without repeatedly executing expensive LLM operators.

9.3. Query Optimization: Toward Semantics- and Uncertainty-Aware Optimization

At the plan level, semantic optimization must account for the fact that both logical rewrites and physical choices can affect answer quality as well as execution cost.

Semantic Rewrite Safety. Unlike relational rewrites, semantic decomposition, fusion, and reordering may change model outputs and therefore answer quality. Existing work such as PLOP [67] establishes equivalence for specific transformations, but a broader theory is needed to determine when semantic rewrites are safe. This may require approximate, probabilistic, or quality-bounded notions of equivalence when exact equivalence is unrealistic.

End-to-End Quality under Uncertainty. Recent optimizers increasingly consider quality together with latency and monetary cost [122], and some provide end-to-end quality constraints [51]. A remaining challenge is to model uncertainty and cross-operator dependencies: errors in an early semantic operator can change the input, difficulty, or available evidence of downstream operators. Optimizers therefore need models that translate local quality estimates and correlated errors into end-to-end plan quality, rather than assuming operator independence.

9.4. Query Execution: Toward Adaptive and Fault-Tolerant Semantic Execution

During execution, uncertain operator behavior creates opportunities for runtime adaptation while also introducing new failure modes.

Semantic Feedback as a Runtime Signal. Traditional adaptive query processing [149,150] reacts mainly to execution statistics such as observed cardinalities. Semantic execution exposes additional signals, including model confidence, validation results, executor disagreement, realized latency and cost, and characteristics of intermediate semantic outputs. Early systems already explore adaptive execution: SEMA [38] dynamically reorders and fuses operators and applies prompt batching at runtime, SPEAR [116] adapts prompts in response to execution-time signals, CAESURA [73] introduces interleaved execution, and QUEST [69] proposes adaptive join ordering. However, these efforts remain isolated. A broader challenge is to define a unified feedback model through which such signals can trigger model switching, threshold adaptation, operator reordering, or partial replanning.

Failure Localization and Selective Recovery. Runtime adaptation should also extend from performance optimization to failure handling. An incorrect final result may originate from an incorrect binding, an unreliable semantic operation, an inappropriate physical choice, or accumulated errors across several operators. Existing studies have proposed basic failure-handling mechanisms, but this line of work remains early. SIC [148] provides one system-level example from the perspective of integrity constraints. Moreover, rather than restarting an entire query, future systems should aim to localize the source of failure and selectively repair the affected operator or subplan. This may require execution provenance that records the data evidence, model invocations, intermediate outputs, constraints, and validation signals contributing to the final result.

9.5. Benchmarking: From Final-Answer Accuracy to Lifecycle-Aware Evaluation

Existing benchmarks increasingly cover semantic operators, pipelines, and data agents, but final-answer quality alone provides limited information about where and why a semantic query processor succeeds or fails. An incorrect answer may result from source discovery, schema grounding, logical planning, physical-plan selection, individual operator failures, or runtime adaptation. Future benchmarks should therefore expose intermediate artifacts throughout the query-processing lifecycle, including source selections, bindings, operator choices, logical rewrites, physical implementations, and recovery decisions. In other words, future evaluation should benchmark not only the answer but also the semantic query-processing process itself.

Benchmark workloads should also reflect the heterogeneous data-lake settings that motivate LLM-SDPs. Many current benchmarks assume a predefined table, database, or document collection and therefore bypass important source-discovery decisions. More realistic workloads should test source discovery, cross-source linking, schema-on-read processing, multimodal integration, and robustness to noisy or incomplete metadata.

9.6. Beyond One-Shot Queries: Toward Semantic Data Runtimes

Most current LLM-SDPs are organized around a one-shot processing lifecycle: a user submits a query, a plan is generated and executed, and a result is returned. Emerging workloads increasingly challenge this assumption. Semantic processing may need to persist as new data arrive, or evolve iteratively as intermediate findings trigger new questions, evidence acquisition, and plan revisions. This suggests a broader evolution from one-shot semantic query engines toward semantic data runtimes that maintain state and continuously coordinate planning, optimization, and execution.

Continuous Semantic Processing. Streaming and production workloads require semantic operators to process continuously arriving data, maintain evolving state, and adapt to data or workload changes. VectraFlow [22] provides an early example by extending semantic operators to long-horizon processing over data and event streams. More generally, continuous LLM-SDPs will require incremental semantic computation, state maintenance, adaptive processing under distribution shifts, and mechanisms for controlling the quality and cost of long-running queries.

Iterative and Agentic Analytics. Deep Research represents another departure from one-shot query processing. Such systems dynamically plan, acquire evidence, and iteratively refine their analysis, offering flexibility that fixed semantic query plans often lack. Conversely, semantic operator systems provide explicit, optimizable execution abstractions but are less suited to dynamically evolving analytical processes. Recent work has begun to bridge these paradigms: Russo and Kraska [151] argue for an analytics runtime combining the flexibility of Deep Research with optimized semantic operators, while AgenticScholar [129] integrates agentic planning with executable operator DAGs over scholarly data. A broader research opportunity is to determine how agentic planning and semantic query processing can be combined without sacrificing optimization, observability, and reliability.

Taken together, these challenges suggest that the next generation of LLM-SDPs will require more than simply more capable LLMs. Metadata and grounding must become planning-aware; semantic operators must expose optimizer-relevant behavior; optimizers must reason about semantics and end-to-end uncertainty; execution must adapt to and recover from runtime failures; and evaluation must make the complete processing lifecycle observable. Addressing these challenges would move semantic data processing from a collection of LLM-enabled operators and pipelines toward a principled data-system abstraction for reasoning over heterogeneous data.

10. Conclusions

LLM-SDPs are emerging as a new paradigm for data analytics over heterogeneous data lakes. By introducing semantic operators into database systems, they enable complex analytical intents over structured and unstructured data to be expressed in natural language, while also fundamentally changing the conventional processing model of databases.

In this survey, we reviewed LLM-SDPs from a query-processing perspective. Instead of summarizing prior work system by system, we organized the field along the major stages of the processing pipeline, including query analysis, semantic operator design, optimization, execution, and benchmarks. Through this view, we summarized representative techniques, their motivations, and their limitations, while bringing a fragmented body of work into a more unified framework. More importantly, this survey highlights several recurring patterns across existing studies, including shared operator abstractions, cross-operator optimization ideas, and connections to analogous techniques in traditional database systems. Our goal is to provide a systematic understanding of this emerging area and to encourage a more integrated view of semantic data processing.

References