Google Cloud Fits AlloyDB with Proxy Models: On-Database AI Judgment, Up to 23,000x Faster

What AlloyDB Is, and What It Means to Put AI Inside a Database

To grasp the weight of this update, you first need two things straight: what AlloyDB actually is, and what "putting Gemini inside it" really changes.

AlloyDB is a relational database on Google Cloud, compatible with PostgreSQL (one of the world's most popular open-source databases — think of it as Google Cloud's enhanced version of PostgreSQL). Businesses use it to store orders, users, product reviews, and system logs, querying it with SQL.

Traditional databases have an inherent limitation: they only do exact literal matching. You can easily query "products priced under 100" or "orders with status completed," but you can't directly query "negative reviews complaining about battery life" or "angry-toned support tickets" — that requires understanding the meaning of text, and a database doesn't understand meaning. Previously, this kind of "meaning-based" analysis meant exporting the data, running it through a separate AI pipeline, and importing the results back — slow, and you had to maintain that whole pipeline yourself.

What AlloyDB has now done is turn Gemini's comprehension ability into a set of SQL functions (called AI functions) and embed them directly into the database itself. So now you can write a judgment about "meaning" in a plain SQL statement, and the database just understands it — picking out the matching rows without moving the data anywhere.

That's the significance here: the database is upgraded from "exact literal matching only" to "meaning-based querying," with semantic search, structured queries, and AI judgment all happening within the same SQL layer. That's a powerful capability, but it brings a new headache: calling a large model for every single row gets slow and expensive fast once row counts climb. So the real centerpiece of this release isn't the functions themselves — it's how to make them powerful, fast, and cheap all at once.

What's Actually New in This Set of AI Functions

First, let's look at what this set of AI functions can do. The most intuitive use case: turning messy, hard-to-parse raw text directly into structured, searchable content — right in SQL.

For example, using ai.generate to automatically break an error log into structured fields:

A messy log line gets automatically split into three queryable, filterable fields: error code, service name, and root cause. Three new functions have been added:

• ai.analyze_sentiment: judges whether a piece of text is positive, negative, or neutral in tone.
• ai.summarize: compresses long text into a summary while preserving the original tone and nuance.
• ai.agg_summarize: an aggregate function that combines a group of rows (paired with GROUP BY) into one unified summary.

Add in ai.generate, ai.rank, ai.if, and ai.forecast — already GA — and that's the full lineup of AI functions. For instance, ai.agg_summarize can roll up a pile of reviews for a product into one overall verdict: "Multiple users praise the 4K resolution, 120Hz frame rate, and ergonomic controller; some complain the fan gets loud during long play sessions. Overall a top-tier console, with heat and noise as minor flaws."

Why It Used to Be Slow and Expensive

The most direct way to apply a large model's judgment across a whole table is to call it once per row. Once row counts climb, that approach falls apart.

InfoQ lays out the problem concretely: for a table of 100,000 products, running a full-table judgment means 100,000 network round trips to Vertex AI. Every single one carries the same system prompt, every one waits on model inference, and every one is billed by token.

These figures come from Google's technical blog post this past May. The conclusion is blunt: for operational databases, this is too slow; for analytics use cases, the cost is high enough that many full-table semantic analyses simply aren't affordable.

Google uses two moves to break through this wall. Here's the overview first, showing how they relate to row-by-row calling — the further right, the less you touch the cloud large model, and the faster and cheaper it gets:

Let's break down each of these two moves.

Move One: Bundling Many Rows into a Single Call

The idea behind Smart Batching is simple: since every call carries the same system prompt, merge many rows into one request so that prompt only gets sent once.

Why not batch at the application layer yourself? Because getting the batch size right is tricky: too small and you barely save on cost or latency; too large and the bloated prompt can trigger hallucinations, or blow past the model's token limit. AlloyDB automatically calculates the optimal batch size for each request, and handles retries automatically too.

It also understands numeric constraints, not just semantic similarity. Take an example: a user on a digital goods marketplace is looking for a camera "rated to dive 60 meters or deeper." A regular hybrid search would just match on semantics and keywords to find the closest fit — it can't grasp a hard numeric constraint, and might recommend a camera only rated to 20 meters. Using ai.if for smart filtering, the database actually understands the "depth" constraint and only returns products that meet or exceed 60 meters. And with ai.if you don't need to specify a batch size yourself — AlloyDB handles that optimization under the hood.

Move Two: The Database Trains Its Own Stand-In Model to Make Judgments

Batching saves on repeated overhead; proxy models go further — letting the database train its own small model that can run locally, taking over most judgments so it doesn't even need to ask the large model anymore.

A proxy model is a tiny model trained specifically for a given problem and a given dataset, used only to quickly answer "yes / no" style judgment questions. When it hits something it can't judge, it automatically hands off to the large model.

The Whole Process Has Two Phases: Train, Then Execute

It splits the work across two SQL statements: PREPARE trains the model in the background, and EXECUTE uses the trained model to run live queries.

Training happens in the background PREPARE phase, and the large-model cost of this step is a one-time expense; the live EXECUTE phase then uses the trained local model directly for judgment. Only when the model's confidence is low, or no matching model can be found, does it automatically fall back to calling the large model as a backstop. Here's an official two-phase SQL example:

-- Phase 1: Train a local proxy model using a data sample + cloud LLM
PREPARE underwater_suitability_proxy FROM
SELECT description FROM products;

-- Phase 2: Execute the query at database speed using the local proxy model
SELECT * FROM products
WHERE ai.if(description, 'suitable for underwater use deeper than 60 meters')
USING proxy(underwater_suitability_proxy);

The condition in that second statement, suitable for underwater use deeper than 60 meters, is the exact judgment the diving camera from earlier needs to pass.

InfoQ frames this pattern as a reversal of the usual database-to-LLM relationship: previously the database was the client, calling out to an external model for every single judgment; now the database acts more like a student, first learning how the model judges on a batch of samples, then applying that at database speed locally. The large model's role shifts from a runtime dependency needed on every query, to a teacher that only shows up during training.

How Can This Stand-In Model Actually Understand Meaning

How can something as simple as logistic regression understand semantics like "interesting plot"? The key is that it isn't fed raw text — it's fed vector embeddings that already carry semantic information.

Put the two together: embedding models like Gemini or Gecko encode semantic concepts — "interesting plot," "great cinematography," "dull," "boring" — into different combinations of dimensions within the vector as they generate it (don't expect a single dimension to just mean "cinematography"). Training a logistic regression is essentially slicing a plane through this (hyper)sphere of embeddings, splitting the semantics into two halves: one side judged TRUE, the other FALSE. Which direction that plane cuts depends on the labels the large model assigned the training samples.

This is also Google's signature diagram from their blog — a green plane slicing through the blue embedding sphere:

The proxy model (green plane) carves out task-relevant semantics by slicing through the embedding space (blue sphere). Image: Google Cloud

This makes the ultra-low latency and cost easy to understand: embeddings only need to be generated once, and every subsequent query reuses them — so the cost of encoding semantics into the data is amortized into a one-time expense; and logistic regression itself runs on an ordinary CPU, no specialized hardware required.

How Accurate Is It? When Does It Fail?

A proxy model is an approximation method, more limited than the large model. It performs well on problems that can be judged through embedding-based semantic patterns, but it's not suitable for complex reasoning or extremely imbalanced samples.

First, accuracy: the SIGMOD 2026 paper compares results across 10 benchmarks, and the proxy model's F1 lands at 90% to 102% of the large model's F1; on the Amazon reviews sentiment classification benchmark, the proxy model hits an F1 of 0.860, beating the large model's 0.739 — that's 116%. The paper's explanation: the proxy model is trained on a batch of samples and has seen the whole picture, while the large model treats every row as a brand-new question judged from scratch.

F1 score ranges from 0 to 1, measuring both "how accurate the judgment is" and "how completely it catches everything it should" — closer to 1 is better. Below are a few specific tests published in the paper:

Now, when it fails — mainly two scenarios:

One more thing that's easy to conflate: a proxy model isn't the same as vector search. It's a classifier that judges each row TRUE / FALSE (or assigns it to a category); vector search does something different — it ranks results by a generic distance function (like cosine distance). And a proxy model is trained specifically for your data and your specific problem — even trying to simulate ai.if with vector search alone would be hard to set up well and would underperform. They're not the same thing.

The Bigger the Data, the Bigger the Advantage

The proxy model's cost and time savings aren't fixed — they grow with data volume. Because training is a one-time cost, that expense gets amortized across every subsequent query, so the more you query, the better the payoff.

The paper's data for an online-training scenario (BigQuery) looks like this:

As the scale grows from a million to ten million rows, both the cost reduction and the latency speedup scale up with it. Applied to AlloyDB, the logic is even more favorable: the large-model cost of the PREPARE phase can be amortized across any number of subsequent query executions — the more the pre-trained model gets used, the lower the per-query cost. The chart below, from the SIGMOD 2026 paper, has table scale on the x-axis: cost reduction and latency speedup both climb with scale, approaching roughly 600x cost and 300x latency at ten million rows:

Cost reduction (token consumption) and latency speedup (query acceleration) both increase as table scale grows, reaching roughly 600x cost / 300x latency at 10 million rows. Image: Google Cloud / SIGMOD 2026 paper

How Usable Is This Right Now, and What Does It Mean for Other Databases

To know how ready this is for real use, you need to separate what's already GA from what's still in preview.

To use proxy models, you have to manually flip a database flag (google_ml_integration.enable_ai_function_acceleration) — it's off by default. InfoQ also cautions: figures like 23,000x and 6,000x are internal test numbers that only hold for ai.if in preview, and don't represent typical performance across all AI functions — they should be tested against your own data distribution and query patterns before going to production.

How Practitioners Are Using It

Starburst architect Raimundas Juodvalkis offered a framing on LinkedIn: treat these as "governed database extensions," not a "magic WHERE clause." His three pieces of advice:

• Start with read-heavy scenarios like review analysis — don't jump straight into heavy-duty workloads with it.
• Be cautious before writing model-derived fields back into core systems.
• Track model costs separately from query costs.

What This Means for Other Databases

InfoQ points out that any database that calls an external model for every row-level judgment hits the same cost and latency wall. Whether competitors like Aurora, Azure SQL, CockroachDB, and PlanetScale will follow with similar "query-time distillation" schemes, or leave users to build this optimization themselves at the application layer, remains to be seen.

This release also comes with a managed MCP server for AlloyDB, letting AI agents query the database via Model Context Protocol (an open protocol that lets AI directly call external tools and data) without having to build their own MCP infrastructure — on top of the existing ScaNN vector index (supporting up to 10 billion vectors). InfoQ sees AlloyDB positioning itself as a PostgreSQL-compatible database where structured queries, semantic search, and AI judgment all coexist within the same SQL layer.