What exactly is this model

Before getting into how LiveKit optimized it, it's worth a minute on Gemma 4 itself. It's an open model family Google DeepMind released this year, built on an open-weight approach: the weights are public under the permissive Apache 2.0 license, so anyone can download and use them, running locally on their own machine instead of going through a cloud API.

LiveKit picked the 31B tier specifically: it's the most capable in the family, without being so large it's impractical to run — a good fit for server-side voice agents. Every number that follows comes from this tier running on LiveKit's platform.

Faster, cheaper, and just as sharp

LiveKit benchmarked Gemma 4 31B against several mainstream commercial models — GPT-4.1, GPT-4.1 mini, and Gemini 2.5 Flash. The takeaway in three words: faster, cheaper, sharper. Let's break each one down.

⚡Fast: near-instant responses

In a text chat, nobody minds if the AI takes two seconds to reply. A phone call is different: a pause over half a second after you stop talking feels awkward, and past a second, the whole call starts to feel fake. The metric for "how fast it starts talking" is called first-token latency — how long the model takes to produce its first word after you finish speaking. Gemma 4 31B needs just 192 milliseconds; GPT-4.1 needs 1006 milliseconds. That 800-plus-millisecond gap is roughly the line between "instant" and "awkward."

💰Cheap: under a fifth of the price

The cost comparison is straightforward with real numbers. Large models charge per token (roughly a word or word-fragment), with separate input and output rates. Here's the list price per million tokens (USD):

Against GPT-4.1, the input price is a fifth (0.4 vs. 2 dollars) and the output price is under a seventh (1.2 vs. 8 dollars). Blended at the roughly 3:1 input-to-output ratio typical of voice workloads, the combined cost comes out to about a sixth of GPT-4.1's — roughly 83% cheaper. Even against GPT-4.1 mini, which is also positioned as a budget option, the output price is still lower. For voice services running around the clock and paying per token, that adds up to real savings.

🧠Sharp: cheap and fast without getting dumber

Cheap and fast naturally raises the question: did it get dumber? LiveKit broke capability down into separate scores using the same hotel front-desk benchmark. The results come in near-perfect across the board: tool-calling accuracy, multi-turn coherence, and factual accuracy all hit 100, instruction-following scores 98, conciseness scores 96, and overall task completion lands at 88. For a voice assistant, following a process step by step and remembering what was said earlier are exactly what these scores measure.

Lay speed, cost, and capability side by side in one table, and Gemma 4 31B sweeps all three: lowest latency, lowest cost, highest capability score. For real-time voice, it's the most cost-effective pick of the bunch.

Why the same model runs faster on LiveKit

The same Gemma 4 31B model can run several times faster or slower depending on the platform. LiveKit says this is a deliberate choice — both its architecture and hardware are optimized specifically for speed.

On the hardware side, LiveKit runs on dedicated GPUs co-located in the same data center, keeping the model and compute close together with fewer hops. In testing, LiveKit takes just 354 milliseconds from the end of your sentence to hearing the first word back; a conventional router like OpenRouter, which forwards requests across multiple providers, takes 1876 milliseconds. Tokens generated per second tell the same story: 158 versus 33.

Worth noting: OpenRouter forwards requests across multiple third-party providers, so latency and availability vary by route — the figure here reflects the best Gemma 4 31B route it could select at test time.

Switch over with one line of code

Integration is simple. If you're already building voice agents with LiveKit Agents, switching to Gemma 4 31B takes one line of configuration pointing at the model — and you're immediately running on LiveKit's GPU infrastructure.

# Gemma 4 31B，跑在 LiveKit 自己的 GPU 上
from livekit.agents import AgentSession

session = AgentSession(
    llm="google/gemma-4-31b-it",
)

LiveKit also spells out its methodology: every number on the page comes from the same open-source "hotel front desk" reference agent (published on GitHub), tested with the same benchmark process, the same latency definition, and real voice-call conditions. First-token latency is sampled turn by turn across every scenario; capability scores come from the same set of conversations, graded task by task. The comparisons are against GPT-4.1, GPT-4.1 mini, and Gemini 2.5 Flash. All figures are LiveKit's own — the official page is the final word.

LiveKit is a voice AI company. It has brought Google's open model Gemma 4 31B (open, meaning the weights are public and free for anyone to use) onto its own platform, tuned specifically for voice, pitching it as the new default brain for voice agents. Ask it the same question, and the response wait drops from over a second to under 200ms, at a sixth of the cost. Integration is still just a one-line change.

But the same Gemma model can run several times faster or slower depending on the platform — the speed advantage isn't about the model itself, it's about where and how it runs.

Why does the same Gemma 4 31B run faster on LiveKit? It comes down to how it's deployed. XiaoHu calls the "LiveKit Hotel" to book a room — after saying "I'd like to book a room," the two routes respond at wildly different speeds:

Set it next to GPT-4.1, the current default for voice, and the most noticeable difference is the wait before a response. Ask the same question, and one feels instant while the other visibly stalls: