Graph View

Gemma 4 Tool-Calling

Tool-calling convention of Google’s Gemma 4 open-weights family (google/gemma-4-*-it). It is a clean break from the prompt-engineered Pythonic tool_code form used by Gemma 3 and hosted Gemini (see gemini.md): Gemma 4 introduces dedicated special tokens and a compact token-delimited brace syntax. Calls and responses each get their own paired markers, and every string value is wrapped in a <|"|> token rather than ASCII quotes. The model emits one call as <|tool_call>call:NAME{key:value,…}<tool_call|>; the developer parses it, runs the tool, and appends <|tool_response>response:NAME{output:…}<tool_response|>.

Verified against the OMP gemma dialect (packages/ai/src/dialect/gemma.ts): the streaming scanner that parses these blocks and the renderAssistantToolCalls / renderToolResults / renderTranscript renderers that produce them. The example streams below match that implementation; the worked model id is google/gemma-4-E2B-it.

Special tokens

Gemma 4 wraps each structural element in a paired token. Note the asymmetric pipe placement — an opener carries the pipe on the left (<|x>) and its closer carries it on the right (<x|>):

OpenClosePurpose
<bos>—Beginning of sequence
`<turn>``<turn
`<tool_call>``<tool_call
`<tool_response>``<tool_response
`<channel>``<channel
`<”>`
<eos>—End of sequence

Because the string delimiter is a token (<|"|>), values may contain raw ASCII quotes and commas without escaping — only a literal <|"|> token sequence cannot appear inside a string.

Thinking variants emit reasoning in a dedicated channel — <|channel>thought\n…<channel|> at the start of the model turn, before any reply text or tool call. The gemma scanner routes that channel to thinking events (keeping it out of the visible reply) and still parses tool calls that follow it; renderThinking round-trips a thought back to the same <|channel>thought\n…<channel|> block. With parseThinking: false the channel is left in the visible text instead.

Roles / turn structure

Each turn is <|turn>{role}\n{body}<turn|>, and turns are concatenated with no separator between them. Roles are system, user, model (a developer message renders as system). With a generation prompt the stream ends at <|turn>model\n and the model continues. Tool calls and the tool responses that follow them are emitted inside one model turn — the response block immediately follows the call block in the re-rendered history.

Tool definitions

The gemma dialect does not put tool schemas on the wire. Tools are advertised in the system prompt by renderInbandToolPrompt (packages/ai/src/dialect/catalog.ts): an OpenAI-style JSON catalog — one object per line inside a <tools></tools> block — followed by the format guide (packages/ai/src/dialect/gemma.md):

<tools>
{"type":"function","function":{"name":"get_current_temperature","description":"Gets the current temperature for a given location.","parameters":{"type":"object","properties":{"location":{"type":"string","description":"The city name, e.g. San Francisco"}},"required":["location"]}}}
</tools>

The verbose system-prompt inventory and /dump additionally render each tool as a # Tool: <name> section — description, a TypeScript-style parameter signature, and native <|tool_call> examples — via renderToolInventory (packages/ai/src/dialect/inventory.ts).

Tool-call format

The model emits one call per <|tool_call>…<tool_call|> block. The body is call:NAME{ARGS}, where ARGS is a comma-separated list of key:value pairs:

<|tool_call>call:get_current_temperature{location:<|"|>London<|"|>}<tool_call|>

Value grammar inside {…}:

Value kindEncodingExample
string`<“
int / floatbarecount:42
boolbareflag:true
nullbareunit:null
list[v,v,…]`tags:[<
nested object{k:v,…}`config:{theme:<

The OMP parser is the streaming GemmaInbandScanner (packages/ai/src/dialect/gemma.ts), not a flat regex. For each <|tool_call> block it:

  1. finds the matching <tool_call|> close, skipping any <|"|>…<|"|> string span so a <tool_call|> sequence that appears inside a string value does not end the block early;
  2. matches the call:NAME{ head, then takes the brace body up to its depth-matched };
  3. splits that body into key:value pairs at top-level commas — bracket depth ([], {}) and <|"|> string spans are skipped — and decodes each value per the grammar above, so nested lists and objects parse correctly (a single-level regex would not).

Multiple / parallel tool calls

Parallel calls are consecutive <|tool_call>…<tool_call|> blocks (one call each), returned in order. The application returns one <|tool_response> per call in the same order.

Tool-result format

Each result is <|tool_response>response:NAME{output:VALUE}<tool_response|>. renderToolResults always wraps the result under a single output key, and JSON.parses the tool’s text first — so JSON output becomes a nested object/array in the brace syntax, while a plain string is wrapped in <|"|>…<|"|>:

<|tool_response>response:get_current_weather{output:{temperature:15,weather:<|"|>sunny<|"|>}}<tool_response|>
<|tool_response>response:read{output:<|"|>FILE<|"|>}<tool_response|>

End-to-end example

renderTranscript output for a weather query. The system turn also carries the <tools> catalog and format guide (see Tool definitions, abbreviated here); the model’s call merges with its tool response into one model turn (response right after the call), and the final answer is the next model turn. Turns are emitted back-to-back with no separator — only the \n after each role is literal:

<bos><|turn>system
You are a helpful assistant.<turn|><|turn>user
Hey, what's the weather in Tokyo right now?<turn|><|turn>model
<|tool_call>call:get_current_weather{location:<|"|>Tokyo, JP<|"|>}<tool_call|><|tool_response>response:get_current_weather{output:{temperature:15,weather:<|"|>sunny<|"|>}}<tool_response|><turn|><|turn>model
The current weather in Tokyo is 15 degrees Celsius and sunny.<turn|>

Parsing notes & gotchas

  • String delimiter is a token, not a quote. Inside <|"|>…<|"|> the bytes " and , are literal data — the example <|"|>The city and state, e.g. "San Francisco, CA"…<|"|> contains both. Split arguments on ,/} only outside a <|"|>…<|"|> span.
  • Asymmetric pipes. The closer is <tool_call|>, not </tool_call> or <|tool_call>. Matching the wrong pipe side will never close the block.
  • One call per block. Unlike a JSON tool_calls[] array, parallelism is “more blocks”, not “more entries in one block”.
  • Bare scalars. A value not wrapped in <|"|> is true/false → bool, null/none → null, numeric → number, otherwise a bare string (e.g. an unquoted enum or type name like STRING).
  • Not Gemma 3 / hosted Gemini. Those use the Pythonic tool_code / default_api form in gemini.md. Gemma 4 replaced it with this token syntax; the two are not interchangeable.

Sources

Properties2
descriptionA guide to the token-delimited brace syntax and special tokens used for tool-calling in Google's Gemma 4 models.
tagsgemma, tool-calling, omp-docs, llm

Backlinks