GPT-5 Header Leak

Historical research note, not a current runtime contract. The statistics below come from the named local stats database snapshot, not from checked-in tests or runtime code.

1. The problem

OpenAI frames tool calls in the Harmony chat protocol:

<|start|>assistant<|channel|>commentary to=functions.<NAME><|message|>{ARGS}<|call|>

<|channel|>commentary to=functions.NAME is the routing header — control tokens consumed by the runtime to dispatch the call. These tokens never appear as content under normal operation; the runtime strips them.

The defect: gpt-5 models occasionally emit, as ordinary content inside {ARGS}, the plain-text shadow of these routing tokens — the same characters without the <|…|> brackets — and continue producing more pseudo-routing structure (channel name, body marker, multilingual spam, fake tool-result framing). The contamination lives inside the visible tool argument and is dispatched to the tool as if it were intended content.

Critical detail. The actual <|start|> / <|channel|> / <|message|> / <|call|> special tokens almost never appear in tool args. What leaks is the bracket-less spelling — analysis to=functions.X code … — because OpenAI applies a logit mask suppressing the control-token IDs inside the args region. The mass that would have gone to those special tokens redistributes onto the un-bracketed plain-text representation the model also learned. This makes the leak structurally invisible to the routing parser and lands it in the tool input verbatim.

Manifestation in tool args (real corpus example):

~      add_function(iso, ctx, ns, "installSystemChangeObserver",
        os_install_system_change_observer);】【"】【analysis to=functions.edit
        code above เงินไทยฟรีuser to=functions.edit code …

The leading code is real and intended. Everything after the first non-Latin token through the next clean structural boundary is corruption.

---

2. Observed statistics & failure modes

Source: ~/.omp/stats.db (ss_tool_calls, ss_assistant_msgs), through 2026-05-10. 1.05M tool calls scanned.

2.1 Rate

Model	Leaks in tool args	Calls	per million
gpt-5.4	37	226,957	163
gpt-5.3-codex	17	112,243	151
gpt-5.5	2	80,750	25
gpt-5.2-codex	0	—	—

Plus 15 hits in assistant visible text / thinking blobs.

2.2 Tool distribution

Tool	Hits
edit	38
eval	11
report_tool_issue	3
grep/read/search/yield	1 each

Concentrated in tools with free-form (non-JSON-schema) argument formats.

2.3 Leak shape (deterministic)

LEAK         ::= JUNK_PREFIX MARKER CHANNEL_BODY (LEAK)?
MARKER       ::= "to=functions." TOOL_NAME
CHANNEL_BODY ::= " code " (SPAM | reasoning_prose | fake_tool_output)*
JUNK_PREFIX  ::= (GLITCH_TOKEN | CHANNEL_WORD | NON_LATIN_RUN | "}" | "】【")+

Cascading is common. Of 96 marker occurrences across 71 contaminated records, 39 contain ≥2 markers and 7 contain ≥3 — the model emits multiple fake to=functions.X code … blocks back-to-back, often with fake code_output\nCell N:\n… framing between them. Once the plain-text scaffolding is in the residual stream, the prefix now looks like a fresh tool envelope start, so the macro prior over continuations keeps voting for more scaffolding. Self-amplifying.

2.4 Glitch tokens

Single-token identifiers in o200k_base whose embeddings appear to be near-init from underrepresentation in post-training. ASCII residue immediately before the marker in the natural corpus:

Surface string	Single-token	Token ID	Hits in corpus
Japgolly	✅	199,745	1
Jsii	✅	114,318	(subtoken of Jsii_commentary)
Jsii_commentary	— (3 toks)	—	2
changedFiles	— (2 toks)	—	8
RTLU	— (2 toks)	—	3

Japgolly is in the last 0.13% of the vocabulary — the same family of GitHub-corpus residue that produced SolidGoldMagikarp in the 2023 GPT-2 vocabulary (Rumbelow & Watkins). SolidGoldMagikarp itself tokenizes to 5 tokens in o200k_base — that specific token was retired, but the class wasn’t.

For the multi-token entries, the corpus-level signature is the surface string; the underlying glitch trigger is a sub-token (e.g. Jsii inside Jsii_commentary). The detector list (G signal) keys on the surface strings.

Stable across unrelated sessions. Treated as a high-precision detector signal.

2.5 Channel-word leakage

analysis (5), assistant (5), commentary (3), user (1) appear directly preceding to=. Always bare words; never <|channel|>analysis or any other bracketed form. Consistent with §1 — the brackets are masked, the words are not.

2.6 Non-Latin spam residue

96 marker hits, by script: CJK 40, Cyrillic 12, Telugu/Kannada/Malayalam 18, Thai 8, Georgian 7, Armenian 7, Arabic 1. Recurring fragments are Chinese gambling SEO (大发时时彩, 天天中彩票), Georgian/Abkhaz junk, and Thai casino spam — well-known low-quality crawl residue.

This is the same script distribution observed in the controlled reproduction (§7.3), independent of the prompt’s natural language.

2.7 Failure-mode breakdown for the edit tool

The edit tool exists in two variants in the corpus:

Variant	Calls	Recovery
Patch-DSL ([PATH#TAG]/anchor/SWAP DEL INS ops)	27	Recoverable by op-truncation (§3.3)
JSON-schema ({path,edits:[…]})	11	Not recoverable — contamination is escaped inside JSON strings, parser accepts it cleanly, content would be written verbatim into source files

For Patch-DSL leaks specifically:

• 20/27 cases: contamination on the last input line; nothing follows.
• 7/27 cases: contamination mid-input; what follows is one of: a duplicate replay of an earlier file/anchor, intended content for a different tool call (the model started its next call inline), or pure hallucination. Post-contamination content is never trustworthy.

2.8 Mechanism (confirmed)

Prior collapse from null-embedding glitch tokens, into a control-token-masked basin whose mass redistributes onto the plain-text shadow of the Harmony protocol.

Step by step:

1. The model is mid-{ARGS} of a Harmony tool call. The runtime applies a logit mask suppressing structural control tokens (<|channel|>, <|message|>, <|call|>, <|start|>, <|end|>) inside the args region. Without this mask, normal generation would constantly hallucinate envelope-closes; with it, those token IDs have logit -∞ in args.
2. A glitch token g is sampled. By construction g was in the BPE merge corpus but barely in LM/RL training, so its input embedding e_g ≈ near-init noise of small norm.
3. At position t+1, the residual update h_{t+1} ≈ LN(h_t + e_g + Attn + MLP) is dominated by the prefix-derived terms; the just-emitted-token signal is effectively absent. Generation diversity normally comes from e_x steering the residual into different sub-regions — stripped here.
4. The next-token distribution therefore collapses onto the conditional prior over continuations of the prefix, with local conditioning removed. In a tool-calling rollout context, that prior is sharply peaked on Harmony scaffolding (control tokens + routing tokens) — that’s what RL trained.
5. The mask zeros the control-token IDs. Mass redistributes onto the next-best continuation: the un-bracketed surface-form spelling of the same protocol (analysis, commentary, to=functions.X, code). This spelling is unmasked because those characters are ordinary tokens.
6. Once a few tokens of plain-text scaffolding land in the residual stream, the prefix now resembles a fresh envelope start. The macro prior keeps voting for more scaffolding. Cascading (§2.3) follows.
7. Multilingual spam after the marker is the same prior-collapse continuation, drawn from the training neighborhood of the glitch token (often ESL/auto-generated multilingual web junk — exactly the crawl residue in §2.6).

Two corollaries the corpus data demanded but only the experiment explained:

• The brackets never appear (§1, §2.5). The mask is what makes the leak land in plain text instead of as a real envelope-close.
• Counterintuitive grammar dependency (§7.4). The leak is worse in formats closest to OpenAI’s training distribution. Off-distribution custom grammars dampen the macro-prior basin; the official *** Begin Patch format is the strongest collapse target.

The 2023 SolidGoldMagikarp paper documented mechanism (1)+(2)+(4). The new piece is (5): when constrained decoding masks the natural collapse target, the mass laundered through the un-masked plain-text shadow becomes a structurally-invisible exfiltration channel.