Chain-of-thought leakage in AI agents: keep reasoning out of user replies
Chain-of-thought leakage happens when an AI agent’s private reasoning trace, scratchpad, tool narration, or <thinking> scaffold reaches a user-visible channel. The fix is not a better prompt alone. Production agents need output boundaries at every delivery layer: model adapter, transcript writer, tool renderer, and channel adapter.
For a chat demo, a leaked thought block is embarrassing. For a long-running agent connected to files, chats, tickets, calendars, and approval flows, it is a security and trust problem. The trace may restate private context, reveal routing decisions, expose tool state, or show an internal instruction that was never meant to leave the agent runtime.
OpenClaw’s 2026.6.5 beta release is a useful case study. QQBot now strips model reasoning and thinking scaffolding before native delivery. That boundary work matters as reasoning models become normal infrastructure.
Why chain-of-thought leakage is different in agents
A chatbot usually has one obvious output: the final answer in the chat window. An agent has more places where text can accidentally escape.
| Surface | What can leak | Why it matters |
|---|---|---|
| Final reply | Raw <thinking> blocks, reasoning summaries, hidden scratchpad text | Users see implementation details instead of the answer |
| Progress updates | Tool plans, partial reasoning, internal status strings | Streaming channels can publish drafts before final sanitization |
| Transcripts and logs | Full traces, tool inputs, copied private context | Stored records become a second exposure path |
| Channel adapters | Platform-specific markdown, cards, replies, voice text | Each adapter can bypass a central output cleanup step |
| Tool renderers | Tool stdout, rich content, provider error text | Tools can return data that was meant for the model, not the user |
The agent part matters because agents bridge contexts. A reasoning trace may mention a user’s email, a ticket title, a file path, an approval target, or why the model decided a person was allowed to perform an action. Even if the final answer is harmless, the intermediate trace can carry sensitive information across boundaries.
This is also why an output-only regex at the end of a request is weak. A progress draft may already have been sent to Discord, Telegram, QQBot, Slack, Matrix, or a web client. The safer pattern is layered: keep private traces separate, sanitize at materialization, and sanitize again at native channel delivery.
What recent research says about reasoning traces
Recent research is converging on a blunt point: reasoning traces are not automatically safe just because they feel internal.
The 2025 paper “Leaky Thoughts” studied reasoning traces in personal-agent settings and argued that traces frequently contain sensitive user data. The authors found that longer reasoning can make final answers more cautious while causing the model to reason more verbosely over private data.
A 2026 paper, “Safer Reasoning Traces,” measured direct inference-time PII leakage and reported a sharp gap between plain prompting and chain-of-thought prompting. Average leakage rose from 52.3% under plain prompting to 86.3% under chain-of-thought, with median chain-of-thought leakage at 100% for many model and PII combinations. The operational lesson is simple: step-by-step reasoning increases the amount of sensitive text the model may generate.
OpenAI and Anthropic add a second lesson. Chain-of-thought can be useful for monitoring, but it is not a clean user-facing explanation. OpenAI suggests using a separate summarizer or sanitizer when user-facing reasoning is needed. Anthropic’s work on faithfulness also warns that stated reasoning may not explain the true cause of a model’s answer.
The policy is straightforward: treat raw reasoning as operational telemetry, not customer content.
Where chain-of-thought leakage starts in production
Most leaks come from boring integration seams, not dramatic jailbreaks.
- A model returns content wrapped in
<thinking>tags. - A provider adapter treats the block as normal assistant text.
- A transcript writer stores the whole assistant message.
- A channel adapter reads from the transcript or stream buffer.
- The channel sends the raw text because it assumes upstream code already cleaned it.
That chain can happen in reverse too. A channel plugin may build a native card or voice response from a partial draft before the final answer exists. A tool may return structured content that the model should read, while a renderer turns it into visible text. A retry path may replay an earlier assistant message that still contains private scaffolding.
The hard part is not recognizing <thinking> once. The hard part is making sure every output path agrees on which text is private, which text is user-visible, and which text is safe to store.
How OpenClaw’s channel boundary fits
OpenClaw already frames agent work around channels, tools, skills, memory, approvals, and transcript boundaries. If you are new to the architecture, the short version is in how OpenClaw works: the runtime routes model work through controlled surfaces instead of letting every integration invent its own agent loop.
The 2026.6.5 beta QQBot change is a channel-boundary fix. The release note says QQBot strips model reasoning and thinking scaffolding before native delivery. That means the channel adapter does not trust that the upstream model output is already clean. It enforces its own boundary right before user delivery.
That is the right instinct for multi-channel agents. A Slack reply, QQBot reply, Telegram progress draft, Discord voice handoff, or web chat message may all have different formatting and delivery semantics. One shared sanitizer helps, but platform-specific adapters still need last-mile protection.
This also connects to broader agent audit design. In an AI agent audit logs checklist, we recommend logging inputs, tools, policy decisions, and outcomes without storing private chain-of-thought by default.
A practical checklist to prevent reasoning leaks
Use this as a pre-release review for any agent channel or provider adapter.
1. classify output before formatting it
Do not pass arbitrary model text directly to channel formatting. Split content into at least four classes:
- private reasoning trace
- user-visible answer
- operational progress
- tool result or attachment content
If the runtime cannot classify a block, default to private or error-visible, not user-visible.
2. strip known scaffolding at channel delivery
Strip or quarantine common wrappers such as <thinking>, <think>, reasoning blocks, provider-specific trace fields, hidden scratchpad markers, and malformed markdown fences before a native channel sends anything. Do this even if an upstream adapter already claims to sanitize.
3. keep progress drafts on a stricter path
Progress messages are risky because they are sent before the agent has finished. Limit progress to short operational states: “checking calendar”, “waiting for approval”, “uploading attachment”. Do not stream model scratchpad text as progress.
4. store summaries, not raw thoughts, by default
For logs and transcripts, store a reason code, tool call, policy decision, and final outcome. Raw reasoning should require an explicit debug mode, a short retention window, and access controls. If you need more observability, sample traces rather than recording everything. See the broader OpenClaw overview for why agent ownership and local control matter here.
5. test with adversarial fixtures
Add tests that feed channel adapters messages like:
<thinking>I should not send this. The user's private file path is /Users/alice/taxes.pdf.</thinking>
Final answer: I found the document and can summarize it.The expected user-visible output should contain only the final answer. The transcript policy should decide whether the private block is dropped, redacted, or stored in a restricted debug artifact.
6. check every channel, not just web chat
A leak fixed in the web UI can still exist in QQBot, Discord, Matrix, Slack, Telegram, WhatsApp, email, or voice. Agent platforms should keep a channel matrix for reasoning-sanitization tests. This is one place where a self-hosted system has an advantage over a closed agent SaaS: you can inspect and test the actual boundary. If you are comparing options, OpenClaw vs alternatives covers the ownership tradeoff.
FAQ
What is chain-of-thought leakage?
Chain-of-thought leakage is the exposure of an AI model’s private reasoning trace, scratchpad, or thinking scaffold to users, logs, tools, or channels that should only receive the final answer or a safe summary.
Should AI agents ever show reasoning to users?
They should show concise explanations and audit trails, not raw reasoning. A useful user-facing explanation says what the agent did, which tools it used, what changed, and where uncertainty remains. It does not need private scratchpad text.
Is stripping <thinking> tags enough?
No. It is a necessary last-mile guard, not a complete policy. You also need output classification, transcript controls, progress-message limits, tool-result boundaries, retention rules, and tests for every channel adapter.
Why does this matter more for multi-channel agents?
Multi-channel agents have many send paths. A web chat sanitizer may not protect a QQBot native reply, a Discord voice handoff, or a Telegram progress draft. Each channel needs its own final boundary before delivery.
The bottom line on chain-of-thought leakage
Chain-of-thought leakage is not just a model behavior. It is a systems problem. Reasoning models generate traces; agent runtimes decide whether those traces become user content, stored records, or private telemetry.
The safest default is simple: raw reasoning stays inside the runtime. Users get final answers, safe summaries, citations, approvals, and audit logs. Channel adapters enforce that rule again at the edge.
Sources: OpenClaw v2026.6.5-beta.2 release notes, Leaky Thoughts: Large Reasoning Models Are Not Private Thinkers, Safer Reasoning Traces: Measuring and Mitigating Chain-of-Thought Leakage in LLMs, OpenAI on chain-of-thought monitoring, Anthropic on reasoning faithfulness