Skip to content

feat(openrouter): native content-block streaming events - #38030

Open
Nick Hollon (nick-hollon-lc) wants to merge 1 commit into
nh/native-content-block-streamingfrom
nh/native-content-block-streaming-openrouter
Open

feat(openrouter): native content-block streaming events#38030
Nick Hollon (nick-hollon-lc) wants to merge 1 commit into
nh/native-content-block-streamingfrom
nh/native-content-block-streaming-openrouter

Conversation

@nick-hollon-lc

Copy link
Copy Markdown
Contributor

Stacked on #37995 (core BlockStreamTracker + Anthropic), which it branches from. Depends only on the core primitive — sibling of the OpenAI chain. Retargets to master once #37995 lands.

Gives ChatOpenRouter native _stream_chat_model_events / _astream_chat_model_events hooks so stream_events(version="v3") maps OpenRouter's stream directly to content-block events.

How it works

A bespoke converter (convert_openrouter_stream / async twin), sibling of the groq converter, builds text, reasoning, and tool-call blocks directly from OpenRouter's raw OpenAI-shaped delta, feeding the shared BlockStreamTracker. Tool-call args stream incrementally and finalize to a parsed dict. It does not depend on langchain-openai — it reuses only OpenRouter's own _create_usage_metadata.

OpenRouter specifics it handles: error chunks are propagated as a ValueError (matching _stream); top-level usage is accumulated; and OpenRouter's cost surfacing (cost/cost_details) plus native_finish_reason/model_name/etc. are carried into message-finish metadata, matching the compat bridge.

The win over the bridge

OpenRouter's chunk parser leaves reasoning in additional_kwargs; the native converter surfaces it as a reasoning block (and tool calls as tool_call blocks).

Worth careful review

  • Error-chunk propagation and the cost/finish metadata parity with _stream (a test asserts cost survives to message-finish).
  • The _StreamState helper shares per-chunk coercion/usage/error logic across the sync/async twins (no event-sequence drift).
  • Scope cut: reasoning_details not yet surfaced as blocks (noted in-code).
  • message_id keyword-only; message-start carries the LangChain run id.

@github-actions github-actions Bot added feature For PRs that implement a new feature; NOT A FEATURE REQUEST integration PR made that is related to a provider partner package integration internal openrouter `langchain-openrouter` package issues & PRs size: L 500-999 LOC labels Jun 10, 2026

@open-swe open-swe Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Open SWE Review found 1 potential issue.

Open in WebView Open SWE trace

Comment on lines +170 to +171
if choice.get("finish_reason"):
state.record_finish(chunk_dict, choice)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Preserve streamed logprobs metadata

When the caller requests streaming logprobs, OpenRouter includes them on choice["logprobs"]. The existing _stream path copies that into generation_info, and the v3 compat bridge merges it into message-finish.metadata; after this native hook is added, stream_events(version="v3", logprobs=True, ...) takes this converter instead and line 170 only records finish metadata, so those logprobs are dropped from the event stream entirely. Please accumulate/propagate choice.get("logprobs") here (and in the async twin) to keep parity with the bridge.

(Refers to lines 170-171)


Your feedback helps Open SWE learn. React with 👍 or 👎 to tell us if this review comment was useful.

@hanu-14

This comment was marked as spam.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feature For PRs that implement a new feature; NOT A FEATURE REQUEST integration PR made that is related to a provider partner package integration internal openrouter `langchain-openrouter` package issues & PRs size: L 500-999 LOC

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants