-
Notifications
You must be signed in to change notification settings - Fork 2.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix(community,aws): Update handleLLMNewToken to include chunk metadata #7568
Conversation
The latest updates on your projects. Learn more about Vercel for Git ↗︎
1 Skipped Deployment
|
Oof, thanks! Keep in mind you should generally be using |
undefined, | ||
undefined, | ||
undefined, | ||
{ chunk } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This will unfortunately need a cast/type guard:
Error: src/chat_models/bedrock/web.ts(927,17): error TS2322: Type 'ChatGeneration' is not assignable to type 'ChatGenerationChunk | GenerationChunk | undefined'.
Property 'concat' is missing in type 'ChatGeneration' but required in type 'ChatGenerationChunk'.
Error: Command failed: tsc --outDir dist-cjs/ -p tsconfig.cjs.json
at ChildProcess.<anonymous> (file:///home/runner/work/langchainjs/langchainjs/libs/langchain-scripts/dist/build/index.js:21:24)
at ChildProcess.emit (node:events:517:28)
at maybeClose (node:internal/child_process:1098:16)
at ChildProcess._handle.onexit (node:internal/child_process:303:5)
Error: command finished with error: command (/home/runner/work/langchainjs/langchainjs/libs/langchain-community) /tmp/xfs-4617c204/yarn run build:internal exited (1)
Error: @langchain/community#build:internal: command (/home/runner/work/langchainjs/langchainjs/libs/langchain-community) /tmp/xfs-4617c204/yarn run build:internal exited (1)
Small fix needed - if you don't have time I should be able to get to it tomorrow |
Sorry for the oversight. I haven't been able to make tsc working locally for langchainjs. I'm basically testing these changes with I pushed another commit to address the same issue in Hope these changes make the CI pass. 🤞 |
undefined, | ||
undefined, | ||
{ | ||
chunk: chunk as ChatGenerationChunk, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this is wrong, this is the chunk from Amazon
I'll go in and fix
); | ||
} else { | ||
// eslint-disable-next-line no-void | ||
void runManager?.handleLLMNewToken(chunk.text); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ahh i see 🤦 . thanks!
* Rename auth method in docs * fix(core): Fix trim messages mutation bug (langchain-ai#7547) * release(core): 0.3.31 (langchain-ai#7548) * fix(community): Updated Embeddings URL (langchain-ai#7545) * fix(community): make sure guardrailConfig can be added even with anthropic models (langchain-ai#7542) * docs: Fix PGVectorStore import in install dependencies (TypeScript) example (langchain-ai#7533) * fix(community): Airtable url (langchain-ai#7532) * docs: Fix typo in OpenAIModerationChain example (langchain-ai#7528) * docs: Resolves langchain-ai#7483, resolves langchain-ai#7274 (langchain-ai#7505) Co-authored-by: jacoblee93 <[email protected]> * docs: Rename auth method in IBM docs (langchain-ai#7524) * docs: correct misspelling (langchain-ai#7522) Co-authored-by: jacoblee93 <[email protected]> * release(community): 0.3.25 (langchain-ai#7549) * feat(azure-cosmosdb): add session context for a user mongodb (langchain-ai#7436) Co-authored-by: jacoblee93 <[email protected]> * release(azure-cosmosdb): 0.2.7 (langchain-ai#7550) * fix(ci): Fix build (langchain-ai#7551) * feat(anthropic): Add Anthropic PDF support (document type) in invoke (langchain-ai#7496) Co-authored-by: jacoblee93 <[email protected]> * release(anthropic): 0.3.12 (langchain-ai#7552) * chore(core,langchain,community): Relax langsmith deps (langchain-ai#7556) * release(community): 0.3.26 (langchain-ai#7557) * release(core): 0.3.32 (langchain-ai#7558) * Release 0.3.12 (langchain-ai#7559) * fix(core): Prevent cache misses from triggering model start callback runs twice (langchain-ai#7565) * fix(core): Ensure that cached flag in run extras is only set for cache hits (langchain-ai#7566) * release(core): 0.3.33 (langchain-ai#7567) * feat(community): Adds graph_document to export list (langchain-ai#7555) Co-authored-by: quantropi-minh <[email protected]> Co-authored-by: jacoblee93 <[email protected]> * fix(langchain): Fix ZeroShotAgent createPrompt with correct formatted tool names (langchain-ai#7510) * docs: Add document for AzureCosmosDBMongoChatMessageHistory (langchain-ai#7519) Co-authored-by: root <root@CPC-yangq-FRSGK> * fix(langchain): Allow pulling hub prompts with associated models (langchain-ai#7569) * fix(community,aws): Update handleLLMNewToken to include chunk metadata (langchain-ai#7568) Co-authored-by: jacoblee93 <[email protected]> * feat(community): Provide fallback relationshipType in case it is not present in graph_transformer (langchain-ai#7521) Co-authored-by: quantropi-minh <[email protected]> Co-authored-by: jacoblee93 <[email protected]> * docs: Add redirect (langchain-ai#7570) * fix(langchain,core): Add shim for hub mustache templates with nested input variables (langchain-ai#7581) * fix(chat-models): honor disableStreaming even for `generateUncached` (langchain-ai#7575) * release(core): 0.3.34 (langchain-ai#7584) * feat(langchain): Add hub entrypoint with automatic dynamic entrypoint of models (langchain-ai#7583) * chore(ollama): Export `OllamaEmbeddingsParams` interface (langchain-ai#7574) * docs: Clarify tool creation process in structured outputs documentation (langchain-ai#7578) Co-authored-by: Sahar Shemesh <[email protected]> Co-authored-by: jacoblee93 <[email protected]> * fix(community): Set awaitHandlers to true in upstash ratelimit (langchain-ai#7571) Co-authored-by: Jacob Lee <[email protected]> * fix(core): Fix trim messages mutation (langchain-ai#7585) * feat(openai): Make only AzureOpenAI respect Azure env vars, remove class defaults, update withStructuredOutput defaults (langchain-ai#7535) * fix(community): Make postgresConnectionOptions optional in PostgresRecordManager (langchain-ai#7580) Co-authored-by: jacoblee93 <[email protected]> * release(community): 0.3.27 (langchain-ai#7586) * release(ollama): 0.1.5 (langchain-ai#7587) * Release 0.3.13 (langchain-ai#7588) * release(openai): 0.4.0 (langchain-ai#7589) * release(core): 0.3.35 (langchain-ai#7590) * fix(ci): Update lock (langchain-ai#7591) * feat(core): Allow passing returnDirect in tool wrapper params (langchain-ai#7594) * release(core): 0.3.36 (langchain-ai#7595) * fix(openai): Revert Azure default withStructuredOutput changes (langchain-ai#7596) * release(openai): 0.4.1 (langchain-ai#7597) * feat(openai): Refactor to allow easier subclassing (langchain-ai#7598) * release(openai): 0.4.2 (langchain-ai#7599) * feat(deepseek): Adds Deepseek integration (langchain-ai#7604) * release(deepseek): 0.0.1 (langchain-ai#7608) * feat: update Novita AI doc (langchain-ai#7602) * Add deployment chat to chat class * feat(langchain): Add DeepSeek to initChatModel (langchain-ai#7609) * Release 0.3.14 (langchain-ai#7611) * fix: Add test for pdf uploads anthropic (langchain-ai#7613) * feat: Update google genai to support file uploads (langchain-ai#7612) * chore(google-genai): Drop .only in test (langchain-ai#7614) * release(google-genai): 0.1.7 (langchain-ai#7615) * Upadate Watsonx sdk * fix(core): Fix stream events bug when errors are thrown too quickly during iteration (langchain-ai#7617) * release(core): 0.3.37 (langchain-ai#7619) * fix(langchain): Fix Groq import for hub (langchain-ai#7620) * docs: update README/intro * Release 0.3.15 * feat(community): improve support for Tavily search tool args (langchain-ai#7561) * feat(community): Add boolean metadata type support in Supabase structured query translator (langchain-ai#7601) * feat(google-genai): Add support for fileUri in media type in Google GenAI (langchain-ai#7621) Co-authored-by: Jacob Lee <[email protected]> * release(google-genai): 0.1.8 (langchain-ai#7628) * release(community): 0.3.28 (langchain-ai#7629) * Rework interfaces in llms as well * Bump watsonx-ai sdk version * Remove unused code * Add fake auth * Fix broken changes --------- Co-authored-by: Jacob Lee <[email protected]> Co-authored-by: Jacky Chen <[email protected]> Co-authored-by: Mohamed Belhadj <[email protected]> Co-authored-by: Brian Ploetz <[email protected]> Co-authored-by: Eduard-Constantin Ibinceanu <[email protected]> Co-authored-by: Jonathan V <[email protected]> Co-authored-by: ucev <[email protected]> Co-authored-by: crisjy <[email protected]> Co-authored-by: Adham Badr <[email protected]> Co-authored-by: Minh Ha <[email protected]> Co-authored-by: quantropi-minh <[email protected]> Co-authored-by: Chi Thu Le <[email protected]> Co-authored-by: fatmelon <[email protected]> Co-authored-by: root <root@CPC-yangq-FRSGK> Co-authored-by: Mohamad Mohebifar <[email protected]> Co-authored-by: David Duong <[email protected]> Co-authored-by: Brace Sproul <[email protected]> Co-authored-by: Matus Gura <[email protected]> Co-authored-by: Sahar Shemesh <[email protected]> Co-authored-by: Sahar Shemesh <[email protected]> Co-authored-by: Cahid Arda Öz <[email protected]> Co-authored-by: Jason <[email protected]> Co-authored-by: vbarda <[email protected]> Co-authored-by: Vadym Barda <[email protected]> Co-authored-by: Hugo Borsoni <[email protected]> Co-authored-by: Arman Ghazaryan <[email protected]> Co-authored-by: Andy <[email protected]>
When using langgraph workflow's
streamEvents
, I expect to seetool_calls
in thedata.chunk
payload of eventon_chat_model_stream
. This works as with ChatOpenAI, however, BedrockChat doesn't expose the actual chunk.After tracing the call stack, I learned that the issue stems from the
handleLLMNewToken
call and this is not in parity with ChatOpenAI's impl:langchainjs/libs/langchain-openai/src/chat_models.ts
Lines 1464 to 1471 in 1546a2e