Skip to content

fix: Updating Langchain handler issues #118

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
May 22, 2025

Conversation

ajaynayak
Copy link
Collaborator

@ajaynayak ajaynayak commented May 1, 2025

Fixing two issues:

  1. Node name extraction (Langchain sometimes passes a null serialized object)
    https://app.shortcut.com/galileo/story/30649/langchain-langgraph-warning-langchain-core-callbacks-manager-error-in-galileocallback-on-chain-start-callback-attributeerror

  2. Allowing users to pass a GalileoLogger with a started trace to the Langchain handler

@ajaynayak ajaynayak marked this pull request as ready for review May 22, 2025 05:49
@ajaynayak ajaynayak requested a review from a team as a code owner May 22, 2025 05:49
Copy link

codecov bot commented May 22, 2025

Codecov Report

Attention: Patch coverage is 92.85714% with 3 lines in your changes missing coverage. Please review.

Project coverage is 78.64%. Comparing base (71d6676) to head (aaebcac).
Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
src/galileo/handlers/langchain/handler.py 88.88% 3 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #118      +/-   ##
==========================================
+ Coverage   78.54%   78.64%   +0.09%     
==========================================
  Files          36       36              
  Lines        2629     2664      +35     
==========================================
+ Hits         2065     2095      +30     
- Misses        564      569       +5     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@ajaynayak ajaynayak changed the title fix: Updating Langchain handler node name extraction fix: Updating Langchain handler issues May 22, 2025
@andriisoldatenko
Copy link
Contributor

@ajaynayak tested and it works!

import os

from langchain_community.document_loaders import WebBaseLoader
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain_core.vectorstores import InMemoryVectorStore # Not used in final setup but present in original imports
from langchain_openai import OpenAIEmbeddings, ChatOpenAI
from langchain.tools.retriever import create_retriever_tool
from langgraph.graph import END, StateGraph, START
from langgraph.graph.message import add_messages
from langgraph.prebuilt import ToolNode, tools_condition
from typing import Annotated, Sequence, TypedDict
from langchain_core.messages import BaseMessage, HumanMessage

from galileo.handlers.langchain import GalileoCallback

os.environ["OPENAI_API_KEY"] = ""
os.environ["GALILEO_API_KEY"] = ""
os.environ["GALILEO_PROJECT"] = "handler-test"
os.environ["GALILEO_LOG_STREAM"] = "handler-test"
os.environ["GALILEO_CONSOLE_URL"] = ""



urls = [
    "https://lilianweng.github.io/posts/2024-11-28-reward-hacking/",
    "https://lilianweng.github.io/posts/2024-07-07-hallucination/",
    "https://lilianweng.github.io/posts/2024-04-12-diffusion-video/",
]

docs = [WebBaseLoader(url).load() for url in urls]

docs[0][0].page_content.strip()[:1000]

docs_list = [item for sublist in docs for item in sublist]

text_splitter = RecursiveCharacterTextSplitter.from_tiktoken_encoder(
    chunk_size=100, chunk_overlap=50
)
doc_splits = text_splitter.split_documents(docs_list)


vectorstore = InMemoryVectorStore.from_documents(
    documents=doc_splits, embedding=OpenAIEmbeddings()
)
retriever = vectorstore.as_retriever()

retriever_tool = create_retriever_tool(
    retriever,
    "retrieve_blog_posts",
    "Search and return information about Lilian Weng blog posts.",
)

retriever_tool.invoke({"query": "types of reward hacking"})

tools = [retriever_tool]

class AgentState(TypedDict):
    # The add_messages function defines how an update should be processed
    # Default is to replace. add_messages says "append"
    messages: Annotated[Sequence[BaseMessage], add_messages]



def agent(state):
    """
    Invokes the agent model to generate a response based on the current state. Given
    the question, it will decide to retrieve using the retriever tool, or simply end.

    Args:
        state (messages): The current state

    Returns:
        dict: The updated state with the agent response appended to messages
    """
    print("--- CALLING AGENT ---")
    messages = state["messages"]
    # Using gpt-4-turbo as it's generally better at tool use
    model = ChatOpenAI(temperature=0, streaming=True, model="gpt-4-turbo-preview") # or "gpt-3.5-turbo"
    model = model.bind_tools(tools)
    response = model.invoke(messages)
    # We return a list, because this will get added to the existing list
    return {"messages": [response]}


retrieve_node = ToolNode([retriever_tool])

# Define the graph
workflow = StateGraph(AgentState)

# Add nodes
workflow.add_node("agent", agent)
workflow.add_node("retrieve", retrieve_node)
image

@ajaynayak ajaynayak merged commit 4304d16 into main May 22, 2025
19 checks passed
@ajaynayak ajaynayak deleted the langchain-handler-decorator-fixes branch May 22, 2025 17:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants