Skip to content

Add Context-Enabled Semantic Caching recipe to semantic cache folder #99

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

redis-phil
Copy link

Add Context-Enabled Semantic Caching Recipe
Overview
This PR adds a comprehensive Jupyter notebook demonstrating Context-Enabled Semantic Caching (CESC) - a three-tier caching architecture that combines Redis vector search with user personalization to deliver tailored LLM responses at 60-80% lower cost and sub-100ms latency.

What's Added

  • Complete production-ready implementation using Redis, OpenAI/Azure OpenAI, and comprehensive telemetry tracking

  • Real-world IT support scenario showcasing semantic similarity matching with user memory integration for personalized responses

Enterprise Impact
Provides quantifiable ROI with 200-400% first-year returns for organizations processing >10K daily LLM queries through intelligent model selection (GPT-4o for new content, GPT-4o-mini for personalization) and Redis-powered vector caching.

@abrookins
Copy link
Contributor

@redis-phil Thanks for the PR and 👋. Can you take a look at the failing build and make any changes to the notebook needed to get it passing? https://github.com/redis-developer/redis-ai-resources/actions/runs/16683036159/job/47230595099?pr=99

Copy link
Contributor

@abrookins abrookins left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @redis-phil! We do want recipes to be runnable outside of Colab, ideally. You can check the other recipes for examples of how this usually works (installing redis, using environment variables for secrets, etc.).

A few notes:

  • Remove broken AzureLLMClient.query() method - references self.generate_embedding, self.search_cache, self.user_context, self.telemetry, self.llm_client, self.store_response, which don't exist on that class. Seems like dead code?

  • Fix vector score logic - Code uses score >= 0.85 but RedisVL with HNSW+cosine returns distance (lower=better) not similarity (higher=better). Need to flip to score <= threshold with different threshold value. Also: why not RangeQuery with threshold? Finally, there is an option, normalize_vector_distance on queries that can flip this from a distance to a similarity value.

  • Look into telemetry issues - summarize() prints DataFrame then returns None, which gets printed as "None". Also raw cache hits log latency_ms=0 instead of measuring actual Redis query time.

  • You might consider caching more fields - Currently only stores content, content_vector, user_id. Missing prompt (original query), model (which model generated it), created_at, ttl for cache policy management.

  • Fix Colab-only secrets handling - Hard imports from google.colab import userdata, crashes outside Colab. Need try/except with fallback to os.getenv(), or just os.getenv().

  • Remove unused Redis client - Creates redis_client in cell 5 but never uses it, only uses one via SearchIndex.connect()

  • (Style nit) Fix inconsistent naming - GPT4_MODEL vs GPT4mini_MODEL inconsistent, should be something like MODEL_GPT4 and MODEL_GPT4_MINI

  • Update Colab badge - Points to private Drive ID that 404s for others. E.g. "<a href=\"https://colab.research.google.com/github/redis-developer/redis-ai-resources/blob/main/python-recipes/vector-search/01_redisvl.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants