[WIP] Migrate from LangChain to LiteLLM (major upgrade) #1426
+3,519
−4,674
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* add new /api/ai/models endpoint * add litellm dependency * add new model ID input component * fix custom model ID input * update ConfigManager to accept litellm model IDs * update Jupyternaut to use litellm * remove LangChain partner packages from dependencies * remove LangChain provider entry points * pre-commit * rename /api/ai/models => /api/ai/models/chat * simplify model settings UI
dlqqq
changed the title
dlqqq
changed the title
This was referenced Jul 23, 2025
* remove LangChain from jupyter_ai_magics * remove unused useServerInfo() hook in frontend * remove LangChain references from jupyter_ai * remove langchain from test & cookiecutter packages * move all completions code to jupyter_ai.completions * remove LangChain provider code from ConfigManager * minor fixes to allow JAI to start
* add EnvSecretsManager and secrets REST API * make ListSecrets endpoint more generic * fix UpdateSecretsRequest type * fix bug where last secret cannot be deleted * remove error log used in dev * add new <SecretsSection /> component * tweak UI appearance * make settings page scrollable, fix #1409 * show no static secrets by default * show something when no secrets are present * adjust language * fix success & error alerts on updating secrets * fix lint * fix bug when .gitignore and .env do not exist * bump to 2s poll interval in EnvSecretsManager * disable model parameters input for now, fix merge conflicts
* Adding `magics` functionality to the `litellm` implementation * updates for initializing aliases and resolving types, mypy * Update magics.py * init alias * Update test_magics.py * Update magics.py * Changed `error` to `fix` in magics * Update index.md * Update test_magics.py * Update magics.py * streamlining magics * removed update command * initial_chat_model * magics pre-commit * Update magics.py
Thanks David for approving this.
* update prettier ignore file * fix prettier errors & warnings * fix mypy errors in jupyter_ai * skip ConfigManager and inline completion tests until fixed in v3 * add tests for secrets manager * fix mypy error * run minimum dependency tests on Python 3.10 * comment out precommit job to be added in future * updated test_magics.py --------- Co-authored-by: Sanjiv Das <srdas@scu.edu>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR contains all of the code changes required for the complete migration from LangChain to LiteLLM as the model library used by Jupyter AI, as proposed in #1418. We will iterate on this through smaller PRs targeting the
litellmbranch. These "sub-PRs" are listed below.This PR will be merged into the
mainbranch once the "issues remaining" (listed below) are addressed and thelitellmbranch is stable for general use. Until then, this PR will be marked as a draft.Issues closed
Closes [v3-beta] EPIC: Define new API for models & model providers #1312
jupyter_aiimport time is too slow #1115Closes Error on chat openai with litellm api for self hosted llm #1308
Closes Add Deepseek support #1434
Issues remaining
jupyter_aiimport time to confirmjupyter_aiimport time is too slow #1115 is resolved.Sub-PRs merged
Summary of changes
LangChain providers (e.g.
ChatOpenAIclass) and provider dependencies (e.g.langchain-openaioptional dependency) have been removed. The only optional dependency isboto3, which is required for using Bedrock models with LiteLLM.Jupyter AI no longer spends 3000-4000ms loading providers from entry points on server startup. Around 1000 models from several providers are available for use without loading entry points or installing optional dependencies.
Model IDs now follow the LiteLLM syntax,
<provider-name>/<model-name>.The AI settings page has been rewritten to be more usable and reliable.