II-Agent is an open-source intelligent assistant designed to streamline and enhance workflows across multiple domains. It represents a significant advancement in how we interact with technology—shifting from passive tools to intelligent systems capable of independently executing complex tasks.
📢 Join Our Discord Channel! Looking forward to seeing you there! 🎉
Intro.to.II-Agent.mp4
II Agent is built around providing an agentic interface to leading language models. It offers:
- A CLI interface for direct command-line interaction
- A WebSocket server that powers a modern React-based frontend
- Integration with multiple LLM providers:
- Anthropic Claude models (direct API or via Google Cloud Vertex AI)
- Google Gemini models (direct API or via Google Cloud Vertex AI)
II-Agent has been evaluated on the GAIA benchmark, which assesses LLM-based agents operating within realistic scenarios across multiple dimensions including multimodal processing, tool utilization, and web searching.
We identified several issues with the GAIA benchmark during our evaluation:
- Annotation Errors: Several incorrect annotations in the dataset (e.g., misinterpreting date ranges, calculation errors)
- Outdated Information: Some questions reference websites or content no longer accessible
- Language Ambiguity: Unclear phrasing leading to different interpretations of questions
Despite these challenges, II-Agent demonstrated strong performance on the benchmark, particularly in areas requiring complex reasoning, tool use, and multi-step planning.
You can view the full traces of some samples here: GAIA Benchmark Traces
- Docker Compose
- Python 3.10+
- Node.js 18+ (for frontend)
- At least one of the following:
- Anthropic API key, or
- Google Gemini API key, or
- Google Cloud project with Vertex AI API enabled
Tip
- For best performance, we recommend using Claude 4.0 Sonnet or Claude Opus 4.0 models.
- For fast and cheap, we recommend using GPT4.1 from OpenAI.
- Gemini 2.5 Pro is a good balance between performance and cost.
You need to set up .env files to run frontend.
Shortcut: Check file .env.example for example of .env file.
For the frontend, create a .env file in the frontend directory, point to the port of your backend:
NEXT_PUBLIC_API_URL=http://localhost:8000
NEXT_PUBLIC_BASE_URL=http://localhost:3000
GOOGLE_API_KEY=<your_google_api_key> # Optional, for Google Drive integration
GOOGLE_CLIENT_ID=<your_google_client_id> # Optional, for Google Drive integration
GOOGLE_CLIENT_SECRET=<your_google_client_secret> # Optional, for Google Drive integration- Clone the repository
- Set up the environment as mentioned in the above step
- If you are using Anthropic Client run
docker compose up
Our II-Agent supports popular models such as Claude, Gemini, and OpenAI. If you’d like to use a model from OpenRouter, simply configure your OpenAI endpoint with your OpenRouter API key. If you are using Vertex, run with these variables
GOOGLE_APPLICATION_CREDENTIALS=absolute-path-to-credential docker compose up
-
Clone the repository
-
Set up Python environment:
python -m venv .venv source .venv/bin/activate # On Windows: .venv\Scripts\activate pip install -e .
-
Set up frontend (optional):
cd frontend npm install
- Start the WebSocket server:
When using Anthropic client:
STATIC_FILE_BASE_URL=http://localhost:8000 python ws_server.py --port 8000When using Vertex:
GOOGLE_APPLICATION_CREDENTIALS=path-to-your-credential STATIC_FILE_BASE_URL=http://localhost:8000 python ws_server.py --port 8000- Start the frontend (in a separate terminal):
cd frontend
npm run dev- Open your browser to http://localhost:3000
ws_server.py: WebSocket server for the frontendsrc/ii_agent/: Core agent implementationagents/: Agent implementationsllm/: LLM client interfacestools/: Tool implementationsutils/: Utility functions
II-Agent is a versatile open-source assistant built to elevate your productivity across domains:
| Domain | What II‑Agent Can Do |
|---|---|
| Research & Fact‑Checking | Multistep web search, source triangulation, structured note‑taking, rapid summarization |
| Content Generation | Blog & article drafts, lesson plans, creative prose, technical manuals, Website creations |
| Data Analysis & Visualization | Cleaning, statistics, trend detection, charting, and automated report generation |
| Software Development | Code synthesis, refactoring, debugging, test‑writing, and step‑by‑step tutorials across multiple languages |
| Workflow Automation | Script generation, browser automation, file management, process optimization |
| Problem Solving | Decomposition, alternative‑path exploration, stepwise guidance, troubleshooting |
The II-Agent system represents a sophisticated approach to building versatile AI agents. Our methodology centers on:
-
Core Agent Architecture and LLM Interaction
- System prompting with dynamically tailored context
- Comprehensive interaction history management
- Intelligent context management to handle token limitations
- Systematic LLM invocation and capability selection
- Iterative refinement through execution cycles
-
Planning and Reflection
- Structured reasoning for complex problem-solving
- Problem decomposition and sequential thinking
- Transparent decision-making process
- Hypothesis formation and testing
-
Execution Capabilities
- File system operations with intelligent code editing
- Command line execution in a secure environment
- Advanced web interaction and browser automation
- Task finalization and reporting
- Specialized capabilities for various modalities (Experimental) (PDF, audio, image, video, slides)
- Deep research integration
-
Context Management
- Token usage estimation and optimization
- Strategic truncation for lengthy interactions
- File-based archival for large outputs
-
Real-time Communication
- WebSocket-based interface for interactive use
- Isolated agent instances per client
- Streaming operational events for responsive UX
The II-Agent framework, architected around the reasoning capabilities of large language models like Claude 4.0 Sonnet or Gemini 2.5 Pro, presents a comprehensive and robust methodology for building versatile AI agents. Through its synergistic combination of a powerful LLM, a rich set of execution capabilities, an explicit mechanism for planning and reflection, and intelligent context management strategies, II-Agent is well-equipped to address a wide spectrum of complex, multi-step tasks. Its open-source nature and extensible design provide a strong foundation for continued research and development in the rapidly evolving field of agentic AI.
We would like to express our sincere gratitude to the following projects and individuals for their invaluable contributions that have helped shape this project:
-
AugmentCode: We have incorporated and adapted several key components from the AugmentCode project. AugmentCode focuses on SWE-bench, a benchmark that tests AI systems on real-world software engineering tasks from GitHub issues in popular open-source projects. Their system provides tools for bash command execution, file operations, and sequential problem-solving capabilities designed specifically for software engineering tasks.
-
Manus: Our system prompt architecture draws inspiration from Manus's work, which has helped us create more effective and contextually aware AI interactions.
-
Index Browser Use: We have built upon and extended the functionality of the Index Browser Use project, particularly in our web interaction and browsing capabilities. Their foundational work has enabled us to create more sophisticated web-based agent behaviors.
We are committed to open source collaboration and believe in acknowledging the work that has helped us build this project. If you feel your work has been used in this project but hasn't been properly acknowledged, please reach out to us.
