Multi-User Chat Engine Workflow

[frk-ti8m]

Problem Statement

I have the following requirements for the chat app that I am implementing:

• supports multiple concurrent user sessions
• chat sessions are persisted to postgres db
• retrieve documents from an index with customizable filter on query
• possibly multiple retrieval steps
• possibly extra llm calls, e.g. for query translation

Llama-index Options

I have identified the following options in llama-index:

1. Maintain a ChatEngine (e.g. CondensePlusContextChatEngine) for each user session with Memory, and Retriever configured, build additional functionality like translation llm calls around ChatEngine
2. Subclass ChatEngine with custom processing logic, maintain an instance of that class for each user session
3. Implement custom processing and chat logic as Workflow, setting user session specific variables as Context, maintain an instance of that class for each user session
4. Implement custom processing and chat logic as Workflow, only instantiate this class once, maintain stateless workflow with respect to user session information and pass all user session related information as Events though the workflow

Question

Which implementation is the recommended one in llama-index?

[dosubot[bot]]

The most recommended and idiomatic approach in LlamaIndex for your requirements is to maintain a separate ChatEngine instance (like CondensePlusContextChatEngine) for each user session, with memory and retriever configured per session (your Option 1). This aligns best with how LlamaIndex manages state, memory, and user context, and makes it straightforward to extend with custom logic like translation LLM calls or multi-step retrievals. This approach also integrates cleanly with persistent storage (e.g., Postgres) for chat history via abstractions like ChatStore, which is designed for long-term session management and retrieval [source].

Option 3 (per-session Workflow with Context) is also valid if you need more complex, multi-step or multi-agent orchestration, but still requires per-session instantiation for clean state management. Option 4 (stateless, single Workflow instance) is only recommended for advanced scenarios where you want to maximize reuse and are comfortable externalizing all session state—this adds significant engineering overhead and is less idiomatic in LlamaIndex [source].

Subclassing ChatEngine (Option 2) is fine for deep customization, but most use cases can be handled by composing or extending existing engines and workflows.

In summary: per-user/session ChatEngine or Workflow instances are the recommended pattern for multi-user chat apps in LlamaIndex. Stateless, single-instance workflows are possible but not generally advised unless you have a strong reason and the engineering resources to manage all session state externally.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}