FinanceAgent - enable on Xeon, remote endpoint, and refactor tests #2032

alexsin368 · 2025-06-04T21:22:21Z

Description

Enable support for running OpenAI models on Xeon
Enable remote endpoints for the LLMs on the agents only (docsum and dataprep will still run LLM locally)
Reorganize set_env.sh environment variables
Refactor tests: since Xeon test steps are similar to Gaudi's, breaking out the tests to make it easier to add new test steps or new tests for hardware

Issues

#1973

Type of change

List the type of change like below. Please delete options that are not relevant.

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds new functionality)
Breaking change (fix or feature that would break existing design and interface)
Others (enhancement, documentation, validation, etc.)

Dependencies

None

Tests

Added new test script.
Verified FinanceAgent is running on the UI.

Signed-off-by: alexsin368 <alex.sin@intel.com>

github-actions · 2025-06-04T21:22:43Z

Dependency Review

✅ No vulnerabilities or license issues found.

Scanned Files

None

for more information, see https://pre-commit.ci

FinanceAgent/docker_compose/intel/hpu/gaudi/README.md

FinanceAgent/docker_compose/intel/hpu/gaudi/set_env.sh

FinanceAgent/docker_compose/intel/hpu/gaudi/README.md

FinanceAgent/README.md

FinanceAgent/tests/step2_start_services.sh

FinanceAgent/docker_compose/intel/cpu/xeon/README.md

FinanceAgent/tests/step2_start_services.sh

Signed-off-by: alexsin368 <alex.sin@intel.com>

…m/alexsin368/GenAIExamples into finance-agent-remote-endpoint-new

for more information, see https://pre-commit.ci

Signed-off-by: alexsin368 <alex.sin@intel.com>

…, fix python cmd Signed-off-by: alexsin368 <alex.sin@intel.com>

Signed-off-by: alexsin368 <alex.sin@intel.com>

…m/alexsin368/GenAIExamples into finance-agent-remote-endpoint-new

Signed-off-by: alexsin368 <alex.sin@intel.com>

FinanceAgent/docker_compose/intel/cpu/xeon/README.md

Copilot

Pull Request Overview

Enables FinanceAgent support for Intel® Xeon® processors with OpenAI models and remote endpoints, while reorganizing the test structure for better maintainability across hardware platforms.

Adds Xeon support with OpenAI model integration and remote endpoint configuration
Refactors test scripts into modular step-based components for reusability
Reorganizes environment variables and documentation for clearer configuration

Reviewed Changes

Copilot reviewed 13 out of 13 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
test_compose_on_gaudi.sh	Refactored to use modular step scripts instead of inline functions
step*.sh	New modular test step scripts extracted from original Gaudi test
_test_compose_openai_on_xeon.sh	New test script for Xeon platform using OpenAI models
set_env.sh (gaudi/xeon)	Environment configuration files with proper variable validation
compose_*.yaml (xeon)	Docker compose files for Xeon deployment with OpenAI and remote endpoints
README.md files	Updated documentation for Xeon support and corrected typos

FinanceAgent/docker_compose/intel/hpu/gaudi/README.md

FinanceAgent/docker_compose/intel/cpu/xeon/README.md

FinanceAgent/tests/step3_validate_ingest_validate_dataprep.sh

FinanceAgent/tests/step4_validate_agent_service.sh

FinanceAgent/README.md

Signed-off-by: alexsin368 <alex.sin@intel.com>

for more information, see https://pre-commit.ci

alexsin368

addressed all comments

FinanceAgent/tests/_test_compose_openai_on_xeon.sh

alexsin368 · 2025-08-12T22:30:31Z

@letonghan @XinyuYe-Intel may I get your review on this PR before the code freeze this Friday?

letonghan

LGTM

louie-tsai · 2025-08-14T17:36:09Z

FinanceAgent/docker_compose/intel/cpu/xeon/README.md

+
+Set the following environment variables.
+
+- `REMOTE_ENDPOINT` is the HTTPS endpoint of the remote server with the model of choice (i.e. https://api.example.com). **Note:** If the API for the models does not use LiteLLM, the second part of the model card needs to be appended to the URL. For example, set `REMOTE_ENDPOINT` to https://api.example.com/Llama-3.3-70B-Instruct if the model card is `meta-llama/Llama-3.3-70B-Instruct`.


do we have instructions how users could figure out API using LiteLLM or not?

louie-tsai · 2025-08-14T17:38:32Z

FinanceAgent/docker_compose/intel/cpu/xeon/README.md

+
+Supervisor Agent multi turn:
+
+```bash


might be good to still have port export here.

Suggested change

```bash

```bash

export agent_port="9090"

louie-tsai · 2025-08-14T17:44:00Z

FinanceAgent/docker_compose/intel/cpu/xeon/compose_remote.yaml

@@ -0,0 +1,18 @@
+# Copyright (C) 2025 Intel Corporation
+# SPDX-License-Identifier: Apache-2.0
+


try to use common environment and remove below section?

Suggested change

x-common-environment:

&common-env

llm_endpoint_url: ${REMOTE_ENDPOINT}

api_key: ${OPENAI_API_KEY}

louie-tsai · 2025-08-14T17:45:02Z

FinanceAgent/docker_compose/intel/cpu/xeon/compose_remote.yaml

+
+services:
+  worker-finqa-agent:
+    environment:


Suggested change

environment:

environment:

<<: *common-env

louie-tsai · 2025-08-14T17:45:22Z

FinanceAgent/docker_compose/intel/cpu/xeon/compose_remote.yaml

+      api_key: ${OPENAI_API_KEY}
+
+  worker-research-agent:
+    environment:


Suggested change

environment:

environment:

<<: *common-env

louie-tsai · 2025-08-14T17:45:38Z

FinanceAgent/docker_compose/intel/cpu/xeon/compose_remote.yaml

+      api_key: ${OPENAI_API_KEY}
+
+  supervisor-react-agent:
+    environment:


Suggested change

environment:

environment:

<<: *common-env

louie-tsai · 2025-08-14T17:56:02Z

FinanceAgent/tests/test_compose_on_gaudi.sh


 echo "=================== #2 Start services ===================="
-start_all_services
-echo "=================== #2 Endpoints for services started===================="
+bash step2_start_services.sh gaudi_vllm


you probably need to change file name when we have new step. I prefer no stepX in file name, but it is ok to keep it this way.

alexsin368 added 9 commits June 3, 2025 17:06

initial commit to enable remote endpoints

a9c1716

Signed-off-by: alexsin368 <alex.sin@intel.com>

fix typo on file name

1f48bcb

Signed-off-by: alexsin368 <alex.sin@intel.com>

add common-env

f12c122

Signed-off-by: alexsin368 <alex.sin@intel.com>

fix environment

68b824d

Signed-off-by: alexsin368 <alex.sin@intel.com>

correct path to top .set_env.sh file

7408871

Signed-off-by: alexsin368 <alex.sin@intel.com>

explicitly set env variables for agent microservices

c3f4e9d

Signed-off-by: alexsin368 <alex.sin@intel.com>

fix typos, initial commit for xeon test

f6f123a

Signed-off-by: alexsin368 <alex.sin@intel.com>

reorganize env variables

62e3fbf

Signed-off-by: alexsin368 <alex.sin@intel.com>

refactor tests for Gaudi and Xeon

de7fe5a

Signed-off-by: alexsin368 <alex.sin@intel.com>

alexsin368 requested review from louie-tsai, yinghu5 and MSCetin37 June 4, 2025 21:22

alexsin368 requested review from ashahba, lkk12014402, minmin-intel, rbrugaro and XinyuYe-Intel as code owners June 4, 2025 21:22

[pre-commit.ci] auto fixes from pre-commit.com hooks

1c60777

for more information, see https://pre-commit.ci

alexsin368 mentioned this pull request Jun 4, 2025

[Feature] Enable remote inference endpoints for examples #1973

Open

36 tasks

MSCetin37 reviewed Jun 4, 2025

View reviewed changes

FinanceAgent/docker_compose/intel/hpu/gaudi/README.md Show resolved Hide resolved

MSCetin37 reviewed Jun 4, 2025

View reviewed changes

FinanceAgent/docker_compose/intel/hpu/gaudi/set_env.sh Show resolved Hide resolved

MSCetin37 reviewed Jun 4, 2025

View reviewed changes

FinanceAgent/docker_compose/intel/hpu/gaudi/README.md Outdated Show resolved Hide resolved

MSCetin37 reviewed Jun 5, 2025

View reviewed changes

FinanceAgent/README.md Show resolved Hide resolved

MSCetin37 reviewed Jun 5, 2025

View reviewed changes

FinanceAgent/tests/step2_start_services.sh Show resolved Hide resolved

MSCetin37 reviewed Jun 5, 2025

View reviewed changes

FinanceAgent/docker_compose/intel/cpu/xeon/README.md Outdated Show resolved Hide resolved

MSCetin37 reviewed Jun 5, 2025

View reviewed changes

FinanceAgent/docker_compose/intel/cpu/xeon/README.md Outdated Show resolved Hide resolved

MSCetin37 reviewed Jun 5, 2025

View reviewed changes

FinanceAgent/tests/step2_start_services.sh Show resolved Hide resolved

alexsin368 added 2 commits June 5, 2025 13:27

address comments, added check for docsum-vllm-xeon in test

bc7f7c4

Signed-off-by: alexsin368 <alex.sin@intel.com>

Merge branch 'finance-agent-remote-endpoint-new' of https://github.co…

234f931

…m/alexsin368/GenAIExamples into finance-agent-remote-endpoint-new

pre-commit-ci bot and others added 6 commits July 3, 2025 16:40

[pre-commit.ci] auto fixes from pre-commit.com hooks

0c5e003

for more information, see https://pre-commit.ci

address comments, added check for docsum-vllm-xeon in test

9598966

Signed-off-by: alexsin368 <alex.sin@intel.com>

make test _ because lack of OPENAI_API_KEY means cannot test in CI/CD…

54bacdb

…, fix python cmd Signed-off-by: alexsin368 <alex.sin@intel.com>

fix typo

d597450

Signed-off-by: alexsin368 <alex.sin@intel.com>

update instructions

9589f08

Signed-off-by: alexsin368 <alex.sin@intel.com>

update link

f56df7c

Signed-off-by: alexsin368 <alex.sin@intel.com>

alexsin368 force-pushed the finance-agent-remote-endpoint-new branch from 90d36d5 to f56df7c Compare July 3, 2025 23:40

alexsin368 added 2 commits July 3, 2025 16:40

Merge branch 'finance-agent-remote-endpoint-new' of https://github.co…

143a76f

…m/alexsin368/GenAIExamples into finance-agent-remote-endpoint-new

add note on HW requirement

ed27bf9

Signed-off-by: alexsin368 <alex.sin@intel.com>

amberjain1 reviewed Jul 31, 2025

View reviewed changes

FinanceAgent/docker_compose/intel/cpu/xeon/README.md Outdated Show resolved Hide resolved

Merge branch 'main' into finance-agent-remote-endpoint-new

42685ba

Copilot AI review requested due to automatic review settings August 5, 2025 22:32

Copilot AI reviewed Aug 5, 2025

View reviewed changes

alexsin368 and others added 2 commits August 5, 2025 16:00

remove denvr reference, fix link to finnhub, fix typos

f1784a9

Signed-off-by: alexsin368 <alex.sin@intel.com>

[pre-commit.ci] auto fixes from pre-commit.com hooks

3e8ab04

for more information, see https://pre-commit.ci

alexsin368 commented Aug 5, 2025

View reviewed changes

letonghan reviewed Aug 6, 2025

View reviewed changes

FinanceAgent/tests/_test_compose_openai_on_xeon.sh Show resolved Hide resolved

chensuyue and others added 2 commits August 11, 2025 13:30

Merge branch 'main' into finance-agent-remote-endpoint-new

bd72480

Merge branch 'main' into finance-agent-remote-endpoint-new

6554f0e

alexsin368 requested review from louie-tsai and letonghan August 12, 2025 20:37

joshuayao added the feature New feature or request label Aug 13, 2025

joshuayao added this to OPEA Aug 13, 2025

joshuayao removed the feature New feature or request label Aug 13, 2025

joshuayao moved this to In review in OPEA Aug 13, 2025

letonghan approved these changes Aug 13, 2025

View reviewed changes

alexsin368 requested a review from amberjain1 August 13, 2025 19:44

joshuayao removed this from the v1.4 milestone Aug 14, 2025

louie-tsai approved these changes Aug 14, 2025

View reviewed changes


		Set the following environment variables.

		- `REMOTE_ENDPOINT` is the HTTPS endpoint of the remote server with the model of choice (i.e. https://api.example.com). Note: If the API for the models does not use LiteLLM, the second part of the model card needs to be appended to the URL. For example, set `REMOTE_ENDPOINT` to https://api.example.com/Llama-3.3-70B-Instruct if the model card is `meta-llama/Llama-3.3-70B-Instruct`.

		@@ -0,0 +1,18 @@
		# Copyright (C) 2025 Intel Corporation
		# SPDX-License-Identifier: Apache-2.0

+x-common-environment:
+  &common-env
+      llm_endpoint_url: ${REMOTE_ENDPOINT}
+      api_key: ${OPENAI_API_KEY}

FinanceAgent - enable on Xeon, remote endpoint, and refactor tests #2032

Are you sure you want to change the base?

FinanceAgent - enable on Xeon, remote endpoint, and refactor tests #2032

Conversation

alexsin368 commented Jun 4, 2025

Description

Issues

Type of change

Dependencies

Tests

Uh oh!

github-actions bot commented Jun 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Dependency Review

Scanned Files

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

alexsin368 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

alexsin368 commented Aug 12, 2025

Uh oh!

letonghan left a comment

Choose a reason for hiding this comment

Uh oh!

louie-tsai Aug 14, 2025

Choose a reason for hiding this comment

Uh oh!

louie-tsai Aug 14, 2025

Choose a reason for hiding this comment

Uh oh!

louie-tsai Aug 14, 2025

Choose a reason for hiding this comment

Uh oh!

louie-tsai Aug 14, 2025

Choose a reason for hiding this comment

Uh oh!

louie-tsai Aug 14, 2025

Choose a reason for hiding this comment

Uh oh!

louie-tsai Aug 14, 2025

Choose a reason for hiding this comment

Uh oh!

louie-tsai Aug 14, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

github-actions bot commented Jun 4, 2025 •

edited

Loading