Skip to content

Test docker runner #2116

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
cf60682
DocSum - add files for deploy app with ROCm vLLM
Feb 13, 2025
1fd1de1
DocSum - fix main
Feb 13, 2025
bd2d47e
DocSum - add files for deploy app with ROCm vLLM
Feb 13, 2025
2459ecb
DocSum - fix main
Feb 13, 2025
4d35065
Merge remote-tracking branch 'origin/main'
Feb 19, 2025
6d5049d
DocSum - add files for deploy app with ROCm vLLM
Feb 13, 2025
9dfbdc5
DocSum - fix main
Feb 13, 2025
a8857ae
DocSum - add files for deploy app with ROCm vLLM
Feb 13, 2025
5a38b26
DocSum - fix main
Feb 13, 2025
0e2ef94
Merge remote-tracking branch 'origin/main'
Feb 25, 2025
30071db
Merge branch 'main' of https://github.com/opea-project/GenAIExamples
Mar 11, 2025
0757dec
Merge branch 'opea-project:main' into main
artem-astafev Mar 20, 2025
9aaf378
Merge branch 'main' of https://github.com/opea-project/GenAIExamples
Mar 26, 2025
9cf4b6e
Merge branch 'main' of https://github.com/opea-project/GenAIExamples
Apr 3, 2025
8e89787
Merge branch 'main' of https://github.com/opea-project/GenAIExamples
Apr 5, 2025
a117c69
Merge branch 'main' of https://github.com/opea-project/GenAIExamples
Apr 11, 2025
7fed5cf
Merge branch 'main' of https://github.com/opea-project/GenAIExamples
Apr 15, 2025
28504e1
Merge branch 'main' of https://github.com/opea-project/GenAIExamples
Apr 15, 2025
4cd6a50
Merge branch 'main' of https://github.com/opea-project/GenAIExamples
chyundunovDatamonsters Apr 21, 2025
9ccf540
DocSum - refactoring README.md
chyundunovDatamonsters Apr 24, 2025
d28db57
DocSum - add files for deploy app with ROCm vLLM
Feb 13, 2025
a644d2a
DocSum - fix main
Feb 13, 2025
fe1a269
DocSum - add files for deploy app with ROCm vLLM
Feb 13, 2025
450ba96
DocSum - fix main
Feb 13, 2025
f5f94b9
DocSum - add files for deploy app with ROCm vLLM
Feb 13, 2025
78700f6
DocSum - fix main
Feb 13, 2025
6b756fd
DocSum - add files for deploy app with ROCm vLLM
Feb 13, 2025
6d63177
DocSum - fix main
Feb 13, 2025
1b6be42
Merge remote-tracking branch 'origin/main'
chyundunovDatamonsters Jul 4, 2025
316d919
test_docker_runner
chyundunovDatamonsters Jul 4, 2025
b0407c0
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 4, 2025
44e6c81
test_docker_runner
chyundunovDatamonsters Jul 4, 2025
59b529e
Merge remote-tracking branch 'origin/test_docker_runner' into test_do…
chyundunovDatamonsters Jul 4, 2025
d6ad48b
test_docker_runner
chyundunovDatamonsters Jul 4, 2025
533b1fd
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 4, 2025
e0553ef
test_docker_runner
chyundunovDatamonsters Jul 4, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 4 additions & 26 deletions DocSum/docker_compose/amd/gpu/rocm/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ This section describes how to quickly deploy and test the DocSum service manuall

### Access the Code

Clone the GenAIExample repository and access the DocSum AMD GPU platform Docker Compose files and supporting scripts:
Clone the GenAIExample repository and access the ChatQnA AMD GPU platform Docker Compose files and supporting scripts:

```bash
git clone https://github.com/opea-project/GenAIExamples.git
Expand All @@ -42,7 +42,7 @@ Some HuggingFace resources, such as some models, are only accessible if you have

### Configure the Deployment Environment

To set up environment variables for deploying DocSum services, set up some parameters specific to the deployment environment and source the `set_env_*.sh` script in this directory:
To set up environment variables for deploying ChatQnA services, set up some parameters specific to the deployment environment and source the `set_env_*.sh` script in this directory:

- if used vLLM - set_env_vllm.sh
- if used TGI - set_env.sh
Expand All @@ -65,7 +65,7 @@ Set the values of the variables:
Setting variables in the operating system environment:

```bash
export HF_TOKEN="Your_HuggingFace_API_Token"
export HUGGINGFACEHUB_API_TOKEN="Your_HuggingFace_API_Token"
source ./set_env_*.sh # replace the script name with the appropriate one
```

Expand Down Expand Up @@ -239,16 +239,13 @@ curl http://${HOST_IP}:${DOCSUM_BACKEND_SERVER_PORT}/v1/docsum \
-F "language=en" \
```

Note that the `-F "messages="` flag is required, even for file uploads. Multiple files can be uploaded in a single call with multiple `-F "files=@/path"` inputs.

### Query with audio and video

> Audio and video can be passed as base64 strings or uploaded by providing a local file path.
> Audio and Video file uploads are not supported in docsum with curl request, please use the Gradio-UI.

Audio:

```bash
# Send base64 string
curl -X POST http://${HOST_IP}:${DOCSUM_BACKEND_SERVER_PORT}/v1/docsum \
-H "Content-Type: application/json" \
-d '{"type": "audio", "messages": "UklGRigAAABXQVZFZm10IBIAAAABAAEARKwAAIhYAQACABAAAABkYXRhAgAAAAEA"}'
Expand All @@ -260,21 +257,11 @@ curl http://${HOST_IP}:${DOCSUM_BACKEND_SERVER_PORT}/v1/docsum \
-F "max_tokens=32" \
-F "language=en" \
-F "stream=True"

# Upload file
curl http://${HOST_IP}:${DOCSUM_BACKEND_SERVER_PORT}/v1/docsum \
-H "Content-Type: multipart/form-data" \
-F "type=audio" \
-F "messages=" \
-F "files=@/path to your file (.mp3, .wav)" \
-F "max_tokens=32" \
-F "language=en"
```

Video:

```bash
# Send base64 string
curl -X POST http://${HOST_IP}:${DOCSUM_BACKEND_SERVER_PORT}/v1/docsum \
-H "Content-Type: application/json" \
-d '{"type": "video", "messages": "convert your video to base64 data type"}'
Expand All @@ -286,15 +273,6 @@ curl http://${HOST_IP}:${DOCSUM_BACKEND_SERVER_PORT}/v1/docsum \
-F "max_tokens=32" \
-F "language=en" \
-F "stream=True"

# Upload file
curl http://${HOST_IP}:${DOCSUM_BACKEND_SERVER_PORT}/v1/docsum \
-H "Content-Type: multipart/form-data" \
-F "type=video" \
-F "messages=" \
-F "files=@/path to your file (.mp4)" \
-F "max_tokens=32" \
-F "language=en"
```

### Query with long context
Expand Down
1 change: 1 addition & 0 deletions DocSum/docker_compose/amd/gpu/rocm/compose.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ services:
ports:
- "${DOCSUM_TGI_SERVICE_PORT:-8008}:80"
environment:
test_env: test
no_proxy: ${no_proxy}
http_proxy: ${http_proxy}
https_proxy: ${https_proxy}
Expand Down
1 change: 1 addition & 0 deletions DocSum/docker_compose/amd/gpu/rocm/compose_vllm.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ services:
ports:
- "${DOCSUM_VLLM_SERVICE_PORT:-8081}:8011"
environment:
test_env: test
no_proxy: ${no_proxy}
http_proxy: ${http_proxy}
https_proxy: ${https_proxy}
Expand Down