Skip to content

Commit 40cfd77

Browse files
committed
add entrypoint for vllm
Signed-off-by: Tsai, Louie <louie.tsai@intel.com>
1 parent e846deb commit 40cfd77

File tree

1 file changed

+1
-0
lines changed

1 file changed

+1
-0
lines changed

ChatQnA/docker_compose/intel/cpu/xeon/compose.perf.yaml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,4 +6,5 @@ services:
66
image: public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.10.0
77
environment:
88
VLLM_CPU_SGL_KERNEL: 1
9+
entrypoint: ["python3", "-m", "vllm.entrypoints.openai.api_server"]
910
command: --model $LLM_MODEL_ID --host 0.0.0.0 --port 80 --dtype bfloat16 --distributed-executor-backend mp --block-size 128 --enforce-eager --tensor-parallel-size $TP_NUM --pipeline-parallel-size $PP_NUM --max-num-batched-tokens $MAX_BATCHED_TOKENS --max-num-seqs $MAX_SEQS

0 commit comments

Comments
 (0)