自编译的 fastdeploy_python-1.0.7-py3.10-linux-x86_64 使用 openvino 作为backend 报错, paddle, ort 工作正常

[jianweimama]

---

温馨提示：根据社区不完全统计，按照模板提问，可以加快回复和解决问题的速度

---

环境

• 【FastDeploy版本】： 说明具体的版本，
  fastdeploy_python-1.0.7-py3.10-linux-x86_64

• 【编译命令】如果您是自行编译的FastDeploy，请说明您的编译方式（参数命令）
  参考: https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/build_and_install/cpu.md
  git clone https://github.com/PaddlePaddle/FastDeploy.git
  git checkout origin/release/1.0.7

cd FastDeploy/python
export ENABLE_ORT_BACKEND=ON
export ENABLE_PADDLE_BACKEND=ON
export ENABLE_OPENVINO_BACKEND=ON
export ENABLE_VISION=ON
export ENABLE_TEXT=ON
export OPENCV_DIRECTORY=/usr/lib/x86_64-linux-gnu/cmake/opencv4

python setup.py build
python setup.py bdist_wheel

-- ***FastDeploy Building Summary

-- CMake version : 3.27.2
-- CMake command : /usr/local/lib/python3.10/dist-packages/cmake/data/bin/cmake
-- System : Linux
-- C++ compiler : /usr/bin/c++
-- C++ compiler version : 12.3.0
-- CXX flags : -Wno-format -g0 -O3
-- EXE linker flags :
-- Shared linker flags :
-- Build type : Release
-- Compile definitions : _GLIBCXX_USE_CXX11_ABI=1;FASTDEPLOY_LIB;CMAKE_BUILD_TYPE=Release;ENABLE_ORT_BACKEND;ENABLE_PADDLE_BACKEND;ENABLE_OPENVINO_BACKEND;ENABLE_VISION;ENABLE_TEXT;ENABLE_PADDLE2ONNX
-- CMAKE_PREFIX_PATH :
-- CMAKE_INSTALL_PREFIX : /usr/local
-- CMAKE_MODULE_PATH :

-- FastDeploy version : 1.0.7
-- ENABLE_ORT_BACKEND : ON
-- ENABLE_RKNPU2_BACKEND : OFF
-- ENABLE_HORIZON_BACKEND : OFF
-- ENABLE_SOPHGO_BACKEND : OFF
-- ENABLE_PADDLE_BACKEND : ON
-- ENABLE_LITE_BACKEND : OFF
-- ENABLE_POROS_BACKEND : OFF
-- ENABLE_TRT_BACKEND : OFF
-- ENABLE_OPENVINO_BACKEND : ON
-- ENABLE_BENCHMARK : OFF
-- ENABLE_VISION : ON
-- ENABLE_TEXT : ON
-- ENABLE_ENCRYPTION : OFF
-- ENABLE_FLYCV : OFF
-- ENABLE_CVCUDA : OFF
-- WITH_GPU : OFF
-- WITH_IPU : OFF
-- WITH_OPENCL : OFF
-- WITH_TESTING : OFF
-- WITH_ASCEND : OFF
-- WITH_DIRECTML : OFF
-- WITH_TIMVX : OFF
-- WITH_KUNLUNXIN : OFF
-- WITH_CAPI : OFF
-- WITH_CSHARPAPI : OFF
-- ONNXRuntime version : 1.12.0
-- Paddle Inference version : 0.0.0.660f781b77
-- OpenVINO version : dev.2023.03.2
-- Python executable : /home/test/py310_paddle_cpu/bin/python
-- Python includes : /usr/include/python3.10

• 【系统平台】: Linux x64(Ubuntu 22.04)
• 【硬件】： 说明具体硬件型号，Xeon-SP CPU
• 【编译语言】： Python(3.10）

  python list

Python 3.10.12
openvino 2024.6.0
openvino-dev 2024.6.0
openvino-telemetry 2025.1.0

问题日志及出现问题的操作流程

测试, 参考:
https://github.com/PaddlePaddle/FastDeploy/tree/develop/examples/vision/ocr/PP-OCR/cpu-gpu/python
看log, openvino 作为backend 已经初始化完成.

FastDeploy/examples/vision/ocr/PP-OCR/cpu-gpu/python# python infer.py --det_model ch_PP-OCRv3_det_infer --cls_model ch_ppocr_mobile_v2.0_cls_infer --rec_model ch_PP-OCRv3_rec_infer --rec_label_file ppocr_keys_v1.txt --image kt.png --device cpu --backend paddle
WARNING: Logging before InitGoogleLogging() is written to STDERR
W0419 17:08:07.493630 923328 analysis_config.cc:971] It is detected that mkldnn and memory_optimize_pass are enabled at the same time, but they are not supported yet. Currently, memory_optimize_pass is explicitly disabled
[INFO] fastdeploy/runtime/runtime.cc(273)::CreatePaddleBackend Runtime initialized with Backend::PDINFER in Device::CPU.
[INFO] fastdeploy/runtime/runtime.cc(273)::CreatePaddleBackend Runtime initialized with Backend::PDINFER in Device::CPU.
[INFO] fastdeploy/runtime/runtime.cc(273)::CreatePaddleBackend Runtime initialized with Backend::PDINFER in Device::CPU.
det boxes: [[6,4],[664,4],[664,18],[6,18]]rec text: KTransformers（KVCache,Al出品）是一个灵活的本地 LLM 推理优化框架，旨在通过高级内 rec score:0.914088 cls label: 0 cls score: 0.940954
det boxes: [[6,29],[664,29],[664,44],[6,44]]rec text: 核优化和模型部署/井行策略提升Transformer模型推理性能(GitHub -kvcache- rec score:0.931263 cls label: 0 cls score: 0.967136
det boxes: [[6,53],[665,53],[665,68],[6,68]]rec text: ai/ktransformers: A Flexible Framework for Experiencing Cutting-edge LLM rec score:0.914593 cls label: 0 cls score: 0.760472
det boxes: [[6,77],[664,78],[664,93],[6,92]]rec text: InferenceOptimizations)。它以注入式设计嵌入在HuggingFaceTransformers接口之 rec score:0.926265 cls label: 0 cls score: 0.960029
det boxes: [[7,102],[665,102],[665,117],[7,117]]rec text: 上，用一行代码即可将模型替换为优化版本，提供与HuggingFace接口兼容的使用方式，以及 rec score:0.941941 cls label: 0 cls score: 0.995538
det boxes: [[7,127],[665,127],[665,142],[7,142]]rec text: OpenAl APl兼容的REsT接和简易 Web Ul (GitHub - kvcache-ai/ktransformers: A rec score:0.904875 cls label: 0 cls score: 0.932776
det boxes: [[6,151],[653,151],[653,166],[6,166]]rec text: Flexible Framework for Experiencing Cutting-edge LLM Inference Optimizations). rec score:0.955298 cls label: 0 cls score: 0.802574

FastDeploy/examples/vision/ocr/PP-OCR/cpu-gpu/python# python infer.py --det_model ch_PP-OCRv3_det_infer --cls_model ch_ppocr_mobile_v2.0_cls_infer --rec_model ch_PP-OCRv3_rec_infer --rec_label_file ppocr_keys_v1.txt --image kt.png --device cpu --backend ort
[INFO] fastdeploy/runtime/runtime.cc(300)::CreateOrtBackend Runtime initialized with Backend::ORT in Device::CPU.
[INFO] fastdeploy/runtime/runtime.cc(300)::CreateOrtBackend Runtime initialized with Backend::ORT in Device::CPU.
[INFO] fastdeploy/runtime/runtime.cc(300)::CreateOrtBackend Runtime initialized with Backend::ORT in Device::CPU.
det boxes: [[6,4],[664,4],[664,18],[6,18]]rec text: KTransformers（KVCache,Al出品）是一个灵活的本地 LLM 推理优化框架，旨在通过高级内 rec score:0.914088 cls label: 0 cls score: 0.940954
det boxes: [[6,29],[664,29],[664,44],[6,44]]rec text: 核优化和模型部署/井行策略提升Transformer模型推理性能(GitHub -kvcache- rec score:0.931263 cls label: 0 cls score: 0.967136
det boxes: [[6,53],[665,53],[665,68],[6,68]]rec text: ai/ktransformers: A Flexible Framework for Experiencing Cutting-edge LLM rec score:0.914593 cls label: 0 cls score: 0.760473
det boxes: [[6,77],[664,78],[664,93],[6,92]]rec text: InferenceOptimizations)。它以注入式设计嵌入在HuggingFaceTransformers接口之 rec score:0.926265 cls label: 0 cls score: 0.960029
det boxes: [[7,102],[665,102],[665,117],[7,117]]rec text: 上，用一行代码即可将模型替换为优化版本，提供与HuggingFace接口兼容的使用方式，以及 rec score:0.941942 cls label: 0 cls score: 0.995538
det boxes: [[7,127],[665,127],[665,142],[7,142]]rec text: OpenAl APl兼容的REsT接和简易 Web Ul (GitHub - kvcache-ai/ktransformers: A rec score:0.904874 cls label: 0 cls score: 0.932776
det boxes: [[6,151],[653,151],[653,166],[6,166]]rec text: Flexible Framework for Experiencing Cutting-edge LLM Inference Optimizations). rec score:0.955297 cls label: 0 cls score: 0.802574

FastDeploy/examples/vision/ocr/PP-OCR/cpu-gpu/python# python infer.py --det_model ch_PP-OCRv3_det_infer --cls_model ch_ppocr_mobile_v2.0_cls_infer --rec_model ch_PP-OCRv3_rec_infer --rec_label_file ppocr_keys_v1.txt --image kt.png --device cpu --backend openvino
[INFO] fastdeploy/runtime/backends/openvino/ov_backend.cc(218)::InitFromPaddle number of streams:1.
[INFO] fastdeploy/runtime/backends/openvino/ov_backend.cc(228)::InitFromPaddle affinity:YES.
[INFO] fastdeploy/runtime/backends/openvino/ov_backend.cc(240)::InitFromPaddle Compile OpenVINO model on device_name:CPU.
[INFO] fastdeploy/runtime/runtime.cc(286)::CreateOpenVINOBackend Runtime initialized with Backend::OPENVINO in Device::CPU.
[INFO] fastdeploy/runtime/backends/openvino/ov_backend.cc(218)::InitFromPaddle number of streams:1.
[INFO] fastdeploy/runtime/backends/openvino/ov_backend.cc(228)::InitFromPaddle affinity:YES.
[INFO] fastdeploy/runtime/backends/openvino/ov_backend.cc(240)::InitFromPaddle Compile OpenVINO model on device_name:CPU.
[INFO] fastdeploy/runtime/runtime.cc(286)::CreateOpenVINOBackend Runtime initialized with Backend::OPENVINO in Device::CPU.
[INFO] fastdeploy/runtime/backends/openvino/ov_backend.cc(218)::InitFromPaddle number of streams:1.
[INFO] fastdeploy/runtime/backends/openvino/ov_backend.cc(228)::InitFromPaddle affinity:YES.
[INFO] fastdeploy/runtime/backends/openvino/ov_backend.cc(240)::InitFromPaddle Compile OpenVINO model on device_name:CPU.
[INFO] fastdeploy/runtime/runtime.cc(286)::CreateOpenVINOBackend Runtime initialized with Backend::OPENVINO in Device::CPU.
Traceback (most recent call last):
File "/home/test/FastDeploy/examples/vision/ocr/PP-OCR/cpu-gpu/python/infer.py", line 211, in
result = ppocr_v3.predict(im)
File "/home/test/py310_paddle_cpu/lib/python3.10/site-packages/fastdeploy_python-1.0.7-py3.10-linux-x86_64.egg/fastdeploy/vision/ocr/ppocr/init.py", line 958, in predict
return self.system_.predict(input_image)
RuntimeError: Primitive descriptor was not found for node linear_1.tmp_1.

[jianweimama]

补充一下: 对于 AI model 我使用的还是 paddle 格式, 如果使用转成 OV 的IR 格式, 直接报错:
FastDeploy/examples/vision/ocr/PP-OCR/cpu-gpu/python# python infer.py --det_model ch_PP-OCRv3_det_infer_ov --cls_model ch_ppocr_mobile_v2.0_cls_infer_ov --rec_model ch_PP-OCRv3_rec_infer_ov --rec_label_file ppocr_keys_v1.txt --image kt.png --device cpu --backend openvino
Traceback (most recent call last):
File "/home/test/FastDeploy/examples/vision/ocr/PP-OCR/cpu-gpu/python/infer.py", line 178, in
det_model = fd.vision.ocr.DBDetector(
File "/home/test/py310_paddle_cpu/lib/python3.10/site-packages/fastdeploy_python-1.0.7-py3.10-linux-x86_64.egg/fastdeploy/vision/ocr/ppocr/init.py", line 209, in init
self._model = C.vision.ocr.DBDetector(
RuntimeError: [ NETWORK_NOT_READ ] Unable to read the model: ch_PP-OCRv3_det_infer_ov/inference.pdmodel Please check that model format: pdmodel is supported and the model is correct. Available frontends: ir tflite onnx paddle tf pytorch

[pioneer12345]

使用官方编译的镜像fastdeploy:1.0.7-cpu-only-21.10,也出现同样的问题，CPU型号也是Xeon