[Cpp Extension] Support Cpp Extension #49893

jiahy0825 · 2023-01-17T08:25:45Z

PR types

New features

PR changes

Others

Describe

飞桨框架提供了丰富的算子库和易用的自定义算子机制，能够满足绝大多数场景的使用需求。
但是在以下场景下，用户可能希望定制化 C++ 实现，从而满足特定需求：

python 函数经常被调用，需要对性能做优化；
即使该函数被调用的次数很少，但是该函数开销很重，希望将实现逻辑放到 C++ 端提高性能；
函数实现依赖或者需要调用 C++ 库。

现有的自定义算子机制在下列场景下无法满足用户需求：

将 C++ 类绑定至 Python；
调用无 Tensor 参数的 C++ 函数。

为此，我们需要提供一种更为灵活的 C++ 扩展机制，以此机制实现的自定义扩展，能够 即插即用 ，不需要重新编译安装飞桨框架。

主要改动：

1. 支持 Cpp Extension 机制

支持 Tensor 作为 pybind11 绑定函数的参数
支持自定义扩展函数和扩展类
支持 Cpp Extension 和自定义算子的混合实现

2. 支持 setup 编译和 JIT 编译

3. 添加 Cpp Extension 单测

添加 setup 编译单测
添加 JIT 单测
验证链接器正确性、验证导入头文件正确性

4. 公开 pybind11 相关头文件

[Update pybind11] update pybind11 TAG, 2.4.3->2.6.0 #50068

将 pybind11 升级至 2.6.0 版本，解决 windows 下 nvcc 编译 pybind11/cast.h 报错的问题
修复 pybind11 升级后，VarType 无法正确打印的问题
公开 pybind11 目录至 site-package/paddle/include/third_party
添加 pybind11 目录至 inference_lib，修改推理头文件
新建 paddle/utils/pybind.h 并添加至 paddle/extension.h 头文件，用于引入 pybind11

目前本 PR 内容相对较多，且作为整体功能比较难拆分。
为便于 PR review 和并行推进剩余工作，后续会通过一些小 PR 继续完善 TODO 项：

1. 添加多个单测，充分验证与测试 Cpp Extension 机制：

添加 Cpp 与 cuda 混合调用单测
添加 Cpp Extension 和自定义算子混合调用单测
添加 STL 与 Tensor 组合输入输出的单测

2. 支持 Mac 和 Windows 平台

PaddlePaddle has provided a large operator library and easy-to-use custom operator mechanism, which can satisfy most scenario requirements. However, in the following scenario, users may hope to customize Cpp implementation and thus meet specific needs:

python functions are called frequently and the performance needs to be optimized;
Even if the function is only called several times, its cost is heavy and you want to move the implementation to Cpp to speed up the function;
The implementation of the function needs to call the Cpp library.

The existing custom operator mechanism can not satisfy users' requirements in the following scenarios:

Binding Cpp class to python;
Calling Cpp functions without Tensor parameter.

Therefore, we need to provide a more flexible Cpp Extension mechanism and this mechanism can plug in the PaddlePaddle framework without recompiling the whole project.

Main Change:

1. Support Cpp Extension mechanism

Support Tensor as parameter binding to pybind11
Support extension functions and extension classes
Support mixed implementation of Cpp extension and custom operator

2. Support setup compile and Just-In-Time (JIT) compile

3. Add Cpp Extension unit test

Add setup compile unit test
Add JIT compile unit test
Verify the accuracy of the linker and the accuracy of including corresponding header files.

4. Disclose pybind11 header files

[Update pybind11] update pybind11 TAG, 2.4.3->2.6.0 #50068

Upgrade pybind11 from 2.4.3 to 2.6.0, which solves the error of compiling pybind11/cast.h with nvcc compiler under windows.
Fix the error of incorrect print message of VarType after upgrading pybind11
Disclose the pybind11 directory to site-package/paddle/include/third_party
Add pybind11 directory to inference_lib and changes inference header files
Add paddle/utils/pybind.h to paddle/extension.h, which helps to import the pybind11 library

This PR is relatively large and hard to split. To simplify the review process and finish remained work parallelly, some small PRs will be created to finish the TODO list.

1. Add multiple unit tests to fully verify the Cpp Extension mechanism

Add the unit test of mixed calling of Cpp and cuda
Add the unit test of mixed calling of Cpp Extension and custom operator
Add the unit test of the combination use of STL and Tensor

2. Support Mac and Windows platform

… support_C++_extension

paddle-bot · 2023-01-17T08:25:49Z

你的PR提交成功，感谢你对开源项目的贡献!
请关注后续CI自动化测试结果，详情请参考Paddle-CI手册。
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

… support_C++_extension

paddle/utils/pybind.h

zyfncg · 2023-02-01T05:13:36Z

cmake/third_party.cmake

-if(WITH_PYTHON)
-  include(external/python) # find python and python_module
-  include(external/pybind11) # download pybind11
-  list(APPEND third_party_deps extern_pybind)
-endif()


这里去掉WITH_PYTHON可开关会不会对推理的编译产生影响？

Thanks，计划将 if(WITH_PYTHON) 改为 if(NOT ON_INFER)

Maybe change if(WITH_PYTHON) to if(NOT ON_INFER) would be better.

zyfncg · 2023-02-01T07:58:16Z

python/paddle/utils/cpp_extension/extension_utils.py

+            if os.name == 'nt' or sys.platform.startswith('darwin'):
+                # Cpp Extension only support Linux now
+                mod = types.ModuleType(__name__)
+            else:


这里的平台限制问题后续可以解决吗？

Windows 和 Mac 平台和 Linux 平台编译的差别较大，需要线下评估这项需求和排期。

There are differences between Windows, Mac, and Linux, we need to assess this requirement offline.

zyfncg · 2023-02-01T08:04:21Z

python/paddle/fluid/tests/cpp_extension/test_cpp_extension_jit.py

+if os.name == 'nt' or sys.platform.startswith('darwin'):
+    # only support Linux now
+    exit()
+
+# Because Windows don't use docker, the shared lib already exists in the
+# cache dir, it will not be compiled again unless the shared lib is removed.
+file = '{}\\custom_cpp_extension\\custom_cpp_extension.pyd'.format(
+    get_build_directory()
+)
+if os.name == 'nt' and os.path.isfile(file):
+    cmd = 'del {}'.format(file)
+    run_cmd(cmd, True)


这部分代码可以封装起来吗？用户在参考示例代码的时候这里看上去可能会有一些干扰

感谢，原本此处代码是留作扩展 Windows 的，可以先删除这部分代码，保持单测的整洁

Thanks, these codes are kept for Windows, I can delete this part temporarily to keep unit test clean.

chenwhql · 2023-02-07T13:33:58Z

paddle/utils/pybind.cc

+// limitations under the License.
+
+#include "paddle/utils/pybind.h"
+#include "paddle/fluid/platform/enforce.h"


这里能否用phi的，utils下的文件是对外放出的，但是platform下的文件没有放出，可能会编译错误

… support_C++_extension

zhangbo9674

LGTM

XieYunshen

LGTM for
set_tests_properties(test_cpp_extension_setup PROPERTIES TIMEOUT 120) set_tests_properties(test_cpp_extension_jit PROPERTIES TIMEOUT 120)

XiaoguangHu01

LGTM

jiahy0825 added 4 commits January 6, 2023 12:55

update include directory

df60146

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

1a8266e

… support_C++_extension

fully support C++ extension, pass unittest

a088e18

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

cfd2b50

… support_C++_extension

jiahy0825 added 25 commits January 17, 2023 10:58

fix include directory

716a7be

support both extension and operator in one file

3c0c182

polish testcase

4c24472

add jit unittest

2352468

update third_party.cmake, pass CI test

98c9ef0

fix cmake

36d39ef

fix setup

c6ff3d7

fix inference, fix unittest precision

8330201

fix unittest precision

1e42f72

fix inference_lib cmake

2b77782

try fix setup, try fix inference_lib

3ea9292

try fix inference_lib pybind

81c3982

fix mix_op_extension, fix inference_lib

e2cbfaa

fix mix_op_extension, fix inference_lib

be02319

change cmake

7cb1dbf

change cmake

2013aaa

add compile flags

21e71ef

add Python.h headerfile

dfa0cb4

add test_custom_plugin_creater cmake

adc6129

comment compile flag

a32ef46

pass all CI

1a55aef

pass all CI

4ef6175

comment compile flag

af05e34

try solve test_custom_plugin_creater link error

4c34a8b

try solve test_custom_plugin_creater link error

7c822c3

jiahy0825 added 4 commits January 28, 2023 13:01

update pybind11, 2.10.0->2.6.0b1

23e63e6

update pybind11, 2.6.0b1->2.6.0, start fix unittest

63cf859

fix pybind11 2.6.0 VarBase print error

95c5be8

fix pybind11 2.6.0 VarBase print error

08f99dc

jiahy0825 mentioned this pull request Jan 30, 2023

[Update pybind11] update pybind11 TAG, 2.4.3->2.6.0 #50068

Merged

jiahy0825 added 2 commits January 30, 2023 03:03

handle PADDLE_ON_INFERENCE

45bc3b8

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

85c4abd

… support_C++_extension

YuanRisheng previously approved these changes Feb 1, 2023

View reviewed changes

YuanRisheng reviewed Feb 1, 2023

View reviewed changes

paddle/utils/pybind.h Show resolved Hide resolved

zyfncg reviewed Feb 1, 2023

View reviewed changes

modify according to reviewer

c193063

jiahy0825 dismissed YuanRisheng’s stale review via c193063 February 1, 2023 09:34

jiahy0825 added 4 commits February 1, 2023 12:47

fix cmake

6a20c8f

cmake decouple pybind_util when not ON_INFER

3063de9

cmake decouple pybind_util when not ON_INFER

8609d90

remove copy of inference_lib.cmake

4b9126e

chenwhql reviewed Feb 7, 2023

View reviewed changes

jiahy0825 added 2 commits February 10, 2023 03:47

change pybind.cc headerfile fluid->phi

68e2e9f

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

941d674

… support_C++_extension

jiahy0825 force-pushed the support_C++_extension branch from 4dc1572 to 941d674 Compare February 15, 2023 03:14

zyfncg approved these changes Feb 17, 2023

View reviewed changes

YuanRisheng approved these changes Feb 17, 2023

View reviewed changes

zhangbo9674 approved these changes Feb 20, 2023

View reviewed changes

XieYunshen approved these changes Feb 20, 2023

View reviewed changes

XiaoguangHu01 approved these changes Feb 20, 2023

View reviewed changes

jiahy0825 merged commit 2a0563f into PaddlePaddle:develop Feb 20, 2023

This was referenced Feb 20, 2023

[Cpp Extension] Add unittest, mixed calling of op and extension #50678

Merged

[Extension Operants] Extension supports tensor operants #50869

Merged

jiahy0825 mentioned this pull request Apr 13, 2023

[CppExtension Cuda] Add cuda unit test for CppExtension #52900

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[Cpp Extension] Support Cpp Extension #49893

[Cpp Extension] Support Cpp Extension #49893

Uh oh!

jiahy0825 commented Jan 17, 2023 •

edited

Loading

Uh oh!

paddle-bot bot commented Jan 17, 2023

Uh oh!

Uh oh!

zyfncg Feb 1, 2023

Uh oh!

jiahy0825 Feb 1, 2023

Uh oh!

zyfncg Feb 1, 2023

Uh oh!

jiahy0825 Feb 1, 2023

Uh oh!

zyfncg Feb 1, 2023

Uh oh!

jiahy0825 Feb 1, 2023

Uh oh!

chenwhql Feb 7, 2023

Uh oh!

jiahy0825 Feb 10, 2023

Uh oh!

zhangbo9674 left a comment

Uh oh!

XieYunshen left a comment

Uh oh!

XiaoguangHu01 left a comment

Uh oh!

Uh oh!

[Cpp Extension] Support Cpp Extension #49893

[Cpp Extension] Support Cpp Extension #49893

Uh oh!

Conversation

jiahy0825 commented Jan 17, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR types

PR changes

Describe

Uh oh!

paddle-bot bot commented Jan 17, 2023

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

zhangbo9674 left a comment

Choose a reason for hiding this comment

Uh oh!

XieYunshen left a comment

Choose a reason for hiding this comment

Uh oh!

XiaoguangHu01 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

jiahy0825 commented Jan 17, 2023 •

edited

Loading