[Inference] Support group-wize quantize for weight_quantize op in GPU

[zeroRains]

PR Category

Inference

PR Types

Performance

Description

给weight_quantize op的group-wize quantize(int8, int4_col, int4_row)添加GPU kernel

deepseek-V2验证成功（已将deepseek-V2 modeling中转成cpu的逻辑删除）

PaddlePaddle/PaddleNLP#10174

python ./predict/predictor.py --model_name_or_path deepseek-ai/DeepSeek-V2-Lite-Chat --dtype float16 --mode dynamic --decode_strategy greedy_search --inference_model 1 --block_attn 1 --append_attn 1 --quant_type weight_only_int4 --weightonly_group_size 64

[paddle-bot]

你的PR提交成功，感谢你对开源项目的贡献!
请关注后续CI自动化测试结果，详情请参考Paddle-CI手册。
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

[paddle-ci-bot]

Sorry to inform you that e6b68d3's CIs have passed for more than 7 days. To prevent PR conflicts, you need to re-run all CIs manually.

[paddle-ci-bot]

Sorry to inform you that 9f7556c's CIs have passed for more than 7 days. To prevent PR conflicts, you need to re-run all CIs manually.

[paddle-ci-bot]

Sorry to inform you that f194185's CIs have passed for more than 7 days. To prevent PR conflicts, you need to re-run all CIs manually.

[paddle-ci-bot]

Sorry to inform you that 51c5483's CIs have passed for more than 7 days. To prevent PR conflicts, you need to re-run all CIs manually.