[Question]: 在paddlenlp部署推理过程中加载Qwen/Qwen2-1.5B等模型报错

[QingdaZ]

请提出你的问题

paddlenlp 3.0.0b4.post20250618
File "D:\PaddleNLP-develop\llm\predict\flask_server.py", line 316, in
predictor = create_predictor(predictor_args, model_args)
File "D:\PaddleNLP-develop\llm\predict\predictor.py", line 1953, in create_predictor
model = AutoModelForCausalLM.from_pretrained(
File "D:\Software\Anaconda\envs\paddle_zqd\lib\site-packages\paddlenlp\transformers\auto\modeling.py", line 799, in from_pretrained
return cls._from_pretrained(pretrained_model_name_or_path, *model_args, **kwargs)
File "D:\Software\Anaconda\envs\paddle_zqd\lib\site-packages\paddlenlp\transformers\auto\modeling.py", line 347, in _from_pretrained
return model_class.from_pretrained(pretrained_model_name_or_path, *model_args, **kwargs)
File "D:\Software\Anaconda\envs\paddle_zqd\lib\site-packages\paddlenlp\transformers\model_utils.py", line 2639, in from_pretrained
state_dict = load_state_dict(resolved_archive_file)
File "D:\Software\Anaconda\envs\paddle_zqd\lib\site-packages\paddlenlp\transformers\model_utils.py", line 500, in load_state_dict
state_dict, scale_dict = _load_part_state_dict(
File "D:\Software\Anaconda\envs\paddle_zqd\lib\site-packages\paddlenlp\transformers\model_utils.py", line 436, in load_part_state_dict
if len(py_safe_slice.shape) == 0:
AttributeError: 'builtins.PySafeSlice' object has no attribute 'shape'

[404814575]

一样的错误

[QingdaZ]

就很奇怪

[404814575]

不要使用最新版的代码进行训练，即develop分支，用release3.0.0的，之后修改一下run_finetune中的quantization_config即可