Skip to content

Conversation

zeroRains
Copy link
Contributor

@zeroRains zeroRains commented Aug 2, 2025

pcard-71500

修复test/layers/test_sampler.py单测执行出错的BUG。

问题分析
PR #2872sampling_metadata中,新增了一个min_p属性默认值为None,同时在forward中添加了一个必定会执行的probs = min_p_sampling(probs, sampling_metadata.min_p) 步骤,又因为test/layers/test_sampler.py中的sampling_metadata没有初始化min_p属性,导致执行到min_p_sampling时传入了min_p=None导致报错。

解决方案
test/layers/test_sampler.py单测中,添加min_p的初始化。

ps: min_p是一个维度为[bs]的Tensor,由于只有显示实例化sampling_metadata时才能知道bs的大小,所以不能给他设置一个确定默认值。

修复前报错内容如下:

Traceback (most recent call last):
  File "/root/paddlejob/workspace/env_run/output/liuyuanle/llj/baidu/paddle_internal/FastDeploy/test/layers/test_sampler.py", line 77, in <module>
    test_sampler()
  File "/root/paddlejob/workspace/env_run/output/liuyuanle/llj/baidu/paddle_internal/FastDeploy/test/layers/test_sampler.py", line 72, in test_sampler
    next_tokens = sampler(logits, sampling_metadata)
  File "/root/paddlejob/workspace/env_run/output/liuyuanle/miniconda3/envs/llj_py310/lib/python3.10/site-packages/paddle/nn/layer/layers.py", line 1571, in __call__
    return self.forward(*inputs, **kwargs)
  File "/root/paddlejob/workspace/env_run/output/liuyuanle/miniconda3/envs/llj_py310/lib/python3.10/site-packages/fastdeploy/model_executor/layers/sample/sampler.py", line 284, in forward_cuda
    probs = min_p_sampling(probs, sampling_metadata.min_p)
  File "/root/paddlejob/workspace/env_run/output/liuyuanle/miniconda3/envs/llj_py310/lib/python3.10/site-packages/fastdeploy/model_executor/layers/sample/ops/top_k_top_p_sampling.py", line 171, in min_p_sampling
    if paddle.count_nonzero(min_p_arr) == 0:
  File "/root/paddlejob/workspace/env_run/output/liuyuanle/miniconda3/envs/llj_py310/lib/python3.10/site-packages/paddle/tensor/math.py", line 2096, in count_nonzero
    bool_tensor = paddle.cast(x, 'bool')
  File "/root/paddlejob/workspace/env_run/output/liuyuanle/miniconda3/envs/llj_py310/lib/python3.10/site-packages/paddle/tensor/manipulation.py", line 237, in cast
    return _C_ops.cast(x, dtype)
ValueError: (InvalidArgument) cast(): argument 'x' (position 0) must be Tensor, but got None (at /root/paddlejob/workspace/env_run/output/liuyuanle/llj/Paddle/paddle/fluid/pybind/eager_utils.cc:1330)

Copy link

paddle-bot bot commented Aug 2, 2025

Thanks for your contribution!

@yuanlehome yuanlehome merged commit 17f51f0 into PaddlePaddle:develop Aug 4, 2025
12 of 14 checks passed
@zeroRains zeroRains deleted the fix_sampler branch August 4, 2025 08:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants