[BUG]: Gemini Plugin，tp=2，TypeError: qwen2model.forward() got an unexpected keyword argument 'force_sp_output_gather'

[happynaruto]

Is there an existing issue for this bug?

• I have searched the existing issues

The bug has not been fixed in the latest main branch

• I have checked the latest main branch

Do you feel comfortable sharing a concise (minimal) script that reproduces the error? :)

Yes, I will share a minimal reproducible script.

🐛 Describe the bug

colossal=0.4.9，Gemini Plugin，在八张A100显卡上训练QWQ32B，当tp=1时程序正常运行，但当tp=2时报错:TypeError: qwen2model.forward() got an unexpected keyword argument 'force_sp_output_gather'

Environment

No response

[eiPI1-0]

Your transformers version might not match with the ColossalAI version

[TongLi3701]

Hi, it is because of the mismatch of the transformer's version.

A quick fix is that you can comment out this line: https://github.com/hpcaitech/ColossalAI/blob/main/colossalai/shardformer/modeling/qwen2.py#L817.

Then you should be able to run successfully.