Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
实现了两个非常有用的 HuggingFace 小工具:
HuggingFace Cache:将当前机器所需的 HuggingFace 权重文件保存在本机的 /dev/shm 中,下次加载的时候从本地高速读取
HuggingFace 缩专家加载:即专家数小于 256 时自动对专家维度进行 slice
注1:如果在加载过程中不慎中断了程序,下次启动前需手动
rm -f /dev/shm/lshrun_*.lock
,否则 Cache 会死锁注2:缩层还需要改
special_cases
,例如: