Skip to content

Issues: vectorch-ai/ScaleLLM

Beta
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

48 Open 43 Closed
48 Open 43 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Assigned to nobody Loading
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

kv cache: integrate nixl for kv cache transfer
#457 opened May 1, 2025 by guocuimi
kv cache: cache aware router
#456 opened May 1, 2025 by guocuimi
kv cache: fp8/int8 kv cache support
#455 opened May 1, 2025 by guocuimi
model: add qwen new models
#454 opened May 1, 2025 by guocuimi
model: add deepseek new models
#453 opened May 1, 2025 by guocuimi
kernel: tile scheduling
#452 opened May 1, 2025 by guocuimi
kernel: attention kernel for sm_75/70
#451 opened May 1, 2025 by guocuimi
feat: disaggregate Prefill and Decoding
#450 opened May 1, 2025 by guocuimi
kernel: deepep integration
#449 opened May 1, 2025 by guocuimi
kernel: grouped gemm for MOE quantization
#448 opened May 1, 2025 by guocuimi
kernel: attention kernel for sm_120
#447 opened May 1, 2025 by guocuimi
kernel: attention kernel for sm_100/101
#446 opened May 1, 2025 by guocuimi
kernel: attention kernel for sm_90
#445 opened May 1, 2025 by guocuimi
feat: EP support for Deepseek models
#444 opened May 1, 2025 by guocuimi
kernel: Grouped GEMM for MOE
#443 opened May 1, 2025 by guocuimi
ScaleAttention: a custom CUDA kernel, optimized for inference. performance Improvements to performance roadmap
#356 opened Jan 1, 2025 by guocuimi
RuntimeError: Timed out
#310 opened Aug 16, 2024 by spongxin
1
Is there any plans to support Int8 weight quant ?
#276 opened Jul 18, 2024 by sitabulaixizawaluduo
2
[Issue] Qwen-14B-Chat init fail and performance issue.
#275 opened Jul 16, 2024 by liutongxuan
2
LoRA: QLoRA/S-LoRA: Serving thousands of LoRA adapters
#166 opened Apr 28, 2024 by guocuimi
Introducing the Mamba model
#165 opened Apr 28, 2024 by guocuimi
Introducing a ring attention mechanism for handling long contexts
#164 opened Apr 28, 2024 by guocuimi
Quantization: Supporting FP8 for both models and KV caches
#163 opened Apr 28, 2024 by guocuimi
Enhancing documentation for improved usability
#162 opened Apr 28, 2024 by guocuimi
@guocuimi
Exploring other chips such as TPU, etc. backlog
#160 opened Apr 28, 2024 by guocuimi
ProTip! What’s not been updated in a month: updated:<2025-04-25.