Implementation of the sparse attention pattern proposed by the Deepseek team in their "Native Sparse Attention" paper
-
Updated
Aug 15, 2025 - Python
#
sparse-attention
Here are 23 public repositories matching this topic...
[ICML2025] SpargeAttention: A training-free sparse attention that accelerates any model inference.
attention vit quantization video-generation mlsys inference-acceleration ai-infra vision-transformer sparse-attention llm sageattention
-
Updated
Sep 27, 2025 - Cuda
LongLive: Real-time Interactive Long Video Generation
-
Updated
Oct 13, 2025 - Python
Fast Multi-dimensional Sparse Attention
-
Updated
Oct 10, 2025 - C++
[NeurIPS 2025] Radial Attention: O(nlogn) Sparse Attention with Energy Decay for Long Video Generation
-
Updated
Sep 18, 2025 - Python
[ICML2025, NeurIPS2025 Spotlight] Sparse VideoGen 1 & 2: Accelerating Video Diffusion Transformers with Sparse Attention
-
Updated
Oct 5, 2025 - Python
Speed Always Wins: A Survey on Efficient Architectures for Large Language Models
moe mamba linear-models state-space-model mixture-of-experts efficient-architectures linear-attention sparse-attention linear-rnn diffusion-llm
-
Updated
Aug 29, 2025
[ICML 2025 Spotlight] ShadowKV: KV Cache in Shadows for High-Throughput Long-Context LLM Inference
-
Updated
May 1, 2025 - Python
Efficient triton implementation of Native Sparse Attention.
-
Updated
May 23, 2025 - Python
[CoLM'25] The official implementation of the paper <MoA: Mixture of Sparse Attention for Automatic Large Language Model Compression>
-
Updated
Jul 11, 2025 - Python
Code for paper: [ICLR2025 Oral] FlexPrefill: A Context-Aware Sparse Attention Mechanism for Efficient Long-Sequence Inference
-
Updated
Oct 13, 2025 - Python
SLA: Beyond Sparsity in Diffusion Transformers via Fine-Tunable Sparse–Linear Attention
transformer video-generation mlsys inference-acceleration ai-infra linear-attention sparse-attention diffusion-transformer train-acceleration sparse-linear-attention
-
Updated
Oct 19, 2025 - Python
[Arxiv 2025] SparseD: Sparse Attention for Diffusion Language Models
-
Updated
Oct 7, 2025 - Python
[TIP-2025] Official Pytorch implementation of "Structural Similarity-Inspired Unfolding for Lightweight Image Super-Resolution"
-
Updated
Jul 8, 2025 - Python
Demo code for CVPR2023 paper "Sparsifiner: Learning Sparse Instance-Dependent Attention for Efficient Vision Transformers"
attention-mechanism fast-inference sparse-neural-networks low-rank vision-transformer efficient-transformers sparse-attention efficient-vision-transformers
-
Updated
Jul 4, 2023 - Python
Dynamic Attention Mask (DAM) generate adaptive sparse attention masks per layer and head for Transformer models, enabling long-context inference with lower compute and memory overhead without fine-tuning.
-
Updated
Jun 16, 2025 - Python
The code implementation of paper "VORTA: Efficient Video Diffusion via Routing Sparse Attention"
-
Updated
Oct 15, 2025 - Python
Building Native Sparse Attention
-
Updated
Feb 20, 2025 - Python
Toy Hydra prototypes: SSM + sparse attention + MoE + memory; synthetic benchmarks. Paper: https://arxiv.org/abs/2508.15099
benchmarking memory pytorch language-model pkm state-space-models mixture-of-experts sparse-attention long-context
-
Updated
Sep 30, 2025 - Python
Classification binaire avec architecture Sparse Attention pour données tabulaires. Optimisation automatique des hyperparamètres via Optuna. Testé sur datasets de churn télécommunications et bancaire.
machine-learning pytorch churn binary-classification telecommunications tabular optuna banki predictive- sparse-attention
-
Updated
Jun 20, 2025 - Jupyter Notebook
Improve this page
Add a description, image, and links to the sparse-attention topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the sparse-attention topic, visit your repo's landing page and select "manage topics."