📝 Publications

NeuIPS 2025
sym

DuSA: Fast and Accurate Dual-Stage Sparse Attention Mechanism Accelerating Both Training and Inference

Chong Wu, Jiawang Cao, Renjie Xu, Zhuoheng Ran, Maolin Che, Wenbo Zhu, Hong Yan

  • A novel and effective dual-stage sparse attention (DuSA) mechanism
  • A mathematical relationship between DuSA and VSA is given
  • DuSA can approximate VSA with a low error
  • DuSA surpasses state-of-the-art (SOTA) methods in different tasks