Skip to content

Releases

fa4-v4.0.0.beta39

fa4-v4.0.0.beta39 Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 11 Nov 08:12
dceb374

What's Changed

  • [mla] pad all dense 2CTA MQA head counts in-kernel by @liangel-02 in #3140
  • [FA2] Fix flash_attn_with_kvcache with cache_leftpad and no cache_seqlens by @andrewliu2001 in #3092
  • [CuTe, SM100] Preserve the 15-bit sm_107 SMEM descriptor address by @drisspg in #3029
  • [CuTe, SM100] Backward: explicit work-tile and head coordinates by @Johnsonms in #3141
  • [CuTe, SM100] Compile autotune candidates for sm_107a by @drisspg in #3121
  • [CuTe, SM100] Make the dynamic varlen forward scheduler bind tiles late and self-reset by @drisspg in #3086
  • [CuTe, SM100] Route padded varlen forward calls to the dynamic persistent scheduler by @drisspg in #3087
  • [CuTe, SM100] Give BF16 2CTA hd128 softmax 192 registers by @drisspg in #3016
  • [CuTe, SM100] Release softmax statistics per stage by default by @drisspg in #3017
  • [CuTe, SM100] Group FP8 descales in a typed scales argument by @drisspg in #3027
  • [CuTe, SM100] Unify hd256 backward (6/7): pack 2 query heads per tile and loop the KV head's remaining heads inside the CTA by @Johnsonms in #3059
  • [FA2] Keep split-KV block steps and accumulator offsets in 64 bits by @0z5a in #3101
  • [CuTe, Bwd, SM100] Fold the hdim64 LSE / dPsum subtractions into the S / dP MMA by @drisspg in #3098
  • [FA2] Check cache_batch_idx length in mha_fwd_kvcache by @bhargavchintam in #3102
  • [CuTe, SM100] hd256 keeps PackGQA when 2CTA is not possible by @Johnsonms in #3148
  • [CuTe, SM100] Unify hd256 backward (7/7): make it the default and delete the dedicated kernels by @Johnsonms in #2968
  • [CuTe, SM100] Add MXFP8 quantization and scale-layout helpers by @drisspg in #3018
  • [CuTe, SM100] Add MXFP8 QK forward with BF16 PV by @drisspg in #3019
  • [CuTe, SM100] hd256 backward follow-ups from #2968 by @Johnsonms in #3155

New Contributors

Full Changelog: fa4-v4.0.0.beta38...fa4-v4.0.0.beta39

fa4-v4.0.0.beta38

fa4-v4.0.0.beta38 Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 04 Nov 08:12
5f7b05a

What's Changed

  • [CuTe, SM100] Unify hd256 backward (5/7): cheaper deterministic dQ semaphore (overlapped release on the M=64 serial schedule, early acquire, relaxed release) by @Johnsonms in #3058
  • [CuTe] Route forward through typed configs by @drisspg in #2733
  • [CuTe, SM100] Add swap-AB split-KV decode by @ighoshsubho in #2973
  • [CuTe, SM100] Run causal forward on CLC by default by @drisspg in #3062
  • [CuTe] Route backward through typed configs by @drisspg in #3044
  • [CuTe] Route MLA forward through typed configs by @drisspg in #3054
  • [CuTe, SM100] Put the forward shared-memory layout in the config by @drisspg in #3056
  • [CuTe] Add explicit forward config benchmark by @drisspg in #2735
  • [CuTe] Tune GB300 forward scheduling by @drisspg in #2738
  • [CuTe] Add explicit forward config autotuning by @drisspg in #2743
  • [mla] expose kernel cta choice for autotuning by @liangel-02 in #3091
  • [mla] pad dense 2CTA MQA heads to the 128-row tile for 65-127 heads by @liangel-02 in #3118

New Contributors

Full Changelog: fa4-v4.0.0.beta37...fa4-v4.0.0.beta38

fa4-v4.0.0.beta37

fa4-v4.0.0.beta37 Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 28 Oct 08:14
98c1cee

What's Changed

  • [CuTe, Fwd, SM100] Prevent NaN contamination from unused V cache entries by @fusheng-ji in #2983
  • [CuTe] Format config.py with ruff so the Lint workflow passes again by @WeiYedi in #3067
  • [CuTe, Bwd, SM100] Bound the K/V TMA loads by seqlen_k so unused rows cannot poison dQ (packed varlen and dense seqused_k) by @yunweili3 in #3012
  • [CuTe, SM90] Bound the K/V TMA loads by seqlen_k (fwd + bwd, packed varlen and dense seqused_k) by @yunweili3 in #3013
  • [CuTe] Bound ragged K/V TMA loads with per-CTA descriptor updates instead of a wrap-around descriptor by @yunweili3 in #3071
  • [CuTe, SM100] Unify hd256 backward (4/7): pack the query heads of a KV head into the Q tile (GQA) by @Johnsonms in #2967
  • [CuTe, SM100] Fix 2CTA CLC tile coords and make forward work-tile coordinates logical by @drisspg in #3034

New Contributors

Full Changelog: fa4-v4.0.0.beta36...fa4-v4.0.0.beta37

fa4-v4.0.0.beta36

fa4-v4.0.0.beta36 Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 21 Oct 08:19
e7a6402

What's Changed

  • Fix paged KV-cache CUDA graph capture by @pkuwangh in #2984
  • [CuTe, SM100] Unify hd256 backward (2/5): 64-row KV tile per CTA (M=64 TMEM layout) with GQA, varlen and deterministic support by @Johnsonms in #2965
  • [Tests] CUDA graph capture/replay coverage for FA4 (CuTe) and FA2 entry points by @yunweili3 in #3007
  • [CuTe, SM100] Unify hd256 backward (3/5): hdim 256 on FlashAttentionBackwardSm100 (opt-in) by @Johnsonms in #2966
  • [CuTe] Add typed forward configurations by @drisspg in #2732

New Contributors

Full Changelog: fa4-v4.0.0.beta35...fa4-v4.0.0.beta36

v2.8.3.post2

Choose a tag to compare

@github-actions github-actions released this 13 Oct 05:41
49774b3

Maintenance release of FlashAttention 2 on top of v2.8.3.post1. No kernel feature changes.

  • Backport #2622: torch.load(..., weights_only=True) in flash_attn/utils/distributed.py and flash_attn/models (CVE-2026-31253 / GHSA-7g5w-pq96-8c5w). #2974
  • Backport #2702: safe tarfile extraction in hopper/setup.py (CVE-2026-62239 / GHSA-w9r7-4gwr-j959). #2974
  • Backport #2260: fix int32 overflow in the FA2 backward dq_accum offsets, which caused an illegal memory access or NaN gradients when batch * round128(seqlen_q) * nheads * headdim exceeded 2^31 (#2987). #2994
  • setup.py: request cu13 prebuilt wheels for CUDA 13 torch.

fa4-v4.0.0.beta35

fa4-v4.0.0.beta35 Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 14 Oct 08:20
1fc0221

What's Changed

New Contributors

Full Changelog: fa4-v4.0.0.beta34...fa4-v4.0.0.beta35

fa4-v4.0.0.beta34

fa4-v4.0.0.beta34 Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 07 Oct 08:23
94e22c9

What's Changed

  • Fix Windows builds with CUDA 13 and PyTorch 2.15 by @tuananhlfc in #2872
  • [CuTe] Allow apache-tvm-ffi 0.1.11 by @felipemello1 in #2948
  • [CuTe, SM100] S/P ping-pong to overlap QK with softmax in decode by @drisspg in #2928
  • [CuTe, SM100] Unify hd256 forward and recover the remaining stack by @drisspg in #2962
  • [CuTe, SM100] Fix hd128 decode crash on SM103: cap other-WG registers at the launch count by @felipemello1 in #2950
  • [Cute,Fwd,Sm100] hd256: S ping-pong on SM103, 2CTA for varlen Q and decode by @felipemello1 in #2951
  • [CuTe, SM100] Allow deterministic=True in the hd256 backward by @felipemello1 in #2952
  • [Cute, Sm100] bwd postprocess: load 2-CTA dQaccum directly into registers by @pchen7e2 in #2960
  • [CuTe, SM100] Unify hd256 backward (1/5): tests first, named 2CTA schedules and TMEM widths by @Johnsonms in #2964
  • [CuTe, SM100] Key MLA forward compilation on the KV head count by @drisspg in #2978

New Contributors

Full Changelog: fa4-v4.0.0.beta33...fa4-v4.0.0.beta34

fa4-v4.0.0.beta33

fa4-v4.0.0.beta33 Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 30 Sep 08:22
fb97d25

What's Changed

  • [CuTe, SM100] Sparse MLA bwd: keep P normalization and dS product in fp32 by @abcdabcd987 in #2893
  • [CuTe, SM100] Sparse MLA training fwd: exact running softmax max by @abcdabcd987 in #2907
  • [CuTe, SM100] Sparse MLA training: dpsum from an O residual by @abcdabcd987 in #2908
  • [CuTe, SM100] Sparse MLA recompute-P: allow 64 Q heads by @abcdabcd987 in #2911
  • [CuTe, SM100] Sparse MLA bwd preprocess: stop the lse_log2 store at the sequence's rows by @abcdabcd987 in #2912
  • [CuTe, SM100] Write real zeros for block-sparse empty tiles by @pashu-cohere in #2906
  • [CuTe, SM90] Fit forward tile to the block-sparse block size by @LiRunGuo in #2903
  • [CuTe, SM90] Skip the causal/local mask on unmasked KV blocks in forward by @LiRunGuo in #2904
  • [ROCm/CK] Fix num_splits heuristic, which could never return more than 1 by @Johnsonms in #2901
  • Support seqused_q/seqused_k in SM100 HD256 backward by @Mellonta in #2891
  • [CuTe, SM100] hd256 fwd: derive lengths and KV ranges from BlockInfo/SeqlenInfoQK by @drisspg in #2917
  • [Cute, Sm100] 1CTA MLA forward (dense + sparse top-k), MLA dispatch heuristic, sparse MLA backward at any head count by @jayhshah in #2938
  • Fix lint failure by @jayhshah in #2940

New Contributors

Full Changelog: fa4-v4.0.0.beta32...fa4-v4.0.0.beta33

fa4-v4.0.0.beta32

fa4-v4.0.0.beta32 Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 23 Sep 08:21
d15f153

What's Changed

  • [CuTe, SM100] Sparse MLA bwd: in-kernel recompute-P + token-chunked backward by @abcdabcd987 in #2816
  • Build CUDA extension with C++20 on PyTorch 2.13+ by @zhang-keliang in #2879
  • ci: add PyTorch 2.14 to the wheel build matrix by @Johnsonms in #2892
  • ci: registry-free GPU job (runner-local SIF) + verify the provisioned overlay from a fresh session by @Johnsonms in #2894
  • Compile with c++20 for pytorch 2.13+ by @cih9088 in #2899
  • [ROCM] add FLASH_ATTENTION_USE_SYSTEM_AITER flag and commit bump by @micmelesse in #2900
  • [CuTe, SM100] Support 1..128 Q heads in sparse MLA via in-kernel TMA padding by @drisspg in #2883

New Contributors

Full Changelog: fa4-v4.0.0.beta31...fa4-v4.0.0.beta32

fa4-v4.0.0.beta31

fa4-v4.0.0.beta31 Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 16 Sep 08:21
0dc2cb4

What's Changed

  • Add an Attention Sink to Flash MLA for SM100 by @floatingtrees in #2768
  • [CuTe, SM100] Stop the non-causal hdim 128 softmax warps spilling by @dawnop in #2869
  • [CuTe, SM100] hd256 2-CTA: support seqused_q/seqused_k and the decoder paged-KV shape by @kzos in #2810
  • [CuTe, SM100] Preserve hd256 output alignment for vectorized stores by @drisspg in #2880
  • One more deprecation; by @drisspg in #2867
  • [CuTe, SM100, HD256] optimize SM100 HD256 varlen scheduling and epilogues by @catwinee in #2807
  • [CuTe, SM90] Fix swapped backward MMA layout and use 2-stage hdim256 config by @SuperGoodGame in #2888
  • [CuTe, Bwd] Return zero gradients for empty Q/K workloads by @guoriyue in #2776
  • [CuTe, Fwd] Fix PackGQA predication for padded head dimensions by @guoriyue in #2775

New Contributors

Full Changelog: fa4-v4.0.0.beta30...fa4-v4.0.0.beta31