Blog Posts

• July 11, 2026 Torch Compile Internals (Part 3) - AOT Autograd
• July 03, 2026 Torch Compile Internals (Part 2) - Dynamo
• June 26, 2026 Torch Compile Internals (Part 1) - FX Graphs
• June 12, 2026 Collective Communication for Distributed Computing
• May 30, 2026 Everything you need to know about Pytorch Distributed Data Parallel(DDP)
• May 22, 2026 A Primer to Understanding Quantization for LLMs
• May 15, 2026 Outperforming cuBLAS on Ada Lovelace using CuTe DSL
• April 22, 2026 Flash Attention Series - Implementing Flash Attention
• April 13, 2026 Flash Attention Series - Tiled Matrix Multiplication
• April 09, 2026 Flash Attention Series - Mastering Softmax
• Mar 21, 2026 Warped Matrix Multiply Accumulate(WMMA) in CUDA
• Mar 07, 2026 Layout Algebra in NVIDIA CuTe
• Feb 21, 2026 A Primer to Understanding Layouts in NVIDIA CuTe
• Jan 10, 2026 Understanding GEMM: From Naive to Optimized Matrix Multiplication in C++