Skip to content

About

CODA: Rewriting Transformer Blocks as GEMM-Epilogue Programs

Resources

Stars

258 stars

Watchers

5 watching

Forks

Latest commit

 

History

747 Commits

Folders and files

Repository files navigation

CODA: GPU Kernels as GEMM-plus-Epilogue Programs

arXiv

CODA is a GPU kernel abstraction that expresses Transformer operators as GEMM-plus-epilogue programs, fusing normalization, activations, residual updates, and reductions into the GEMM output tile before it is written to global memory, combining framework-level productivity with hardware-level efficiency. CODA is built on CUTLASS CuTeDSL and targets NVIDIA Hopper (H100) GPUs.

Updates

  • October 6, 2026. Released v0.3.1, pinning the quack fork to v0.6.5+fork.1.
  • September 28, 2026. Released v0.3.0.
  • July 19, 2026. Released v0.2.
  • June 23, 2026. We are restructuring CODA. For legacy version, please check v0.1 tag.

Installation

Note: the coda-kernels package on PyPI is deprecated and no longer updated. Install from GitHub instead.

pip install git+https://github.com/open-lm-engine/coda-kernels.git@v0.3.1

Or from source:

git clone https://github.com/open-lm-engine/coda-kernels.git
cd coda-kernels
pip install -e .

CODA depends on a fork of quack, pinned to the tag v0.6.5+fork.1. Its version differs from PyPI's quack-kernels 0.6.5, so installing CODA replaces a PyPI install.

Functional API

About

CODA: Rewriting Transformer Blocks as GEMM-Epilogue Programs

Resources

Stars

258 stars

Watchers

5 watching

Forks

Releases

Packages

Contributors

Languages