Skip to content

Implement BitLinear Layer #178

Closed
Closed

Description

@thejhh

Develop a core linear algebra routine for BitNet’s binary-weight linear layer (BitLinear). This function takes an input vector (or matrix) of 8-bit activations and multiplies it by a weight matrix with 1.58-bit (ternary) weights to produce an 8-bit output
ar5iv.org.

To do this efficiently, use bitwise operations: unpack the weight matrix bits (as prepared in step 3) on the fly, interpret 2-bit codes as -1, 0, or +1, and compute the dot-products. Exploit binary arithmetic where possible (e.g., treat -1 as +1 with a sign bit to use XNOR or bit-count tricks if inputs were binary, though here inputs are 8-bit). Accumulate results in a higher precision type (e.g. int32) then quantize/clamp the output back to int8 range. No bias terms should be added in these projections
ar5iv.org.

Make sure the BitLinear function is general and can handle weight matrices of various shapes (it will be used for Q, K, V projections, feed-forward layers, etc.). Optimize inner loops with bit-level operations and verify correctness on small test matrices.

Activity

  1. added a commit that references this issue on May 21, 2025
  2. added theissue type on May 21, 2025
  3. self-assigned this
    on May 21, 2025
  4. added 2 commits that reference this issue on May 21, 2025
    750678e
    8ee212a
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bitnetBitNet implementationtask

Type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions