Quantized Token Embedding Layer #175

Implement the embedding layer that maps input token IDs to their initial hidden vectors. Use the embedding matrix from the BitNet weights (this is typically a learned matrix of size vocab_size × hidden_size). Since the weights are quantized, apply the same 2-bit ternary {–1, 0, +1} decoding for the embedding matrix or treat the embedding as a special case of a BitLinear operation (a one-hot vector multiplied by ternary weights). In practice, this can be done by looking up the precomputed vector for each token or by decoding the relevant weight bytes for that token. This produces the model’s initial hidden state for each input token in the sequence.

After decoding to int8 values (–1, 0, +1), multiply the vector by the per-matrix abs-mean scale stored with the embedding weights to obtain float32 values before further processing.

BitNet omits bias terms, so no bias is applied in the embedding lookup.