You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Use the attention weights to compute the attention output for each head. For each attention head, multiply the weight matrix (from step 12) by the head’s Value matrix $V$ (shape [seq_len, head_dim]). This yields the context matrix for that head (of shape [seq_len, head_dim]), where each token now contains a weighted sum of value vectors from other tokens. Implement this multiplication in an efficient manner, again using higher precision for accumulation if $V$ is 8-bit. The result should then be quantized/clamped back to 8-bit range. After computing the context for all heads, concatenate the context vectors of all heads together for each token (resulting in a [seq_len, hidden_size] matrix, since hidden_size = num_heads * head_dim).
Activity
fix(test): resolve failing Go tests for scaled attention (#183)
Optimize attention weights multiplication with values (#211)