ADFA-5188 | Enable KV cache quantization and flash attention - #76

Merged

jatezzz merged 17 commits into

mainfrom

fix/ADFA-5188-kv-cache-quantization

Sep 2, 2026

Commits on Aug 26, 2026

feat(ai-agent-local): size n_ctx per model and device
jatezzz
committed
b4efd66
feat(ai-agent-local): store the KV cache as q8_0 to double the context
jatezzz
committed
c45cce1

Commits on Aug 27, 2026

fix(ai-agent-local): charge model weights against the KV budget
jatezzz
committed
1706b80
merge: bring ADFA-5187 weights accounting into the q8_0 KV cache
jatezzz
committed
24b31ac
fix(ai-agent-local): retry a failed context at the 4096 floor
jatezzz
committed
f570d09
fix(ai-agent-local): parse the GGUF header once and fail safe on absurd sizes
jatezzz
committed
5d181b3
merge: bring ADFA-5187 single-parse header into the q8_0 KV cache
jatezzz
committed
5d1f91c

Commits on Sep 1, 2026

Merge remote-tracking branch 'origin/main' into fix/ADFA-5188-kv-cache-quantization
jatezzz
committed
c5e58e7

Commits on Sep 2, 2026

Merge remote-tracking branch 'origin/main' into fix/ADFA-5188-kv-cache-quantization
jatezzz
committed
012f064