Skip to content

Gemma4 ov int4 and stateful fixes - #260

Draft
cavusmustafa wants to merge 5 commits into
ravi9:dev_backend_openvinofrom
cavusmustafa:gemma4_ov_int4_and_stateful_fixes
Draft

Gemma4 ov int4 and stateful fixes#260
cavusmustafa wants to merge 5 commits into
ravi9:dev_backend_openvinofrom
cavusmustafa:gemma4_ov_int4_and_stateful_fixes

Conversation

@cavusmustafa

@cavusmustafa cavusmustafa commented Jul 18, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator
  • enables gemma4 dense stateful support (limited support, up to 1k context)
  • optional int4 downscale for requant: sacrifice accuracy for performance (env to enable - GGML_OPENVINO_INT4_REQUANT)
  • optional disk spill when weight loading to reduce initial RSS peak for low memory systems (env to enable: GGML_OPENVINO_SPILL_DIR)

@cavusmustafa
cavusmustafa force-pushed the gemma4_ov_int4_and_stateful_fixes branch from 530aa62 to cbb8242 Compare July 21, 2026 19:26
@wine99
wine99 force-pushed the dev_backend_openvino branch from 4250de7 to 175f422 Compare July 27, 2026 04:33
@cavusmustafa
cavusmustafa force-pushed the gemma4_ov_int4_and_stateful_fixes branch from 53d099b to 61c08f4 Compare August 11, 2026 21:16
@wine99
wine99 force-pushed the dev_backend_openvino branch 2 times, most recently from 37b164f to 2bacf9e Compare August 14, 2026 05:24
@cavusmustafa
cavusmustafa force-pushed the gemma4_ov_int4_and_stateful_fixes branch from 61c08f4 to 4acdec3 Compare August 14, 2026 18:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@wine99 wine99 Awaiting requested review from wine99 wine99 will be requested when the pull request is marked ready for review wine99 is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@cavusmustafa