[QNN] Improve GPU StaticLLM pass - #2633
Conversation
|
Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run. |
* Update search.max_length in genai_config.json to inform onnxruntime-genai when to stop decode, which is required for the QNN GPU exports with fixed max seq len. * Remove the "update_genai_config" option. * Handle new ModelBuilder behavior with symbolic "kv_cache_dim". * Add "inherit_session_options" to pipeline models in the genai config.
a654895 to
3cc8454
Compare
|
Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run. |
There was a problem hiding this comment.
Pull request overview
This PR updates the QNN GPU StaticLLM export flow so generated genai_config.json better reflects fixed-shape decoding requirements (fixed max sequence length), aligns with newer ModelBuilder symbolic dim behavior (kv_cache_dim), and standardizes session options inheritance across pipeline entries.
Changes:
- Update QNN GPU genai pipeline generation to set
inherit_session_optionsand to derivemodel.context_length/search.max_lengthfrom the fixed KV-cache input shape. - Remove the
update_genai_configoption and always perform genai_config updates in the relevant QNN GPU paths. - Add QNN GPU-focused unit tests that validate prefill/decode pipeline structure and enforce static I/O shapes.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
test/passes/onnx/test_static_llm.py |
Adds QNN GPU coverage validating genai pipeline structure and ensuring exported model I/O shapes are fully static. |
olive/passes/onnx/static_llm.py |
Handles new ModelBuilder symbolic kv_cache_dim by mapping it using head_size from genai_config.json; removes update_genai_config gating. |
olive/passes/onnx/context_binary.py |
Removes update_genai_config option and updates QNN GPU context-binary path to update genai pipeline entries unconditionally. |
olive/passes/onnx/common.py |
Adds inherit_session_options to pipeline entries and updates genai max sequence length fields based on the fixed KV-cache input shape. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.
Suppressed comments (1)
olive/passes/onnx/context_binary.py:312
- update_llm_pipeline_genai_config_gpu_ctxbin is now called unconditionally for QNN GPU, but
_generate_context_binaryacceptsmodel=Noneand ModelHandlers can havemodel_attributes=None(or missingadditional_files). In those cases this call will raise before any genai_config update can be attempted. Guard the call so non-GenAI/non-LLM QNN GPU flows still work.
update_llm_pipeline_genai_config_gpu_ctxbin(model, ctxbin_path)
|
Jambay Kinley (@jambayk), PTAL. This is needed for QNN. Thanks! |
Describe your changes
Checklist before requesting a review
lintrunner -a