Skip to content

fix(quant): export modelopt_state.pth after quantization - #256

Draft
mutichung wants to merge 4 commits into
NVIDIA:mainfrom
mutichung:mutic/fix-modelopt-state
Draft

mutichung wants to merge 4 commits into
NVIDIA:mainfrom
mutichung:mutic/fix-modelopt-state

Conversation

@mutichung

Copy link
Copy Markdown
Contributor

Problem

The original quantization pipeline did not create the corresponding modelopt_state.pth that is presented in the released FP8 checkpoints. The file is required for HuggingFace transformers/diffusers to load the and run inference on the quantized checkpoints.

Changes

  • Export the missing modelopt_state.pth files for transformers and diffusers.
  • Fix updated functions imported by legacy mode.
  • Add mixed precision W8A16 policy configuration.

Blocks NVIDIA/cosmos#298

Signed-off-by: Mu-Ti Chung <mutic@nvidia.com>
Signed-off-by: Mu-Ti Chung <mutic@nvidia.com>
Signed-off-by: Mu-Ti Chung <mutic@nvidia.com>
Signed-off-by: Mu-Ti Chung <mutic@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant