SFG provides inference-time guidance for image and video generation, without fine-tuning or additional learned weights.
Paper: Images Talk Back: Guiding MMDiTs by Suppressing Image-to-Text Feedback
Supported models: SD3 Medium · SD3.5 Medium · FLUX.1-dev · FLUX-de-distill · HunyuanVideo-1.5 (T2V / I2V)
Use Python 3.10+ and a CUDA GPU.
git clone https://github.com/yyc1215lll-creator/SFG.git
cd SFG
python -m venv .venv
source .venv/bin/activate
pip install --upgrade pip
pip install -r requirements-image.txtFor video generation, use a separate environment:
pip install -r video/third_party/HunyuanVideo-1.5/requirements.txt
pip install flash-attn==2.8.3 --no-build-isolationInstall a PyTorch build compatible with your CUDA environment before installing FlashAttention.
Download the model and pass its local directory with --weights.
| Model | Weights |
|---|---|
| SD3 Medium | stabilityai/stable-diffusion-3-medium-diffusers |
| SD3.5 Medium | stabilityai/stable-diffusion-3.5-medium |
| FLUX.1-dev | black-forest-labs/FLUX.1-dev |
| FLUX-de-distill transformer | InstantX/flux-dev-de-distill-diffusers |
| HunyuanVideo-1.5 | Upstream model setup |
Use complete Diffusers model directories for image models. FLUX-de-distill
additionally requires a de-distilled transformer directory with
guidance_embeds=false; pass it with --transformer.
For HunyuanVideo, use the native Base 480p_t2v or 480p_i2v weights.
For example, after accepting any required model access terms:
hf auth login
hf download stabilityai/stable-diffusion-3.5-medium --local-dir models/sd35m
hf download black-forest-labs/FLUX.1-dev --local-dir models/flux-dev
hf download InstantX/flux-dev-de-distill-diffusers --local-dir models/flux-de-distill
hf download tencent/HunyuanVideo-1.5 --local-dir models/HunyuanVideo-1.5The downloaded models/flux-de-distill directory itself is the transformer
directory to pass to --transformer.
Provide a text file with one prompt per line. Example prompts are included in examples/prompts.txt.
# SD3.5 Medium
python run_image.py --model sd35m --weights /path/to/SD3.5-medium \
--prompt-file examples/prompts.txt --output outputs/sd35m
# SD3 Medium
python run_image.py --model sd3m --weights /path/to/SD3-medium \
--prompt-file examples/prompts.txt --output outputs/sd3m
# FLUX.1-dev
python run_image.py --model flux-dev --weights /path/to/FLUX.1-dev \
--prompt-file examples/prompts.txt --output outputs/flux-dev
# FLUX-de-distill
python run_image.py --model flux-de-distill --weights /path/to/FLUX.1-dev \
--transformer /path/to/de-distilled/transformer \
--prompt-file examples/prompts.txt --output outputs/flux-de-distillImages are saved to OUTPUT/result/. Use a fresh output directory for each run.
Use --batch-size 1 to reduce memory usage.
Create a prompt manifest, then launch generation:
# Text-to-video
python prepare_video_manifest.py \
--prompt 'A boat moves across a calm lake.' --output t2v.jsonl
torchrun --standalone --nproc_per_node=1 run_video.py --task t2v \
--weights /path/to/HunyuanVideo-1.5 \
--manifest t2v.jsonl --output outputs/t2v
# Image-to-video
python prepare_video_manifest.py \
--prompt 'The subject slowly turns its head.' \
--reference-image /path/to/reference.png --output i2v.jsonl
torchrun --standalone --nproc_per_node=1 run_video.py --task i2v \
--weights /path/to/HunyuanVideo-1.5 \
--manifest i2v.jsonl --output outputs/i2v
# Image-to-video with text-only guidance
torchrun --standalone --nproc_per_node=1 run_video.py --task i2v-text \
--weights /path/to/HunyuanVideo-1.5 \
--manifest i2v.jsonl --output outputs/i2v-textUse --offloading to enable model offloading or --resume to resume a run.
For image generation, replace python with
torchrun --standalone --nproc_per_node=8.
For video generation, set --nproc_per_node=8 with a multi-row manifest.
Each GPU loads a complete model and processes independent samples.
If you find this work useful, please cite our paper:
@article{yang2026imagestalkback,
title={Images Talk Back: Guiding {MMDiTs} by Suppressing Image-to-Text Feedback},
author={Yang, Yucheng and Sun, Xiaoxiao and Yang, Shusheng and Chen, Jiakang and Yeung-Levy, Serena},
journal={arXiv preprint arXiv:2610.13941},
year={2026},
doi={10.48550/arXiv.2610.13941},
url={https://arxiv.org/abs/2610.13941}
}Built on Diffusers, Stable Diffusion, FLUX, and HunyuanVideo-1.5. Third-party code and model weights retain their respective licenses; see THIRD_PARTY_NOTICES.md.