Skip to content
BranchesTags

Repository files navigation

META FIT - AI Virtual Try-On

AI-powered virtual try-on system that generates realistic images of clothing on a user's body from a single photo.

Two modern API-based engines and one legacy GAN-based engine are available.

Important Notice

This repository contains components under different licenses.

  • PASTA-GAN++ (legacy): Non-commercial research and educational purposes only. See License Notice below.
  • Nano Banana / Vertex AI VTO (current): Commercial use allowed under Google API terms.

Please read the license section carefully before using any part of this project.

Quick Start

Option A: Nano Banana (Recommended)

The simplest and most capable option. Uses Gemini API for virtual try-on.

Prerequisites

pip install google-genai python-dotenv pillow

Get API Key

  1. Go to Google AI Studio
  2. Create an API key
  3. Create .env file in the project root:
GEMINI_API_KEY=your-api-key-here

Run Try-On

# Clothing mode: product image -> person
python3 try_on_test.py --person test_data/person/woman_standing4.jpg \
                       --clothing test_data/clothing/red_dress.jpg

# Transfer mode: source person's clothes -> target person
python3 try_on_test.py --mode transfer \
                       --person test_data/person/woman_standing3.jpg \
                       --source test_data/person/woman_standing5.jpg

# With MediaPipe preprocessing (usually not needed for high-res images)
python3 try_on_test.py --mode transfer \
                       --person test_data/person/man_standing.jpg \
                       --source test_data/person/man_standing2.jpg \
                       --preprocess

Results are saved to test_results/nano_banana/.

Option B: Vertex AI Virtual Try-On

Google's dedicated virtual try-on model. Best for clothing product images (flat lay / white background).

Prerequisites

pip install google-genai google-auth python-dotenv pillow

GCP Setup

  1. Create a GCP project (or use an existing one) at Google Cloud Console

  2. Enable Vertex AI API:

  3. Create a Service Account:

  4. Download JSON key:

    • Click on the created service account
    • Go to "Keys" tab -> "Add Key" -> "Create new key" -> JSON
    • Save the downloaded JSON file to configs/ directory (gitignored)

  5. Configure .env:

GOOGLE_CLOUD_PROJECT=your-project-id
GOOGLE_CLOUD_LOCATION=us-central1
GOOGLE_APPLICATION_CREDENTIALS=configs/your-key-file.json

Note: GCP offers $300 free credit for new accounts. Virtual Try-On costs approximately $0.02-0.04 per image.

Tip: If gcloud auth application-default login fails with scope errors (common with older gcloud versions), the service account method above is more reliable.

Run Try-On

# Connection test
python3 test_vertex_vto.py

# Clothing mode (Vertex VTO's strength)
python3 test_vertex_vto.py test_data/person/woman_standing4.jpg test_data/clothing/red_dress.jpg

# Generate multiple samples (up to 4)
python3 test_vertex_vto.py test_data/person/man_standing.jpg test_data/clothing/hoodie.jpg 4

Results are saved to test_results/vertex_vto/.

Option C: PASTA-GAN++ (Legacy / Deprecated)

DEPRECATED: This is the legacy GAN-based approach. It is no longer actively maintained and no support is provided. The results are significantly inferior to Nano Banana / Vertex VTO. Included for research reference only.

LICENSE: PASTA-GAN++ and its dependencies (StyleGAN2, OpenPose) are non-commercial research only. Do NOT use for commercial purposes.

Prerequisites

Setup

  1. Download model weights (not included in this repo — download at your own responsibility):

    Model File Download
    PASTA-GAN++ network-snapshot-004408.pkl Google Drive (from official repo)
    OpenPose body_pose_model.pth (~209MB) HuggingFace
    Graphonomy inference.pth (~167MB) HuggingFace

    Note: These weights are provided by their respective authors. Please review each project's license before use.

  2. Place weights in the following structure:

    metafit/
└── weights/
    ├── pasta-gan++/network-snapshot-004408.pkl
    ├── openpose/body_pose_model.pth
    └── graphonomy/inference.pth

  3. Prepare test data:

    metafit/
└── test_datas/
    ├── image/       (320x512px, white background images)
    ├── keypoints/   (OpenPose output, generated by test.py)
    ├── parsing/     (Graphonomy output, generated by test.py)
    └── test_pairs.txt
    • Place your test images in test_datas/image/ (must be 320x512px, full-length, white background)
    • Create test_datas/test_pairs.txt with space-separated pairs: 
      target_person.jpg source_model.jpg
    • keypoints/ and parsing/ are generated automatically if OpenPose/Graphonomy calls are uncommented in test.py

  4. Build and run Docker:

    make build && make run

  5. Run inference (inside Docker container):

    python3 test.py --config configs/test_config.yaml

    Note: OpenPose and Graphonomy preprocessing are currently commented out in test.py. If you need to generate keypoints/ and parsing/ from scratch, uncomment lines 28-40 in test.py.

    Results are saved to test_results/full/.

Test Configuration

Edit configs/test_config.yaml:

dataroot: test_datas
testtxt: test_pairs.txt
network: weights/pasta-gan++/network-snapshot-004408.pkl
outdir: test_results/full
batchsize: 1
testpart: full          # full / upper / lower
use_sleeve_mask: false

Input Requirements

  • Image size: 320x512 pixels (width x height)
  • Full-length photo on white background
  • Supported parts: full body, upper body, lower body

Test Data

Person images (test_data/person/)

Various body types, poses, and genders for comprehensive testing. All images are from Unsplash (free for commercial use, no attribution required).

Clothing images (test_data/clothing/)

File Type Source
tshirt_black.png Black T-shirt -
sckirt.png Skirt -
red_dress.jpg Red dress Unsplash
denim_jacket.jpg Denim jacket Unsplash
hoodie.jpg Grey hoodie Unsplash
jeans.jpg Jeans Unsplash
striped_shirt.jpg Striped shirt Unsplash
suit_blazer.jpg Navy suit Unsplash

Engine Comparison

Feature PASTA-GAN++ (Legacy) Nano Banana (Current) Vertex AI VTO
Approach GAN (local GPU) Gemini API (cloud) Dedicated VTO model (cloud)
Clothing mode (product -> person) N/A Good Best (faithful reproduction)
Transfer mode (person -> person) Poor Best Not supported
Body type diversity Poor (slim bias) Best (faithful) Good
Complex patterns Poor Best Good
Shoes N/A Poor Best
Safety filter None (local) Strict (blocks exposed clothing) Moderate (inconsistent)
Setup complexity High (GPU + Docker) Low (API key only) Medium (GCP project)
Cost Free (local) Free tier available ~$0.02-0.04/image
Commercial use No Yes Yes

License Notice

Non-Commercial Components (PASTA-GAN++ legacy pipeline)

The following components are included for research and educational purposes only. They MUST NOT be used for commercial purposes:

Component License Repository
PASTA-GAN++ Non-commercial research only Try-on generation
StyleGAN2 (NVIDIA) NVIDIA Source Code License-NC Generator backbone
OpenPose (CMU) CMU Academic Non-Commercial Pose estimation

Affected directories: torch_utils/, dnnlib/, training/, src/generate_keypoints.py, src/body.py

If you wish to use OpenPose commercially, a license is available through CMU FlintBox (~$25,000/year).

Commercial-Friendly Components

Component License Usage
Graphonomy MIT Body segmentation
MediaPipe Apache 2.0 Face/Pose detection
PyTorch BSD ML framework
OpenCV Apache 2.0 Image processing
Pillow MIT-like (HPND) Image processing
Gemini API (Nano Banana) Google API Terms Try-on generation
Vertex AI VTO Google Cloud Terms Try-on generation

For Commercial Use

If you plan to use this project commercially, use only the Nano Banana (try_on_test.py) and/or Vertex AI VTO (test_vertex_vto.py) pipelines. These do not depend on any non-commercial components.

Commercial-safe pipeline:
  Photo -> Gemini API (Nano Banana) -> Try-on image     OK
  Photo -> Vertex AI VTO API        -> Try-on image     OK

NOT commercial-safe:
  Photo -> OpenPose -> PASTA-GAN++ -> Try-on image      NG

Project Site

https://suzuki-shoten.dev/projects/metafit/

About

AI-powered virtual try-on system - from PASTA-GAN++ to Gemini Nano Banana

Resources

Readme
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

Packages

Contributors

Languages