PrAda: Few-Shot Visual Adaptation for Text-Prompted Segmentation

Gabriele Rosi^1,2 · Fabio Cermelli² · Carlo Masone^1,2 · Barbara Caputo^1,2

¹ Politecnico di Torino · ² Focoos AI

PrAda adapts frozen text-prompted segmentation models to new domains using just a few labeled examples.

• 🔍 Root-cause analysis: we study open-vocabulary segmentation under domain shift and find that misclassification, rather than mask quality, is the dominant failure mode.
• 🧩 Rich prototypes: class-specific visual prototypes capture complementary spatial and semantic information, enabling effective adaptation from only a few examples.
• ⚡ Lightweight adaptation: only the prototypes and a single importance scalar are learned, adding as little as +0.02% parameters while keeping the full model frozen.
• 📈 Strong performance: PrAda consistently improves over state of the art across semantic, instance, and panoptic segmentation on 5 benchmarks (28 datasets).
• ♾️ Zero-shot preserved: visual and text scores are fused at inference, retaining the original open-vocabulary capability out of the box.

Whether you want to reproduce the paper or adapt the idea to a new segmentation domain, this repository provides the complete path from support images to prototypes, training, compact adapters, and evaluation.

• Reproduce the paper experiments
• Train and evaluate one experiment
• Bring your own dataset

How PrAda works

PrAda starts from a frozen FC-CLIP model. Given a small labeled support set, it extracts mask-aware visual features and averages them into one prototype per target class. During adaptation, the original text score is combined with prototype similarity:

adapted logits = FC-CLIP logits + alpha × cosine(mask-pooled query, class prototype)

Only the class prototypes and the scalar alpha are optimized. The backbone, pixel decoder, transformer decoder, mask prediction, and text classifier stay frozen. The trained adapter is therefore small, easy to store, and tied to an explicit prototype bank and support split.

Installation

PrAda builds on FC-CLIP, Detectron2, and Mask2Former. Python 3.11, PyTorch 2.5.1 with CUDA 11.8, Detectron2 0.6, OpenCLIP 2.24.0, and timm 1.0.14 were used for the released experiments.

Install a PyTorch and Detectron2 build compatible with your CUDA version, then install the remaining dependencies and compile the deformable-attention operator:

cd PrAda
pip install -r requirements.txt
cd fcclip/modeling/pixel_decoder/ops
sh make.sh
cd ../../../..

Download the two official FC-CLIP checkpoints following weights/README.md:

weights/
├── fcclip_cocopan.pth
└── fcclip_cocopan_r50.pth

Datasets

Set DETECTRON2_DATASETS to the directory containing the datasets:

export DETECTRON2_DATASETS=/path/to/datasets

The released registrations expect the following layout:

$DETECTRON2_DATASETS/
├── ADEChallengeData2016/
├── cityscapes/
├── mapillary_vistas/
├── seginw/
└── ShowOrTell/

Few-shot support splits are already included under fcclip/data/datasets/fewshot_files/. Dataset-specific preparation otherwise follows the upstream FC-CLIP conventions.

Reproducing the paper

The main experiments use five support seeds and the released 5-shot prototype banks. A single ADE20K ConvNeXt-L run is:

python tools/run_prada.py ade20k \
  --backbone cnextl --shots 5 --seed 0 \
  --weights-root weights --num-gpus 2

Run all five seeds with the matrix runner:

# ConvNeXt-L
python tools/train_matrix.py ade20k --backbone cnextl --shots 5 \
  --seeds 0 1 2 3 4 --weights-root weights --num-gpus 2
python tools/train_matrix.py cityscapes --backbone cnextl --shots 5 \
  --seeds 0 1 2 3 4 --weights-root weights --num-gpus 2
python tools/train_matrix.py mapillary --backbone cnextl --shots 5 \
  --seeds 0 1 2 3 4 --weights-root weights --num-gpus 2

# R50
python tools/train_matrix.py ade20k --backbone r50 --shots 5 \
  --seeds 0 1 2 3 4 --weights-root weights --num-gpus 2
python tools/train_matrix.py cityscapes --backbone r50 --shots 5 \
  --seeds 0 1 2 3 4 --weights-root weights --num-gpus 2
python tools/train_matrix.py mapillary --backbone r50 --shots 5 \
  --seeds 0 1 2 3 4 --weights-root weights --num-gpus 2

For the cross-dataset suites:

python tools/train_matrix.py seginw --all-datasets \
  --seeds 0 1 2 3 4 --weights-root weights --num-gpus 2

python tools/train_matrix.py sot --all-datasets \
  --seeds 0 1 2 3 4 --weights-root weights --num-gpus 2

Training always performs a final evaluation and writes per-run results.json and results.csv files plus a matrix-level JSON/CSV summary. The paper's ADE20K, Cityscapes, and Mapillary seed-wise results are available in reference_results.csv.

For the ConvNeXt-L shot ablation, use --shots 1, --shots 2, or --shots 10 on ADE20K and Cityscapes. R50 and the remaining benchmarks use the released 5-shot setting.

Mapillary note: full-resolution evaluation is memory intensive. Two 24 GB GPUs are close to the limit; larger-memory GPUs are recommended if evaluation OOMs.

Training and evaluation

The experiment registry selects the support split, prototype bank, class IDs, learning rate, iteration count, and batch size from the requested benchmark, backbone, shot count, and seed.

Train Cityscapes with R50:

python tools/run_prada.py cityscapes \
  --backbone r50 --shots 5 --seed 0 \
  --weights-root weights --num-gpus 2

Train one SegInW or Show-or-Tell dataset:

python tools/run_prada.py seginw --dataset Chicken \
  --seed 0 --weights-root weights --num-gpus 2

python tools/run_prada.py sot --dataset Pizza \
  --seed 0 --weights-root weights --num-gpus 2

Outputs use deterministic paths containing every experiment axis:

output/ade20k/cnextl/5shot/seed_0/
├── adapter_final.pth
├── metrics.json
├── results.json
└── results.csv

Evaluate a compact adapter without retraining:

python tools/run_prada.py ade20k \
  --backbone cnextl --shots 5 --seed 0 \
  --weights-root weights \
  --adapter output/ade20k/cnextl/5shot/seed_0/adapter_final.pth \
  --eval-only --num-gpus 2

Periodic checkpoints contain only PrAda parameters plus optimizer/scheduler state required for resuming. Final checkpoints contain only the learned prototypes, alpha, and metadata identifying the base checkpoint, prototype bank, classes, and support split. W&B is disabled unless --wandb-project is supplied.

Computing prototypes

The released prototype banks are ready to use. To build one again from its support images:

python tools/compute_prototypes.py ade20k \
  --backbone cnextl --shots 5 --seed 0 \
  --datasets-root "$DETECTRON2_DATASETS" \
  --weights-root weights \
  --output-root generated_prototypes

Each artifact stores raw 256-dimensional support features and their class IDs:

{
    "prototypes": FloatTensor[N, 256],
    "prototype_classes": LongTensor[N],
}

The adjacent JSON sidecar records the resolved experiment, support-list checksum, extraction statistics, and artifact checksum. The notebook-free implementation preserves the support ordering and matching behavior used by the original prototype-generation workflow.

Bring your own dataset

PrAda can be applied to a new semantic segmentation dataset without modifying the model or experiment registry. Define the dataset, classes, paths, and schedule in one JSON file; select support images in one CSV; then use the same prototype, training, and evaluation commands as the built-in benchmarks.

The complete walkthrough is in CUSTOM_DATASETS.md, with ready-to-edit examples in examples/custom_semantic_dataset.json and examples/support_5shot_seed0.csv.

Citation

If you find this work useful, please consider citing:

@inproceedings{rosi2026prada,
  title     = {PrAda: Few-Shot Visual Adaptation for Text-Prompted Segmentation},
  author    = {Rosi, Gabriele and Cermelli, Fabio and Masone, Carlo and Caputo, Barbara},
  booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Findings},
  year      = {2026}
}

Acknowledgements

This codebase is built on FC-CLIP, Detectron2, and Mask2Former. We thank their authors for releasing their excellent work. Upstream copyright notices are retained in derived files; see LICENSE and NOTICE.