An end-to-end Machine Learning security firewall detecting visual prompt injection, typographic camouflage, and adversarial instruction overrides targeting Multimodal Large Language Models (MLLMs).
As Multimodal Large Language Models (MLLMs such as GPT-4V, Claude 3.5 Sonnet, and Gemini) become widely integrated into enterprise systems, they introduce a critical attack surface: Visual Prompt Injection (VPI).
Attackers can embed covert text instructions inside images—ranging from visible memos and signage to extreme low-contrast typographic camouflage (
This project implements an independent, lightweight ML security gateway deployed upstream of the VLM. The firewall:
- Normalizes and enhances image contrast via Computer Vision (adaptive Otsu binarization and thresholding) to defeat typographic camouflage.
- Extracts textual payloads using Optical Character Recognition (OCR).
- Analyzes semantic intent with n-gram TF-IDF vectorization.
- Classifies threats with a Linear Support Vector Machine (LinearSVC) tuned specifically to maximize Recall, preventing zero-day adversarial payloads from compromising the multimodal model.
Input Image ──► [CV Preprocessing & Binarization] ──► [Tesseract OCR] ──► [TF-IDF Vectorizer] ──► [Linear SVM] ──► SAFE (Forward to VLM)
│
└──► MALICIOUS (Block & Alert)
The pipeline is validated against both In-Distribution synthetic datasets (5,000 images incorporating real-world AdvBench jailbreaks) and Out-of-Distribution real-world benchmarks (Meta's official CyberSecEval 3 dataset).
| Dataset / Benchmark | Accuracy | Precision | Recall | F1-Score | ROC-AUC |
|---|---|---|---|---|---|
| Synthetic Test Set (In-Distribution) | 100.0% | 100.0% | 100.0% | 100.0% | 1.000 |
| Meta CyberSecEval 3 (Out-of-Distribution) | 86.7% | 100.0% | 80.0% | 88.9% | 1.000 |
Note: Recall is strictly prioritized over precision to minimize False Negatives, ensuring malicious payloads never leak through to the target VLM.
Evaluated on the synthetic test partition with zero false negatives and zero false positives.
Unsupervised clustering (
- Cluster 0: Benign Notices & Signage (meeting, invoice, exit, caution)
- Cluster 1: System Prompt Exfiltration (system, instructions, verbatim, print, reveal)
- Cluster 2: Command Injection & Jailbreaks (dan, ignore, bypass, curl, bash, override)
Linear SVM coefficient analysis highlights discriminative keywords driving adversarial detection:
- Python 3.10 or higher
- System Tesseract-OCR engine:
# Debian/Ubuntu sudo apt-get install tesseract-ocr # macOS brew install tesseract
# Clone the repository
git clone https://github.com/odai-dev/visual_prompt_injection_detection.git
cd visual_prompt_injection_detection
# Create and activate virtual environment
python3 -m venv .venv
source .venv/bin/activate
# Install dependencies
pip install -r requirements.txtThe CLI tool src/main.py provides complete control over training, live inference, and evaluation:
Generates synthetic data, executes OCR extraction, fits TF-IDF vectorizers, trains Linear SVM & Random Forest classifiers, performs K-Means clustering, and renders all diagnostic figures:
python src/main.py --trainPass any candidate image through the preprocessing, OCR, and classification pipeline:
python src/main.py --predict path/to/sample_image.pngOutput: Returns safety verdict (SAFE vs MALICIOUS), confidence probability, and extracted payload.
Re-evaluates the serialized model (models/best_supervised_model.joblib) against the synthetic test set and Meta's CyberSecEval 3:
python src/main.py --evaluatepytest tests/├── Ar-rasheed_uni_logo.png # University insignia
├── Visual_Prompt_Injection_Detection_Report.pdf # Final compiled academic thesis/report
├── Visual_Prompt_Injection_Detection.ipynb # Interactive demonstration notebook
├── report.md # Markdown source for the academic paper
├── requirements.txt # Python dependencies
├── src/
│ ├── main.py # CLI interface entry point
│ ├── data_collection.py # Automated synthetic data generation engine
│ ├── preprocessing.py # Computer Vision filtering & OCR pipeline
│ ├── feature_extraction.py # TF-IDF n-gram vectorization
│ ├── modeling.py # Supervised Linear SVM & Random Forest
│ └── clustering.py # Unsupervised K-Means semantic grouping
├── models/
│ ├── best_supervised_model.joblib # Serialized Linear SVM classifier
│ ├── tfidf_vectorizer.joblib # Fitted TF-IDF feature extractor
│ └── kmeans_model.joblib # Serialized K-Means clustering model
├── results/
│ ├── figures/ # Generated charts (ROC, Confusion Matrix, PCA)
│ ├── predictions/ # Inference outputs & CyberSecEval 3 predictions
│ └── reports/ # Detailed metrics JSON and final summary markdown
└── tests/ # Unit and integration test suites
- Author: Odai Gubran
- Institution: Ar-Rasheed Smart University
- Supervisor: Eng. Sondos Saif
- Subject: Visual Prompt Injection Detection in Large Vision-Language Models
For comprehensive methodology details and formal literature review, see Visual_Prompt_Injection_Detection_Report.pdf and report.md.





