Status — Research-active browser extraction toolkit. PHANTOM is being retained for deterministic browser automation, structured extraction, scheduling, and data collection. Older fingerprint/behavior-simulation code remains experimental and does not claim reliable anti-bot bypass.
A browser-automation research prototype originally designed as a Hermes agent skill. PHANTOM automates browsing, scraping, page monitoring, form interaction, and structured extraction. Its fingerprint and behavior-simulation layers are experimental; effectiveness against modern anti-bot systems has not been validated.
The active wedge is provider-agnostic browser extraction reliability, not stealth. PHANTOM can now validate structured payloads from its own Playwright workflows or from external browser agents/providers against deterministic extraction contracts. Current work prioritizes extraction contracts, canonical evidence fingerprints, locale-aware parsing, structured-data extraction, retries/queues, and replayable quality evidence. Fingerprint and behavior-simulation modules are retained as experiments and are not treated as a security or anti-detection guarantee.
Browser automation succeeding does not prove the extracted payload is usable. PHANTOM separates those concerns.
POST /validate/extraction accepts any JSON payload plus a deterministic contract:
{
"data": {
"product": {
"title": "Widget",
"price": {"amount": 29.99}
}
},
"contract": {
"required_fields": ["product.title", "product.price.amount"],
"optional_fields": ["product.seller"],
"min_required_coverage": 1.0
}
}The response includes required-field coverage, missing/blank fields, field-level evidence, and canonical SHA-256 fingerprints for both the contract and payload. The validator has no Playwright dependency, so the same contract can verify results produced by PHANTOM, an MCP browser tool, Browserbase, Cloudflare Browser Rendering, or another provider.
This is the active product thesis: browser agents execute; PHANTOM verifies the extraction contract.\n\nPOST /validate/extraction-drift additionally compares a baseline/current payload under the same contract and reports changed fields, fields missing now, newly present fields, stable fields, and whether the current payload still passes.
┌─────────────────────────────────────┐
│ PHANTOM FastAPI (8001) │
│ /browse /scrape /extract /jobs │
└────────────────┬────────────────────┘
│
┌──────────────────────────▼─────────────────────────────┐
│ Core Engine │
│ │
│ ┌──────────────┐ ┌──────────────┐ ┌───────────┐ │
│ │ Stealth │ │ Browser │ │ Queue │ │
│ │ Layer │ │ Layer │ │ Layer │ │
│ │ │ │ │ │ │ │
│ │ fingerprint │ │ session.py │ │ TaskQueue │ │
│ │ humanizer │ │ navigator.py │ │ Scheduler │ │
│ │ evasion │ │ extractor.py │ │ │ │
│ └──────────────┘ └──────────────┘ └───────────┘ │
│ │
│ ┌──────────────────────────────────────────────────┐ │
│ │ Extractors │ │
│ │ price.py article.py structured.py │ │
│ └──────────────────────────────────────────────────┘ │
│ │
│ ┌──────────────┐ ┌─────────────────────────────┐ │
│ │ Cache │ │ Hermes Skill │ │
│ │ Redis/Mem │ │ PHANTOM.md (8 tools) │ │
│ └──────────────┘ └─────────────────────────────┘ │
└─────────────────────────────────────────────────────────┘
│
┌──────────────────────────▼──────────────────────────────┐
│ Playwright (Chromium) │
│ Context per task • Evasion scripts • Fingerprinting │
└─────────────────────────────────────────────────────────┘
- Fingerprint randomization — Realistic Chrome 120–125 User-Agents, weighted viewport sizes (1366×768, 1440×900, 1920×1080, 2560×1440), timezone/locale pairing, full HTTP header sets
- Human behavior simulation — Bezier-curve mouse paths, random micro-delays (50–300ms), ease-in-out scroll patterns, typo-and-correct form fill simulation
- WebDriver evasion — Hides
navigator.webdriver, fakeschrome.runtime, spoofsnavigator.plugins, overridesnavigator.languages, fixes iframe detection, spoofs hardware concurrency
- Async Playwright (Chromium) — one browser, isolated contexts per task
- Smart wait strategies:
networkidle,domcontentloaded, custom selector wait - Screenshot capture on-demand and on error
- JavaScript injection and evaluation
- Price — Regex + CSS for USD/EUR/GBP/JPY, Amazon ASIN detection, eBay listings, sale vs. original price, stock status
- Article — Readability-style extraction: title, author, date, full content, word count, images
- Structured — JSON-LD, OpenGraph, Twitter Cards, Schema.org microdata, product schema
- Async priority queue (HIGH / MEDIUM / LOW)
- Max 3 concurrent tasks (configurable)
- Exponential backoff retry (max 3 attempts)
- Full lifecycle tracking:
pending → running → done | failed | retrying
- FastAPI with lifespan management
- Full OpenAPI docs at
/docs - Redis cache with transparent in-memory fallback
# 1. Clone and install
git clone https://github.com/kOs-tile/phantom.git
cd phantom
pip install -r requirements.txt
# 2. Install Chromium
playwright install chromium
# 3. Configure environment
cp .env.example .env
# Edit .env as needed
# 4. Start the API
uvicorn phantom.api.main:app --host 0.0.0.0 --port 8001
# 5. Run the demo (no browser needed)
python scripts/demo.pyNavigate to a URL and return full page content.
curl -X POST http://localhost:8001/browse \
-H "Content-Type: application/json" \
-d '{"url": "https://news.ycombinator.com", "capture_screenshot": true}'Extract specific data fields using CSS selectors.
curl -X POST http://localhost:8001/scrape \
-H "Content-Type: application/json" \
-d '{
"url": "https://books.toscrape.com",
"selectors": {
"title": "h1",
"books": ".product_pod h3 a"
},
"multiple": {"books": true}
}'Extract product pricing from any e-commerce URL.
curl -X POST http://localhost:8001/extract/price \
-H "Content-Type: application/json" \
-d '{"url": "https://www.amazon.com/dp/B09G9FPHY6"}'Extract article content from news and blog pages.
curl -X POST http://localhost:8001/extract/article \
-H "Content-Type: application/json" \
-d '{"url": "https://techcrunch.com/2024/06/15/example/"}'Capture a full-page screenshot.
curl -X POST http://localhost:8001/screenshot \
-H "Content-Type: application/json" \
-d '{"url": "https://stripe.com/pricing", "full_page": true}'List all task queue jobs.
curl http://localhost:8001/jobs?status_filter=running&limit=10Get a specific job status and result.
curl http://localhost:8001/jobs/3f8b2c1a-...Service health check.
curl http://localhost:8001/healthPHANTOM ships with a complete Hermes skill definition at hermes_skill/PHANTOM.md.
It exposes 8 tools to any Hermes agent:
| Tool | Description |
|---|---|
browse_url |
Navigate + extract page content |
scrape_page |
CSS selector-based extraction |
extract_price |
E-commerce price extraction |
extract_article |
News/blog article extraction |
take_screenshot |
Full-page PNG capture |
schedule_monitor |
Recurring scrape jobs (cron) |
list_jobs |
View task queue |
get_job_result |
Get job result by ID |
# In a Hermes agent
price = phantom.extract_price(url="https://www.amazon.com/dp/B09XS7JWHH")
if price["discount_percent"] > 25:
oracle.alert(f"Price drop: {price['product_title']} now ${price['current_price']}")PHANTOM is the stealth browser backbone for DealHarvest deal discovery pipelines:
# DealHarvest → PHANTOM integration
deals = phantom.scrape_page(
url="https://slickdeals.net/deals/",
selectors={
"titles": ".bp-c-card-deal-title",
"prices": ".bp-c-card-deal-price",
"stores": ".bp-c-card-store-name"
},
multiple={"titles": True, "prices": True, "stores": True}
)| Variable | Default | Description |
|---|---|---|
PHANTOM_API_URL |
http://localhost:8001 |
Base URL of PHANTOM service |
PHANTOM_API_KEY |
(empty) | Optional API key |
REDIS_URL |
redis://localhost:6379/1 |
Redis connection URL |
MAX_CONCURRENT_TASKS |
3 |
Max parallel browser tasks |
BROWSER_HEADLESS |
true |
Run browser in headless mode |
DEFAULT_TIMEOUT_MS |
30000 |
Navigation timeout (ms) |
SCREENSHOT_DIR |
./screenshots |
Screenshot save directory |
RATE_LIMIT_DELAY_MS |
1500 |
Min delay between requests |
USER_AGENT_ROTATION |
true |
Enable UA rotation |
PROXY_URL |
(empty) | Optional HTTP proxy |
LOG_LEVEL |
INFO |
Logging level |
# Install dev dependencies
pip install -r requirements.txt
# Run tests
pytest tests/ -v
# Run with coverage
pytest tests/ -v --cov=phantom --cov-report=html
# Run demo
python scripts/demo.pyphantom/
├── phantom/ # Core library
│ ├── config.py # Pydantic Settings
│ ├── models.py # All Pydantic v2 data models
│ ├── stealth/ # Fingerprint, humanizer, evasion
│ ├── browser/ # Playwright session, navigator, extractor
│ ├── queue/ # Async task queue + APScheduler
│ ├── extractors/ # Price, article, structured data
│ ├── api/ # FastAPI app + routes
│ └── cache/ # Redis/memory result store
├── hermes_skill/ # PHANTOM.md Hermes skill definition
├── scripts/demo.py # Rich CLI demo
├── tests/ # 70+ tests (no real browser)
├── .env.example
├── requirements.txt
└── pyproject.toml
MIT © Hermes Agent Team
PHANTOM's primary metric is contract false-pass rate. The executable v0.1 mutation corpus is defined in docs/VALIDATION.md. Current CI checkpoint: 20/20 expected outcomes, including 0 false-passes across 13 broken/incomplete payloads, 0 false-fails across 7 valid payloads, and 20/20 payload-drift expectation matches. Cross-provider browser validation remains a separate unfinished gate.