██████╗ ███╗   ██╗██╗ ██████╗ ███╗   ██╗
██╔═══██╗████╗  ██║██║██╔═══██╗████╗  ██║
██║   ██║██╔██╗ ██║██║██║   ██║██╔██╗ ██║
██║   ██║██║╚██╗██║██║██║   ██║██║╚██╗██║
╚██████╔╝██║ ╚████║██║╚██████╔╝██║ ╚████║
 ╚═════╝ ╚═╝  ╚═══╝╚═╝ ╚═════╝ ╚═╝  ╚═══╝

███████╗ ██████╗██████╗  █████╗ ██████╗ ███████╗██████╗
██╔════╝██╔════╝██╔══██╗██╔══██╗██╔══██╗██╔════╝██╔══██╗
███████╗██║     ██████╔╝███████║██████╔╝█████╗  ██████╔╝
╚════██║██║     ██╔══██╗██╔══██║██╔═══╝ ██╔══╝  ██╔══██╗
███████║╚██████╗██║  ██║██║  ██║██║     ███████╗██║  ██║
╚══════╝ ╚═════╝╚═╝  ╚═╝╚═╝  ╚═╝╚═╝     ╚══════╝╚═╝  ╚═╝

        Cybersecurity • Automation • AI • Research

🧅 Onion Scraper

Tor-Aware Directory Listing Downloader
Resume-Capable · Duplicate-Safe · Progress-Tracked

---

📖 Project Description

Onion Scraper is a powerful desktop application built with Python and Tkinter designed to recursively download directory listings from .onion hidden services over the Tor network. It automates what would otherwise require hundreds of manual clicks through a browser.

With an elegant dark-mode GUI and a robust multi-threaded backend, Onion Scraper can gracefully recover from Tor circuit drops, resume interrupted sessions without re-downloading existing files, and provide a comprehensive "Scan First" mode to inspect trees before initiating downloads.

Whether you prefer a seamless graphical interface or a sleek CLI (powered by rich), Onion Scraper has you covered.

---

✨ Key Features

• 🕵️ Tor Network Integration: Auto-detects Tor Browser (port 9150) and Tor Daemon (port 9050) automatically.
• 📂 Smart Directory Detection: Automatically recognizes standard directory listings (Apache, QData Data Browser, IIS, nginx autoindex).
• 🔄 Intelligent Resume: State tracking saves your progress. Safely resume interrupted downloads without fetching already completed files.
• 🛡️ Circuit Drop Recovery: Pauses all downloads when a Tor circuit fails and waits for recovery instead of spamming retries.
• 🔍 Playbook Mode ("Scan First"): Scan an entire directory tree via BFS without downloading anything. Select exactly what you want via checkboxes and download selectively.
• 🎨 Dual Interfaces: Features both a beautifully designed Dark-Mode Tkinter GUI and a visually rich Command-Line Interface.

---

📸 Screenshots

Main dashboard featuring Tor auto-detection, URL history, and download progress tracking.


"Scan First" playbook mode allowing you to select specific files and folders before downloading.

---

🚀 Setup & Usage

📋 Prerequisites

• Python: Version 3.9 or higher is required.
• Tor: You must have the Tor Browser running in the background (or a standalone Tor daemon).

⚙️ Installation

For Windows Users: The easiest way to install all required dependencies is using the provided batch script.

:: Double-click the file, or run it from the command line:
install.bat

For Linux/macOS Users (or manual install): Install the necessary Python packages using pip.

pip install "requests[socks]" beautifulsoup4 rich lxml PySocks

🛠️ Configuration

No complex configuration is needed! Onion Scraper automatically detects standard Tor proxy ports:

• Port 9150: Default Tor Browser port.
• Port 9050: Default Tor Daemon port.

If you use a custom port, you can specify it directly within the GUI or when prompted in the CLI.

🏃 Running the Project

Graphical User Interface (GUI): Launch the visually appealing dark-mode application.

:: On Windows, just double-click:
run.bat

:: Alternatively, run the script directly:
python onion_scraper_gui.py

Command Line Interface (CLI): Prefer the terminal? Run the rich CLI version.

python onion_scraper.py

🚑 Troubleshooting

• ❌ Tor Not Found Error: Ensure the Tor Browser is open and fully connected to the Tor network. If you are inside a Virtual Machine, Tor must be running inside the VM, not just on the host.
• ❌ Connection Timeouts / Circuit Errors: The Tor network can be slow or unstable. The app automatically pauses and waits for the circuit to recover. Just leave it running, or safely stop and resume the session later.
• ❌ Missing Libraries: If the app fails to start due to ImportError, ensure you ran install.bat (Windows) or installed the dependencies via pip.

---

🤝 Contribution Guidelines

Contributions are always welcome! To contribute:

1. Fork the repository.
2. Create a new branch (git checkout -b feature/amazing-feature).
3. Commit your changes (git commit -m 'Add amazing feature').
4. Push to the branch (git push origin feature/amazing-feature).
5. Open a Pull Request.

Please ensure your code follows standard Python PEP-8 guidelines and doesn't break the Tor proxy configurations.

---

📄 License

This project is open-source and available under the MIT License. Feel free to use, modify, and distribute as you see fit.

---

_{Built with 💻 and 🧅 by the open-source community.}