Skip to content

About

free proxy from free-proxy-list.net

Resources

Stars

2 stars

Watchers

1 watching

Forks

Latest commit

 

History

92 Commits

Folders and files

Repository files navigation

ProxyGenerator

Secure HTTP proxy fetcher with automated validation, concurrent processing, and hourly refresh.

Features

  • Security-First: HTTP validation with comprehensive input validation and SSRF protection
  • Input Validation: IP address and port range validation with private/loopback filtering
  • Concurrent Processing: Multi-threaded proxy validation for improved performance
  • Automated Refresh: GitHub Actions scheduled execution every hour
  • Production Quality: 100% test coverage (34/34 tests) with 10/10 Pylint score

Usage

poetry install
poetry run proxygenerator

Fresh proxy data: Download JSON

Data Access

# Direct download
curl -O https://kouni.github.io/ProxyGenerator/proxies.json

# Use in Python
import requests
data = requests.get('https://kouni.github.io/ProxyGenerator/proxies.json').json()
metadata = data['metadata']  # Generated time, count, source info
proxies = data['proxies']    # List of proxy objects

Security Features

  • SSRF Protection: Trusted host whitelist prevents server-side request forgery
  • Protocol Validation: HTTP testing optimized for proxy compatibility
  • Input Sanitization: IP address format validation and private network filtering
  • Dependency Scanning: Automated security vulnerability checks in CI/CD

Architecture

  • Modular Design: Separated concerns for fetching, validation, and file handling
  • Concurrent Validation: ThreadPoolExecutor for parallel proxy testing
  • Error Handling: Specific exception types with comprehensive logging
  • Caching Strategy: Local cache with fallback for network failures
  • Code Quality: 10/10 Pylint score with production-ready standards

Implementation Details

  • Source: free-proxy-list.net (whitelisted for security)
  • Validation: 5s timeout with HTTP IP verification for broad compatibility
  • Refresh: GitHub Actions hourly with security scanning
  • Format: JSON with metadata and generation timestamps
  • Performance: Concurrent validation with configurable worker threads

Development

# Install dependencies
poetry install

# Run tests with coverage (34/34 tests, 100% coverage)
poetry run pytest --cov=src tests/

# Security scan (requires SAFETY_API_KEY environment variable)
export SAFETY_API_KEY=your_api_key
poetry run safety scan

# Note: In CI/CD, the workflow automatically handles interactive prompts

# Code quality check (achieves 10/10 score)
poetry run pylint src/

# Format checking
poetry run isort --check-only src/ tests/

Quality Metrics

  • Test Coverage: 100% (34/34 tests passing)
  • Code Quality: 10.00/10 Pylint score (production standard)
  • Security: Zero known vulnerabilities with automated scanning
  • Performance: Concurrent proxy validation with configurable workers

Requirements

  • Runtime: Python 3.13+, Poetry package manager
  • Core Dependencies: beautifulsoup4, lxml
  • Development: safety, pylint, pytest, pytest-cov, isort
  • CI/CD: GitHub Actions with automated testing and security scanning

About

free proxy from free-proxy-list.net

Resources

Stars

2 stars

Watchers

1 watching

Forks

Releases

Packages

Used by

Contributors

Languages