Scanline — Web Scraper (Form → Backend → JSON)

A tiny full-stack tool: an HTML page with a form where you paste a URL, a Flask backend that fetches and parses that page, and a JSON result shown back in the browser.

How it fits together

[index.html form]  --POST url-->  [Flask /api/scrape]  --requests+BeautifulSoup-->  [target website]
       ^                                                                                    |
       |____________________________ JSON result ___________________________________________|

1. Set up the backend

cd backend
python3 -m venv venv
source venv/bin/activate      # on Windows: venv\Scripts\activate
pip install -r requirements.txt
python app.py

You should see Flask running at http://localhost:5000.

2. Open the frontend

Just open frontend/index.html directly in your browser (double-click it, or right-click → Open With → your browser). It talks to the backend at http://localhost:5000/api/scrape.

If you prefer serving it instead of opening the file directly:

cd frontend
python3 -m http.server 8000

Then visit http://localhost:8000.

3. Use it

1. Paste a full URL (e.g. https://example.com) into the field.
2. Click Extract.
3. The page shows a summary (link/image/paragraph counts, title, headings) and a Show JSON button that reveals the full raw JSON payload.

What gets extracted

• Page title & meta description
• H1/H2/H3 headings
• All paragraph text
• All links (resolved to absolute URLs)
• All images (resolved to absolute URLs, with alt text)
• HTTP status code and fetch time

Notes & limitations

• This uses requests + BeautifulSoup, so it only sees the raw HTML the server sends — it will not see content that's injected by JavaScript after page load (common on React/Vue-heavy sites). For those, you'd swap the backend's fetch step for Playwright or Selenium.
• Respect robots.txt and each site's terms of service before scraping it. Add delays between requests if you loop over many URLs, and avoid hammering any one server.
• CORS is enabled on the backend (flask-cors) so the browser is allowed to call localhost:5000 from the index.html file.
• If a target site blocks scripted requests, you may see an HTTP error (403/429) — that's the site's own protection, not a bug in this tool.