Make external service clients optional

[Gerrrr]

Why

pip install apache-otava currently installs every database and notification client, even when the caller only needs the analysis library, bundled CLI, built-in CSV and JSON sources, or the bundled NoSQLBench HistoStat and Graphite importers. This PR makes the default installation useful on its own while letting users opt into only the external services they need.

Measured impact

Fresh measurements from the wheel built from this branch:

install / extra	covers	client dependency	installed size	cost over core
default	analysis, CLI, CSV, JSON, NoSQLBench HistoStat, Graphite	—	148.1 MB / 19 pkgs	core
influxdb	importer	influxdb3-python	283.0 MB / 24 pkgs	+134.9 MB / +5 pkgs
bigquery	importer + --update-bigquery	google-cloud-bigquery	221.2 MB / 41 pkgs	+73.1 MB / +22 pkgs
slack	--notify-slack	slack-sdk	152.1 MB / 20 pkgs	+4.0 MB / +1 pkg
grafana	--update-grafana, remove-annotations	requests	151.7 MB / 24 pkgs	+3.5 MB / +5 pkgs
postgres	importer + --update-postgres	pg8000	149.7 MB / 22 pkgs	+1.5 MB / +3 pkgs
all	every integration	all clients	360.3 MB / 48 pkgs	+212.2 MB / +29 pkgs

The default installation avoids 212.2 MB (58.9%) and 29 packages (60.4%) compared with the full integration set.

NoSQLBench HistoStat and Graphite support remain bundled because their implementations add no external Python dependencies, so making them extras would not reduce installation size. We may still extract them into separate packages or plugins as follow-up work.

Measured on Python 3.14.7 on macOS arm64 using wheel-only installs with bytecode compilation and a fresh environment for each row. Installed size is the sum of file sizes under site-packages. Absolute results vary by platform, but these measurements closely match those reported in #175.

Installation examples

# Analysis library, CLI, and built-in sources
pip install apache-otava

# One external service
pip install 'apache-otava[bigquery]'

# Combine only the services needed
pip install 'apache-otava[bigquery,postgres]'

# Existing all-integrations experience
pip install 'apache-otava[all]'

The production Docker image continues to include every integration by installing [all].

Contributor-only tools now live in the local dev dependency group rather than a
published [dev] extra. A full contributor environment is installed with:

uv sync --all-extras --dev

What changed

• Move BigQuery, PostgreSQL, InfluxDB, Grafana, and Slack clients into named extras, plus an all extra.
• Move test, lint, and build tools out of the published [dev] extra and into the standardized local dev dependency group.
• Declare pytz, which Otava imports directly at runtime, as a core dependency instead of relying on dateparser to install it transitively.
• Import optional clients only when their features are used.
• Report actionable errors that name the missing extra and installation command.
• Keep Slack optional even for code paths that do not send notifications.
• Add a clean-wheel core installation smoke test to CI.
• Install [all] in the production Docker image.
• Update the README, installation guide, getting-started guide, and integration-specific documentation.

Note

Exercising the documented workflows during testing also uncovered pre-existing
documentation issues, including stale paths, dates and expected output. Those
corrections and their executable CSV regression coverage are included in a
separate commit.

Verification

• 246 passed across the complete test and performance suite, including Docker-backed Graphite, InfluxDB, and PostgreSQL end-to-end tests.
• Lint and license checks pass through tox.
• Clean default wheel: 19 packages and the core smoke test passes.
• [all] wheel: 48 packages and all service client imports pass.
• Production Docker image builds and otava --help runs successfully.
• Documentation examples were exercised for CSV, Graphite, Grafana, PostgreSQL, and InfluxDB; their commands and deterministic output now match the bundled fixtures.

Closes #55
Closes #175

[Copilot]

Pull request overview

Makes external service clients optional while keeping core analysis, CLI, and built-in sources available by default.

Changes:

• Adds per-integration extras and lazy client imports.
• Adds missing-dependency handling and core-install tests.
• Updates Docker, CI, and installation documentation.

Reviewed changes

Copilot reviewed 20 out of 21 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
pyproject.toml	Defines integration extras and local dev dependencies.
uv.lock	Locks the revised dependency structure.
otava/_optional.py	Adds optional-import error handling.
otava/bigquery.py	Lazily imports BigQuery clients.
otava/grafana.py	Lazily imports Requests.
otava/importer.py	Lazily creates BigQuery parameters.
otava/influxdb.py	Lazily imports the InfluxDB client.
otava/main.py	Lazily initializes Slack and handles missing extras.
otava/postgres.py	Lazily imports pg8000.
otava/slack.py	Lazily imports the Slack SDK.
tests/core_install_smoke.py	Tests a clean core installation.
tests/optional_dependencies_test.py	Tests optional-import behavior.
.github/workflows/python-app.yml	Adds the core-wheel CI job.
Dockerfile	Installs all integrations in production images.
README.md	Documents installation extras.
docs/INSTALL.md	Lists available extras.
docs/GETTING_STARTED.md	Updates setup instructions.
docs/BIG_QUERY.md	Documents the BigQuery extra.
docs/GRAFANA.md	Documents the Grafana extra.
docs/INFLUXDB.md	Documents the InfluxDB extra.
docs/POSTGRESQL.md	Documents the PostgreSQL extra.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Gerrrr]

@henrikingo

[MrlixiangWE]

@Gerrrr 245 tests pass on each of Python 3.10, 3.11, 3.12, 3.13 and 3.14, including the Docker-backed Graphite, PostgreSQL and InfluxDB e2e tests, and core_install_smoke.py passes against a freshly built wheel on all five.

pytz is imported at runtime in util.py, data_selector.py, grafana.py, slack.py and main.py, but a default install only gets it through dateparser. Declare it in [project.dependencies]?

[Gerrrr]

Thanks for the review @MrlixiangWE! This is a valid suggestion; fixed.

[Gerrrr]

@henrikingo I am going to merge this PR and let you comment async. As usual, I'll be happy to fix-forward any feedback.

[henrikingo]

I'm keeping an eye on things, can't comment yet. Glad to see project so active! Hopefully I can help do a release soon and hopefully the deterministic variation is in it.