Architecture: Make online HTTP transport modular

[mveril]

Context

rustiq-core currently uses Reqwest for its optional online basis-set functionality. Because Reqwest is Tokio-based, enabling that functionality currently introduces a Tokio dependency into the core.

PR #58 explored replacing Reqwest with Isahc/libcurl in order to remove that runtime dependency. The approach works, but it also introduces native libcurl/TLS integration and additional Nix/CI/platform build complexity.

For the current RustiQ architecture this tradeoff is not justified yet:

• the RustiQ CLI already uses Tokio;
• rustiq-python is likely to integrate naturally with Tokio when async support is introduced;
• Reqwest is already working and provides a convenient pure-Rust HTTP stack when used with Rustls;
• replacing Reqwest directly with Isahc would exchange a runtime coupling for a native libcurl coupling without making the transport itself modular.

Therefore Reqwest remains the current/default HTTP implementation, and runtime independence is deferred until the online layer is redesigned around a transport boundary.

Desired outcome

Make the online basis-set functionality independent of one concrete HTTP client at the architectural boundary.

The intended direction is conceptually:

                     rustiq-core
                         │
                 online basis service
                         │
                 HTTP transport boundary
                    /             \
                   /               \
          Reqwest transport     other transport
              (Tokio)          (e.g. libcurl)

The scientific/domain API should not need to change when the HTTP implementation changes.

This issue should be implemented when there is a concrete reason to support runtime-independent consumers, multiple transports, or a consumer for which Tokio is an undesirable dependency. It does not need to block the current rustiq-core split.

Design direction

Prefer a small transport abstraction over replacing one global HTTP client with another.

The abstraction should be located as close as practical to the online/download layer and should expose only the operations RustiQ actually needs. It should not attempt to reproduce the full Reqwest API.

Possible implementations may include:

• Reqwest as the default transport;
• a libcurl-based transport through Isahc or the curl crate;
• a caller-provided/custom transport where useful for embedding or tests.

The exact abstraction (trait, generic provider, feature-gated implementation, etc.) should be decided when this issue is implemented based on the consumers that actually exist at that time.

Do not introduce abstraction solely for theoretical flexibility: the resulting API should remain small and idiomatic.

Runtime independence

Runtime independence is a goal of this future transport work, not a current invariant of rustiq-core.

Until this issue is implemented, it is acceptable for the optional Reqwest-based online functionality to depend on Tokio.

When runtime-independent online usage becomes necessary, the design should allow a transport that does not require Tokio without forcing all users to depend on libcurl or another native HTTP stack.

Acceptance criteria

• Online basis-set/network functionality is separated from the concrete HTTP client by a narrow transport boundary.
• Scientific/domain APIs do not expose Reqwest-, libcurl-, Isahc-, or Tokio-specific types.
• Reqwest can remain available as a supported/default implementation.
• The architecture can support a non-Tokio HTTP implementation without changing the scientific API.
• A non-Reqwest implementation can be added without forcing its native/runtime dependencies on users that do not select it.
• Download progress, HTTP status handling, redirects, interrupted transfers, atomic destination replacement, and cancellation semantics remain covered as appropriate for supported transports.
• Nix, CI, and packaging account only for the dependencies required by the selected transport(s).
• The final design documents the runtime and native-library requirements of each supported transport.

Out of scope

• replacing Reqwest immediately only to satisfy a dependency-graph invariant;
• forcing libcurl on every RustiQ consumer;
• changing scientific algorithms or basis-set formats;
• changing the public calculation API introduced by Define the reusable calculation API (Part 3 of #54) #57.

Related work

• Engineering: Split the reusable core from the RustiQ CLI #54 — split and stabilize rustiq-core
• Part 4: Remove async runtime dependencies from rustiq-core #58 — explored the Isahc/libcurl approach; closed in favor of keeping Reqwest for now and revisiting the problem through a modular transport design.