Skip to content

Add dynamic service registration + GitHub Actions reindex example - #99

Merged
GoodbyePlanet merged 1 commit into
mainfrom
feat/dynamic-service-registration
Jul 27, 2026
Merged

Add dynamic service registration + GitHub Actions reindex example#99
GoodbyePlanet merged 1 commit into
mainfrom
feat/dynamic-service-registration

Conversation

@GoodbyePlanet

Copy link
Copy Markdown
Owner

Summary

Closes #83.

config.yaml is a static, curated list — fine for a handful of repos, but indexing hundreds this way means hundreds of hand-maintained entries. This adds a second, self-service way to tell semcode about a repo, and a ready-to-use GitHub Actions workflow built on it.

  • Dynamic service registration: POST /reindex accepts github_repo/github_ref/root/exclude inline (alongside the existing service/force). Providing github_repo registers service with that repo definition — persisted in a new Qdrant collection (service_registry, see server/store/service_registry.py) rather than a file, so it's safe to write from an unauthenticated request and survives restarts.
    • Services resolve through load_effective_services() (config.yaml ∪ registry) everywhere a service name is looked up — IndexPipeline, GitHistoryPipeline, get_code_context, index_stats. config.yaml always wins on a name collision, so a stray/malicious registration can never silently repoint a curated service.
    • index_stats now lists "From config.yaml" and "Registered dynamically via API" separately.
    • No new auth was added — /reindex is unauthenticated the same way it already was for config.yaml services.
  • examples/github-actions/reindex-on-merge.yml: drop-in workflow for the indexed repo's own CI. On push to main/master it self-registers (default service name = owner-repo, collision-safe across orgs) and reindexes in the same call — no central file to edit, no server restart.
  • config.yaml is now genuinely optional:
    • Settings.load_services() treats a missing or empty config.yaml as zero services instead of raising FileNotFoundError.
    • If CONFIG_PATH resolves to a directory (the classic Docker bind-mount-of-a-missing-file footgun), it now raises a clear, actionable error instead of a cryptic IsADirectoryError.
    • Dockerfile no longer bakes config.yaml in at build time.
    • docker-compose.yaml no longer mounts config.yaml by default; docker-compose.config-yaml.yml is a small overlay that adds it back, wired up via new -with-config Makefile targets, for anyone who wants curated services alongside or instead of dynamic registration.
  • Bug fix found while wiring this up: get_code_context only ever resolved services from config.yaml, so it would report a dynamically-registered service as "no longer in config.yaml" even while it was actively indexed. Fixed to use the same merged resolution as everything else.
  • Docs updated: docs/configuration.md gets a full "Dynamic Service Registration" section; docs/README.md, docs/ingestion.md, docs/retrieval-rrf.md, and the root README.md updated to describe both paths and how they interact.

Test plan

  • uv run pytest -q — 289 passed
  • uvx ruff check . / uvx ruff format --check . — clean
  • Verified docker compose -f docker-compose.yaml -f docker-compose.config-yaml.yml config merges the config.yaml volume mount correctly on top of the base file
  • Manual smoke test against a live stack: POST /reindex with github_repo for a repo not in config.yaml, confirm it indexes and shows up in list_indexed_services/index_stats, and survives a container restart

🤖 Generated with Claude Code

…xample

config.yaml doesn't scale past a handful of repos — indexing hundreds means
hundreds of hand-maintained entries. POST /reindex now accepts a repo
definition inline (github_repo/github_ref/root/exclude alongside service),
registering it in a new Qdrant-backed ServiceRegistry instead of requiring a
config.yaml entry. Registered services persist across restarts, resolve
through the same load_effective_services() used everywhere config.yaml
services are looked up (config.yaml wins on name collisions), and show up in
index_stats/list_indexed_services like any other service.

Includes examples/github-actions/reindex-on-merge.yml, a drop-in workflow
that self-registers a repo and reindexes it on every merge to main/master —
no central config edit needed.

Also, since config.yaml is now genuinely optional:
- load_services() treats a missing/empty config.yaml as zero services
  instead of crashing, and raises a clear error (instead of a cryptic
  IsADirectoryError) if CONFIG_PATH resolves to a directory — the failure
  mode of a Docker bind mount pointing at a config.yaml that doesn't exist.
- docker-compose.yaml no longer mounts config.yaml by default; a new
  docker-compose.config-yaml.yml overlay (via `-with-config` Makefile
  targets) adds it back for curated/static setups.
- Dockerfile no longer requires config.yaml to exist at build time.

Fixed get_code_context, which only ever resolved services from config.yaml
and would 404 dynamically-registered ones despite them being indexed.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@GoodbyePlanet
GoodbyePlanet merged commit 0a591ce into main Jul 27, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add GA example that will run every time merge to main/master happen and scan that repo to run indexing if neccessary

1 participant