Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/dashboard.yml
Original file line number Diff line number Diff line change
Expand Up @@ -35,3 +35,4 @@ jobs:
events: "2026-10-10: integrase-bot (CI shepherd) on"
max-wait-p90: "15"
cost-max: "10"
clones-db: https://raw.githubusercontent.com/squid-protocol/squid-telemetry/main/traffic_metrics.db
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,11 +24,11 @@ squid-bot keeps the swarm inside its habitat's carrying capacity. What it actual

## 📊 Live Fleet Dashboard

![GitGalaxy promoter-bot fleet over time: delivery, cumulative bot PRs, speed, cost, spend and efficiency](https://raw.githubusercontent.com/squid-protocol/squid-bot/dashboard/dashboard.svg)
![GitGalaxy promoter-bot fleet over time: delivery, cumulative repo clones, speed, CI load, dollar spend, cost per PR and efficiency](https://raw.githubusercontent.com/squid-protocol/squid-bot/dashboard/dashboard.svg)

Live numbers from the GitHub API for [GitGalaxy](https://github.com/squid-protocol/gitgalaxy), the repository squid-bot runs on, refreshed every 6 hours. Raw data: [`metrics.json`](https://github.com/squid-protocol/squid-bot/blob/dashboard/metrics.json).

It's made by this repo's **[Fleet CI report action](actions/fleet-report)**. The dashboard lines up six panels over time, with solid lines where integrase-bot (Oct 10) and promoter-bot (Oct 11) went live: merged PRs per day by origin; promoter-bot's merged PRs, cumulative since it started; job queue wait; runner hours per merged PR, with the before and with-bot averages; runner hours spent per day; and the share of run minutes saved by early cancellation against the share of runs that failed. The weekly report also includes reruns by workflow and, when the orchestrator publishes its own data, agent factory time and slot use. It does not yet include historical host pressure or post-merge rework.
It's made by this repo's **[Fleet CI report action](actions/fleet-report)**. The dashboard lines up seven panels over time, with solid lines where integrase-bot (Oct 10) and promoter-bot (Oct 11) went live: merged PRs per day by origin; clones of the repos the fleet works in, added up since promoter-bot started (GitHub's counts, via [squid-telemetry](https://github.com/squid-protocol/squid-telemetry)); job queue wait; CI runner hours per merged PR, with the before and with-bot averages; promoter-bot's total spend in dollars since it started; its dollars per merged PR each day, with the average; and the share of run minutes saved by early cancellation against the share of runs that failed. The weekly report also includes reruns by workflow and, when the orchestrator publishes its own data, agent factory time and slot use. It does not yet include historical host pressure or post-merge rework.

---

Expand Down
12 changes: 7 additions & 5 deletions actions/fleet-report/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

A GitHub Action that shows what an agent bot changed in your repo. It publishes a live dashboard (SVG, light and dark) you can embed in a README, and writes a report to each run's summary page.

Every chart shows one mark per UTC day: merged PRs by origin, job queue wait, runner time per merged PR and per day, and the share of run minutes saved by early cancellation and of runs that failed. Name your bot's branch prefixes and its PRs get their own color and a cumulative chart; a line marks the day it started, through every chart, and the cards compare each measure's average with the bot against the 14 days before it. Without a bot, they compare the last 7 days with the 7 before. It also publishes its source data and explains which measures are exact or sampled.
Every chart shows one mark per UTC day: merged PRs by origin, job queue wait, runner time per merged PR, and the share of run minutes saved by early cancellation and of runs that failed. Name your bot's branch prefixes and its PRs get their own color, and its dollar spend and repo clones are added up from the day it started; a line marks the day it started, through every chart, and the cards compare each measure's average with the bot against the 14 days before it. Without a bot, they compare the last 7 days with the 7 before. It also publishes its source data and explains which measures are exact or sampled.

![Example: GitGalaxy's agent fleet](https://raw.githubusercontent.com/squid-protocol/squid-bot/dashboard/dashboard.svg)

Expand Down Expand Up @@ -48,7 +48,9 @@ The first run backfills 30 days of exact counts, plus the sampled detail for the
| `detail-backfill` | `4` | Older days to fill with sampled detail per run |
| `max-wait-p90` | off | Warn when the 7-day average p90 queue wait (min) exceeds this |
| `max-runner-hours-per-pr` | off | Warn when 7-day runner hours per merged PR exceed this |
| `cost-max` | fit the data | Top of the cost-per-merged-PR axis, in hours |
| `cost-max` | fit the data | Top of the runner-hours-per-merged-PR axis |
| `clones-db` | none | Path or https URL of a squid-telemetry `traffic_metrics.db`; adds the cumulative repo clones chart |
| `clones-repos` | the fleet's CI targets | Comma-separated `owner/repo` to count in the clones chart |
| `fail-on-alert` | `false` | Fail the step on a warning |

Output: `dashboard-url`, the raw URL of the published SVG.
Expand All @@ -69,10 +71,10 @@ Output: `dashboard-url`, the raw URL of the published SVG.
### The charts

- **Delivery:** merged PRs per day, split by recorded origin (bot, other agents, other, automation). Attribution depends on configured branch prefixes or PR-body markers.
- **Bot PRs, cumulative** (with a bot): the bot's merged PRs added up day by day since it started.
- **Repo clones, cumulative** (with `clones-db`): GitHub's clone counts for the repos the fleet works in, added up since the bot started, from a [squid-telemetry](https://github.com/squid-protocol/squid-telemetry) database. GitHub publishes clone traffic about two days late, so the line ends on its last reported day.
- **Speed:** each day's sampled p90 job queue wait.
- **Cost:** each day's estimated runner hours per merged PR, with dashed lines at the average before and with the bot (the cards' figures). Set `cost-max` to cap the axis; days above it are drawn at the top and marked with their value.
- **Spend:** each day's estimated runner hours in total, the bot's days in its color and labelled.
- **CI load:** each day's estimated runner hours per merged PR, with dashed lines at the average before and with the bot (the cards' figures). Set `cost-max` to cap the axis; days above it are drawn at the top and marked with their value.
- **Spend** and **Cost per PR** (when the orchestrator publishes `factory.json` with `usd`): the bot's dollar cost added up since it started, and each day's dollars per merged bot PR with the overall average.
- **Efficiency:** two percentages per day. **Runs failed:** failed ÷ (passed + failed) workflow runs, exact. **Run minutes saved (est.):** a run cancelled early (usually because a newer push replaced it) would have taken about as long as that day's median successful run of the same workflow; what it saved is that minus what it ran. The chart shows saved ÷ (spent + saved), in run wall-clock minutes. Runs of the configured `bot-workflows` are left out of both sides, and cancelled runs of a workflow with no successful run that day aren't priced. It is an estimate, labelled as one.

Today is drawn faded, as incomplete. The "with bot" averages and cards use finished days only.
Expand Down
18 changes: 18 additions & 0 deletions actions/fleet-report/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,14 @@ inputs:
cost-max:
description: Top of the cost-per-merged-PR chart axis, in runner hours; days above it are drawn at the top and marked. Empty = fit the data.
default: ""
clones-db:
description: >-
Path or https URL of a squid-telemetry traffic_metrics.db. Adds a panel of the fleet's repo clones (GitHub's counts),
added up since the bot started. Empty = no clones panel.
default: ""
clones-repos:
description: Comma-separated owner/repo to count in the clones panel. Empty = every repo the fleet's CI fetches.
default: ""
fail-on-alert:
description: Fail the step when an alert fires.
default: "false"
Expand Down Expand Up @@ -111,6 +119,8 @@ runs:
I_MAX_H: ${{ inputs.max-runner-hours-per-pr }}
I_FAIL: ${{ inputs.fail-on-alert }}
I_COST_MAX: ${{ inputs.cost-max }}
I_CLONES_DB: ${{ inputs.clones-db }}
I_CLONES_REPOS: ${{ inputs.clones-repos }}
run: |
args=(--repo "$I_REPO" --days "$I_DAYS" --out "$OUT" --title "$I_TITLE" --bot-workflows "$I_BOTS"
--agent-branch-prefixes "$I_PREFIXES" --sample-runs "$I_SAMPLE" --detail-backfill "$I_BACKFILL"
Expand All @@ -119,6 +129,14 @@ runs:
--cost-max "$I_COST_MAX")
[ -n "$I_MARKERS" ] && args+=(--agent-markers "$I_MARKERS")
[ "$I_FAIL" = "true" ] && args+=(--fail-on-alert)
if [ -n "$I_CLONES_DB" ]; then
db="$I_CLONES_DB"
if [[ "$db" == https://* ]]; then
db="$RUNNER_TEMP/clones.db"
curl -sfL --retry 3 -o "$db" "$I_CLONES_DB" || { echo "::warning title=Fleet CI report::could not download clones-db"; db=""; }
fi
[ -n "$db" ] && args+=(--clones-db "$db" --clones-repos "$I_CLONES_REPOS")
fi
python3 "$GITHUB_ACTION_PATH/fleet_report.py" "${args[@]}"

- name: Publish
Expand Down
Loading
Loading