Repository navigation
build: replace the makefile with mise tasks - #633
Merged
Merged
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
🟢 No regressionCompared with the newest nightly run of internal/graph ·
|
| benchmark | unit | baseline | this run | vs base | |
|---|---|---|---|---|---|
| Search/depth0 | sec/op | 16.2 ms | 12.26 ms | -24.29% | p=0.000 |
| Search/depth1 | sec/op | 16.53 ms | 12.58 ms | -23.89% | p=0.000 |
| Search/depth2 | sec/op | 16.36 ms | 12.6 ms | -23.00% | p=0.000 |
| Search/vector | sec/op | 14.24 ms | 10.12 ms | -28.98% | p=0.000 |
| Search/conversation | sec/op | 17.01 ms | 14.65 ms | -13.91% | p=0.000 |
| Search/depth0 | B/op | 6.22 MiB | 6.22 MiB | +0.00% | p=0.027 |
| Search/depth1 | B/op | 6.399 MiB | 6.399 MiB | ~ | p=0.487 |
| Search/depth2 | B/op | 6.399 MiB | 6.399 MiB | +0.00% | p=0.017 |
| Search/vector | B/op | 4.228 MiB | 4.228 MiB | ~ | p=0.706 |
| Search/conversation | B/op | 6.868 MiB | 6.868 MiB | ~ | p=0.140 |
| Search/depth0 | allocs/op | 281 | 281 | ~ | p=1.000 |
| Search/depth1 | allocs/op | 450 | 450 | ~ | p=1.000 |
| Search/depth2 | allocs/op | 450 | 450 | ~ | p=1.000 |
| Search/vector | allocs/op | 392 | 392 | ~ | p=1.000 |
| Search/conversation | allocs/op | 645 | 645 | ~ | p=1.000 |
internal/hash · all benchmarks
| benchmark | unit | baseline | this run | vs base | |
|---|---|---|---|---|---|
| T1haHash | sec/op | 10.44 ns | 10.57 ns | +1.20% | p=0.000 |
| XxHash | sec/op | 16.15 ns | 16.04 ns | -0.68% | p=0.000 |
| T1haHash | B/s | 4,118,090,000 | 4,070,180,000 | -1.16% | p=0.000 |
| XxHash | B/s | 2,663,070,000 | 2,680,365,000 | +0.65% | p=0.000 |
| T1haHash | B/op | 0 | 0 | ~ | p=1.000 |
| XxHash | B/op | 0 | 0 | ~ | p=1.000 |
| T1haHash | allocs/op | 0 | 0 | ~ | p=1.000 |
| XxHash | allocs/op | 0 | 0 | ~ | p=1.000 |
internal/index · all benchmarks
| benchmark | unit | baseline | this run | vs base | |
|---|---|---|---|---|---|
| BTreeIndexSearch | sec/op | 957.6 µs | 900.9 µs | -5.92% | p=0.000 |
| BTreeIndexSearchWindow | sec/op | 3.183 ms | 2.911 ms | -8.55% | p=0.000 |
| RPTreeIndexInsert | sec/op | 72.11 µs | 63.34 µs | -12.17% | p=0.000 |
| RPTreeIndexSearch | sec/op | 377.3 µs | 357.2 µs | -5.32% | p=0.000 |
| BTreeIndexSearch | B/op | 1.353 MiB | 1.353 MiB | ~ | p=0.596 |
| BTreeIndexSearchWindow | B/op | 2.284 MiB | 2.284 MiB | ~ | p=0.675 |
| RPTreeIndexInsert | B/op | 3.677 KiB | 3.652 KiB | -0.68% | p=0.000 |
| RPTreeIndexSearch | B/op | 179.1 KiB | 179.1 KiB | ~ | p=0.839 |
| BTreeIndexSearch | allocs/op | 24 | 24 | ~ | p=1.000 |
| BTreeIndexSearchWindow | allocs/op | 41 | 41 | ~ | p=1.000 |
| RPTreeIndexInsert | allocs/op | 48 | 48 | ~ | p=1.000 |
| RPTreeIndexSearch | allocs/op | 2,115 | 2,115 | ~ | p=1.000 |
internal/query · all benchmarks
| benchmark | unit | baseline | this run | vs base | |
|---|---|---|---|---|---|
| RememberCommit/size-100 | sec/op | 8.53 µs | 7.73 µs | -9.38% | p=0.000 |
| RememberCommit/size-10000 | sec/op | 8.655 µs | 7.792 µs | -9.98% | p=0.000 |
| RecallCommit/size-1000 | sec/op | 1.082 ms | 1.035 ms | -4.37% | p=0.000 |
| RecallCommit/size-10000 | sec/op | 14.65 ms | 12.2 ms | -16.72% | p=0.000 |
| RememberCommit/size-100 | B/op | 2.486 KiB | 2.477 KiB | -0.39% | p=0.002 |
| RememberCommit/size-10000 | B/op | 2.504 KiB | 2.465 KiB | -1.58% | p=0.000 |
| RecallCommit/size-1000 | B/op | 381.5 KiB | 381.5 KiB | -0.00% | p=0.000 |
| RecallCommit/size-10000 | B/op | 3.53 MiB | 3.526 MiB | -0.12% | p=0.000 |
| RememberCommit/size-100 | allocs/op | 38 | 38 | ~ | p=1.000 |
| RememberCommit/size-10000 | allocs/op | 38 | 38 | ~ | p=1.000 |
| RecallCommit/size-1000 | allocs/op | 103 | 103 | ~ | p=0.474 |
| RecallCommit/size-10000 | allocs/op | 449 | 404 | -10.02% | p=0.000 |
pkg/server · BenchmarkRetrievalQuality
| benchmark | unit | baseline | this run | vs base | |
|---|---|---|---|---|---|
| RetrievalQuality/text/all | sec/op | 193.4 ms | 173.6 ms | ~ | p=1.000 |
| RetrievalQuality/text/multi-hop | sec/op | 31.74 ms | 27.33 ms | ~ | p=1.000 |
| RetrievalQuality/text/temporal | sec/op | 36.44 ms | 32.64 ms | ~ | p=1.000 |
| RetrievalQuality/text/open-domain | sec/op | 11.76 ms | 9.715 ms | ~ | p=1.000 |
| RetrievalQuality/text/single-hop | sec/op | 66.5 ms | 60.56 ms | ~ | p=1.000 |
| RetrievalQuality/text/adversarial | sec/op | 46.72 ms | 41.59 ms | ~ | p=1.000 |
| RetrievalQuality/vectors/all | sec/op | 265.1 ms | 234.9 ms | ~ | p=1.000 |
| RetrievalQuality/vectors/multi-hop | sec/op | 44.31 ms | 38.37 ms | ~ | p=1.000 |
| RetrievalQuality/vectors/temporal | sec/op | 50.93 ms | 43.92 ms | ~ | p=1.000 |
| RetrievalQuality/vectors/open-domain | sec/op | 15.4 ms | 13.38 ms | ~ | p=1.000 |
| RetrievalQuality/vectors/single-hop | sec/op | 94.08 ms | 83.64 ms | ~ | p=1.000 |
| RetrievalQuality/vectors/adversarial | sec/op | 62.59 ms | 55.93 ms | ~ | p=1.000 |
| RetrievalQuality/text/all | turn-f1@10 | 0.1512 | 0.1512 | 0.00% | exact |
| RetrievalQuality/text/multi-hop | turn-f1@10 | 0.1059 | 0.1059 | 0.00% | exact |
| RetrievalQuality/text/temporal | turn-f1@10 | 0.1671 | 0.1671 | 0.00% | exact |
| RetrievalQuality/text/open-domain | turn-f1@10 | 0.09091 | 0.09091 | 0.00% | exact |
| RetrievalQuality/text/single-hop | turn-f1@10 | 0.158 | 0.158 | 0.00% | exact |
| RetrievalQuality/text/adversarial | turn-f1@10 | 0.1734 | 0.1734 | 0.00% | exact |
| RetrievalQuality/vectors/all | turn-f1@10 | 0.1666 | 0.1666 | 0.00% | exact |
| RetrievalQuality/vectors/multi-hop | turn-f1@10 | 0.1506 | 0.1506 | 0.00% | exact |
| RetrievalQuality/vectors/temporal | turn-f1@10 | 0.1818 | 0.1818 | 0.00% | exact |
| RetrievalQuality/vectors/open-domain | turn-f1@10 | 0.1543 | 0.1543 | 0.00% | exact |
| RetrievalQuality/vectors/single-hop | turn-f1@10 | 0.1606 | 0.1606 | 0.00% | exact |
| RetrievalQuality/vectors/adversarial | turn-f1@10 | 0.1773 | 0.1773 | 0.00% | exact |
| RetrievalQuality/text/all | turn-p@1 | 0.3299 | 0.3299 | 0.00% | exact |
| RetrievalQuality/text/multi-hop | turn-p@1 | 0.125 | 0.125 | 0.00% | exact |
| RetrievalQuality/text/temporal | turn-p@1 | 0.5946 | 0.5946 | 0.00% | exact |
| RetrievalQuality/text/open-domain | turn-p@1 | 0.09091 | 0.09091 | 0.00% | exact |
| RetrievalQuality/text/single-hop | turn-p@1 | 0.3286 | 0.3286 | 0.00% | exact |
| RetrievalQuality/text/adversarial | turn-p@1 | 0.3191 | 0.3191 | 0.00% | exact |
| RetrievalQuality/vectors/all | turn-p@1 | 0.3858 | 0.3858 | 0.00% | exact |
| RetrievalQuality/vectors/multi-hop | turn-p@1 | 0.25 | 0.25 | 0.00% | exact |
| RetrievalQuality/vectors/temporal | turn-p@1 | 0.6486 | 0.6486 | 0.00% | exact |
| RetrievalQuality/vectors/open-domain | turn-p@1 | 0.09091 | 0.09091 | 0.00% | exact |
| RetrievalQuality/vectors/single-hop | turn-p@1 | 0.3857 | 0.3857 | 0.00% | exact |
| RetrievalQuality/vectors/adversarial | turn-p@1 | 0.3404 | 0.3404 | 0.00% | exact |
| RetrievalQuality/text/all | turn-recall@10 | 0.7652 | 0.7652 | 0.00% | exact |
| RetrievalQuality/text/multi-hop | turn-recall@10 | 0.3047 | 0.3047 | 0.00% | exact |
| RetrievalQuality/text/temporal | turn-recall@10 | 0.9189 | 0.9189 | 0.00% | exact |
| RetrievalQuality/text/open-domain | turn-recall@10 | 0.2727 | 0.2727 | 0.00% | exact |
| RetrievalQuality/text/single-hop | turn-recall@10 | 0.8571 | 0.8571 | 0.00% | exact |
| RetrievalQuality/text/adversarial | turn-recall@10 | 0.9362 | 0.9362 | 0.00% | exact |
| RetrievalQuality/vectors/all | turn-recall@10 | 0.8236 | 0.8236 | 0.00% | exact |
| RetrievalQuality/vectors/multi-hop | turn-recall@10 | 0.4141 | 0.4141 | 0.00% | exact |
| RetrievalQuality/vectors/temporal | turn-recall@10 | 1 | 1 | 0.00% | exact |
| RetrievalQuality/vectors/open-domain | turn-recall@10 | 0.5455 | 0.5455 | 0.00% | exact |
| RetrievalQuality/vectors/single-hop | turn-recall@10 | 0.8714 | 0.8714 | 0.00% | exact |
| RetrievalQuality/vectors/adversarial | turn-recall@10 | 0.9574 | 0.9574 | 0.00% | exact |
| RetrievalQuality/text/all | B/op | 65.01 MiB | 65.27 MiB | ~ | p=1.000 |
| RetrievalQuality/text/multi-hop | B/op | 10.95 MiB | 10.82 MiB | ~ | p=1.000 |
| RetrievalQuality/text/temporal | B/op | 12.4 MiB | 12.57 MiB | ~ | p=1.000 |
| RetrievalQuality/text/open-domain | B/op | 3.823 MiB | 3.724 MiB | ~ | p=1.000 |
| RetrievalQuality/text/single-hop | B/op | 22.59 MiB | 22.69 MiB | ~ | p=1.000 |
| RetrievalQuality/text/adversarial | B/op | 15.32 MiB | 15.48 MiB | ~ | p=1.000 |
| RetrievalQuality/vectors/all | B/op | 91.77 MiB | 91.62 MiB | ~ | p=1.000 |
| RetrievalQuality/vectors/multi-hop | B/op | 15.15 MiB | 15.13 MiB | ~ | p=1.000 |
| RetrievalQuality/vectors/temporal | B/op | 17.29 MiB | 17.47 MiB | ~ | p=1.000 |
| RetrievalQuality/vectors/open-domain | B/op | 5.427 MiB | 5.31 MiB | ~ | p=1.000 |
| RetrievalQuality/vectors/single-hop | B/op | 31.88 MiB | 31.87 MiB | ~ | p=1.000 |
| RetrievalQuality/vectors/adversarial | B/op | 21.8 MiB | 21.72 MiB | ~ | p=1.000 |
| RetrievalQuality/text/all | allocs/op | 108,540 | 108,379 | ~ | p=1.000 |
| RetrievalQuality/text/multi-hop | allocs/op | 16,889 | 16,861 | ~ | p=1.000 |
| RetrievalQuality/text/temporal | allocs/op | 19,402 | 19,450 | ~ | p=1.000 |
| RetrievalQuality/text/open-domain | allocs/op | 6,156 | 6,108 | ~ | p=1.000 |
| RetrievalQuality/text/single-hop | allocs/op | 37,564 | 37,605 | ~ | p=1.000 |
| RetrievalQuality/text/adversarial | allocs/op | 25,126 | 25,217 | ~ | p=1.000 |
| RetrievalQuality/vectors/all | allocs/op | 205,314 | 205,002 | ~ | p=1.000 |
| RetrievalQuality/vectors/multi-hop | allocs/op | 33,161 | 33,143 | ~ | p=1.000 |
| RetrievalQuality/vectors/temporal | allocs/op | 38,227 | 38,263 | ~ | p=1.000 |
| RetrievalQuality/vectors/open-domain | allocs/op | 11,855 | 11,773 | ~ | p=1.000 |
| RetrievalQuality/vectors/single-hop | allocs/op | 70,773 | 70,665 | ~ | p=1.000 |
| RetrievalQuality/vectors/adversarial | allocs/op | 48,312 | 48,284 | ~ | p=1.000 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pull Request
Type of change
Description of the changes
The Makefile is replaced by mise:
mise.tomlpins the toolchain (Go 1.26, uv 0.12, pre-commit 4.7.0) and holds every task, namedgroup:target, somake lint-gois nowmise run lint:go. CI installs the toolchain withjdx/mise-action, and every step it runs is amise run, the three releases included (check:tag:go|py|mcp, thenrelease:go|py|mcp), as well as the MCP registry publish, the docs sync to FraiseHQ/www, govulncheck and the example image builds. The benchmarks page moves from the docs intoCONTRIBUTING.md.Motivation for the changes
One file now sets the toolchain locally and in CI, where setup-go read
go.modand setup-uv readpyproject.toml, and a contributor installs all of it withmise install. A CI step that is a task runs the same way on a laptop, with its arguments listed bymise run <task> --help. The docs at docs.getfraise.dev are for users of fraise; how the benchmark gates run and how to accept a regression is for contributors.Breaking change?
If yes, fill this in
What breaks:
Before → after:
Migration:
How it was tested
mise run lint:all: every pre-commit hook passes;mise run lintpasses the hooks no language owns and skips the rest.mise run test:pyandmise run test:mcp: both unit suites pass.mise run test:integration(18 passed) andFRAISE_E2E_PORT=9879 mise run test:e2e: both pass, and the e2e container is torn down after.mise run bench:historyon sample benchmark outputs: byte-identical to whatmake bench-historywrote.mise run build:go: the same version stamp asmake build-go.mise run check:tag:go|py|mcp <tag>, and with the tag inGITHUB_REF_NAME: a server tag onmainand package tags naming the package's version pass; a tag offmainand a mismatched version fail.GORELEASER_CURRENT_TAG=v0.3.7 mise run release:go --snapshot --skip=publish,sign,announce: builds every archive, package and the Homebrew formula.mise run build:pyandmise run build:mcp: each lands in its package's owndist/.mise run check:vuln: no reachable vulnerabilities.mise run build:examples openai-agents: the image builds.mise run docs:syncagainst a local bare repository standing in for FraiseHQ/www: removed pages are deleted, files outsidecontents/docskept, the commit is authored byfraise-bot[bot]with its id address, and a second run commits nothing.release:registrywas not run whole, since it publishes. Its read-only steps were: theserver.jsonread at v0.3.6 is rejected for v0.3.7, and at v0.3.7 the image is pinned toghcr.io/fraisehq/fraise:0.3.7andmcp-publisher validateaccepts the record.main; every other job runs on this pull request.Performance regression
Checklist
Notes for reviewers
.pre-commit-config.yamlsetslanguage_version: systemongolangci-lint-full. pre-commit does not count a go under the home directory, where mise installs it, as the system's, and builds the hook with the latest Go release instead, which golangci-lint v2.12.2 cannot read.mise run lintruns the hooks no language owns, the pull request workflow's slice;lint:go,lint:pyandlint:dockerrun in their own workflows, andlint:allruns every hook, the check before a pull request.check:tag:*andrelease:go:provenancedefault the tag toGITHUB_REF_NAME,release:registry <tag> --ref <sha>,docs:sync <www> --bot <slug>,build:examples [example]....mise.lock: goreleaser and cosign (release:go), gh, jq and mcp-publisher (release:registry), govulncheck (check:vuln). They replacegoreleaser/goreleaser-action,sigstore/cosign-installer,golang/govulncheck-actionandstefanzweifel/git-auto-commit-action. Only mise 2026.11 or later writes task-scoped tools to the lockfile.workflow_runismain, so the publish credential only runsmain's tasks; the release'sserver.jsonis still read as data. mcp-publisher stays pinned by checksum, now inmise.lock.check:tag:*: release-please writes the tags.check:tag:gochecks the tag is onmain,check:tag:pyandcheck:tag:mcpthat it names the package's version.mise run bench --bench-baseline=<dir>. The gate workflow passes--bench-always, which now namesmise.toml, since it sets the Go version the benchmarks build with.build:pyandbuild:mcpwrite to the package's owndist/. A bareuv buildin this workspace writes to the root's, whereuv publishrun from the package finds nothing.install-tools,help,clean-alland the Go-onlyreleaseprep have no task:mise install,mise tasksandcleancover them.