Test phase 8: graph checked on monorepo layouts; known gaps listed

.github/workflows/graph-oracle.yml

@@ -1,10 +1,13 @@
 name: graph oracle
 
 # Does the dependency graph draw the edges the languages' own resolvers
-# draw? examples/graph_oracle.rs (the comparison `fcs check` runs) compares fcs's import graph with grimp on
-# Django and with TypeScript's resolveModuleName on Vite, and fails when a
-# change misses or adds more edges than the recorded baseline. Lower the
-# baselines when a fix closes a gap.
+# draw? examples/graph_oracle.rs (the comparison `fcs check` runs) compares
+# fcs's import graph with grimp on Django, generated deep trees and
+# generated monorepo layouts, and with TypeScript's resolveModuleName on
+# Vite and a generated workspace monorepo. Every edge fcs misses or draws
+# wrongly must be a known gap listed in benches/graph/known_gaps.toml (each
+# tied to a row of the book's graph-support page), with its exact count: a
+# new miss fails, and so does a fix until its gap is lowered or removed.
 
 on:
   push:
@@ -15,6 +18,7 @@ on:
       - "src/**"
       - "examples/graph_oracle.rs"
       - "scripts/graph-oracle/**"
+      - "benches/graph/**"
       - "tests/python_codebase.rs"
       - "tests/support/**"
       - "Cargo.toml"
@@ -77,21 +81,36 @@ jobs:
         run: |
           python src/check/oracle/py.py bench-corpus/django > py.json
           ./target/release/examples/graph_oracle bench-corpus/django --oracle py.json --lang py \
-            --max-missing 0 --max-extra 0 --markdown graph-py.md
+            --gaps benches/graph/known_gaps.toml --corpus django --markdown graph-py.md
 
       - name: Python (grimp on a package nested 14 levels deep)
         run: |
           python scripts/graph-oracle/deep_python_tree.py deep-py 14
           python src/check/oracle/py.py deep-py > py-deep.json
           ./target/release/examples/graph_oracle deep-py --oracle py-deep.json --lang py \
-            --max-missing 0 --max-extra 0 --markdown graph-py-deep.md
+            --gaps benches/graph/known_gaps.toml --corpus deep-py --markdown graph-py-deep.md
 
       - name: Python (grimp on a repository whose root folder has __init__.py)
         run: |
           python scripts/graph-oracle/deep_python_tree.py root-init-py 4 --root-init
           python src/check/oracle/py.py root-init-py > py-root-init.json
           ./target/release/examples/graph_oracle root-init-py --oracle py-root-init.json --lang py \
-            --max-missing 0 --max-extra 0 --markdown graph-py-root-init.md
+            --gaps benches/graph/known_gaps.toml --corpus root-init-py --markdown graph-py-root-init.md
+
+      # Repositories laid out like real monorepos, generated with seed 1
+      # (scripts/graph-oracle/layouts.py): projects side by side each with
+      # its own src/, a root folder that is a package, and a namespace
+      # package split over two folders. oracle.env tells grimp what is on
+      # sys.path.
+      - name: Python (generated monorepo layouts)
+        run: |
+          for layout in monorepo root-init namespace; do
+            python scripts/graph-oracle/layouts.py "$layout" "py-$layout" --seed 1
+            (set -a; . "py-$layout/oracle.env"; set +a
+             python src/check/oracle/py.py "py-$layout" > "py-$layout.json")
+            ./target/release/examples/graph_oracle "py-$layout" --oracle "py-$layout.json" --lang py \
+              --gaps benches/graph/known_gaps.toml --corpus "py-$layout" --markdown "graph-layout-$layout.md"
+          done
 
       # Seeded generated Python trees whose generator knows every edge,
       # definition and reference (tests/support/pygen.rs). grimp and Python
@@ -111,8 +130,17 @@ jobs:
         run: |
           (cd scripts/graph-oracle && node ../../src/check/oracle/ts.mjs ../../bench-corpus/vite) > ts.json
           ./target/release/examples/graph_oracle bench-corpus/vite --oracle ts.json --lang ts \
-            --max-missing 7 --max-extra 1 --markdown graph-ts.md
+            --gaps benches/graph/known_gaps.toml --corpus vite --markdown graph-ts.md
+
+      # npm workspaces with tsconfig paths and extends, barrels, .mts/.cts,
+      # installed as symlinks (scripts/graph-oracle/ts_layouts.mjs).
+      - name: TypeScript (generated workspace monorepo)
+        run: |
+          node scripts/graph-oracle/ts_layouts.mjs ts-workspaces --seed 1
+          (cd scripts/graph-oracle && node ../../src/check/oracle/ts.mjs ../../ts-workspaces) > ts-workspaces.json
+          ./target/release/examples/graph_oracle ts-workspaces --oracle ts-workspaces.json --lang ts \
+            --gaps benches/graph/known_gaps.toml --corpus ts-workspaces --markdown graph-ts-workspaces.md
 
       - name: Report
         if: always()
-        run: cat graph-py.md graph-py-deep.md graph-py-root-init.md graph-ts.md >> "$GITHUB_STEP_SUMMARY" 2>/dev/null || true
+        run: cat graph-py.md graph-py-deep.md graph-py-root-init.md graph-layout-*.md graph-ts.md graph-ts-workspaces.md >> "$GITHUB_STEP_SUMMARY" 2>/dev/null || true

CHANGELOG.md

@@ -162,6 +162,18 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   made up from the corpus (`--generate-regex`), and
   `tests/discovery_oracle.rs` compares the indexed files and the FILES box
   with `rg`.
+- **The import graph is checked on monorepo layouts.** Generated
+  repositories shaped like real ones (`scripts/graph-oracle/layouts.py`,
+  `ts_layouts.mjs`): Python projects side by side each with its own
+  `src/`, a root folder that is a package, a namespace package split over
+  two folders, and npm workspaces with tsconfig `paths` and `extends`,
+  barrels and `.mts`/`.cts`. grimp and TypeScript give the answer. Every
+  edge fcs misses or draws wrongly, there and on Django and Vite, is now
+  a known gap listed with its reason and exact count in
+  `benches/graph/known_gaps.toml`, instead of a count to stay under. The
+  Python oracle (`fcs check` included) now builds one graph across
+  `ROOT` and `ROOT/src`, so imports from one into the other are compared
+  too.
 - **Kill and restart.** `examples/chaos.rs` kills the server binary with
   SIGKILL at random moments (during a build, a load from disk, a stream of
   edits, a periodic save, a burst of changes, and the save at shutdown),

benches/graph/known_gaps.toml

@@ -0,0 +1,85 @@
+# Every import edge the graph oracle (examples/graph_oracle.rs) finds fcs
+# missing, or drawing wrongly, on each CI corpus, sorted by reason. Each gap
+# names the row of docs/book/src/reference/graph-support.md it is, so the
+# list is the known limitations, not a number to stay under.
+#
+# The run fails when an edge fits no gap (a bug, or a limitation to
+# document here and on that page) and when a gap's count changes (a fix
+# lowers it: lower the count or remove the gap; a rise is a regression).
+#
+# `spec`, `from` and `to` are regexes on the import text, the importer and
+# the target as the report prints them (the corpus folder's name first).
+# Generated corpora are written with seed 1 (see the workflow).
+
+# --- Vite 6.3.5 (TypeScript's resolveModuleName) ---
+
+[[gap]]
+corpus = "vite"
+kind = "missing"
+spec = '^#'
+count = 4
+doc = "package.json `imports` (`#name`)"
+
+[[gap]]
+corpus = "vite"
+kind = "missing"
+spec = '^\./(bar)?$'
+count = 2
+doc = "A folder's own package.json `main`"
+
+[[gap]]
+corpus = "vite"
+kind = "missing"
+spec = '\.js$'
+to = '\.ts$'
+count = 1
+doc = "`./x.js` when both x.js and x.ts exist: fcs takes x.js (as Node and bundlers do), TypeScript x.ts"
+
+[[gap]]
+corpus = "vite"
+kind = "extra"
+to = 'syntax-error\.js$'
+count = 1
+doc = "`./x.js` when both x.js and x.ts exist: fcs takes x.js (as Node and bundlers do), TypeScript x.ts"
+
+# --- Generated: npm workspaces (scripts/graph-oracle/ts_layouts.mjs) ---
+
+[[gap]]
+corpus = "ts-workspaces"
+kind = "missing"
+spec = '^#'
+count = 1
+doc = "package.json `imports` (`#name`)"
+
+[[gap]]
+corpus = "ts-workspaces"
+kind = "missing"
+spec = '^@acme/ui/theme$'
+count = 1
+doc = "`exports` subpath maps (here a package importing itself by name)"
+
+# --- Generated: Python layouts (scripts/graph-oracle/layouts.py) ---
+
+[[gap]]
+corpus = "py-monorepo"
+kind = "missing"
+to = '/libs/lib_\w+/src/'
+count = 330
+doc = "Other sys.path entries: another project's `src/` in the same repository"
+
+[[gap]]
+corpus = "py-namespace"
+kind = "missing"
+from = '^py-namespace/src/acme/'
+to = '^py-namespace/plugins/src/acme/'
+count = 51
+doc = "Other sys.path entries: a namespace package's portion in another folder"
+
+[[gap]]
+corpus = "py-namespace"
+kind = "missing"
+from = '^py-namespace/plugins/src/acme/'
+spec = '^from \.'
+to = '^py-namespace/src/acme/'
+count = 13
+doc = "Other sys.path entries: a relative import into a namespace package's portion in another folder"

docs/book/src/how-it-works/testing.md

@@ -28,7 +28,7 @@ wrote every line and knows the answer.
 | `tests/lock_sim.rs` | the engine lock and the watcher's install steps under minutes of made-up load (back-to-back half-second searches, one 20 s search among quick ones, a busy server during a branch switch): every search answered, no search waits past its hold-back plus one step, backlogs drain at half speed or better, a slow search delays only the writer | bounds derived from the protocol, on a simulated clock (`fast_code_search::sim`): the real lock code, the same schedule on every run and OS, four minutes in about a second | `cargo test --test lock_sim` (`-- --ignored --nocapture` prints the tuning table) |
 | `tests/fault_injection.rs` | failed reads (permission, out of file handles, a file gone between listing and reading), a slow disk, panics while reading, parsing or applying changes, lost changes and lost events, a save onto a full disk or a failed rename: each is retried, reported or repaired, searches are not held up, and the saved index stays whole | expected recovery | `cargo test --features failpoints --test fault_injection` (fault points are listed in `src/faults.rs`) |
 | `examples/rg_backtest.rs` | fixed and generated queries and regexes on tokio, Django and fixtures, with paging and default limits, on an index built in one go, restarted from disk (`--history reload`) and filled by the watcher (`--history watch`) | `rg` | see [Correctness against ripgrep](../benchmarks/correctness.md) |
-| `examples/graph_oracle.rs` | the import graph on Django, Vite and generated deep trees | grimp, TypeScript's resolver | CI job "graph oracle" |
+| `examples/graph_oracle.rs` | the import graph on Django, Vite, generated deep trees and generated monorepo layouts (projects side by side with their own `src/`, a root folder that is a package, a split namespace package, npm workspaces with tsconfig `paths` and `extends`); every edge missed or wrong must be a known gap listed with its reason and count | grimp, TypeScript's resolver | CI job "graph oracle", gaps in `benches/graph/known_gaps.toml` |
 | `examples/ref_oracle.rs`, `examples/impact_oracle.rs` | Python references and diff impact on real repositories | Jedi | CI jobs "ref oracle", "impact oracle" |
 | `examples/scoreboard.rs` | the server binary over HTTP in seven scenarios (cold build, warm restart, branch switch, ten-minute stream, slow mix, saves under load, event-queue overflow): searches turned away, latency, time until the index matches the disk, memory, CPU, diagnostic captures, crashes, `/api/check` | targets in `benches/scoreboard/targets.toml` | nightly and on pull requests that touch locking or watching (workflow "Scale scoreboard"), see [Scale scoreboard](../benchmarks/scoreboard.md) |
 | `examples/chaos.rs` | `kill -9` of the server binary at random moments during a build, a load, a stream of edits, a periodic save, a burst and the shutdown save; after each, a restart must load the saved index (never reject it), leave no temp file, catch up with every change and pass `/api/check` | the files on disk | nightly and on pull requests that touch saving or watching (workflow "Scale scoreboard", job "Kill and restart"), see [below](#killed-at-any-moment) |

docs/book/src/reference/graph-support.md

@@ -67,7 +67,8 @@ a file `foo.rs` that is not the module. Config read: `Cargo.toml`.
 | `src/` layouts (`tests/` importing `src/pkg`) | yes: each of those folders' `src/` is tried too |
 | Absolute imports from a file nested any number of folders deep | yes: every ancestor folder is tried |
 | A repository whose root folder has an `__init__.py`, run with that folder on `sys.path` (`import app.models`) | yes: when nothing else resolves the import, the outermost package folder is tried as a root if it is the repository root (`.git`) or no folder above it holds an indexed file |
-| Other `sys.path` entries (`pyproject.toml`, `setup.cfg`, `PYTHONPATH`) | **no**: no configuration is read, so a package in a sibling folder (`lib/mypkg/` imported from `apps/x/main.py`, with `lib` put on `PYTHONPATH` by your setup) is not found and the import counts as external |
+| Other `sys.path` entries (`pyproject.toml`, `setup.cfg`, `PYTHONPATH`) | **no**: no configuration is read, so a package in a sibling folder (`lib/mypkg/` imported from `apps/x/main.py`, with `lib` put on `PYTHONPATH` by your setup) is not found and the import counts as external. This includes a monorepo of projects side by side, each with its own `src/` and installed into one environment (`services/billing` importing `libs/lib_db/src/lib_db`): imports within a project resolve, imports of another project do not |
+| Namespace packages (no `__init__.py`) | within one folder, yes; a namespace package split over several folders on `sys.path` (`src/acme/core` and `plugins/src/acme/x`) only where one portion's folder is an ancestor's `src/` of the other, so imports into the other portion, absolute or relative, are mostly missed |
 | `.pyi` stubs as import targets | **no** |
 | Standard library, installed packages | no edge, by design |
 
@@ -81,7 +82,7 @@ a file `foo.rs` that is not the module. Config read: `Cargo.toml`.
 | `import x = require('./x')` (TypeScript) | yes |
 | Relative specifiers (`./`, `../`, `.`, `..`) | yes |
 | `@/x`, `~/x` | yes: tsconfig `paths` when it maps them, else from the nearest folder with a `package.json` or `tsconfig.json` (else the repository root), trying `src/x` then `x` |
-| Extension probing | the exact file; `.js`/`.jsx` written for a `.ts`/`.tsx` file, `.mjs` for `.mts`, `.cjs` for `.cts`; then `.ts .tsx .js .jsx .mjs .cjs .mts .cts .d.ts .json` appended; then `index` with each of those extensions |
+| Extension probing | the exact file; `.js`/`.jsx` written for a `.ts`/`.tsx` file, `.mjs` for `.mts`, `.cjs` for `.cts`; then `.ts .tsx .js .jsx .mjs .cjs .mts .cts .d.ts .json` appended; then `index` with each of those extensions. When both `x.js` and `x.ts` exist, `./x.js` is `x.js` (as Node and bundlers load it), where TypeScript picks `x.ts` |
 | Imports of stylesheets, `.vue`/`.svelte` components and other assets | yes, when the file is indexed (a bundler resolves them; TypeScript does not) |
 | A folder's own `package.json` `main` (`import './lib'` where `lib/package.json` names the entry) | **no** |
 | `package.json` `imports` (`#name`) and a package importing itself by name | **no** |
@@ -102,16 +103,30 @@ wrongly:
 | Language | Oracle | Repository | Result |
 |---|---|---|---|
 | Python | [grimp](https://github.com/python-grimp/grimp) 3.17 (`src/check/oracle/py.py`) | Django 5.2 | 3,041 of 3,041 edges, none extra |
-| TypeScript, JavaScript | TypeScript 5.9 `ts.resolveModuleName` under each file's tsconfig (`src/check/oracle/ts.mjs`) | Vite 6.3.5 | 99.6% of edges, 1 extra; 4 of the 7 misses are `package.json` `imports` (`#name`); 221 workspace-package edges the oracle cannot see without an install are listed apart |
-
-CI runs both, and fails if a change misses or adds more edges than that.
+| Python | grimp | a package nested 14 levels deep; a repository whose root folder has `__init__.py` (`scripts/graph-oracle/deep_python_tree.py`) | every edge, none extra |
+| Python | grimp, with every project's `src/` on `sys.path` | generated: services and libraries side by side, each its own project with `src/` (`layouts.py monorepo`) | 368 of 698 edges, none extra; the 330 missed are imports of another project (other `sys.path` entries, above) |
+| Python | grimp | generated: a root folder that is a package, three top-level packages importing each other (`layouts.py root-init`) | 468 of 468 edges, none extra |
+| Python | grimp | generated: a namespace package split over `src/` and `plugins/src/` (`layouts.py namespace`) | 98 of 162 edges, none extra; the 64 missed cross from one portion to the other (namespace packages, above) |
+| TypeScript, JavaScript | TypeScript 5.9 `ts.resolveModuleName` under each file's tsconfig (`src/check/oracle/ts.mjs`) | Vite 6.3.5 | 1,613 of 1,620 edges, 1 extra: 4 `package.json` `imports` (`#name`), 2 folders with their own `package.json` `main`, and one `./x.js` where `x.ts` exists too (the extra is its other side); 221 workspace-package edges the oracle cannot see without an install are listed apart |
+| TypeScript, JavaScript | TypeScript | generated: npm workspaces installed as symlinks, a root `tsconfig.base.json` with `paths` that each project `extends`, barrels, `.mts`/`.cts`, `import type`, `import()`, `require` (`scripts/graph-oracle/ts_layouts.mjs`) | 388 of 390 edges, none extra: one `#name` import and one `exports` subpath |
+
+Every edge fcs misses or draws wrongly on these is listed, with its reason
+and count, in `benches/graph/known_gaps.toml`. CI ("graph oracle") fails
+when an edge is not on that list (a bug, or a gap to document here) and
+when a listed count changes, so a fix shows up as a gap to lower or
+remove. The generated repositories are written with seed 1.
 
 ```sh
 python3 src/check/oracle/py.py bench-corpus/django > py.json
 cargo run --release --example graph_oracle -- bench-corpus/django --oracle py.json --lang py
 (cd scripts/graph-oracle && npm ci)   # pins TypeScript
 (cd scripts/graph-oracle && node ../../src/check/oracle/ts.mjs ../../bench-corpus/vite) > ts.json
-cargo run --release --example graph_oracle -- bench-corpus/vite --oracle ts.json --lang ts
+cargo run --release --example graph_oracle -- bench-corpus/vite --oracle ts.json --lang ts \
+    --gaps benches/graph/known_gaps.toml --corpus vite
+python3 scripts/graph-oracle/layouts.py monorepo py-monorepo --seed 1
+(set -a; . py-monorepo/oracle.env; set +a; python3 src/check/oracle/py.py py-monorepo) > py-monorepo.json
+cargo run --release --example graph_oracle -- py-monorepo --oracle py-monorepo.json --lang py \
+    --gaps benches/graph/known_gaps.toml --corpus py-monorepo
 ```
 
 To run the same comparison on your own code, use `fcs check` (see