Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ The schema follows a few rules, informed by how agent-maintained wikis actually
- **Contested is a state to exit.** The documented failure mode of agent wikis is contradictions accumulating faster than they resolve. Lint flags contested pages older than 30 days; the reconcile workflow rewrites in place, moving losing claims to a dated "Superseded claims" section instead of deleting them.
- **Autonomous but reversible.** The maintenance workflow runs unattended on a `maintenance` branch with an exhaustively-listed set of safe actions (mechanical fixes, index rebuild, unambiguous cross-links); everything else becomes a proposal. The human reviews the branch diff and merges. Nothing automated ever lands on main directly.
- **Native OKF conformance.** Every wiki is an [OKF v0.2](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/okf/SPEC.md) bundle: markdown files with YAML frontmatter, ordinary markdown links as the edge form (bundle-absolute `[title](/dir/page.md)`, the form v0.2 §6.1 recommends), reserved `index.md` (stamped with `okf_version` frontmatter by `rebuild-index`, the mechanism §12 specifies) and `log.md` (date-grouped headings, bold-action-word entries). `check_okf` enforces the spec's three conformance rules: parseable frontmatter on every non-reserved `.md`, a non-empty `type`, and reserved-file structure. Provenance uses the spec's `sources` shape (§5.1): mapping entries with a required `resource`, enforced by `check_sources`, with pre-0.2 string entries downgraded to warnings. Scriptorium's schema is a strict superset of OKF's (the spec's `generated.at` is our `updated`; its `resource` is our `source_path`; its `title` is optional, with the index prettifying filenames when absent). One deliberate deviation: `raw/` is excluded from conformance because prime directive 1 makes those sources immutable, and OKF has no concept of a non-concept directory.
- **Opt-in extension points for non-wiki trees.** The engine can also lint markdown trees that aren't wikis (a findings folder, a labs journal): `okf_conformance: False` turns off the OKF rules for trees that aren't bundles, `non_page_allowed` accepts glob patterns, `index_file`/`index_body_fn` relocate and reshape the generated index (`index_file: None` disables it), `extra_secret_patterns`/`secret_allow_res` extend the secrets scan, and `extra_checks` runs custom callables. Every knob's default lives once in `wikilint/settings.py` (`DEFAULTS`) and a wiki's `lint.py` lists a key only to override it. `DEFAULTS` covers *every* key the engine reads, in two documented tiers: `EXTENSION_DEFAULTS` (the extension points above) preserves the original wiki behavior, so a wiki written before an extension existed keeps behaving as it did; `CORE_DEFAULTS` (the schema knobs — page dirs, required fields, staleness, ADRs, mermaid, coverage, ...) defaults to the neutral/disabled value, so a default can only ever silence a check, never invent one. A minimal config is therefore a handful of lines rather than a full key list. Bad values (a malformed regex, an out-of-tree `index_file`, a non-callable check) are rejected at startup with a clear message rather than a mid-run traceback.
- **Opt-in extension points for non-wiki trees.** The engine can also lint markdown trees that aren't wikis (a findings folder, a labs journal): `okf_conformance: False` turns off the OKF rules for trees that aren't bundles, `non_page_allowed` accepts glob patterns, `index_file`/`index_body_fn` relocate and reshape the generated index (`index_file: None` disables it), `extra_secret_patterns`/`secret_allow_res` extend the secrets scan, `extra_checks` runs custom callables, and `extra_commands` registers extra `lint.py <verb>` subcommands (`{verb: (callable(root) -> exit code, help)}`) which dispatch ahead of the wiki-root guard and appear in `lint.py help` alongside the engine's own verbs, so a wiki never has to intercept `argv` and end up with a second, partial usage string. Every knob's default lives once in `wikilint/settings.py` (`DEFAULTS`) and a wiki's `lint.py` lists a key only to override it. `DEFAULTS` covers *every* key the engine reads, in two documented tiers: `EXTENSION_DEFAULTS` (the extension points above) preserves the original wiki behavior, so a wiki written before an extension existed keeps behaving as it did; `CORE_DEFAULTS` (the schema knobs — page dirs, required fields, staleness, ADRs, mermaid, coverage, ...) defaults to the neutral/disabled value, so a default can only ever silence a check, never invent one. A minimal config is therefore a handful of lines rather than a full key list. Bad values (a malformed regex, an out-of-tree `index_file`, a non-callable check) are rejected at startup with a clear message rather than a mid-run traceback.

## Unattended maintenance

Expand Down
170 changes: 170 additions & 0 deletions tests/test_extra_commands.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,170 @@
"""The extra_commands registration seam: a wiki registers custom CLI verbs
with the engine instead of intercepting argv before wikilint.main(), so one
usage string lists every verb and every registration is validated at startup.
"""

import tempfile
import unittest
from pathlib import Path

from helpers import VARIANTS, load_variant_config


class CommandTest(unittest.TestCase):
def setUp(self):
from wikilint.settings import CONFIG
saved = dict(CONFIG)
self.addCleanup(lambda: (CONFIG.clear(), CONFIG.update(saved)))

def wiki_root(self):
tmp = tempfile.TemporaryDirectory()
self.addCleanup(tmp.cleanup)
root = Path(tmp.name)
(root / "CLAUDE.md").write_text("---\ntype: tooling\n---\n\n# stub\n")
return root


class TestDispatch(CommandTest):
def test_registered_command_is_dispatched_with_the_root(self):
from wikilint import main
calls = []

def hello(root):
calls.append(root)
return 0

root = self.wiki_root()
rc = main(
{"page_dirs": [], "extra_commands": {"hello": (hello, "say hi")}},
lambda fields: "", argv=["hello"], root=str(root),
)
self.assertEqual(rc, 0)
self.assertEqual(calls, [root])

def test_registered_command_returns_its_own_exit_code(self):
from wikilint import main
rc = main(
{"page_dirs": [], "extra_commands": {"hello": (lambda root: 7, "hi")}},
lambda fields: "", argv=["hello"], root=str(self.wiki_root()),
)
self.assertEqual(rc, 7)

def test_registered_command_needs_no_wiki_root(self):
"""Registered verbs dispatch ahead of the CLAUDE.md root guard, so a
pure-reporting verb (a schema dump, say) works from any cwd."""
from wikilint import main
with tempfile.TemporaryDirectory() as tmp: # no CLAUDE.md in it
rc = main(
{"page_dirs": [], "extra_commands": {"hello": (lambda root: 7, "hi")}},
lambda fields: "", argv=["hello"], root=tmp,
)
self.assertEqual(rc, 7)

def test_help_exits_zero_without_a_wiki_root(self):
from wikilint import main
with tempfile.TemporaryDirectory() as tmp:
for verb in ("help", "-h", "--help"):
self.assertEqual(
main({"page_dirs": []}, lambda fields: "", argv=[verb], root=tmp),
0, verb)

def test_root_guard_still_applies_to_engine_verbs(self):
"""Only registered verbs and `help` skip the guard."""
from wikilint import main
with tempfile.TemporaryDirectory() as tmp:
for verb in ("check", "rebuild-index", "nonsense"):
self.assertEqual(
main({"page_dirs": []}, lambda fields: "", argv=[verb], root=tmp),
2, verb)

def test_unknown_verb_still_exits_two(self):
from wikilint import main
rc = main({"page_dirs": []}, lambda fields: "",
argv=["nonsense"], root=str(self.wiki_root()))
self.assertEqual(rc, 2)


class TestUsage(CommandTest):
def test_usage_lists_engine_and_registered_verbs(self):
from wikilint.cli import usage
from wikilint.settings import configure
configure(
{"page_dirs": [], "extra_commands": {"hello": (lambda root: 0, "say hi")}},
lambda fields: "",
)
text = usage()
for verb in ("check", "rebuild-index", "reverse-deps", "coverage", "help", "hello"):
self.assertIn(verb, text)
self.assertIn("say hi", text)

def test_usage_covers_every_builtin(self):
"""usage() renders BUILTIN_COMMANDS, so the verb list has exactly one
declaration and cannot drift from what main() dispatches."""
from wikilint.cli import usage
from wikilint.settings import BUILTIN_COMMANDS, configure
configure({"page_dirs": []}, lambda fields: "")
text = usage()
for verb, help_text in BUILTIN_COMMANDS:
self.assertIn(verb, text)
self.assertIn(help_text, text)


class TestRegistrationValidation(CommandTest):
def configure_with(self, commands):
from wikilint.settings import configure
configure({"page_dirs": [], "extra_commands": commands}, lambda fields: "")

def test_verb_shadowing_a_builtin_is_rejected(self):
from wikilint.settings import BUILTIN_COMMANDS, ConfigError
for verb, _help in BUILTIN_COMMANDS:
with self.assertRaises(ConfigError, msg=verb):
self.configure_with({verb: (lambda root: 0, "shadow")})

def test_non_callable_handler_is_rejected(self):
from wikilint.settings import ConfigError
with self.assertRaises(ConfigError):
self.configure_with({"hello": ("nope", "desc")})

def test_malformed_entry_is_rejected(self):
from wikilint.settings import ConfigError
with self.assertRaises(ConfigError):
self.configure_with({"hello": lambda root: 0})

def test_empty_help_is_rejected(self):
from wikilint.settings import ConfigError
for help_text in ("", " ", None):
with self.assertRaises(ConfigError, msg=repr(help_text)):
self.configure_with({"hello": (lambda root: 0, help_text)})

def test_empty_verb_is_rejected(self):
from wikilint.settings import ConfigError
for verb in ("", " "):
with self.assertRaises(ConfigError, msg=repr(verb)):
self.configure_with({verb: (lambda root: 0, "desc")})

def test_non_dict_table_is_rejected(self):
from wikilint.settings import ConfigError
with self.assertRaises(ConfigError):
self.configure_with([("hello", (lambda root: 0, "desc"))])

def test_a_bad_registration_fails_before_any_dispatch(self):
from wikilint import main
rc = main({"page_dirs": [], "extra_commands": {"check": (lambda r: 0, "x")}},
lambda fields: "", argv=["check"], root=str(self.wiki_root()))
self.assertEqual(rc, 2)


class TestTemplateUnaffected(unittest.TestCase):
def test_the_shipped_template_registers_no_command(self):
"""The seam is additive: the template registers nothing, so its usage
output and dispatch are unchanged."""
from wikilint.settings import CONFIG, configure
for variant in VARIANTS:
config, extra = load_variant_config(variant)
self.assertNotIn("extra_commands", config, variant)
configure(config, extra)
self.assertEqual(CONFIG["extra_commands"], {}, variant)


if __name__ == "__main__":
unittest.main()
45 changes: 33 additions & 12 deletions wiki/wikilint/cli.py
Original file line number Diff line number Diff line change
@@ -1,20 +1,32 @@
"""CLI entry point: subcommand dispatch and the check orchestrator."""
"""CLI entry point: subcommand dispatch and the check orchestrator.

usage() renders the engine's verbs (settings.BUILTIN_COMMANDS) plus whatever
the active wiki registered through `extra_commands`, so one usage string
covers every verb and a wiki never has to intercept argv for itself.
"""

import sys
from pathlib import Path

from . import checks
from .derived import check_index_drift, rebuild_index, run_coverage, run_reverse_deps
from .model import Report, discover_pages
from .settings import CONFIG, ConfigError, configure
from .settings import BUILTIN_COMMANDS, CONFIG, ConfigError, configure

USAGE = """\
Usage:
python3 lint.py check run all mechanical checks, exit 1 on errors
python3 lint.py rebuild-index regenerate the index from page frontmatter
python3 lint.py reverse-deps print the derived reverse-dependency maps
python3 lint.py coverage write coverage.md (variants with coverage enabled)
"""

def usage():
"""The single authoritative verb list: engine subcommands plus whatever
the active wiki registered through extra_commands."""
rows = list(BUILTIN_COMMANDS) + [
(verb, help_text)
for verb, (_fn, help_text) in sorted(CONFIG.get("extra_commands", {}).items())
]
width = max(len(verb) for verb, _help in rows)
lines = ["Usage:"] + [
f" python3 lint.py {verb.ljust(width)} {help_text}"
for verb, help_text in rows
]
return "\n".join(lines) + "\n"


def gather_report(root):
Expand Down Expand Up @@ -70,11 +82,20 @@ def main(config, index_entry_extra, argv=None, root=None):
print(f"lint config error: {e}", file=sys.stderr)
return 2
root = Path(root) if root else Path.cwd()
args = sys.argv[1:] if argv is None else argv
command = args[0] if args else "check"
# Registered verbs and `help` dispatch ahead of the wiki-root guard, so a
# pure-reporting verb works from any cwd. A registered command that does
# need the wiki root must check for it itself.
handler = CONFIG["extra_commands"].get(command)
if handler is not None:
return handler[0](root)
if command in ("help", "-h", "--help"):
print(usage(), end="")
return 0
if not (root / "CLAUDE.md").is_file():
print("run from the wiki root (CLAUDE.md not found here)", file=sys.stderr)
return 2
args = sys.argv[1:] if argv is None else argv
command = args[0] if args else "check"
if command == "check":
return run_check(root)
if command == "rebuild-index":
Expand All @@ -86,5 +107,5 @@ def main(config, index_entry_extra, argv=None, root=None):
print("coverage is not enabled for this variant")
return 2
return run_coverage(root)
print(USAGE)
print(usage(), end="")
return 2
40 changes: 40 additions & 0 deletions wiki/wikilint/settings.py
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,19 @@
"inbox_warn_age_days": 14,
}

# The engine's own subcommands, as (verb, one-line help). Declared here rather
# than in cli.py because this module is the validation boundary and needs the
# reserved names to reject an `extra_commands` verb that would shadow a
# built-in; cli.usage() renders this same table, so the verb list has exactly
# one declaration.
BUILTIN_COMMANDS = (
("check", "run all mechanical checks, exit 1 on errors"),
("rebuild-index", "regenerate the index from page frontmatter"),
("reverse-deps", "print the derived reverse-dependency maps"),
("coverage", "write coverage.md (variants with coverage enabled)"),
("help", "print this usage"),
)

# Engine extension points. Unlike the core knobs above, these default to the
# ORIGINAL wiki behavior — including the non-off `orphans: True`,
# `log_file: "log.md"`, `okf_conformance: True` and `types_glossary: True` —
Expand Down Expand Up @@ -110,6 +123,10 @@
"log_file": "log.md",
# Callables(pages, report, root) run at the end of every check pass.
"extra_checks": [],
# Variant subcommands: {verb: (callable(root) -> exit code, help text)}.
# Registered verbs dispatch ahead of the wiki-root guard and appear in the
# one usage string, so a variant never intercepts argv for itself.
"extra_commands": {},
}

# Every key the engine reads, with its default. Reading these bare as
Expand Down Expand Up @@ -137,6 +154,28 @@ def _compile(pattern, key):
raise ConfigError(f"{key}: invalid regex {pattern!r}: {e}")


def _validate_commands(commands):
"""Validate the extra_commands registration table so a misregistered verb
fails here, once, instead of at dispatch time."""
reserved = {verb for verb, _help in BUILTIN_COMMANDS}
if not isinstance(commands, dict):
raise ConfigError("extra_commands must be a {verb: (callable, help)} dict")
for verb, entry in commands.items():
if not isinstance(verb, str) or not verb.strip():
raise ConfigError(f"extra_commands verbs must be non-empty strings: {verb!r}")
if verb in reserved:
raise ConfigError(f"extra_commands verb {verb!r} shadows an engine subcommand")
try:
fn, help_text = entry
except (TypeError, ValueError):
raise ConfigError(
f"extra_commands[{verb!r}] must be a (callable, help) pair: {entry!r}")
if not callable(fn):
raise ConfigError(f"extra_commands[{verb!r}] handler must be callable: {fn!r}")
if not isinstance(help_text, str) or not help_text.strip():
raise ConfigError(f"extra_commands[{verb!r}] needs a non-empty help string")


def _validate(cfg):
"""Validate user-supplied extension values at the config boundary and
precompile the secret regexes so a bad pattern fails here, once, with a
Expand Down Expand Up @@ -168,6 +207,7 @@ def _validate(cfg):
for fn in cfg["extra_checks"]:
if not callable(fn):
raise ConfigError(f"extra_checks entries must be callable: {fn!r}")
_validate_commands(cfg["extra_commands"])

compiled_extra = []
for item in cfg["extra_secret_patterns"]:
Expand Down
Loading