Add the intake reader and close test gaps found in the first audit

docs/DECISIONS.md

@@ -2,6 +2,7 @@
 
 Choices the owner has made, newest first, with the date each was made. Later work follows them without asking again. Bigger design choices have their own notes in `docs/adr`, and plans live in `docs/ROADMAP.md`.
 
+- **2026-11-08 · Intake from other tools needs the user's yes.** The owner's words: "Build it; yes before joining". Another tool (first, a console app with action items from meeting notes) appends JSON Lines to `data/intake/<tool>.jsonl` and never edits the list or the Notes. `engine/tools/intake.py` reads it; new items are offered to the user and only accepted ones join the master list. Not the Notes inbox, because that is the user's own words. Wiring it into the workflows (RULES.md) waits for the owner's yes in a session.
 - **2026-11-07 · Copyright.** `LICENSE` reads "Copyright (c) 2026 EXTANT APE". In prose the org is written EXTANT APE; GitHub URLs keep `ExtantApe`. The leak check allows that exact line through `scripts/leak-allow.txt`.
 - **2026-11-07 · Name every way data leaves the Mac.** The manifesto lists each one: Claude Code, connectors you turn on, Apple's own sync, and a wall display if you connect one. Anything new that sends data off the Mac is named there first.
 - **2026-11-07 · iPhone plans** (not started; details in `docs/ROADMAP.md`). The encrypted backup uses a recovery key you keep, and Theseus suggests saving it in Apple's Passwords. First run suggests iCloud backup, and staying offline is a choice. The backup covers `config.json`, `profile.md` and `data/`. The lockdown button hides and locks, replaces what the Today and Master List notes show, and warns when FileVault is off.

docs/test-audit.md

@@ -0,0 +1,35 @@
+# Test audit
+
+How the tests line up with what Theseus promises. Redo it after every feature or fix, and at least every 14 days while the repo has commits.
+
+## 2026-11-09 · first audit
+
+What exists: `tests/test_engine.py` (134 tests, stdlib, run by `scripts/ci-checks.sh`), `apps/desktop/src/*.test.ts` (22, `node --test`), 2 Rust tests in the desktop app, the desktop and wall end-to-end runs in CI (`apps/desktop/e2e/ui.mjs`, `apps/wall/e2e.mjs`), and `apps/wall/sim/test_connection.py`. CI also runs the leak check, syntax checks for every script and tool, `swiftc -typecheck` on `ek.swift`, and the shared-file check.
+
+### Always covered (security, data loss, privacy, fail-closed checks)
+
+| Promise | Proved by |
+| --- | --- |
+| Nothing is sent without the user; connectors can read but not send, delete or change things in unattended runs | `Guard` (send_guard: sends and deletes refused, one bad action refuses the batch, empty batch blocked, fails closed, works from any folder, settings run the guard on every connector, unattended runs can sync but not send) |
+| The iMessage channel answers only the user's own self-chat | `test_the_imessage_channel_can_answer_you`, `…_cannot_reach_anyone_else`, `…_needs_your_handles` |
+| Personal databases (Messages, Mail, Things) are only read, through a copy | **new** `PersonalDataReadOnly`: every live database is opened `mode=ro`; no reader runs SQL that writes or can send; a snapshot leaves the source byte for byte |
+| No copies of personal data stay behind | `test_a_run_leaves_no_copies_of_personal_databases_behind`, `test_a_run_clears_its_hidden_temp_exports` |
+| Secrets are off limits; what runs read is data, not instructions | `test_secrets_are_off_limits_everywhere`, `test_runs_are_told_what_they_read_is_data`, `test_both_hosts_keep_runs_off_the_web_and_out_of_settings` |
+| The wall display sends only what the user chose, and the Pi only accepts it | `WallDisplay` and `WallReceive` (only the list is read, notes and ids stay behind, Done and Communications never appear, titles off sends no titles, a bad list sends nothing, the key can do nothing but deliver or clear, host identity changes are never accepted) |
+| The master list is never lost or half written | app-edits batches (`test_a_change_survives_a_run_that_never_finished`, `test_a_batch_from_a_dead_run_comes_back`), `test_state_is_never_half_written`, desktop `master.test.ts` round trips, undo refusing when the list changed |
+| An install never overwrites the user's local engine changes | **new** `LocalChangesGuard`: a patch the source lacks stops the install, a missing patch file stops it, an applied patch lets it through, entries without a patch don't block, the session note says the file is data |
+| Items other tools hand Theseus join the list only with the user's yes | **new** `Intake`: only open items for the user are offered, a non-date due is never used, nothing joins without a decision and each is decided once, a later done line reaches its task once, the tool's file is never written |
+
+### Gaps added in this audit
+- `engine/tools/local_changes.py` had no tests, though `guard` is the check that keeps an install from overwriting local changes. Added `LocalChangesGuard` (5 tests).
+- Read-only access to personal databases was true in the code but unproven. Added `PersonalDataReadOnly` (3 tests).
+- `engine/tools/intake.py` is new (the Console intake, decided 2026-11-08). Added with `Intake` (6 tests). It isn't wired into the workflows yet.
+
+### Gaps skipped, and why
+- `alarms.py` (reads a file a Shortcut writes; no writes, nothing sent) and `things_export.py` (one-time migration). Their read-only side is covered by `PersonalDataReadOnly`; parsing details aren't worth a test until they change.
+- `ek.swift` beyond the type check: it needs real Reminders and Calendar access, so it can't run in CI. Its writes go through the send guard's rules and the workflows' logging.
+- `mail.py` and `imsg.py` output formatting: covered where it matters (search windows, contacts fallback, seen marks); the rest is presentation.
+- Framework internals of the desktop app (Tauri, Svelte) and trivial getters.
+
+### Next audit
+By 2026-11-23, or with the next feature or fix, whichever comes first. Check then: the intake wiring once the workflows use it.

engine/tools/intake.py

@@ -0,0 +1,153 @@
+#!/usr/bin/env python3
+"""
+intake.py: items other tools hand Theseus (data/intake/<tool>.jsonl). Nothing joins the master list without the user's yes.
+
+Each tool only appends, one JSON object per line:
+  {"id": "console-…", "created": "<ISO 8601>", "title": "…", "owner_is_me": true,
+   "meeting": {"title": "…", "date": "YYYY-MM-DD"},
+   "context": "…", "due": "YYYY-MM-DD", "category": "#…", "link": "…"}      (last four optional)
+A later line with the same id and "status": "done" or "cancelled" updates the item.
+
+  pending              new items waiting for the user's yes, numbered, plus lines that were skipped and why
+  accept ID --task tNNNN   the user said yes and the item is now that master task
+  decline ID           the user said no
+  updates              accepted items whose tool later marked them done or cancelled, not yet carried to the list
+  applied ID           that update has been carried to the list
+
+Decisions live in data/intake-state.json. This tool never writes to the tools' files.
+"""
+import datetime as dt
+import json
+import os
+import re
+import sys
+from pathlib import Path
+
+WS = Path(os.environ.get("THESEUS_HOME") or Path(__file__).resolve().parents[2])
+INTAKE = WS / "data/intake"
+STATE = WS / "data/intake-state.json"
+REQUIRED = ("id", "created", "title", "owner_is_me", "meeting")
+DATE = re.compile(r"^\d{4}-\d{2}-\d{2}$")
+
+
+def load_state():
+    try:
+        return json.loads(STATE.read_text())
+    except (OSError, ValueError):
+        return {"items": {}}
+
+
+def save_state(state):
+    tmp = STATE.with_suffix(".tmp")
+    tmp.write_text(json.dumps(state, indent=1, ensure_ascii=False) + "\n")
+    os.replace(tmp, STATE)
+
+
+def read_items():
+    """Fold every tool's lines by id: the first full line is the item, later status lines update it."""
+    items, skipped = {}, []
+    for path in sorted(INTAKE.glob("*.jsonl")) if INTAKE.exists() else []:
+        source = path.stem
+        for n, line in enumerate(path.read_text().splitlines(), 1):
+            if not line.strip():
+                continue
+            where = f"{path.name}:{n}"
+            try:
+                obj = json.loads(line)
+            except ValueError:
+                skipped.append({"line": where, "why": "not JSON"})
+                continue
+            if not isinstance(obj, dict) or not isinstance(obj.get("id"), str) or not obj["id"]:
+                skipped.append({"line": where, "why": "no id"})
+                continue
+            key = f"{source}:{obj['id']}"
+            status = obj.get("status")
+            if status in ("done", "cancelled"):
+                if key in items:
+                    items[key]["status"] = status
+                    items[key]["status_line"] = where
+                else:
+                    skipped.append({"line": where, "why": f"{status} for an id not seen yet"})
+                continue
+            if status is not None:
+                skipped.append({"line": where, "why": f"unknown status {status!r}"})
+                continue
+            if key in items:
+                continue   # same id again: the first one stands
+            missing = [f for f in REQUIRED if f not in obj or obj[f] in (None, "")]
+            if missing:
+                skipped.append({"line": where, "why": "missing " + ", ".join(missing)})
+                continue
+            if obj["owner_is_me"] is not True:
+                continue   # someone else's action item
+            if obj.get("due") and not DATE.match(str(obj["due"])):
+                obj = {**obj, "due_raw": obj["due"], "due": None}
+            items[key] = {"key": key, "source": source, "line": where, "status": "open",
+                          **{k: obj.get(k) for k in ("id", "created", "title", "context", "due", "due_raw",
+                                                    "category", "meeting", "link")}}
+    return items, skipped
+
+
+def pending():
+    items, skipped = read_items()
+    decided = load_state()["items"]
+    new = [i for k, i in items.items() if k not in decided and i["status"] == "open"]
+    new.sort(key=lambda i: str(i["created"]))
+    for n, i in enumerate(new, 1):
+        i["n"] = n
+    print(json.dumps({"pending": new, "skipped": skipped}, indent=1, ensure_ascii=False))
+
+
+def decide(key, decision, task=None):
+    items, _ = read_items()
+    if key not in items:
+        sys.exit(f"no intake item {key!r} (use the key from `pending`, e.g. console:console-123)")
+    state = load_state()
+    if key in state["items"]:
+        sys.exit(f"{key} was already {state['items'][key]['decision']}")
+    entry = {"decision": decision, "at": dt.datetime.now().isoformat(timespec="seconds"),
+             "title": items[key]["title"]}
+    if task:
+        entry["task"] = task
+    state["items"][key] = entry
+    save_state(state)
+    print(json.dumps({key: entry}, ensure_ascii=False))
+
+
+def updates():
+    items, _ = read_items()
+    state = load_state()["items"]
+    out = [{"key": k, "task": d.get("task"), "title": d["title"], "status": items[k]["status"]}
+           for k, d in state.items()
+           if d["decision"] == "accepted" and k in items and items[k]["status"] != "open" and not d.get("applied")]
+    print(json.dumps({"updates": out}, indent=1, ensure_ascii=False))
+
+
+def applied(key):
+    state = load_state()
+    if key not in state["items"]:
+        sys.exit(f"no decision recorded for {key!r}")
+    state["items"][key]["applied"] = dt.datetime.now().isoformat(timespec="seconds")
+    save_state(state)
+    print(json.dumps({key: state["items"][key]}, ensure_ascii=False))
+
+
+def main(argv):
+    if not argv:
+        sys.exit(__doc__)
+    cmd, rest = argv[0], argv[1:]
+    if cmd == "pending" and not rest:
+        return pending()
+    if cmd == "updates" and not rest:
+        return updates()
+    if cmd == "accept" and len(rest) == 3 and rest[1] == "--task" and re.match(r"^t\d{4,}$", rest[2]):
+        return decide(rest[0], "accepted", rest[2])
+    if cmd == "decline" and len(rest) == 1:
+        return decide(rest[0], "declined")
+    if cmd == "applied" and len(rest) == 1:
+        return applied(rest[0])
+    sys.exit(__doc__)
+
+
+if __name__ == "__main__":
+    main(sys.argv[1:])