docs: retro for the WebServer stall work (ADFA-5172/5175/5176)

.claude/skills/retro/SKILL.md

@@ -29,7 +29,9 @@ This affects Steps 3 and 5 below. All other steps run the same regardless of mod
    - Find the transcript: glob `~/.claude/projects/<converted-cwd>/*.jsonl` sorted by modification time (convert cwd slashes to dashes, e.g., `/Users/me/myproject` → `-Users-me-myproject`). Pick the most recent.
    - Run the script: `python3 scripts/analyze_transcript.py <path-to-jsonl>` — the script lives in `scripts/` alongside this SKILL.md.
    - The script outputs: per-turn breakdown (full user text, assistant word count, tools, errors) and timing stats (reading at 150 wpm, typing at 60 wpm, 1 min buffer per turn, overlapping turns merged).
-   - System-injected messages (skill injections, /mcp outputs, system reminders) are automatically filtered out.
+   - System-injected messages (skill injections, /mcp outputs, system reminders) are automatically filtered
+     out, as are the agent's own image reads -- a screenshot the agent took arrives as a `user` message and
+     would otherwise be counted as a prompt the human typed, inflating hands-on time.
 
    **What you do with the output:**
    - Read the turn-by-turn output to understand what happened
@@ -39,7 +41,7 @@ This affects Steps 3 and 5 below. All other steps run the same regardless of mod
 
    Present as a time breakdown table with proportional bars and a metrics summary:
 
-   | Started | Phase | 👤 Hands-On Time | 🤖 Agent Time | Problems |
+   | Started | Phase | 👤 Hands-On Time | 🤖 Span (agent + away) | Problems |
    |---------|-------|-----------------|---------------|----------|
    | Feb 10 10:00am | Build (engine restart, voice recog, UI tweaks) | ██████ 60m | ███ 30m | ⚠ 5 fix cycles |
    | Feb 10 11:30am | Research (BT routing for AirPods + external mics) | | █████ 45m | |
@@ -127,7 +129,7 @@ This affects Steps 3 and 5 below. All other steps run the same regardless of mod
    ## YYYY-MM-DD - [Brief context of what we worked on]
 
    ### Time Breakdown
-   | Started | Phase | 👤 Hands-On Time | 🤖 Agent Time | Problems |
+   | Started | Phase | 👤 Hands-On Time | 🤖 Span (agent + away) | Problems |
    |---------|-------|-----------------|---------------|----------|
    | ... | ... | ... | ... | ... |
 

.claude/skills/retro/scripts/analyze_transcript.py

@@ -25,6 +25,11 @@
   - Skill injections ("Base directory for this skill:")
   - Local command outputs (<command-name>, <local-command-)
   - System reminders (<system-reminder>)
+  - Image tool results ("[Image: original ...]"), which are the agent's own Read of a
+    screenshot arriving in the human role -- counting those as human turns adds reading
+    and buffer time nobody spent. Measured on one session that drove a device: 7.5 min of raw
+    hands-on, 4.5 min adjusted, all of it typing and per-turn buffer. Reading time does not
+    change -- the assistant words are conserved, just re-attributed to the prompt that caused them.
 """
 
 from __future__ import annotations
@@ -55,6 +60,7 @@
 
 SYSTEM_MESSAGE_PATTERNS = [
     re.compile(r"^Base directory for this skill:"),
+    re.compile(r"^\[Image: original \d+x\d+"),
     re.compile(r"^<(command-name|local-command|system-reminder)"),
     re.compile(r"^<local-command-caveat>"),
     re.compile(r"^This session is being continued from a previous conversation"),

CLAUDE.md

@@ -60,6 +60,7 @@ See **[ARCHITECTURE.md](ARCHITECTURE.md)** — the single source of truth for th
 - **Protect the two Android system bars** in any UI work: the top status bar (clock, notifications, status icons) and the bottom navigation bar (home, back, recents). Don't draw over or intercept them.
 - **Every screen must survive 2x font scale.** Users with low vision run large system fonts, and a screen that clips or hides content at 2.0 is broken for them. Verify any new or changed screen at font scale **1.0 and 2.0** (see Build & test, Emulator / device) and say in the PR that you did. Text grows, so: use `sp` for text and `dp` for spacing — never an `sp` dimen as a margin or padding; don't box text in a fixed `dp` height or width; give content that can grow somewhere to scroll; and reserve `maxLines`/`singleLine`/`ellipsize` for text that is genuinely disposable.
 - **Plan and size before building.** Prefer **one PR per ticket/use case** — don't force-split a coherent change (splitting has its own overhead when later edits span the pieces). When a change is large, break it into **reviewable commits** — mechanical/refactor commits separate from behavioral ones — and offer review-by-commit. Treat ~500 LOC / ~10 files as a signal to reach for that commit structure, not a hard cap; the ceiling rises as LLM-assisted review matures. For staged multi-commit refactors (e.g. removing a dependency across many files/modules), order stages easiest-to-hardest and independently compile/test each stage (see Build & test's fast-iteration guidance) before moving to the next, so a failure is isolated to the stage that caused it.
+- **Before designing a way to make a costly operation cheaper, ask whether the operation is needed at all.** When a problem's severity scales with how often something happens — connections opened per second, requests issued per page, calls made per frame — eliminating the operations usually beats optimizing each one. Check whether the platform offers a way to do the work without them before writing the plan for the optimized version. ADFA-5172/5176 is the cautionary case: the server opened one TCP connection per documentation request to our own process, and `WebViewClient.shouldInterceptRequest` served the same content with no connection at all, which HTTP keep-alive would only have made cheaper per connection.
 - **Keep docs in step with code.** When you change code, update the docs that describe it in the same change — a module's `README.md`, `ARCHITECTURE.md`, or an ADR — so a doc never outlives the API it documents (see REVIEW.md, Code quality). If the doc fix is out of scope, file a ticket rather than let it drift.
 - `.androidide_root` is a sentinel file tests use to locate the project root — don't delete it.
 - Avoid http or https links which go off-device. When such links are unavoidable, warn the user beforehand and offer to cancel the action.

docs/process/learnings.md

@@ -12,12 +12,23 @@
 - Before pushing a follow-up commit to a community PR, check `gh pr view <n> --json headRepositoryOwner` — the PR head is usually on the contributor's **fork**, so a same-named push to `origin` doesn't touch the PR and just creates a confusing dead branch that has to be deleted.
 
 ## Android / Kotlin
+- A config data class whose **default** values call framework APIs (e.g. `ServerConfig`'s paths default to `Environment.getExternalStorageDirectory()`) makes itself unconstructable in a JVM unit test that doesn't stub those APIs — `RuntimeException: Method ... not mocked`, thrown from the constructor before your test body runs. Such a test has to pass *every* framework-backed parameter explicitly (or `mockkStatic` the API, or run under Robolectric), which is easy to miss when copying a config from a test that already does. Prefer lazily-resolved paths in new config types.
 - `Handler.removeCallbacks(Runnable)` only removes callbacks posted by that *exact* `Handler` instance, not just the same `Looper` — `Handler(Looper.getMainLooper()).removeCallbacks(x)` won't cancel something posted via a *different* `Handler` bound to the same looper. Any post/cancel pair needs to share one `Handler` instance (see `TaskExecutor.mainThreadHandler`, added when replacing blankj's `ThreadUtils.getMainHandler()`).
 
+## Serving content to a WebView
+- A WebView can be handed content **in-process** through `WebViewClient.shouldInterceptRequest`, returning a `WebResourceResponse` built from a stream — no socket, no port, no handshake. It sees the http(s) requests the page makes, whatever the URL, so an existing `http://localhost:PORT/...` URL space needs **no rewriting**: strings.xml entries, link builders and even a published plugin-API contract keep working while the transport underneath changes (ADFA-5176 turned 31 TCP connections per documentation page into 0 this way). It is not a hook on *every* URL — `blob:`, `javascript:` and `file:///android_asset/` requests don't reach it — so anything routed that way still needs its own path.
+- A WebView does **not** decode an intercepted response, so hand back decompressed bytes and don't bother with `Content-Encoding`. Give `WebResourceResponse` the bare MIME type with the charset as its own argument, and pass `null` for binary types — claiming a charset on an image makes the WebView try to decode it as text.
+- `shouldInterceptRequest` never sees a POST body: a request with one has to go to the network. Range requests are answerable in principle — `WebResourceResponse`'s six-argument constructor takes a status code and headers, so 206 is expressible — but a handler that ignores `Range` and returns 200 with the whole body is not; ours does, which was fine because the WebView asked for a whole 407 KB PDF rather than ranges.
+- A WebView does not render PDFs itself: pointing one at an `application/pdf` URL shows a blank page, identically over HTTP or in-process (verified on Android 13 with byte-identical screenshots). Showing one in-app needs a real renderer — `PdfRenderer`, or handing the file to an external viewer. See ADFA-5180.
+
+## Android system SQLite
+- Don't assume the JSON1 extension. On a Samsung Android 13 device, `JSON_OBJECT`/`JSON_GROUP_ARRAY` fail at runtime with `no such function: JSON_OBJECT` even though the same query runs fine against the same database file under a desktop sqlite3. Any query using JSON functions needs either a fallback or a documented minimum, and a JSON-based endpoint can be dead on real hardware while passing every desktop test.
+
 ## Reverse-engineering a library before porting it
 - When writing a same-name drop-in for a third-party utility (to remove the dependency without changing call-site behavior), don't guess its semantics from memory/docs — extract the AAR's `classes.jar` and run `javap -c` against the actual bytecode to confirm exact chaining/wrapping behavior, especially for fluent/reflection-style APIs where a subtle mismatch (e.g., wrapping a field's *declared* type vs. its *runtime* class) changes behavior at existing call sites.
 
 ## MockK
+- To unit-test code that touches WebView plumbing without Robolectric: `mockkStatic(android.os.Environment::class)` for `getExternalStorageDirectory()`, and a plain `mockk<Uri>` stubbing only `host`/`port`/`path`. Keep the framework *construction* out of the unit under test — a `WebResourceResponse` constructor throws `Stub!` in a JVM test, so split the decision (which content answers this request) from the wrapping, and test the decision.
 - Migrating a mocked call from a Java static method (`mockkStatic(SomeClass::class)`) to a Kotlin top-level extension function requires `mockkStatic("com.package.FileNameKt")` (the compiled JVM facade class name) instead — `mockkStatic(ExtensionReceiver::class)` doesn't work for extension functions.
 
 ## Measuring a real before/after delta

docs/process/retrospective.md

@@ -53,6 +53,71 @@ Caveat on the last phase: 154m of "hands-on" counts ~6,900 words of machine-gene
 | Instrumented tests, fixtures, stale approvals, Spotless double cost | Doc | Four new entries in `docs/process/learnings.md` |
 | Reviewer-side revert check; dismiss stale approvals on `stage` | Deferred | Both change artifacts other people rely on (REVIEW.md, branch protection) — raised, not applied |
 
+## 2026-08-18 - ADFA-5172/5175/5176: the local WebServer's 1 s stall, and removing the socket instead
+
+### Time Breakdown
+
+Each phase's span runs from its first prompt to the next one, so the spans sum to the wall clock
+below, to within a minute of rounding. A span holds both agent work and any time nobody was at
+the keyboard; only the totals in Metrics attempt that split, and only as an estimate. Hands-on is per-phase raw, so it sums slightly
+above the adjusted total, which merges overlapping turns into one buffer.
+
+| Started | Phase | 👤 Hands-On Time | 🤖 Span (agent + away) | Problems |
+|---------|-------|-----------------|------------------------|----------|
+| Aug 17 9:42pm | Ticket read + accept-loop instrumentation | ▊ 7m | █▏ 12m | |
+| Aug 17 9:54pm | Build, drive, root-cause the stall | ▌ 5m | ███ 34m | ⚠ HelpActivity not exported, so the measurement needed a throwaway manifest tweak; one flaky arm |
+| Aug 17 10:28pm | Keep-alive design + ADFA-5175 filed | █ 10m | █ 10m | |
+| Aug 17 10:37pm | ADFA-5175 stage 1, transport pivot, ADFA-5176 spike | ▊ 8m | ███████████ 109m | ⚠ 3 Spotless whole-file reformats; direction changed mid-implementation |
+| Aug 18 12:26am | Extraction onto the ADFA-5153 base | ▌ 5m | █████████████ 133m | ⚠ merge conflicts, plus a stale KDoc and dangling brace from moving code by script |
+| Aug 18 2:39am | Tests, Pebble move, cleanup, two PRs | ▊ 8m | ██████████████ 142m | ⚠ tests written just before the API they cover moved |
+| Aug 18 5:01am | Review fixes, CodeRabbit replies, retro | █▏ 12m | ██████████████████ 182m | |
+
+### Metrics
+
+| Metric | Duration |
+|--------|----------|
+| Total wall-clock (first prompt to last) | 10h 21m (621m) |
+| Hands-on | 53m (9%) |
+| Automated agent time (estimated) | ~6h 20m (380m, 61%) |
+| Idle/testing/away (estimated) | ~3h 8m (188m, 30%) |
+| Retro analysis time | 6 min |
+| Cost | $347.53 (495 API calls, 618K output tokens) |
+
+Wall-clock is exact, from the message timestamps. Hands-on is the transcript script's adjusted
+figure. The last two are an estimate of how the 568 minutes that are not hands-on divide, since
+nothing in the transcript marks when the agent stopped working and the user walked away; they are
+sized from the work performed (build and test runs, device measurements, an adb pull of a 267 MB
+database) and add up to the wall clock rather than being measured independently.
+
+13 user messages, most of them one to three words. Only user-message timestamps are exact, so the agent/idle split is estimated from the work performed.
+
+### Key Observations
+- The two longest unattended stretches were the most productive: "build and drive" (30m, root cause established with kernel counters and a control-listener comparison) and "proceed" (130m, a cross-module extraction, built and device-verified). Three-word prompts, high leverage.
+- **The most valuable question came from the user, and should have come from the agent.** "Could we use a different transport?" arrived *after* ADFA-5175 was filed and keep-alive was already being built. The agent's own evidence -- drop rate scaling with connection *rate* -- pointed at "open fewer connections", and `shouldInterceptRequest` was the obvious mechanism. It designed a way to tune the mechanism instead of asking whether the mechanism was needed. Result: a filed ticket whose plan was invalidated a day later, and the keep-alive work stopped after stage 1.
+- Rework was formatting tax and transplant fixups, not logic: three whole-file Spotless reformats (~500 whitespace lines, kept out of behavioral diffs by hand), and 4-5 failed python patch asserts from over-long match anchors.
+- Zero substantive corrections from the user across 13 messages. Steering, not fixing.
+- The device work needed a temporary `android:exported="true"` on HelpActivity to be scriptable at all; it was kept on a throwaway branch and reverted, but it is a recurring cost of driving activities that are (correctly) not exported.
+- The retro script counted the agent's own screenshot reads as user turns. Fixing it moved hands-on **down**, from 57.4 to 52.9 minutes, as predicted -- six phantom turns lose their per-turn buffer and typing time, while their assistant output is re-attributed to the real prompt that caused it, so reading time is unchanged at 41.6 either way. (An earlier version of this entry reported 51 -> 53 and explained the rise; both numbers came from runs against different lengths of a transcript that was still growing, since the script always reads the whole file. Re-run against one fixed slice, the metric can only fall: reading is conserved by construction and the other two components shrink.)
+
+### Feedback
+**What worked:** Autonomy. The long unattended stretches were where the value was.
+**What didn't:** The transport question should have come from the agent, not the user.
+
+### Actions Taken
+
+| Issue | Action Type | Change |
+|-------|-------------|--------|
+| Designed keep-alive to tune a mechanism before asking whether the mechanism could go | CLAUDE.md | "Plan and size before building": new bullet -- when the evidence scales with a rate or volume, check whether the platform can remove the mechanism before planning the tuned version, citing ADFA-5172/5176 |
+| Ratchet reformats risk burying behavioral diffs | CLAUDE.md | Code style: land a whole-file reformat as its own commit, before the behavioral one. Superseded on `stage` by the fuller version in the Spotless paragraph, which also says to reformat first |
+| `ServerConfig`-style defaults that call framework APIs break any new JVM test | learnings.md | Added under Android / Kotlin, with the failure mode (constructor throws before the test body runs) |
+| Testing WebView interception without Robolectric | learnings.md | Added under MockK: `mockkStatic(android.os.Environment::class)` plus a mocked `Uri`, and split the decision from the framework construction |
+| How in-process WebView serving actually behaves | learnings.md | New "Serving content to a WebView" section: interception matches any URL so existing URL spaces need no rewriting; no response decoding; no POST body; no 206; WebView cannot render a PDF |
+| Android system SQLite may lack JSON1 | learnings.md | New "Android system SQLite" section, cross-referenced to ADFA-5179 |
+| Retro script counted screenshot reads as user turns | Skill | `analyze_transcript.py`: filter `[Image: original NxN...]` tool results out of the human role |
+| Bookshelf 500s where SQLite lacks JSON1 | Ticket | ADFA-5179 (Bug), linked to ADFA-5176 |
+| Documentation PDFs render blank in HelpActivity | Ticket | ADFA-5180 (Bug), linked to ADFA-5176 |
+| Tests written just before the API they cover moved | No action | One-off: the risk was flagged and the order was chosen deliberately; cost was ~10 lines of test edits |
+
 ## 2026-08-13 - ADFA-5088: individual Preferences/Plugin Manager tooltips + docdb SQL scripts
 
 ### Time Breakdown