fix: accuracy pass on the public surfaces before sharing (v3.4.0.1)

.claude-plugin/plugin.json

@@ -1,6 +1,6 @@
 {
   "name": "idstack",
-  "version": "3.4.0.0",
+  "version": "3.4.0.1",
   "description": "Evidence-based instructional design pipeline. 11 skills backed by peer-reviewed research across 11 domains. Analyze, design, build, review, and export courses with every recommendation citing its evidence tier.",
   "author": {
     "name": "Philippos Savvides",

CHANGELOG.md

@@ -1,5 +1,33 @@
 # Changelog
 
+## v3.4.0.1 (2026-08-07)
+
+Accuracy pass over the public surfaces before sharing the project more widely. No skill behavior changes.
+
+To update: `cd` into your idstack clone, then `git pull && ./setup`.
+
+### Fixed — three evidence cards claimed better evidence than idstack holds
+
+- **Two cards on idstack.org advertised T2 for domains whose strongest reference is T3**, and a third advertised T5 for a domain that bottoms out at T4. Needs Analysis showed `T2–T5` when all 7 of its references are T3; Evaluation Models showed `T2–T5` against an actual `T3–T5`; Online Course Quality Frameworks showed `T1–T5` against an actual `T1–T4`. The other eight cards were correct, and every study count already matched.
+
+  Overstating a tier is the one inaccuracy this project cannot ship — labelling evidence honestly is the entire claim. The cards are no longer hand-maintained: `test/check-evidence-cards.py` derives every count and tier span from `evidence/references.md` and smoke-test fails on any disagreement, naming the domain and flagging when a card overstates. Two mutations pin it, one per direction (tier drift, count drift).
+
+### Fixed — PRIVACY.md did not disclose two outbound calls
+
+- **`/idstack:course-export` uploads to Canvas and the privacy policy never said so.** PRIVACY.md stated "No data is sent to external servers by idstack" and carved out only `/idstack:course-import` *fetching* from Canvas. But `course-export` POSTs modules, pages, assignments, and discussions to the Canvas instance you point it at. That is your institution's server rather than ours, and it only runs when you invoke the skill and confirm the target course, but an undisclosed upload in a privacy policy is a defect regardless of where the bytes land.
+- **The hourly update check was also undisclosed.** `bin/idstack-update-check` runs `git fetch` against this repository. It carries nothing but the fetch and never touches course data, and it only runs for git installs, but it is a network call and now says so.
+- Both are listed under Third-party services. Audited by enumerating every outbound host across `bin/`, `templates/`, and `skills/`: the Canvas API and that `git fetch` are the only two.
+
+### Fixed — the Windows install path could not work
+
+- **`README.md` told Windows users to open PowerShell and run `./setup`.** `setup` is an extensionless bash script; PowerShell cannot execute it, and no `.ps1` ships. The ZIP instructions now name the shell that works (WSL or Git Bash) and say why PowerShell does not. This lived in the collapsed "Download ZIP" block, the path written for people who cannot use `git clone`.
+
+### For contributors
+
+- `ROADMAP.md` claimed "18 mutations" against an actual 24. The count is gone rather than corrected, so it cannot drift a third time; run the suite to see the number.
+- `TODOS.md`'s landing-page demo item described a "See it work" transcript section that a redesign had already removed, so the task read as an upgrade to something that no longer existed. Rewritten against the page as it stands.
+- v3.4.0.0 was merged but never tagged, so `.github/workflows/release.yml` never fired and GitHub still listed v3.3.0.4 as the latest release. Tagged retroactively at its merge commit.
+
 ## v3.4.0.0 (2026-08-06)
 
 idstack is a Claude Code plugin now, and only that. The OpenAI Codex CLI target that shipped in v2.5.0.0 is removed, and the Gemini CLI target that was planned is off the roadmap. Nothing about the 11 skills, the evidence base, the manifest schema, or the report contract changes.

PRIVACY.md

@@ -20,13 +20,20 @@ Designer profile and cross-project learnings are stored locally:
 - `~/.idstack/profile.yaml` — your experience level preference
 - `~/.idstack/global/learnings.jsonl` — learnings promoted across projects
 
-No data is sent to external servers by idstack. No analytics, no tracking, no telemetry.
+idstack adds no analytics, no tracking, and no telemetry. Two things do reach the network, both only when you ask for them, and both are listed under Third-party services below: the Canvas API calls made by `/idstack:course-import` and `/idstack:course-export`, and an update check against this repository on GitHub.
 
 ## Third-party services
 
 idstack runs inside Claude Code, which is operated by Anthropic. Your conversation with Claude Code is subject to [Anthropic's privacy policy](https://www.anthropic.com/privacy). idstack itself does not add any data collection beyond what Claude Code already does.
 
-If you use `/idstack:course-import` with the Canvas API, your Canvas access token is used locally to fetch course data. idstack does not store or transmit your token beyond the API call.
+**Canvas API.** Two skills talk to the Canvas instance you point them at, using the access token you supply:
+
+- `/idstack:course-import` **downloads** your course data from Canvas.
+- `/idstack:course-export` **uploads** generated course content to Canvas — modules, pages, assignments, and discussions are POSTed to your Canvas instance so they appear in the course.
+
+Both run only when you invoke that skill and confirm the target course. Your token is used for those API calls and is neither stored by idstack nor sent anywhere else. The receiving Canvas instance is your institution's, not ours.
+
+**Update check.** On skill startup idstack runs `git fetch` against this repository to see whether a newer version exists, at most once an hour. That is a request to GitHub carrying nothing but the fetch itself; it never uploads your course data. It only runs for git installs, and removing the repo's `.git` directory disables it.
 
 ## Questions
 

README.md

@@ -123,7 +123,8 @@ Claude clones the repo, runs setup, and confirms the skills are registered.
 
 1. Click the green "Code" button on GitHub, then "Download ZIP"
 2. Unzip the file
-3. Open Terminal (Mac) or PowerShell (Windows)
+3. Open a bash shell: Terminal on macOS or Linux; WSL or Git Bash on Windows.
+   `./setup` is a bash script, so PowerShell and `cmd` cannot run it.
 4. Navigate to the folder: `cd path/to/idstack`
 5. Run: `./setup`
 </details>

ROADMAP.md

@@ -21,7 +21,7 @@ What's coming next for idstack. Priorities are shaped by user feedback. [Tell us
 - The suite had never run automatically. GitHub Actions runs every suite on each push and pull request, across ubuntu (Python 3.9 + 3.12) and macOS — eight suites at v3.3.0.0, ten today.
 - `./setup` — the primary deliverable — went from zero coverage to 17 behavioral tests.
 - `bin/idstack-doctor` and `bin/idstack-status --readiness` gained their first execution coverage in v3.3.0.4. Both are what a user reaches for when something has already gone wrong, and neither had any.
-- `test/mutation-test.sh` reintroduces each fixed defect and asserts its guarding test fails, which is how a test that only appeared to test something gets caught. 18 mutations, all guarded.
+- `test/mutation-test.sh` reintroduces each fixed defect and asserts its guarding test fails, which is how a test that only appeared to test something gets caught. Every mutation in it is guarded; the suite prints the current count when you run it.
 
 ### Install through the Claude Code plugin marketplace (v3.2.0.0)
 - `./setup` registers idstack as a Claude Code plugin marketplace and installs from there. Recent Claude Code versions stopped discovering plugins from the bare symlink older setups created, so `/idstack:<skill>` commands silently never appeared in the slash picker. If that happened to you, pull the latest and re-run `./setup`.

TODOS.md

@@ -90,12 +90,13 @@ vs project-local learnings interact.
 **Priority:** P2
 
 ## v2: Interactive Landing Page Demo
-Upgrade the "See it work" section on idstack.org from static text to an interactive
-pre-install demo. The current section shows a text transcript of /course-import and
-/course-quality-review. A richer version could include: animated terminal replay
-(CSS-only, building on the existing typing animation), clickable pipeline explorer
-showing what each of the 11 skills does, or expanded sample output with evidence
-citations. The goal is to let prospective users experience the value before installing.
+Give idstack.org a pre-install demo. The redesign removed the old "See it work"
+transcript section, so today a visitor sees the evidence cards, the pipeline diagram,
+and a sample report excerpt, but never watches a skill run. Options: animated terminal
+replay (CSS-only, building on the existing typing animation), a clickable pipeline
+explorer showing what each of the 11 skills does, or expanded sample output with
+evidence citations. The goal is to let prospective users experience the value before
+installing.
 **Why:** CEO review killed a post-install /demo command because IDs always have a course.
 The real TTHW problem is pre-install: visitors on idstack.org need to understand the
 value before committing to install. The landing page demo is where that conversion happens.

VERSION

@@ -1 +1 @@
-3.4.0.0
+3.4.0.1

docs/index.html

@@ -902,7 +902,7 @@
           <span class="dot" aria-hidden="true"></span>
           <span>MIT</span>
           <span class="dot" aria-hidden="true"></span>
-          <span>v3.4.0.0 — Claude Code only</span>
+          <span>v3.4.0.1 — Claude Code only</span>
         </p>
         <h1 id="hero-title">Evidence-based instructional design, transparently cited.</h1>
         <p class="lede">108 peer-reviewed studies across 11 research domains. Every recommendation tagged with its evidence tier — <span class="tier tier-1">T1</span> meta-analyses and RCTs through <span class="tier tier-5">T5</span> expert opinion. Runs in Claude Code.</p>
@@ -950,7 +950,7 @@ <h3 class="evidence-card-title">Constructive Alignment &amp; Learning Objectives
         <article class="evidence-card" role="listitem">
           <h3 class="evidence-card-title">Needs Analysis</h3>
           <p class="evidence-card-body">Three layers — organizational, task, learner — analyzed before any content gets written. When this step is skipped, courses tend to teach what the designer found interesting rather than what the audience came for.</p>
-          <p class="evidence-card-meta">7 studies<span class="meta-sep" aria-hidden="true"></span><span class="meta-tier">T2–T5</span></p>
+          <p class="evidence-card-meta">7 studies<span class="meta-sep" aria-hidden="true"></span><span class="meta-tier">T3</span></p>
         </article>
 
         <article class="evidence-card" role="listitem">
@@ -980,7 +980,7 @@ <h3 class="evidence-card-title">Learner Analysis &amp; Differentiation</h3>
         <article class="evidence-card" role="listitem">
           <h3 class="evidence-card-title">Evaluation Models</h3>
           <p class="evidence-card-body">Evaluation only measures what it was designed to measure, so the design has to happen alongside the course. Kirkpatrick's four levels (reaction, learning, behavior, results) map what's possible; most courses report only the first.</p>
-          <p class="evidence-card-meta">8 studies<span class="meta-sep" aria-hidden="true"></span><span class="meta-tier">T2–T5</span></p>
+          <p class="evidence-card-meta">8 studies<span class="meta-sep" aria-hidden="true"></span><span class="meta-tier">T3–T5</span></p>
         </article>
 
         <article class="evidence-card" role="listitem">
@@ -992,7 +992,7 @@ <h3 class="evidence-card-title">Rapid Prototyping &amp; Design-Based Research</h
         <article class="evidence-card" role="listitem">
           <h3 class="evidence-card-title">Online Course Quality Frameworks</h3>
           <p class="evidence-card-body">Quality Matters' eight standards and the Community of Inquiry framework (teaching, social, cognitive presence) are the two rubrics with the most published validation studies behind them. idstack's quality audit runs against both.</p>
-          <p class="evidence-card-meta">10 studies<span class="meta-sep" aria-hidden="true"></span><span class="meta-tier">T1–T5</span></p>
+          <p class="evidence-card-meta">10 studies<span class="meta-sep" aria-hidden="true"></span><span class="meta-tier">T1–T4</span></p>
         </article>
 
         <article class="evidence-card" role="listitem">
@@ -1146,10 +1146,11 @@ <h2 id="whatsnew-title">v3.4.0.0 — idstack is a Claude Code plugin.</h2>
       </header>
 
       <article class="release-card">
-        <p class="release-meta">August 6, 2026 · v3.4.0.0</p>
+        <p class="release-meta">August 6, 2026 · v3.4.0.0 · patched through v3.4.0.1</p>
         <h3>One host, supported properly.</h3>
         <p>idstack shipped an OpenAI Codex CLI build from v2.5.0.0 through v3.3.0.4. It is removed. Every skill, the evidence base, the project manifest, and the HTML reports are unchanged — idstack now targets Claude Code and nothing else, which is the only place it was ever tested end to end.</p> <!-- IDSTACK_CLI_LEAK_ALLOW: dated release note, not a capability claim -->
         <p>To update: <code>cd</code> into your idstack clone, then <code>git pull &amp;&amp; ./setup</code>. If you had the Codex build installed, <code>./setup</code> can no longer clean it up — the <a href="https://github.com/savvides/idstack/blob/main/CHANGELOG.md" rel="noopener">release notes</a> carry the two paths to remove.</p> <!-- IDSTACK_CLI_LEAK_ALLOW: dated release note, not a capability claim -->
+        <p>Patched in v3.4.0.1 (August 7): three evidence cards on this page overstated their domain's evidence tier. They are derived from <code>evidence/references.md</code> now, and the test suite fails if they ever disagree again.</p>
       </article>
 
       <article class="release-card">

test/check-evidence-cards.py

@@ -0,0 +1,130 @@
+#!/usr/bin/env python3
+"""Assert the landing page's evidence cards agree with evidence/references.md.
+
+docs/index.html carries one card per research domain, each restating two facts
+from evidence/references.md: how many studies the domain holds, and the span of
+evidence tiers across them. Both were hand-typed, and three had drifted by
+v3.4.0.0 — two cards advertised T2 for domains whose strongest reference is T3.
+
+Overstating tier strength is the one inaccuracy idstack cannot afford: the
+product's entire claim is that it labels evidence honestly. So the cards are
+derived from the reference file here rather than trusted.
+
+Usage: check-evidence-cards.py <repo-root>
+Prints one line per mismatch and exits 1; prints nothing and exits 0 when clean.
+Runs on Python 3.9 (the macOS system interpreter) — see test/test-preamble-python.sh
+for why that floor matters.
+"""
+
+import html
+import io
+import os
+import re
+import sys
+
+# A domain heading: "## Domain 7: Learner Analysis & Differentiation"
+DOMAIN_RE = re.compile(r"^## Domain \d+: (.+?)\s*$")
+# A reference key: "[Alignment-14]" — one per study.
+CITATION_RE = re.compile(r"\[[A-Za-z][A-Za-z-]*-\d+\]")
+TIER_RE = re.compile(r"\bT([1-5])\b")
+# One card: title, study count, tier range. Order-independent; matched by title.
+CARD_RE = re.compile(
+    r'evidence-card-title">(.*?)</h3>'
+    r".*?"
+    r'evidence-card-meta">(\d+)\s+studies'
+    r".*?"
+    r'meta-tier">(.*?)</span>',
+    re.S,
+)
+
+
+def tier_span(tiers):
+    """Render a sorted tier set the way the cards do: 'T3' or 'T1-T4' (en dash)."""
+    lo, hi = tiers[0], tiers[-1]
+    if lo == hi:
+        return "T%d" % lo
+    return "T%d–T%d" % (lo, hi)
+
+
+def parse_domains(path):
+    text = io.open(path, encoding="utf-8").read().split("\n")
+    starts = []
+    for i, line in enumerate(text):
+        m = DOMAIN_RE.match(line)
+        if m:
+            starts.append((i, m.group(1)))
+    starts.append((len(text), None))
+
+    domains = []
+    for k in range(len(starts) - 1):
+        begin, name = starts[k]
+        end = starts[k + 1][0]
+        body = "\n".join(text[begin:end])
+        tiers = sorted(set(int(t) for t in TIER_RE.findall(body)))
+        domains.append((name, len(CITATION_RE.findall(body)), tiers))
+    return domains
+
+
+def main():
+    if len(sys.argv) != 2:
+        print("usage: check-evidence-cards.py <repo-root>")
+        return 2
+    root = sys.argv[1]
+    refs = os.path.join(root, "evidence", "references.md")
+    landing = os.path.join(root, "docs", "index.html")
+
+    for p in (refs, landing):
+        if not os.path.isfile(p):
+            print("missing file: %s" % p)
+            return 1
+
+    domains = parse_domains(refs)
+    if not domains:
+        print("no '## Domain N:' sections found in evidence/references.md")
+        return 1
+
+    cards = CARD_RE.findall(io.open(landing, encoding="utf-8").read())
+    by_title = {}
+    for raw_title, count, tier in cards:
+        by_title[html.unescape(raw_title).strip()] = (int(count), tier.strip())
+
+    problems = []
+    if len(cards) != len(domains):
+        problems.append(
+            "card count %d != domain count %d — a domain was added or removed "
+            "without updating docs/index.html" % (len(cards), len(domains))
+        )
+
+    for name, count, tiers in domains:
+        if name not in by_title:
+            problems.append("no evidence card for domain %r" % name)
+            continue
+        card_count, card_tier = by_title.pop(name)
+        if card_count != count:
+            problems.append(
+                "%s: card says %d studies, references.md has %d"
+                % (name, card_count, count)
+            )
+        if not tiers:
+            problems.append("%s: references.md lists no evidence tiers" % name)
+            continue
+        want = tier_span(tiers)
+        if card_tier != want:
+            note = ""
+            if int(card_tier[1]) < tiers[0]:
+                note = "  <- OVERSTATES: claims stronger evidence than the domain holds"
+            problems.append(
+                "%s: card says %s, references.md spans %s%s"
+                % (name, card_tier, want, note)
+            )
+
+    for leftover in sorted(by_title):
+        problems.append("evidence card %r matches no domain in references.md" % leftover)
+
+    for line in problems:
+        print(line)
+    return 1 if problems else 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())