fix(studio): restore metadata pick-lists, unblock repo vocab reads, correct token docs (v0.70)

.github/workflows/validate-protocol-roundtrip.yml

@@ -73,6 +73,11 @@ jobs:
       - name: Run G6 wire encoder tests
         run: node tests/test-arena-wire-g6.js
 
+      - name: Validate configs/metadata controlled-vocab YAML
+        # These pick-lists are edited in the GitHub UI; a bad indent silently
+        # empties the Arena Studio experimenter dropdown (2026-08-27).
+        run: node tests/test-metadata-yaml.js
+
       - name: Run G6 sequence runner tests
         run: node tests/test-arena-runner-g6.js
 

CLAUDE.md

@@ -226,9 +226,17 @@ fix flows to every page automatically; two hand-written HTML pages never will.
   `fetchCourseGenotypes` / the generic `fetchCourseVocab(file, key, srcId, apply)`.
   **Any NEW controlled vocab MUST follow this course-first, site-fallback pattern:**
   add a site YAML under `configs/metadata/`, load it in `populateMetaDatalists`, AND
-  add a `fetchCourseVocab(...)` call in `refreshCourseMeta` + seed the file into the
-  course repo root. The course repo (`reiserlab/cshl-2026-course`) is PRIVATE, so
-  the course override needs a token; the site files are same-origin (always work).
+  add a `fetchCourseVocab(...)` call in `refreshCourseMeta`, ADD THE FILENAME TO
+  `READABLE_EXACT` in `js/studio-github.js` (the read allowlist — a miss throws inside
+  the loader's try/catch and the override silently falls back; that is exactly how
+  ages/sexes/fly_numbers were broken until v0.70), + seed the file into the repo root.
+  The course override runs only when a token is stored (token + repo = "course-repo
+  mode") even though `reiserlab/cshl-2026-course` has been PUBLIC since 2026-08; the
+  site files are same-origin (always work). `configs/metadata/*.yaml` are parsed by
+  `tests/test-metadata-yaml.js` (pixi + CI) because they get edited in the GitHub UI.
+  Token types: org members use a fine-grained PAT scoped to the one repo; the shared
+  course guest account (`cshl-2026`, an outside collaborator) can only use a CLASSIC
+  token — see `docs/development/data-repo-token-runbook.md`.
 - **Session rig (#135, v0.4):** `Studio.currentRig` (`{name, arenaConfig,
   explicit}`) is THE bench rig for all three views — one top-bar selector,
   locked by default. Always change it via the module block's

arena_studio.html

@@ -2917,7 +2917,7 @@ <h2 id="modalTitle">Import error</h2>
 </div>
 
 <footer id="footer">
-  <span class="foot-left">Arena Studio v0.69 | 2026-07-21 16:46 ET · <a href="https://github.com/reiserlab/webDisplayTools" target="_blank" rel="noopener" title="webDisplayTools on GitHub: source, issues, and release notes (docs/development/arena-studio-release-notes.md)">GitHub</a></span>
+  <span class="foot-left">Arena Studio v0.70 | 2026-09-04 12:14 ET · <a href="https://github.com/reiserlab/webDisplayTools" target="_blank" rel="noopener" title="webDisplayTools on GitHub: source, issues, and release notes (docs/development/arena-studio-release-notes.md)">GitHub</a></span>
   <!-- Course-repo quick-links: open protocols / logs / patterns in a new tab.
        Hrefs built from the configured repo + bench id (updateGhQuickLinks). -->
   <span id="ghQuickLinks" title="Open the course repo on GitHub (new tab)">
@@ -8184,13 +8184,19 @@ <h2 id="modalTitle">Import error</h2>
     e.stopPropagation();
     const signInBtn = $('ghSignInBtn');
     const pat = prompt(
-        'Paste a GitHub fine-grained personal access token for ' + ghTargetRepo().full +
-        '\nFor private read-only access: Contents read. For saving: Contents read/write' +
-        ' + Pull requests read/write. Public protocol/replay reads need no token.\n\n' +
+        'Paste a GitHub personal access token for ' + ghTargetRepo().full + '\n\n' +
+        'Lab / org members: a FINE-GRAINED token — Resource owner = the repo\'s org, ' +
+        'Only select repositories = this repo, Contents: Read and write (Pull requests ' +
+        'read/write only if you save as PR). Shared course account or outside ' +
+        'collaborators: a CLASSIC token (scope public_repo for a public repo, repo for ' +
+        'a private one) — fine-grained tokens cannot target an org you are not a ' +
+        'member of. Public protocol/replay reads need no token.\n\n' +
         'Stored in sessionStorage (cleared when this tab closes). The follow-up ' +
-        '"remember" option keeps it in localStorage — REQUIRED for course bench ' +
-        'setup (a sessionStorage-only token evaporates when the tab closes); ' +
-        'do NOT use it on a personal shared machine.'
+        '"remember" option keeps it in localStorage — REQUIRED on a dedicated bench/rig ' +
+        'computer (a sessionStorage-only token evaporates when the tab closes); ' +
+        'say NO on a shared or borrowed machine. Tokens expire: when the Studio reports ' +
+        'the stored token is no longer valid, create a new one (see ' +
+        'docs/development/data-repo-token-runbook.md).'
     );
     if (!pat) return;
     const token = pat.trim();
@@ -8392,7 +8398,7 @@ <h2 id="modalTitle">Import error</h2>
             signIn.classList.toggle('gh-signin-while-locked', locked);
             signIn.title = locked
                 ? 'Sign in for course-repo read access; repo, bench, and write settings remain instructor-locked'
-                : 'Sign in to GitHub with a fine-grained personal access token';
+                : 'Sign in to GitHub with a personal access token (fine-grained for org members; classic for the shared course account) — see docs/development/data-repo-token-runbook.md';
         }
         if (ghLock) {
             ghLock.textContent = locked ? '🔒' : '🔓';
@@ -14882,7 +14888,7 @@ <h2 id="modalTitle">Import error</h2>
     '#fmOpenCourse': 'Open a protocol this bench saved (or one shared with the course) from the course repo.',
     '#fmSave': 'Save the protocol. Where it goes depends on the GitHub settings below — a local file, a Pull Request, or straight to the course repo.',
     '#fmPromote': 'Copy this bench’s saved protocol (and its patterns) to the course’s shared folder so other benches can use it.',
-    '#ghSignInBtn': 'Sign in with a GitHub token so the Studio can read and write the course repo.',
+    '#ghSignInBtn': 'Sign in with a GitHub token so the Studio can read and write the course repo. Tokens expire — if the Studio says the stored token is no longer valid, make a new one and sign in again.',
     '#ghLock': 'Instructor lock — the repo and bench settings stay fixed during a course.',
     '#ghRepoInput': 'The GitHub repo saves go to (owner/name).',
     '#ghBenchId': 'This bench’s name — saves and run logs are filed under it.',

configs/metadata/people.yaml

@@ -13,5 +13,5 @@ people:
     name: "Hannah Santos"
   - id: loeschef
     name: "Frank Loesche"
- - id: shubhamTR
+  - id: shubhamTR
     name: "Shubham Rathore"

course/cshl-2026/docs/arena-studio.md

@@ -82,8 +82,9 @@ format is:
 - `&advanced=0` forces safe mode on a shared machine.
 - These links always open in the **Run** view.
 
-Loading a protocol from the private course repo needs you to be **signed in to
-GitHub once** on that browser (File ▾ → GitHub). After that, the link just works.
+Loading a protocol from the course repo works signed-out now that the repo is
+public; saving to it still needs the bench to be **signed in to GitHub once** on that
+browser (File ▾ → GitHub). After that, the link just works.
 
 ## What safe mode blocks
 

course/cshl-2026/docs/github-overview.md

@@ -16,10 +16,11 @@ each other's data.
 
 ## The course repo
 
-Everything lives in one repository: **`reiserlab/cshl-2026-course`** (private —
-you need to be added, and Arena Studio needs to be signed in once per browser).
+Everything lives in one repository: **`reiserlab/cshl-2026-course`** (public since
+August 2026 — anyone can read it; writing from Arena Studio still needs the bench to
+be signed in once per browser).
 
-Each of the 7 bench rigs has a **bench id** (`bench00`, `bench01`, …). The repo
+Each of the 7 bench rigs has a **bench id** (`bench01` … `bench07`). The repo
 is organized so no two benches ever overwrite each other's files:
 
 ```

dashboard/data-browser/README.md

@@ -8,7 +8,7 @@ CSHL 2026 course.
 - Opens one or more local JSONL files.
 - Loads a same-origin JSONL URL.
 - Browses a locally served course-repository checkout.
-- Signs in to the private GitHub course repository with the same fine-grained
+- Signs in to the GitHub course repository with the same personal access
   personal-access-token flow and storage keys used by Arena Studio.
 - Indexes run metadata without downloading complete runlogs, then loads full
   files only when selected.
@@ -101,9 +101,9 @@ relative-heading pages instead of failing import.
 
 ## Public page, private data
 
-The dashboard itself can be public while its URL is distributed through private
-course documentation. Private runlogs remain in
-`reiserlab/cshl-2026-course`.
+The dashboard itself is public. Runlogs live in `reiserlab/cshl-2026-course`,
+which was private during the course and has been public since 2026-08 (reads work
+signed-out; writes still need a token).
 
 The sign-in flow uses:
 
@@ -112,8 +112,10 @@ The sign-in flow uses:
 - `Authorization: Bearer <token>` headers to `api.github.com`
 
 The token is never placed in a URL, request body, runlog, CSV export, or console
-message. The course token should be a fine-grained PAT restricted to the course
-repository with Contents read/write access, matching Arena Studio.
+message. The shared course token is a **classic** PAT from the `cshl-2026` guest
+account (scope `public_repo`); lab members use a fine-grained PAT scoped to the one
+repo with Contents read/write — matching Arena Studio. See
+`docs/development/data-repo-token-runbook.md`.
 
 The dashboard currently performs read operations only. Write permission is
 retained on the shared course token so the same sign-in remains compatible with

dashboard/data-browser/github-repo.js

@@ -124,7 +124,7 @@
     async function signIn(repoValue) {
         const repo = parseRepo(repoValue || currentRepo());
         const pat = prompt(
-            `Paste a GitHub fine-grained personal access token for ${repo.full}.\n` +
+            `Paste a GitHub personal access token for ${repo.full} (fine-grained for org members; classic for the shared course account).\n` +
                 'The course token should have Contents read/write access.\n\n' +
                 'It is stored in sessionStorage first. The next prompt can remember it on this browser.'
         );

docs/development/arena-studio-release-notes.md

@@ -4,6 +4,25 @@ The Studio's footer used to carry the full changelog inline; it now shows one li
 history lives here. Newest first. (Per-session engineering detail stays in
 `arena-studio-handover.md` and the design docs — this file is the user-facing what-changed list.)
 
+## v0.70 (2026-09-04) · Metadata pick-lists restored; course age/sex/fly# lists load; token guidance
+
+- **The Experimenter dropdown is populated again when no data repo is signed in.**
+  A GitHub-UI edit to the site library's `configs/metadata/people.yaml` (2026-08-27)
+  left an indentation error; the Studio swallowed the parse failure and showed an
+  empty list, which blocked recorded runs. Fixed, and the metadata files are now
+  parsed in the test suite and CI so a bad edit fails loudly.
+- **Fly age / sex / fly-number lists now really load from the connected repo.** The
+  read allowlist only permitted `roster.yaml` and `genotypes.yaml`, so the three other
+  root vocab files were refused and the site list was silently kept. The opt-in
+  post-run SD pattern archive (`pattern-sets/…`) was blocked by the write allowlist the
+  same way; both are allowed now.
+- **Sign-in text explains which token to use.** Org members: a fine-grained token
+  scoped to the one repo (Contents read/write). Shared course account / outside
+  collaborators: a classic token (`public_repo` for a public repo) — GitHub does not let
+  a non-member own a fine-grained token for an org repo. Also says what an expired token
+  looks like and where the renewal runbook is
+  (`docs/development/data-repo-token-runbook.md`).
+
 ## v0.69 (2026-07-21) · ISP batch retries a failed panel twice
 
 - **A failed panel flash now gets up to two retries** (was one) before being

docs/development/cshl-pipeline-test-plan.md

@@ -81,30 +81,44 @@ in one place and its writes are attributed to a course identity.
 - Org note: if `reiserlab` restricts outside collaborators, an org owner may
   need to approve the invite first.
 
-**1c. Generate the shared fine-grained PAT** (signed in as `cshl-2026`)
-1. Settings → Developer settings → **Fine-grained tokens** → **Generate new
-   token**.
-2. **Token name**: `cshl-2026-benches`.
-3. **Expiration**: set it to just past the course (e.g. the Monday after) so a
-   forgotten token can't linger.
-4. **Resource owner**: `cshl-2026` (the guest account itself).
-5. **Repository access** → **Only select repositories** →
-   `reiserlab/cshl-2026-course`. (If the repo isn't listed, the invite in
-   1b wasn't accepted yet.)
-6. **Permissions** → Repository permissions → **Contents: Read and write**
-   (leave everything else "No access"). Metadata auto-selects read-only —
-   that's fine.
-7. Generate; **copy the `github_pat_…` string now** (shown once). Store it in
-   the shared inbox / a password manager.
-8. This single token is what you paste into each bench in P3. Students never
+**1c. Generate the shared token — it must be a CLASSIC token** (signed in as
+`cshl-2026`)
+
+> **Correction (2026-09-04).** An earlier version of this step described a
+> *fine-grained* token. That cannot work here: `cshl-2026` is an *outside
+> collaborator* of `reiserlab`, and GitHub only lets a fine-grained token name
+> an organization as resource owner when the token's owner is an org *member*
+> ("Only personal access tokens (classic) have write access for public
+> repositories that are not owned by you"). Do NOT make the guest account an
+> org member to work around this — the org grants members write on every repo.
+> The course token that ran the 2026 course was a classic token. Full recipe
+> and the lab-member (fine-grained) alternative:
+> `docs/development/data-repo-token-runbook.md`.
+
+1. Settings → Developer settings → **Personal access tokens → Tokens (classic)**
+   → **Generate new token (classic)**.
+2. **Note**: `cshl-2026-benches`.
+3. **Expiration**: pick a date and put it in the calendar. For the course, just
+   past the course (e.g. the Monday after) so a forgotten token can't linger;
+   for ongoing post-course use, ≤ 1 year.
+4. **Scopes**: `reiserlab/cshl-2026-course` is **public** (since 2026-08), so tick
+   only **`public_repo`** (read/write code on public repos). Only if the repo is
+   ever made private again would the full **`repo`** scope be needed.
+5. Generate; **copy the `ghp_…` string now** (shown once). Store it in the lab
+   password manager. (The account itself was registered to Michael's Janelia
+   address, not a separate shared inbox — password resets go there.)
+6. This single token is what you paste into each bench in P3. Students never
    see it.
 
-**1d. Revoke path** (know it before the course): signed in as `cshl-2026` →
-Developer settings → Fine-grained tokens → the token → **Revoke**. All benches
-stop writing immediately; issue a new one and re-paste per P3.
+**1d. Revoke / renew path**: signed in as `cshl-2026` → Developer settings →
+Tokens (classic) → the token → **Delete** (or **Regenerate** to get a new value
+with a new expiry). All benches stop writing immediately — the Studio shows
+"The stored GitHub token is no longer valid. Sign in again from File ▾." on the
+next load — issue a new one and re-paste per P3.
 
 ### P2. Repo — DONE
-- `reiserlab/cshl-2026-course` exists (private), seeded with `README.md`,
+- `reiserlab/cshl-2026-course` exists (private during the course; made **public**
+  in 2026-08 — reads no longer need a token, writes still do), seeded with `README.md`,
   `roster.yaml` (test entries: michael/frank/isabel/hannah_marie + guest) and a
   course `genotypes.yaml` (lab set + wild-type + none), plus the
   `protocols/shared/` + `runlogs/` skeleton. Edit `roster.yaml` for the real