fix: accuracy pass on the public surfaces before sharing (v3.4.0.1) - #69
Merged
Conversation
Three evidence cards on idstack.org claimed better evidence than the repo holds. Needs Analysis showed T2-T5 when all 7 of its references are T3; Evaluation Models showed T2-T5 against an actual T3-T5; Online Course Quality Frameworks showed T1-T5 against an actual T1-T4. The other eight cards were right and every study count already matched. Overstating a tier is the one inaccuracy this project cannot ship, so the fix is not the three numbers. test/check-evidence-cards.py derives every count and tier span from evidence/references.md, smoke-test fails on any disagreement and names the domain, and it flags the overstating direction specifically. Two mutations pin it, one per failure axis. PRIVACY.md said "No data is sent to external servers by idstack" and then carved out only course-import fetching from Canvas. Two calls were undisclosed: course-export POSTs modules, pages, assignments and discussions to your Canvas instance, and idstack-update-check runs git fetch against this repo hourly. Both documented. Verified complete by enumerating every outbound host across bin/, templates/ and skills/ -- the Canvas API and that git fetch are the only two. README's ZIP install told Windows users to open PowerShell and run ./setup. It is an extensionless bash script and no .ps1 ships, so that path could never work. Now names WSL or Git Bash and says why. Also: ROADMAP's "18 mutations" was 24, so the count is gone rather than corrected; TODOS described a landing section a redesign had removed. smoke-test 353 -> 354, mutations 22 -> 24. All ten suites green. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Share-readiness audit found three things wrong on public surfaces. Fixed, each with a guard so it cannot come back.
Three evidence cards claimed better evidence than the repo holds
The other eight were correct and every study count already matched (they sum to 108).
Overstating a tier is the one inaccuracy this project cannot ship, so the fix is not the three numbers.
test/check-evidence-cards.pyderives every count and tier span fromevidence/references.md; smoke-test fails on disagreement, names the domain, and flags the overstating direction specifically. Two mutations pin it, one per failure axis (tier drift, count drift).PRIVACY.md did not disclose two outbound calls
It said "No data is sent to external servers by idstack" and carved out only
course-importfetching from Canvas./idstack:course-exportPOSTs modules, pages, assignments and discussions to your Canvas instance (skills/course-export/SKILL.md:813onward).bin/idstack-update-checkrunsgit fetchagainst this repo, at most hourly.Both now documented under Third-party services. Verified complete by enumerating every outbound host across
bin/,templates/andskills/— the Canvas API and thatgit fetchare the only two.The Windows install path could not work
README.mdtold Windows users to open PowerShell and run./setup. It is an extensionless bash script and no.ps1ships, so PowerShell cannot execute it. Now names the shell that works (WSL or Git Bash) and says why PowerShell does not.Also
ROADMAP.mdclaimed "18 mutations" against an actual 24. Count removed rather than corrected so it cannot drift a third time.TODOS.mddescribed a "See it work" landing section that a redesign had already removed.release.ymlnever fired. Tagged retroactively atb795d02; the release is now live.Verification
smoke-test.shintegration-test.shtest-setup.shtest-doctor.shtest-status.shtest-manifest-merge.shtest-version-classifier.shtest-plugin-status.shtest-preamble-python.shmutation-test.sh🤖 Generated with Claude Code