Fix concept reassignment discarding and recreating unchanged BC entities

[pendingintent]

Summary

Reassigning the same Biomedical Concept code(s) to an activity via POST /soa/{soa_id}/activities/{activity_id}/concepts deleted every activity_concept row for that activity and recreated it with a fresh UID — even when nothing had actually changed. Because the next UID was computed as MAX(live rows)+1 immediately after the activity's own row was deleted, this could reissue the exact number that was just freed. The subsequent orphan-cleanup step then unconditionally deleted the biomedical_concept row for that UID, wiping out the freshly recreated entity (and its properties) moments after creation, and leaving the activity's concept assignment pointing at nothing.

This surfaced while investigating a real report: after reassigning concept C114209 to a study's RANDOMIZATION activity, its USDM export showed the Biomedical Concept with no properties. Root-caused, reproduced, and fixed against the live data — confirmed the fix makes an unchanged reassignment a byte-for-byte no-op (same activity_concept.id and biomedical_concept.id before/after).

Changes

• set_activity_concepts now reconciles instead of replacing (src/soa_builder/web/routers/activities.py): diffs the requested codes against the activity's current assignment. Codes that are unchanged keep their row and UID completely untouched — no delete, no reinsert, no new UID, no re-triggered CDISC Library enrichment/property-population background tasks. Only genuinely added/removed codes are touched.
• Durable uid_counter table (new migration in migrate_database.py): all 11 get_next_*_uid generators in utils.py now draw from a persistent per-(soa_id, prefix) counter instead of recomputing MAX(live rows)+1 on every call, so a freed number is never reissued — even when rows are genuinely removed and re-added within the same request. Each generator lazily seeds the counter once from its original max-scan logic, so existing SoAs pick up exactly where their current numbering left off.
• Guarded _cleanup_orphaned_concept_rows (app.py): won't delete a biomedical_concept row that's still referenced by a live activity_concept row — defense-in-depth alongside the two fixes above.
• Removed a dead, unreachable duplicate of set_activity_concepts in app.py (shadowed at the HTTP layer by the router version, confirmed via routing order + live testing). It carried the same bug and never scheduled the biomedical-concept-property background population task at all — so concepts assigned via the HTML form UI (ui_set_activity_concepts, which called this dead copy directly) never got their properties populated at all. The UI form handler now calls the fixed router implementation directly.

Test plan

• pytest — full suite passes (668 tests), including two new regression tests: reassigning an unchanged code is a true no-op, and adding+removing codes in one call preserves the kept code's UID.
• Fixed two pre-existing tests for this endpoint that posted the wrong request-body shape and only asserted status_code in (200, 422) (i.e. never actually verified success).
• flake8 clean on all changed files.
• Verified uid_counter migration is idempotent (created once, no errors, no duplicate rows on a second run against a DB copy).
• Verified live against the originally-reported production data: reassigning C114209 to the RANDOMIZATION activity is now a no-op (activity_concept.id and biomedical_concept.id identical before/after), with its 3 properties intact.

🤖 Generated with Claude Code

[pendingintent]

@claude tests failed on the latest push