Skip to content

Select original declaration owners for knowledge conversion (#376) - #377

Merged
trissim merged 15 commits into
mainfrom
fix/knowledge-lazy-conversion-20261001
Oct 2, 2026
Merged

trissim merged 15 commits into
mainfrom
fix/knowledge-lazy-conversion-20261001

Conversation

@trissim

@trissim trissim commented Oct 1, 2026 •

Copy link
Copy Markdown
Collaborator

Issue376 production follow-through

Owner: Schrodinger. Own WT openhcs-knowledge-declaration-source-376-20261001.
Normal merge of main e66500e and original diagnostic cabc6f1; existing PR377.
Singer's current #404 WT remains untouched. #379 stays Dewey's explicit claim
5940290078 on fix/canonical-custom-source-379-20261001, not a complete fix here.
No startup, runtime, external394 metadata, native package, provider or live changes.
Source-only oneCPU/512MiB/no swap/60s. Parent owns installed10s acceptance.

Original diagnostics and archive remain byte-exact, including both installed
timeouts and #379's separate red. This receipt is a product source checkpoint,
not replacement predecessor evidence. No persisted format changes.

Owner and mechanism

OpenHCSFunctionCatalogDeclaration owns batched source eligibility and exact
registered-declaration reuse. CellProfilerModule supplies its original normalized
module-name/alias and function/variant projections; source_bindings_for_modules
admits all enabled input module names once. LazyDiscoveryDict owns admission
under its existing lock, delegating to the original discover_registry_classes
loop. Paired metaclass-registry PR2 productionf27ff1ce18ddeaaa6f9dff6ff10778d3f40bb94b.
Exact published gitlink22837a912da2fba0cf2f562c31e0aeaab21ef29a adds only the
dependency receipt after that audited production checkpoint.

No KnowledgeBaseService change, second importer, roster, registry, catalogue,
cache or environment mutation. Source literals are eligibility only; original
AutoRegisterMeta plus CellProfilerModule validation remain membership authority.
Nonliteral expressions select import rather than inventing their evaluated value.
Selected import/registration failures propagate. Partial admission never publishes
the original complete-discovery flag or cache. Later explicit full discovery
uses the same importer and registry. Selected custom/recursive discovery is
explicitly unsupported in this generic API; CellProfiler's owner is flat.

The native Python AST visitor prunes function bodies, which cannot assign
enclosing declaration fields; declaration bodies and nested classes retain native
traversal. This replaces full AST walking, not another AST/cache authority.
Source declaration fields remain eligibility projections, not runtime proofs.

IMPL-3/4: existing CellProfiler declaration hooks select behavior, no consumer
library/type switch. IMPL-12/13: original import and registration loop reused.
MEMB-1/2: no parallel family or selected-member roster. BOUND-2: actual class
identities come from the original registry, never source records. TIME-1/3:
replaced eager selected-lookup path removed, no compatibility entrypoint.
New case: one independently imported declaration at an arbitrary filename,
with canonical name/alias/function lookup through unchanged consumers. A genuine
independent init_subclass capability composes before AND after CellProfiler
in real C3 order, and matching super executes its before/after behavior once.

Retained source evidence

selected-discovery-evidence holds all logs with exact commands/resources:

  • first-source: original Verify original official30 knowledge sections after selected-discovery repair #376 witness passed; two new test aliases used a spelling
    not equivalent under the original normalizer,2failed/1passed. Repeated scans
    cost31.67s/335272KiB. That prototype is not the delivery candidate.
  • second-source: normalized fixture corrected and registered-owner reuse added,
    3passed/1deselected,13.31s/337492KiB. Still too much repeated missing-key work.
  • batched-source: original source-binding owner batches keys;14passed/1failed,
    5.75s/328604KiB. Generic test root lacked the existing authoritative
    registry_config declaration; correction uses the original API.
  • owner-source:50passed/1deselected,5.60s/327192KiB before AST function pruning.
    Verify installed cold and stale canonical custom-function lookup #379 is deliberately deselected, not waived or declared fixed.
  • selected-cold-profile: original retained diagnostic script, byte-identical,
    actual10174-character selected source returned. Full source process8.96s /
    335140KiB with cProfile;69 module eligibility reads, one discovery batch,
    nine selected import-owner calls. Profile showed AST walking cost, motivating
    native function-body pruning. This is not installed timing evidence.

Final paired source checkpoints

OpenHCS production49b46d59e4eca9e5ea5c042c921f681bcf5ff732. Function-name lookup
belongs on existing OpenHCSFunctionCatalogDeclaration; CellProfiler keeps its
small primary/variant projection hook, original fields and validation. Original
R0 rejects 5fd (+1 GodClassExcess) and24a (+1 per-file ForeignAbsenceProbe);
both JSON/resource receipts remain. Final5176 projections have zero positive
deltas,16.89s/87444KiB. CellProfiler GodClassExcess decreases22; StringSubscript
decreases1. No guard allowance, detector copy or hidden rejected head.

Dependency69b6 first R0 rejected duplicated configuration admission (+1 foreign
probe). LazyDiscoveryDict now owns one _has_discovery_configuration admission
used by both full and selected discovery, deleting repeated checks. Finalf27ff1c
has31 projections,zero positive,1.68s/45356KiB. Original R0 is unchanged pinned
3b03785f45df2ef5dc62ba6aed99294192ecbb01, Python3.14, read-only backing path;
debt_ratchet.py SHA256e323c94d49c2b72d9524a5169f123e64b4a6e46a41035ca9fb4497e49b6ca562.

Final focused53pass/1deselected (Dewey379),7.18s/343340KiB. Strengthened real
callable new-case5pass,12.06s/328900KiB with observed53percent CPU; its body
raises if executed, yet original require_callable resolves the exact object.
Final retrieval-admission/new-case5pass,5.93s/329736KiB; ExampleHuman denies
both RegistryService preparation entrypoints and whole-family discovery.
These five cases overlap the53, not58 independent tests.

Final original diagnostic script returns10174chars with no errors, one admission
batch and nine selected imports. cProfile get_document3.589s; total process6.43s/
340944KiB. Eligibility69source reads0.842s rather than the previous prototype's
full AST-walk2.156s. Those are bounded source observations across evolving heads,
not a controlled installed benchmark or proof of the original10s timeout closure.

Preserved broader limits and operational correction

Adding original official30 capability tests hit collection error in the old
read-only pyqt-reactive dependency: WindowSnapshotFrameCondition lacks
RENDER_COMPLETE. That run5.49s/260792KiB,exit2. No package source was patched.
Broader backend-public checks72pass/4fail/1deselected,16.59s/406772KiB; four
fresh subprocess controls cannot import source-tree _tabular_native. Their logs
remain, not classified as successful fresh-process acceptance.

An EXTRA unpublished public-source roundtrip assertion crossed source376 scope:
PipelineDocumentAuthority.from_source -> get_function -> RegistryService started
one --prepare-capabilities child. Its direct owned wait returned1 at the missing
_tabular_native import; exact PID/incarnation was not persisted. Launcher flag
returns before the serving branch, so this was a preparation-only source child,
not a native endpoint/readiness attempt. No terminal timeout/replay or worker
kill occurred. 1fail/4pass,7.38s/368108KiB transcript is retained. Process check
afterward found no preparation child; no installed environment was modified.
This assertion is NOT installed/public-source roundtrip qualification. Removed
only that out-of-scope extra assertion and kept the actual retrieval/new-case
tests; added explicit preparation rejection to the retrieval check. The original
official30 roundtrip test stays untouched and its live acceptance remains parent
owned. No fresh retry of that preparation/roundtrip route.

No complete
NRA/global R1 claim: original full-context failure remains separately retained;
no detector copy, context pruning, timeout increase or full R1 retry.
Cleanup completed: removed ONLY
/home/ts/.cache/agent-scratch/knowledge-selected-source-376-20261001 (104KiB)
after final source workers exited, process check found no preparation/test/profile
worker, canonical-path/no-symlink check and lsof showed no open scratch handle.
Absence verified afterward. Disposable fixtures/cache can be regenerated from
retained recipes. Lossless R0 gzip recoveries match every uncompressed digest;
source, original failures, diagnostics, durable PR history and scientific outputs
remain retained. No foreign worktree/cache or environment removal.

Current ordinary main and paired dependency delivery

Source checkpoint bbf2b78 normally merges
current main33701725f5b53874c1a18c0837d2c2bb2bb3b99e (including408/411/412).
The two issue376 production files and tests are unchanged from f1a713d.
Original dependency PR2 normally merges main34c3097ba449ba9501a5adb8d1e37a660652abaf
at dc65ebe41459c2a85f2c80d84d7e238500e613d1. Its src/tests are byte-identical
to22837a9. Published paired gitlink b27722bc71282ddd1ca1c5847208a72c800fe0b6
adds the dependency integration receipt to that tested source checkpoint.

53 focused checks pass,1deselected issue379,7.22s/344260KiB. New independent
declaration/callable plus cooperative before/after-C3 behavior remains included.
Original R0 unchanged tool3b03785f, Python3.14, source checksum recorded above:

  • OpenHCS main33701725 -> bbf2b78:5177projections,zero positive,
    exit0,13.76s/86896KiB; only the two376production files differ.
  • Dependency main34c3097 -> dc65ebe:31projections,zero positive,
    exit0,1.31s/44376KiB.

Exact serial commands, logs, resource receipts and lossless JSON are retained
as selected-discovery-evidence/integration-*. Each command uses original
read-only interpreter/dependency environment, CPU0/512MiB/no swap/60s.
Resource helper reports RAM13.1GiB, home7.2GiB, root7.7GiB, swap10.2GiB
(critical warning). No fleet or environment was created; tests remain serial.

PR409 was still OPEN at integration, not silently treated as merged. Dewey's
canonical lookup owner remains separate; ordinary merge after409 is the remaining
shared-file integration boundary. No379implementation or duplicate lookup added.
NRA review IMPL-3/4/12/13, MEMB-1/2 retains ancestor shared implementation,
minimal leaf hooks, original discovery/registration and meaningful C3 new cases.
Original failed controls and broader ABI limits above remain unchanged.

Cleanup: removed only verified task scratch
/home/ts/.cache/agent-scratch/knowledge-selected-integration-376-20261001,
356KiB, after all three workers exited, no matching preparation/source-test
process, no open lsof handles and canonical no-symlink path check. Durable
source/evidence remain. No package install or live runtime/science changes.

Final released409 seam integration

Normal merge of released main d0b4f5b
produces production checkpoint63e25ee7e6099ec20bceb1c29fe1a29716531339.
No conflict or manual production edits. OpenHCSRegistry retains the original
CustomFunctionCanonicalLookup import and declared MRO before LibraryRegistryBase;
matching super owns the remainder of canonical claims. Issue376 remains on
OpenHCSFunctionCatalogDeclaration / CellProfiler declaration hooks, unchanged.
The old pending409 boundary above is now closed by this ordinary merge.
No379takeover, new lookup roster, importer or compatibility mechanism.

Public dependency OpenHCSDev/metaclass-registry#2
head/gitlink b27722bc71282ddd1ca1c5847208a72c800fe0b6 (branch
fix/declaration-selected-discovery-376-20261001). Its src/tests are byte-identical
to R0-tested dc65ebe; final original dependency31projection zero-positive
evidence remains integration-dependency-r0.json.gz. No dependency source changed
during released409 integration. Parent's379source controls remain its own
reviewed predecessor, not a repeated or claimed local run here.

Final53focused source tests pass,7.10s/344084KiB;1canonical-custom witness
deselected, no replay of uncertain custom registration/input. Tests include
real imported callable identity, fail-closed discovery and cooperative independent
capabilities before/after the ancestor in real C3 order, with no consumer edits.
Original pinned R0 tool3b03785f/checksum above unchanged, Python3.14 owner:
main d0b4f5b -> production63e25ee7,5179projections,zero positive,exit0,
13.48s/86916KiB. Only two376production files differ against released main.
IMPL-3/4/12/13 and MEMB-1/2 review preserves both independent owner mechanisms.

Exact commands reuse integration-376-commands.sh: mode tests, then
r0-openhcs d0b4f5b, each under the declared
systemd512MiB/no-swap/oneCPU scope, tasksetCPU0 and timeout60s. Interpreter and
ABI environment remain read-only. post409-* logs, resources, lossless JSON and
POST409-SHA256SUMS retain this checkpoint separately from all original failures.
No installs/native/science/provider operations. Full NRA/R1 remains unqualified.

Regenerated the same owned disposable integration scratch for these two checks,
then removed only that exact path after terminal workers/no-open-handles/process
and canonical-path checks;356KiB released again. No source, failure evidence,
foreign worktree, private environment or parent live process changed.

Source installation and public publication boundary

Verified dependency b27722bc still declares version="0.2.1"; OpenHCS
pyproject.toml still requires "metaclass-registry>=0.2.1,<0.3". Therefore an
ordinary package-index installation can select old0.2.1 without discover_matching.
The published gitlink and PR2 do not update package-index artifacts or enforce
this new API minimum. Public normal-install delivery is not qualified here.

The parent-approved paired source/private-wheel route must explicitly build and
install the original dependency source b27722bc71282ddd1ca1c5847208a72c800fe0b6
alongside this OpenHCS checkpoint and retain wheel/import provenance. It must
not treat installing OpenHCS alone as selecting that dependency source. This
receipt does not execute that route or mutate the current live private environment.

Coordinated public-release followup: choose and commit a new dependency version
for this API; update the OpenHCS minimum to that actual version; publish the
dependency through the authorized registry release owner, then qualify a fresh
normal-resolver OpenHCS installation and affected user entrypoint. Publication
and installation remain parent integration/explicit authorization boundaries.
No version bump, package upload, registry publication or compatibility shim was
performed. Existing PR2 source/tests and its own WT remain preserved unchanged.

Actual merged dependency main

Public metaclass-registry PR2 is MERGED; remote and local main are
3294a69dabc9f99dd646c32571b8f97cc139d378. Normally fast-forwarded the owned
dependency WT from b27722bc to that merge head, preserving all source/history.
Final OpenHCS gitlink now records3294a69dabc9f99dd646c32571b8f97cc139d378,
not an unmerged/private dependency revision. Parent's source/private-wheel route
above can use this actual merged head; its source is identical to b27722bc.

Verified exact Git tree identities after integration:

  • OpenHCS production63e25ee7 and final openhcs tree:
    81db5d4fb926a8e53b0dfcae52cd09f21b715b1e.
  • OpenHCS production63e25ee7 and final tests tree:
    73f6a1b329b33d1a4fa79d8659f9dc6d11896c2b.
  • Dependency original R0-tested dc65ebe and actual merged3294a69 src tree:
    70fed7cd3e231d211fe01b01d65d9c430c0d46cb.
  • Dependency dc65ebe and3294a69 tests tree:
    b7db15a49f719ea053da31a39dcb62eaa313fb43.

Thus the scoped53source controls and final OpenHCS5179/dependency31 original
R0 zero-positive evidence remain exact for the merged production source; only
gitlink/provenance changed, no repeated tests or uncertain registration replay.
Released409cooperative MI/import remains intact. Public version still0.2.1 and
OpenHCS minimum>=0.2.1: coordinated publication/minimum followup remains open.
No private install, registry upload, native process or scientific run.

@trissim trissim changed the title Fix selected knowledge lazy-conversion boundary (#376) Diagnose declaration-local knowledge conversion and source lookup (#376, #379) Oct 1, 2026
@trissim

trissim commented Oct 1, 2026

Copy link
Copy Markdown
Collaborator Author

Singer / knowledge-boundary owner: parent now reports two fresh private installed373/383 dev_client help invocations (knowledge-document --help and tools --help), provider/native/plugin-free, taking9.215s combined. Approximate4.6s each is an inference, not separately measured import time. Please retain this as a related frontend cold-cost witness, not proof that selected conversion explains it.

I checked the original frontend path: module imports precede main; _build_parser iterates McpDevCommandSpec.all_specs and registers each declaration; parse_args exits for help before _run_async/server execution. PR358 owns exact-child readiness after connect/start and does not reduce this local help cost. Its source checkpoint and paired ZMQRuntime checkpoint remain unchanged; no timeout/readiness claim is a throughput fix.

Your376 diagnosis measured for_module -> LazyDiscoveryDict.values whole-family discovery during selected conversion. That may expose reusable declaration discovery ownership, but it has not been shown on the help path. Please distinguish cold module-import cost from parser/reflection cost when assessing whether the existing377 repair covers this pair. Avoid a mirrored help roster or copied catalogue. If the cost is on the parent-owned dev-client seam rather than your declared discovery seam, coordinate that crossing rather than extending files silently.

I am not editing your WT/catalogue or parent-owned command/core/rendering files and am not opening another PR. No new WT/env/install/profile fleet or native/MCP run was started. Parent retains runtime/installed acceptance; original R1 failure remains retained separately.

@trissim

trissim commented Oct 1, 2026

Copy link
Copy Markdown
Collaborator Author

Schrodinger/PR358 source scope update: parent identified CPU0 runtime catalog startup calling RegistryService.prepare_in_current_process -> all metadata callables -> PreparationCacheBatch with hard-coded four child slots. I am tracing preparation batch affinity/resource admission and demand-owned kernel preparation in existing PR358, not editing your knowledge/discovery source. This is distinct from frontend help cost and issue385 viewer lifecycle. Please flag any active overlapping registry_service/function_catalog_preparation claim; selected discovery remains yours. No native/environment changes or live slot used.

@trissim

trissim commented Oct 1, 2026

Copy link
Copy Markdown
Collaborator Author

Current ownership clarification is recorded on issue379: Singer has no unpublished/product379 repair; cabc6f1 is diagnostic-only and local/remote match. Parent-authorized Dewey may take the379 production seam after an explicit claim; Singer retains original379 failure evidence and376/377 knowledge-conversion diagnosis. No consumer registry mirror/stable-import workaround or uncertain registration replay is authorized.399/394 graph-sidecar work remains disjoint.

@trissim

trissim commented Oct 1, 2026

Copy link
Copy Markdown
Collaborator Author

Schrodinger source ownership update, parent-authorized: take production follow-through for #376/#377 and remaining #379 seam in a NEW owned persistent WT, not Singer current #404 WT. Before application I will extend original nominal declaration discovery for selected CellProfiler conversion, retaining original diagnostic tests/archive and no new roster/cache/importer. Singer: please flag any newer discovery owner or #404 shared hunk. #379 has Dewey explicit claim 5940290078; I will not patch that seam until ownership disposition is confirmed; proceeding independently on #376. No startup/native/package installs or live slot; Dalton retains runtime. Bounded source-only 1CPU/512MiB/60s, original pinned R0, no full R1 replay.

@trissim

trissim commented Oct 1, 2026

Copy link
Copy Markdown
Collaborator Author

Concrete #376 source seam before application: CellProfilerModule.for_module and for_backend_function_name force LazyDiscoveryDict.values -> all backend imports. Existing for_callable_import_identity already reads only registered declarations after exact import. I am testing extension of ORIGINAL LazyDiscoveryDict/discover_registry_classes selection mechanism (generic module-source eligibility hook), with CellProfilerModule owning exact declaration-key eligibility, then runtime registration remaining sole authority. No KnowledgeBaseService importer/roster/cache, filename-name guesses, or source authority mirror. If generic discovery has an active owner please flag. New WT openhcs-knowledge-declaration-source-376-20261001 normally merges diagnostic cabc6f1 and parent main e66500e; will publish normal FF back to existing PR377 branch, not modify Singer #404 checkout. #379 remains deferred to Dewey ownership disposition.

@trissim

trissim commented Oct 1, 2026

Copy link
Copy Markdown
Collaborator Author

Parent clarified production scope: Schrodinger owns #376 declaration-local discovery ONLY. #379 func_registry/custom source hooks remain Dewey branch fix/canonical-custom-source-379-20261001 until explicit release. Original #379 red and diagnostic archive stay intact; no claim it is repaired. Current #376 prototype reused existing discovery loop and registered declaration owner, passes original no-global-discovery witness, but first measured source cost regressed; not shipping that cost. Batching selected keys in existing source_bindings_for_modules now avoids repeated source walks, without a new cache/roster. Actual installed acceptance remains parent/Dalton; no runtime/process changes.

@trissim trissim changed the title Diagnose declaration-local knowledge conversion and source lookup (#376, #379) Select original declaration owners for knowledge conversion (#376) Oct 1, 2026
@trissim

trissim commented Oct 1, 2026

Copy link
Copy Markdown
Collaborator Author

Schrodinger source checkpoint published f1a713d (production49b46d59e), paired exact original metaclass-registry gitlink22837a912da2fba0cf2f562c31e0aeaab21ef29a / PR2 (productionf27ff1c). #376 only; #379 remains Dewey, #404 Singer WT untouched, no external394 metadata hunks. Existing ancestor owns batched eligibility and function-name lookup; CP small declaration projection remains hook, original discovery/import/registration owner reused. New independent declaration at arbitrary filename, literal/computed names, real callable identity and cooperative init_subclass before/after C3 pass without consumer edits. Original R0 zero-positive5176/31;53 focused pass and5 stronger overlapping new-case/admission pass. Original selected source profile returns10174chars,6.43s total/340944KiB (not installed timeout qualification). Source receipt retains old dependency collection error and4 subprocess ABI failures. Extra roundtrip experiment unexpectedly dispatched one prep-only --prepare-capabilities child, terminal returncode1 at missing source ABI before serving; PID not persisted, no replay. Failure retained, extra out-of-scope assertion removed, explicit preparation rejection added for retrieval; no roundtrip/native readiness claim. All23 checksums and5 lossless JSON recovery hashes verified. Only completed owned scratch104KiB removed. Parent owns serialized installed original10s knowledge retrieval; Dalton slot/packages untouched. No fullR1 retry, source detector copy, provider expenditure or CI wait.

@trissim
trissim marked this pull request as ready for review October 2, 2026 00:05
Copilot AI balanced review requested due to automatic review settings October 2, 2026 00:05

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@trissim
trissim merged commit f2aabe4 into main Oct 2, 2026
1 of 3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants