WIP: align File::Temp stat modes and investigate goto sub cleanup

dev/design/goto-tailcall-parity-handoff.md

@@ -0,0 +1,236 @@
+# `goto &sub` parity handoff
+
+## Objective
+
+Complete Perl-compatible `goto &sub` behavior on the JVM and bytecode interpreter backends. The acceptance target is every applicable assertion in `perl5_t/t/op/goto-sub.t`, with identical JVM and interpreter results.
+
+## Current state
+
+Both backends use `RuntimeCode.resolveTailCalls()` for tail-call marker dispatch. Named-target `AUTOLOAD`, chained calls, recursion, localized/replaced `@_`, and absent ARRAY slots after `undef *_` and `local *_` pass on both backends.
+
+The two UAT regressions are resolved on both backends:
+
+- `RuntimeCode.resolveTailCalls()` drains only deferred referents represented
+  by the marker's ownership carrier after a completed replacement call, so
+  destructor ordering is correct without touching borrowed caller values.
+- Eval-scoped tail-call markers are resolved inside the bytecode interpreter's
+  active eval boundary. This preserves the named-undefined-target-before-eval
+  diagnostic order and lets the catcher populate `$@`.
+- Dynamic `goto $coderef` now carries compile-time eval scope in its bytecode
+  operand rather than inferring eval-string provenance from a caller's runtime
+  eval depth. A normal sub can therefore tail-call through a tied coderef when
+  invoked by an eval block.
+- Dynamic tail calls retain their saved coderef identity. Only markers emitted
+  for literal named gotos perform a fresh symbol lookup, so a wrapper using
+  `goto &$original_stub` cannot recurse through its replacement CODE slot.
+
+Completed since the initial handoff:
+
+- Tail-call markers carry explicit named-source identity, deferred eval scope, and the original `@_` container.
+- A named target is freshly resolved after source-frame cleanup; core assertion 2 now passes with the required `Goto undefined subroutine ... at file line N` form.
+- Literal `@_` uses the live/current-frame-localized argument container in both emitters; sparse `$_[0]` reification now passes core assertion 24.
+- `src/test/resources/unit/goto_tailcall_cleanup.t` passes on system Perl, JVM, and interpreter with 60-second process timeouts.
+
+Focused validation now passes on system Perl, JVM, and interpreter:
+`goto_tailcall_cleanup.t`, `goto_named_redefinition.t`, all 44 assertions in
+`goto-sub.t`, and all four assertions in `uni/goto.t`. The DBIC lifecycle
+regression `refcount/dbic_try_tiny_goto_schema_backref.t` also passes on
+system Perl, JVM, and interpreter. A successful immutable full `make` gate
+remains required before the PR can be updated.
+
+## Required implementation
+
+### Tail-call marker
+
+Extend `ControlFlowMarker` / `RuntimeControlFlowList` so a TAILCALL marker stores the original coderef, optional named-symbol identity, source file/line, the unchanged `RuntimeArray` argument container, and ownership of marker-created aliases.
+
+Only a source `goto &name` or `goto &Pkg::name` supplies named-symbol identity. Do not infer it from `RuntimeCode.packageName` / `subName`: anonymous closures and dynamic coderefs can carry those fields. Preserve normal behavior for coderefs, globs, strings, overload, `AUTOLOAD`, and eval.
+
+Do not validate in the marker constructor. After the source frame cleanup, `RuntimeCode.resolveTailCalls()` must freshly look up an explicitly named marker target through normal dispatch, preserving `AUTOLOAD`. For an undefined target, use marker file/line and the exact form:
+
+```
+Goto undefined subroutine &Pkg::name at file line N
+```
+
+Undefined-target handling precedes any eval-scope error.
+
+### Argument ownership
+
+Always pass the actual current or current-frame-localized `@_` container; never clone it for cleanup control. Audit `RuntimeCode.apply(..., "tailcall", ...)` and `RuntimeCode.resolveTailCalls()` together: each currently has tail-call cleanup paths. Assign cleanup ownership to one layer, consume each marker once, and release only marker-owned temporary aliases after the target returns or yields its next marker. This must fix core assertions 7 and 9 without changing aliasing.
+
+### Sparse arguments and ARRAY slots
+
+Trace `RuntimeArray.getTailCallArrayOfAlias()`, `RuntimeCode.getGotoArgs()`, and `utf8::encode`. A sparse `@_` created with `$#_++` must remain the same container through the handoff, so `utf8::encode($_[0])` reifies its missing element as `""`.
+
+Keep glob slot reads non-vivifying: reads of `*glob{ARRAY}` use a peek; writes and true array dereferences may create a slot. Do not use slot vivification to solve sparse-argument reification.
+
+## Regression coverage
+
+Keep and extend project-owned tests under `src/test/resources/unit` for the exact named-target cleanup diagnostic, repeated destructor ordering, sparse `@_` reification through `utf8::encode`, deferred named-target `AUTOLOAD`, and absent ARRAY slots after `undef *_` / `local *_`.
+
+Run new or modified Perl tests with system Perl first. Do not alter existing core tests.
+
+## Validation
+
+Capture complete output to files and wrap every `jperl` invocation in `timeout`.
+
+1. Run focused unit tests on system Perl, JVM, and interpreter.
+2. Run `perl5_t/t/op/goto-sub.t` on JVM and interpreter; require no `not ok` lines and normal exit.
+3. Run relevant `goto`, subroutine, typeglob, and UTF-8 tests on both backends.
+4. Update `docs/about/changelog.md` under `## Work in progress` when runtime behavior is complete.
+5. On an immutable final commit, run `make`, inspect its complete log, then update PR #1205 and monitor CI before UAT.
+
+## Progress Tracking
+
+### Current Status: UAT regression fixed; final PR validation pending (2026-09-02)
+
+### Completed Phases
+
+- [x] Marker identity and late named-target lookup
+  - Files: `ControlFlowMarker.java`, `RuntimeControlFlowList.java`, `RuntimeCode.java`, JVM and bytecode emitters.
+- [x] Live `@_` handoff for literal `goto &name`
+  - Files: `CompileOperator.java`, `BytecodeInterpreter.java`, `RuntimeArray.java`.
+  - Core sparse argument assertion 24 passes on both backends.
+- [x] Tail-call cleanup timing and exercised eval restriction diagnostics
+  - `RuntimeCode.resolveTailCalls()` flushes deferred source-frame decrements
+    after consuming marker ownership and emits the exact eval diagnostics.
+  - Core `goto-sub.t` assertions cover the eval-string diagnostic; the focused
+    regression covers target lookup and repeated destructor ordering.
+- [x] Direct JVM eval-string trampoline and sparse argument handoff
+  - `evalStringWithInterpreter()` now resolves tail-call markers at the eval
+    execution boundary, matching `EvalStringHandler`.
+  - `CompileOperator` recognizes a list-wrapped literal `@_` and preserves the
+    live argument container for bytecode tail calls.
+  - Expanded `goto_tailcall_cleanup.t` covers eval strings, late `AUTOLOAD`,
+    sparse argument reification, and absent ARRAY slots after `undef *_` and
+    `local *_` on system Perl, JVM, and interpreter.
+- [x] Top-level anonymous-coderef invocation
+  - `Variable.parseCoderefVariable()` now lowers bare `&{sub {...}}` to an
+    auto-call sharing `@_`, while `\&{sub {...}}` remains reference-taking.
+  - Added `top_level_coderef_call.t`; system Perl, JVM, and interpreter pass
+    invocation side-effect, scalar-return, and reference-taking assertions.
+- [x] Rebase regression repair
+  - Tail-call scope cleanup now drains only the retired frame's mortal entries;
+    it no longer releases caller-owned deferred `Sub::Quote` metadata.
+  - Restored refcount-aware ARRAY/HASH typeglob detachment so saved slots can
+    be re-installed after `undef`.
+  - `sub_quote_qsub_metadata.t`, `typeglob_undef_slot_semantics.t`, and
+    `goto_tailcall_cleanup.t` pass on system Perl, JVM, and interpreter.
+- [x] JVM undefined Unicode tail-marker diagnostic
+  - Top-level marker resolution and named-target preservation now report
+    `Goto undefined subroutine &main::因` rather than an internal escaped-marker error.
+  - `perl5_t/t/uni/goto.t` passes all four assertions on the JVM backend.
+- [x] Completed handoff cleanup and eval-boundary parity
+  - `RuntimeCode.resolveTailCalls()` drains only marker-owned pending
+    referents after the replacement call completes, fixing core destructor
+    assertions 7 and 9 without regressing `Sub::Quote` metadata or caller
+    lifetimes.
+  - `GOTO_TAILCALL` resolves eval-scoped markers inside the interpreter's
+    catcher; `GOTO_DYNAMIC` carries compile-time eval scope to avoid treating
+    normal subs called from eval as eval-string code.
+  - `goto_tailcall_cleanup.t` adds destructor-ordering, Unicode eval-block,
+    and dynamic tied-coderef regressions; it passes on system Perl, JVM, and
+    interpreter. Core `goto-sub.t` (44 assertions) and `uni/goto.t` (4)
+    pass on both backends.
+- [x] CI named-redefinition timeout repair
+  - The stalled `goto_named_redefinition.t` shard was traced to a generic
+    tail-call coderef rewrite that turned `goto &$original_stub` into the
+    replacement wrapper. Fresh lookup is now limited to explicit named-marker
+    targets.
+  - The existing six-case project regression passes on system Perl, JVM, and
+    interpreter, alongside the 13-case cleanup regression.
+- [x] Borrowed-argument lifetime repair
+  - The first final gate exposed an early DBIC schema `DESTROY`: the completed
+    handoff drain considered every live `@_` alias, including caller-owned
+    values.
+  - The drain now consumes only the marker's ownership carrier. The existing
+    `dbic_try_tiny_goto_schema_backref.t` regression passes 3/3 on system
+    Perl, JVM, and interpreter without weakening tail-call cleanup coverage.
+- [x] Windows filehandle stat identity repair
+  - Windows handle stat now retains the open-time `BasicFileAttributes` file
+    key, so a renamed handle has a distinct synthetic inode from a replacement
+    at its former path while an unchanged handle and pathname agree.
+  - The handle also resolves the current identity-validated Windows mode record
+    at its original pathname, preserving File::Temp's default `0600` mode
+    without sacrificing the captured inode after a rename.
+  - Existing `file_temp_stat_mode.t` and `stat_filehandle_after_rename.t`
+    regressions pass on system Perl, JVM, and interpreter. The immutable full
+    `make` gate passed in 4m24s.
+- [x] Hosted cross-platform validation
+  - PR #1205 run 33570088728 passed on Ubuntu and Windows. Windows completed
+    the full build plus focused Perl thread gate; Ubuntu completed the full
+    build, Perl thread compatibility gate, and SBOM generation.
+- [x] Destructor-ordering ownership provenance
+  - The tail-call drain now recognizes only an abandoned argument referent's
+    explicit `bless mortal temporary` birth hold when that deferred entry has
+    no scalar owner. This retires inline constructor arguments before the
+    replacement sub starts, without releasing borrowed DBIC schema aliases.
+  - `goto_tailcall_core_destroy_order.t` (six assertions) and
+    `refcount/dbic_try_tiny_goto_schema_backref.t` (three assertions) pass on
+    system Perl, JVM, and interpreter. Core `op/goto-sub.t` passes on both
+    backends; the final immutable `make` gate passed in 4m27s.
+- [x] Closed-`STDERR` unhandled-die UAT repair
+  - `Main` now writes uncaught Perl diagnostics through the active Perl
+    `main::STDERR` handle instead of directly to Java `System.err`, preserving
+    Perl-level close/redirection semantics.
+  - Added `closed_stderr_unhandled_die.t`; it passes on system Perl, JVM, and
+    interpreter. `run/fresh_perl.t` test 72 passes on both backends in focused
+    core runs.
+
+### Next Steps
+
+1. Commit the closed-`STDERR` repair and focused regression.
+2. Update PR #1205 and restart UAT after reviewing the complete `make` log.
+3. Monitor hosted CI and resolve any remaining unrelated baseline failures.
+
+### Validation note
+
+Earlier gates exposed a named-redefinition loop and a borrowed DBIC schema
+lifetime regression. Both have focused system-Perl, JVM, and interpreter
+coverage and now pass. The final immutable `make` gate on `ddd1160a6` passed
+in 4m25s (856 tests, 3 skips, zero failures); its complete log is
+`/tmp/pr1205-owner-drain-final-make.log`.
+
+The direct JVM one-liner (`sub target{}; eval q{goto &target}`) is now covered
+by the focused regression and reports the expected eval-string diagnostic on
+both backends.
+
+The closed-`STDERR` regression was validated on system Perl, JVM, and
+interpreter. The immutable `make` gate at `acdf0c442` passed in 4m37s; its
+complete log is `/tmp/make-fresh-perl-stderr.log`. Focused `run/fresh_perl.t`
+runs report test 72 as `ok` on both backends; the file continues into older
+unrelated failures after that assertion.
+
+UAT passed on `72cca717e`. Its hosted Ubuntu CI job also passed, but Windows
+exposed an unrelated `File::Temp` handle/path `stat` representation mismatch
+(device, inode, and mode). The first repair made unchanged paths agree but
+revealed that renamed handles must retain their open-time identity. The final
+repair records `BasicFileAttributes` at channel open and derives the same
+synthetic inode for pathname and handle stat without losing renamed-handle
+identity. It also resolves the remembered Windows mode against the original
+path only while that identity still matches, so File::Temp handle and pathname
+stat retain the same `0600` mode.
+
+### Reopened destructor-ordering investigation
+
+UAT again reports `perl5_t/t/op/goto-sub.t` assertions 7 and 9 one iteration
+late on both JVM and interpreter. The exact core test reproduces the result.
+`goto_tailcall_core_destroy_order.t` is the new focused project regression:
+it passes on system Perl and exposes the interpreter variant. The broad
+live-argument cleanup restores the core ordering but destroys DBIC's borrowed
+weak schema too early. The marker-owned-only cleanup preserves DBIC but leaves
+the core object late. A provisional owner/frame-provenance implementation and
+last-counted-owner cleanup did not change either core failure, so the next
+phase is trace-led rather than extending that heuristic.
+
+## Relevant files
+
+- `src/main/java/org/perlonjava/runtime/runtimetypes/RuntimeCode.java`
+- `src/main/java/org/perlonjava/runtime/runtimetypes/ControlFlowMarker.java`
+- `src/main/java/org/perlonjava/runtime/runtimetypes/RuntimeControlFlowList.java`
+- `src/main/java/org/perlonjava/backend/bytecode/BytecodeInterpreter.java`
+- `src/main/java/org/perlonjava/runtime/runtimetypes/RuntimeGlob.java`
+- `src/test/resources/unit/goto_tailcall_cleanup.t`
+- `src/test/resources/unit/closed_stderr_unhandled_die.t`
+- `perl5_t/t/op/goto-sub.t`
+- `perl5_t/t/run/fresh_perl.t`

docs/about/changelog.md

@@ -25,11 +25,18 @@ Release history of PerlOnJava. See [Roadmap](roadmap.md) for future plans.
   avoiding alias-package misattribution in compatibility reports.
 - Fix parser diagnostics, Unicode split and global-regex progression, and
   persistent-app closure cleanup while preserving DBIx::Class leak behavior.
+- Complete `goto &sub` tail-call parity, including eval diagnostics, sparse
+  `@_` reification, late `AUTOLOAD`, completed-handoff temporary cleanup,
+  dynamic-coderef calls from eval, preserved saved-coderef identity across
+  named redefinition, and top-level anonymous-coderef invocation.
+- Route uncaught Perl diagnostics through the active `STDERR` handle, so a
+  closed `STDERR` suppresses a bare `die` like standard Perl.
 - Preserve process-pipe descriptors through returned and argument-aliased
   aggregates, and align compound-assignment lvalue order across both backends.
 - Keep Windows `sysopen` raw unless lexical `use open` applies, preserve exact
-  emulated mode bits in `stat`, and pass Ubuntu/Windows CI run `33223173108` on
-  runtime head `b1b0494cd`.
+  emulated mode bits in `stat`, make tempfile handle stats reflect creation
+  modes, and pass Ubuntu/Windows CI run `33223173108` on runtime head
+  `b1b0494cd`.
 - Restore the post-acceptance core UAT baseline on `9b2377b6f`: value-producing
   `defer` bodies remain verifier-safe, `PerlIO->import` rejects code injection
   without inheriting `UNIVERSAL` export errors, and repeated `$#array` lvalues

src/main/java/org/perlonjava/app/cli/Main.java

@@ -6,6 +6,7 @@
 import org.perlonjava.runtime.runtimetypes.GlobalVariable;
 import org.perlonjava.runtime.runtimetypes.PerlExitException;
 import org.perlonjava.runtime.runtimetypes.PerlRuntime;
+import org.perlonjava.runtime.runtimetypes.RuntimeIO;
 import org.perlonjava.runtime.runtimetypes.RuntimeScalar;
 import org.perlonjava.runtime.regex.RuntimeRegex;
 
@@ -161,8 +162,17 @@ private static void run(String[] args) {
             }
 
             String errorMessage = ErrorMessageUtil.stringifyException(t);
-            System.err.print(errorMessage);
-            System.err.flush();
+            // Unhandled Perl errors follow the current Perl STDERR handle. In
+            // particular, `close STDERR; die` must remain silent; writing to
+            // Java's process stderr bypasses Perl-level handle state.
+            RuntimeIO stderr = GlobalVariable.getGlobalIO("main::STDERR").getRuntimeIO();
+            if (stderr != null) {
+                stderr.write(errorMessage);
+                stderr.flush();
+            } else {
+                System.err.print(errorMessage);
+                System.err.flush();
+            }
             RuntimeRegex.emitPendingFailedCompileDebugFreeTraces();
 
             // Match system perl behavior for unhandled die:

src/main/java/org/perlonjava/app/scriptengine/PerlLanguageProvider.java

@@ -603,6 +603,12 @@ private static RuntimeList executeCodeImpl(RuntimeCode runtimeCode, Node ast, Em
                 WarningBitsRegistry.setCallSiteBits(savedCallSiteBits);
             }
 
+            // Top-level code normally has no generated call-site trampoline.
+            // Resolve a propagated goto &sub marker here so an undefined named
+            // target reports its Perl diagnostic instead of escaping as an
+            // internal control-flow marker.
+            result = RuntimeCode.resolveTailCalls(result, executionContext);
+
             try {
                 if (isMainProgram) {
                     // Flush deferred mortal decrements from file-scoped lexical cleanup.