A type alias that contains itself is an error, not a stack overflow

[ASDAlexander77]

Fixes #370: a type alias that refers to itself crashed the compiler with a stack overflow (0xC00000FD, no message). It's now a compile error.

Cause

Every mention of a type alias expanded its declaration again, and nothing noticed when the same alias was already being expanded higher up the stack.

Fix

• Detection: an alias being expanded is remembered (expandTypeAlias). This covers the plain resolution and both generic ones: resolveTypeByNameInNamespace, getTypeByTypeReference and resolveGenericTypeInNamespace. An alias that meets itself is reported:

  error: type alias 'Node' circularly references itself; use an interface or a class for a recursive type
  

• Why an error, not a recursive type: a type here has a fixed shape, because a tuple holds its fields in place. So no alias can contain itself, not even behind Reference<T>: its tuple type would have to name itself, and the dialect's tuples are structural. A real recursive alias would need a tuple type that can refer to itself, which is a separate design. An interface or a class is the recursive type today. tsbindgen's rule of mapping self-pointers to Opaque therefore stays.
• One error only: errors that merely follow from it are left out. Those raised while the expansion unwinds are swallowed by a scoped handler until the outermost expansion of that alias ends. At later mentions, "can't find type", "can't resolve name" and "generic type … can't be found" are skipped for an alias already reported.
• Exported aliases: an exported alias is now registered before it is resolved, so it is found, and found circular, the same way.
• Aside: the old code tried to cache the resolved alias, but its condition was inverted and it wrote into a copy, so aliases were never cached. The dead code is gone. Aliases stay uncached, which is effectively what happened before, because a resolution can differ between MLIRGen passes.

Tests

• type-alias-circular/ has nine shapes, each passing only on the message and failing on the follow-on messages or a stack dump:
  • a tuple through Reference;
  • an object;
  • an optional field;
  • a declared function's parameter;
  • two mutually recursive aliases;
  • an exported alias;
  • a generic List<T>;
  • a union through an array;
  • an alias inside a namespace.
• positive.ts: aliases that name other aliases, and nested generic ones (Pair<Pair<number>>, Box<Box<T>>), run JIT and AOT.
• Windows Release: 3264/3264.
• Linux (WSL): 3250/3250.
• DefaultLib rebuilt with this compiler: 157/157, compile and JIT (release, gc). The previous DefaultLib build was restored afterwards.

Also: debug build fix (separate commit)

MLIRGenCast.cpp didn't compile in a Debug build after #407. The MLIRRTTIHelperVC.h include undefines DEBUG_TYPE, and the file uses LLVM_DEBUG. It now defines DEBUG_TYPE again after that include, as MLIRGenStatements.cpp already does.

🤖 Generated with Claude Code