Devlog

[ehartford]

with migrate is complete. I used it to port pcre2 to with, which is 100% working, all tests pass.
I had to accept goto into With. For implementation, I followed golang's example. https://github.com/withlang-dev/with/blob/main/docs/completed/goto.md

the reason is that pcre2's gotos are irreducable. Therefore, I was not able to convert them to With using the Beyond Relooper algorithm. https://github.com/withlang-dev/with/blob/main/lib/std/cfg/stackify.w

[ehartford]

I am currently working on removing &mut. this is a major change to the language. probably the biggest refactor I've ever done.
about 80% done. 4 days in. This is part of my push to get all major language syntax changes in, before i move on to the stdlib.

the spec for &mut removal:

https://github.com/withlang-dev/with/blob/main/docs/completed/mut.md

[ehartford]

this is finished - but it revealed another layer.

#185 (comment)

[ehartford]

Devlog: Three Block Body Forms

I kept seeing this pattern and it bothered me:

fn double(x: i32) -> i32:
    x * 2

A single expression taking three lines. It didn't feel like With.

We already had two block forms — indented colon and braces — but neither solved the trivial-body problem cleanly. So I added a third: inline colon.

Now this is valid:

fn double(x: i32) -> i32: x * 2

I wrote up the full spec for the three interchangeable forms (inline colon, indented colon, braced) and made the rule universal across every construct that takes a body.

this is now completed

https://github.com/withlang-dev/with/blob/main/docs/completed/three-block-types.md

[ehartford]

Devlog: Mutability and Calling Convention

Increasing complexity in the name of simplicity.

What changed

Six independent axes now:

1. Binding stability (let vs var)
2. Value mutation (through methods or places)
3. Calling convention (default share-place, copy, or move)
4. Receiver mode (&self, mut self, move self)
5. Effect summary (inferred: read / write / consume / escape_value / escape_view)
6. Effect pinning (@[effect(...)] for stable contracts)

The big shift: share-place by default

For non-Copy types, f(x) now gives the function an ephemeral shared-place alias — read + write during the call, caller keeps ownership. No &mut ceremony.

fn append(b: Buffer, byte: u8):
    b.data.push(byte)     // mutates caller's buffer

let buf = Buffer.new()
append(buf, 42)           // OK — share-place
print(buf.data.len())     // 1

For Copy types, default is still copy (as expected).

When you actually want isolation or ownership transfer, you say so:

store(move user)          // ownership transfer
store(copy user)          // independent clone

Why this feels right

• Common case (read or mutate during call) needs zero annotation
• Ownership transfer and isolation are explicit and visible
• Effect summaries are inferred from the body — no signature/body fights
• &T still exists for the niche cases (FFI, stable read-only contracts)
• Methods require explicit receiver modes so API contracts are obvious at a glance

The old model forced too many decisions into signatures. This one pushes the decisions to the call site where they belong, while still giving the compiler everything it needs for safety.

https://github.com/withlang-dev/with/blob/main/docs/completed/mutability.md

This is now implemented.

[ehartford]

Devlog: Implicit Main

I added implicit main.

This means simple With programs no longer need ceremony:

let x = 3
print(int_to_string(double(x) + 1))

fn double(n: i32) -> i32:
    n * 2

The compiler now treats top-level executable statements in the build/run entry file as the body of a synthesized fn main.

This is only for entry files. Imported modules, with check, tests, and normal library parsing still use the declaration-only top-level model. That boundary is important. I wanted the script experience without weakening module semantics.

The annoying part was top-level let / var.

At first I tried the obvious thing: in implicit-main mode, parse top-level let / var as statements. But that breaks files that already have an explicit main plus globals:

var g_counter: i32 = 0

fn main:
    g_counter = g_counter + 1

The parser sees the file incrementally, so it can encounter var g_counter before it knows there is a later fn main. If it classifies that var too early, it silently turns a module global into a local statement and then reports the wrong conflict.

So the final rule is:

the parser does a cheap top-level pre-scan for fn main.

If there is an explicit main anywhere in the entry file, top-level let / var keep their existing global meaning. If there is no explicit main, top-level executable statements accumulate into a synthesized main.

The implementation is entirely in the frontend/parser path. The parser builds a normal NK_FN_DECL named main with an NK_BLOCK body, so sema, MIR, and codegen do not need special cases. Once parsing is done, the rest of the compiler sees ordinary With.

I added coverage for:

• basic implicit main behavior
• declaration hoisting / forward reference visibility
• top-level let locality
• with check still rejecting top-level executable statements
• imported modules still rejecting top-level executable statements
• explicit fn main plus top-level executable statements being a hard error

The regression that mattered was explicit-main files with top-level globals. That is fixed by the pre-scan.

This feature enables with repl and with notebook

https://github.com/withlang-dev/with/blob/main/docs/completed/implicit-main.md

This is now implemented.

[ehartford]

Devlog: Do-While

Another concession to with migrate - I have decided to add do-while loops, due to the complexity and bugs of migrating c code without it, it's easier to add do-while than to resist it.

and it does no true harm besides unnecessarily expanding the lexicon of With.

it's a pragmatic tradeoff and I am ok with it.

[ehartford]

PCRE2 is in. Regular expressions integrated. Implicit main, one-liners implemented.

$ printf '[INFO] boot\n[WARN] slow query\n[ERROR] db timeout\n' | with -n 'if line =~ /^\[(?<level>ERROR|WARN)\]\s+(?<msg>.*)$/: print(f"{nr}: {$level} {$msg}")'
2: WARN slow query
3: ERROR db timeout

[ehartford]

Right now I'm working on updating the build system. Inspired by Jai's.

[ehartford]

Devlog: Three Months

It's mid-May. With is three months old.

I keep having to remind myself of that, because the project feels older than it is. Not in a bad way. In the way that a thing feels older when you've lived inside it. I've been thinking in With syntax for weeks now. When I read C code I mentally translate it. When I see a Python script I think about what it would look like without the runtime. That's either a good sign or a warning sign, and I'm choosing not to examine which.

---

The 48-hour sprint in late February was the most productive weekend of my life, and I don't think I fully understood at the time what I was doing. I wasn't building a compiler. I was building a commitment. Once you have 400 tests passing and a language that can express structs, enums, traits, generics, closures, pattern matching, and async, even if most of those are skeletal, you've crossed a line. You can't look at that and say "this was just an experiment." You have to either abandon it or finish it.

I chose to finish it.

---

Self-hosting was the first real milestone. Not because it was technically the hardest thing (it wasn't. The MIR pipeline was harder), but because it changed the nature of the project. Before self-hosting, With was a program I was writing. After self-hosting, With was the tool I was using to write itself. Every bug in the compiler was now a bug in my development environment. Every missing feature was something I personally needed, right now, to keep working.

That pressure is clarifying. You stop designing features in the abstract and start building what you actually need next. The whole March sprint. MIR, generics, C interop, f-strings, the compiler refactors. All of that was driven by "I need this to keep going." I didn't sit down and plan a MIR pipeline because it was architecturally elegant. I built it because the AST codegen was falling over on real code and I needed something that worked.

The day I deleted the AST codegen path entirely, March 16 , felt like burning a bridge. In a good way. There was no fallback. MIR was the compiler now. If it didn't work, nothing worked.

---

The C interop story surprised me. I expected it to be a necessary chore, something you grind through so you can call printf and move on. Instead it became one of the most compelling parts of the language.

c_import reaching Zig parity in four days was not something I planned for. It happened because the test cases kept revealing gaps, and each gap had a clean fix, and before I knew it the thing actually worked on real headers. Then with get happened and suddenly you could pull in raylib with one command and start drawing things. That's when With stopped feeling like a compiler project and started feeling like a language.

The BearSSL port for the package manager was a fun detour. People will probably read the timeline and think "they wrote a TLS stack in a day," which sounds insane. What actually happened is that BearSSL is a beautifully written, compact C library, and the migration tool handled most of the translation. The hard part was getting the networking right, not the crypto. Thomas Pornin did the hard part years ago. I just moved it to a different language.

---

April was the PCRE2 month.

73,000 lines of C. Not clean C, either. PCRE2 is a serious regex engine with computed gotos, macro-heavy codegen, and bit-manipulation code that makes you question whether the author was writing a program or composing a spell. Migrating it was the hardest test the migration tool has faced, and the migration tool failed that test repeatedly before it started passing.

The bugs were educational, though. Every PCRE2 test failure pointed at a real compiler or migrator defect. Sign extension in integer codegen. Comma expressions losing side effects. C integer promotion rules not being applied to small types. sizeof on pointer types resolving to the pointee instead of the pointer. Each one was a four-line fix once you found it, and each one took hours to find.

The moment the upstream PCRE2 test suite started passing was quietly one of the best moments of the project. No fanfare. Just a test run that ended with "All tests passed" instead of a crash. I stared at it for a while.

---

The mutability model redesign in late April was the biggest philosophical decision I've made in With.

The earlier model used &mut T -- Rust-style mutable references as a first-class type. It worked, technically. But every time I used it, I felt the same friction I feel writing Rust. The lifetime propagation. The borrow checker fights. The feeling that the language is spending more time checking my work than helping me do it.

So I threw it out. mut self receivers. Ephemeral borrows only. No stored references in safe code. Handles for everything else.

People will tell me this is too restrictive. They're not wrong. There are patterns you can express in Rust that you can't express in safe With. But there are also entire categories of bugs and complexity that simply don't exist in With's model. No lifetime annotations. No borrow checker error messages that take a PhD to parse. No 'a in your function signatures.

I think that's worth it. Ask me again in a year.

---

May has been about refinement. Implicit main, open source, regex literals, CLI one-liners, the build system, derive macros. Each of these is small compared to the March and April sprints, but they're the features that make the language feel finished. Or at least, feel like it's heading somewhere that people would want to use it.

The derive macros are a good example. @[derive(Serialize)] is not a hard feature to implement once you have comptime transforms. But it's the kind of feature that tells a user "this language respects your time." Nobody wants to write JSON serialization by hand. The language should do it for you.

Same with CLI one-liners. cat log.txt | with -n 'if line =~ /error/: print(line)' is not technically impressive. It's a wrapper that generates a source file, compiles it, and runs the binary. But it means you can reach for With the same way you'd reach for Perl or awk. That's a workflow thing, not a language thing, and workflow things matter more than language things in practice.

---

Looking back at three months, what surprises me most is how much of the design held up.

The spec I wrote in February, the original one, before any code existed, is still recognizably the language I'm building. The syntax barely changed. The ownership model evolved (the &mut redesign), but the core idea of "persistent ownership, ephemeral borrowing, handles for relationships" was there from the start. The type system is the same. The error handling is the same. The async model is the same.

That's either because I got lucky with the initial design, or because I've been unconsciously steering the implementation to match the spec even when it would have been easier to diverge. Probably both.

What I didn't anticipate is how much of the work would be C interop. I thought C interop would be a feature. It turned out to be half the project. The migration tool alone is thousands of lines of compiler code. The PCRE2 migration consumed most of April. The BearSSL port, the c_import infrastructure, the Conan integration. All of that is "C interop" in the broad sense, and all of it was more work than I expected.

But it's also the reason the language is useful today, not in some hypothetical future. You can write a real program in With right now, pulling in real C libraries, because the interop actually works. That's worth the investment.

---

What's next? The obvious answer is "a lot." The REPL and notebook are release gates. The query language for JSON/YAML processing needs a real design. The ML framework Crux is still just a name and a motivation. The regex engine works but the language integration needs more polish. The build system is functional but young.

But the foundation is solid. The compiler bootstraps. The type system is coherent. The ownership model is simple and enforced. The C interop is real. The fixpoint holds.

https://github.com/withlang-dev/with/blob/main/docs/blog_posts/timeline.md

[ehartford]

Devlog: With native build and emitting C

Prerequite to cross-platform, is emitting C. this is a big feature, I wanna discuss what it is and why I'm doing it.

The opposite of with migrate (our killer app) we also have an "--emit-c" option. Here we compile With code, and emit C code.

Why would we do such a thing?

• To flex, because we can.
• It enables us to seed new platforms with trusted compiler
• It stress tests with migrate (essentially self-hosting it)

How do we prove that it works?

1. First we build the compiler normally
2. Then we --emit-c and build the emitted compiler.
3. Then we with migrate the emitted code, and build the migrated code.
4. All test cases pass on both emitted and migrated compiler
5. use the migrated compiler to compile the compiler.
6. output must be byte-equal to the compiler built in step 1.

In addition: We are adopting Jai-style build. (the build logic is written in With). this eliminates the need for Makefile and build scripts.

All of this paves the path to true cross-platform.

This work is currently spec'd and in-progress.

[ehartford]

Of desire for purity, and to enable cross platform build with minimal dependencies, I am adding a native build system to With inspired by Jai, and deprecating my Makefile and the dozens of shell scripts that previoualy functioned as my build system.

This effort required me to expand the concept of comptime beyond Zig's (which I previously ported to With)

Capability-Bearing Comptime

What it is

Capability-bearing comptime is a generalization of pure compile-time evaluation. A comptime function declares the capabilities it requires through a with clause:

comptime with ToolFs:
    fn embed_template(path: str) -> str:
        fs.read_text(path)

Each capability binds to a name in scope (fs for ToolFs, proc for ProcessRunner, ctx for BuildCtx and ActionCtx, with as name to rebind). Method calls on a capability dispatch to compiler-native handlers that perform the corresponding effect: filesystem access, process execution, source emission, diagnostics.

A function with no with clause is pure comptime in the Zig sense. It computes values and types, specializes generics, folds constants. It cannot do I/O. It cannot forge a capability. It can only call other functions whose capability requirements are a subset of its own.

The capability set determines what the function is allowed to do. The mechanism is the same evaluator either way.

Motivation

Zig's comptime is powerful but strictly pure. Anything requiring effects, build orchestration in particular, lives outside the language as a separate program. Jai's metaprogramming runs compile-time code with full ambient authority. The first is too restrictive for build systems; the second gives up the safety property that makes compile-time evaluation tractable.

Capability-bearing comptime resolves this without splitting the language. Build scripts, action functions, code generators, and compile-time queries are all ordinary comptime functions. They differ only in what capabilities they declare. There is no separate build language, no generated runner binary, no graph text as transport between phases.

The same property gives capability-bearing comptime its safety: a function cannot acquire a capability it did not declare. The driver mints capabilities at well-defined entry points (build, action invocation, code-gen hooks) and passes them in. Everywhere else, the capability set is whatever the caller passes through, never more.

Implementation

The compiler maintains a registry of capability types (BuildCtx, ProjectInfo, Diagnostics, SourceEmitter, ToolFs, ProcessRunner, ActionCtx). Semantic analysis uses this registry to enforce three rules: capability values cannot be constructed by user code, cannot escape to runtime contexts, and cannot be called without a matching with clause at the call site.

A comptime evaluator interprets MIR with capability-aware dispatch. When a method is called on a capability value, the evaluator validates the handle (correct kind, live generation, owned by the current evaluator) and routes the call to a compiler-native handler. The handler performs the effect (read a file, run a process, emit a diagnostic) and returns a value the evaluator can use.

The driver mints capabilities at entry points and invokes user-declared comptime functions through the evaluator. build(ctx) returns a typed Build value; a materializer converts that value directly into the driver's internal build graph. Action targets store the action function as a typed reference; when the target runs, the driver mints an ActionCtx scoped to the target's declared outputs and evaluates the action through the same evaluator.

Compile-time evaluation is unified. The same machinery handles const X = comptime fib(10), comptime with BuildCtx: pub fn build, and every capability-bearing function in between. Only the capability set changes.

Why expand comptime instead of building a tool mode

An earlier draft of this design proposed a separate execution context called "tool mode." Build scripts and action functions would run in this mode, distinct from both runtime and ordinary comptime. The compiler would have three execution tiers: pure comptime for constants and types, tool mode for build orchestration with driver-minted authority, and runtime for the compiled program.

The reframing to capability-bearing comptime came from noticing that tool mode would have been comptime with a specific, fixed set of capabilities. The "tool" in tool mode was just shorthand for "comptime with BuildCtx, or one of the things BuildCtx hands out." Once that was visible, the separate mode stopped earning its complexity.

[ehartford]

With 0.14.0 Released.

massive progress.

Added native build, one liners, and install script, regex, with migrate, --emit-c, and so much more.

Release
https://github.com/withlang-dev/with/releases/tag/v0.14.0

Install (on mac - windows and linux are coming soon)

curl -fsSL https://github.com/withlang-dev/with/releases/latest/download/install.sh | sh

Demo

with -e 'if "err42" =~ /(\D+)(\d+)/: print(f"{$1}:{$2}")'

[siriobalmelli]

Very interested in the direction this is going; kudos. Bringing into nixpkgs NixOS/nixpkgs#525186