Repository navigation
Runtime, GC and os changes from a large program running under TinyGo: which would you take? #5847
Description
Running a large networked Go program (skywire) natively and in the browser under TinyGo, we collected runtime and os changes in our fork that may be useful here. Bug fixes are already filed separately (#5840 to #5846). These are larger, so I would like to know which ones you want before preparing PRs with tests.
Threads scheduler and timers
- Timers in a binary min heap instead of a sorted list. A program with thousands of goroutines and net deadlines spent about a quarter of its CPU in removeTimer. e7845741
- Reuse timer nodes after they fire or stop, so Reset does not allocate. a025435f 0842c639
- gopark, goready and the runtime semaphores for linknamed code, which gvisor's netstack needs to link. 33d553e9
Memory and GC (bdwgc)
- sync.Pool keeps every item forever. Age items over two collections as Go does. 1e98f1ba
- A few thousand goroutines abort with "Unexpected mark stack overflow". Start bdwgc with a larger mark stack. a4bbda9a
- Unmap free blocks after 3 collections instead of 7. A long running program kept about 300 MB of free heap mapped. e30ddf37
- Sampled heap profile, debug.SetGCPercent and FreeOSMemory for bdwgc, and Alloc and HeapAlloc in MemStats. 8188f830 d9804254
Diagnostics
- pprof goroutine, threadcreate and heap profiles, and goroutines grouped by start function. 1c3551a8 4f7a929e
- debug.SetCrashOutput writes panics and fatal signals to the file. c870d267
os on linux
- os/exec: start the child from C with clone(CLONE_VM|CLONE_VFORK), and implement Dir, Files, Wait, Kill and Signal. Today exec.Cmd cannot run anything, and the fork runs Go code in the child of a multithreaded process. a0453257
- os.Hostname via uname. 3e455308
- File.SyscallConn for files with a descriptor. c98baf45
js target
Activity
dkegel-fastly commentedon Oct 8, 2026
the os.Hostname fix seems like low hanging fruit
dgryski commentedon Oct 9, 2026
Need to think more about the rest but:
Timers in a binary min heap instead of a sorted list.I would be interested in the code size impact of this. I think most programs built with tinygo aren't hitting this extreme use case and have only a few timers, and I don't want to have a significant binary bloat for them. Alternatively, we could consider having the timer queue be swappable with a build flag, but that might just add bloat that we're not testing.
Reuse timer nodes after they fire or stop, so Reset does not allocate.This one seems safe.
sync.Pool keeps every item forever. Age items over two collections as Go does.This one seems safe.
0pcom commentedon Oct 14, 2026
Code size of the timer min-heap against dev,
-size shortcode bytes:So it costs about 650 bytes on Cortex-M for any program that uses a timer. The cost I measured only shows up with thousands of goroutines and net deadlines, so one option is to use the heap only under the threads scheduler and keep the sorted list elsewhere. That leaves microcontrollers unchanged, at the price of two implementations. Or I can drop it.
The other three are on branches off dev, each tested on linux/amd64 on top of #5782:
testdata/timers.gomatches under both-scheduler=threadsand-scheduler=tasks.tinygo test syncpasses with the default GC (bdwgc), with-gc=precise, and on wasip1. The fork version touched a fork-only file, so the bdwgc hook is now in its own file.os.Hostnamereturns the kernel hostname.I'm at the open PR limit, so I'll open these as slots free up.
0pcom commentedon Oct 20, 2026
I split the items here into their own issues, each with a branch on current dev: #5941 gopark and semaphores, #5942 bdwgc mark stack, #5943 bdwgc unmap, #5944 runtime/pprof profiles, #5945 SetGCPercent and FreeOSMemory, #5946 SetCrashOutput, #5947 File.SyscallConn, #5948 js argv and env, #5949 js exit, #5956 timer heap, #5957 timer node reuse, #5958 sync.Pool aging, #5959 Hostname. os/exec is on #5695. #5956 has the code size numbers you asked about.