[dgryski]

qbs flashes a few small programs onto a connected board and checks the results over its serial port. It is meant to find obvious problems on many boards quickly, not to test any one feature in depth.

go run ./tools/qbs -target <target>

The tool builds with the standard Go compiler. The programs it flashes build with TinyGo.

Classes:

• timing covers sleep accuracy, a monotonic clock, loop speed, and the device clock against the host clock.
• compute runs the same computations on the device and the host and compares the results. It covers integers, bits, sorting, maps, hashes, encodings, strings, float32 and float64, math, strconv and fmt.
• gc covers large blocks, linked lists, slice and string growth, map churn, fragmentation and collection cycles. Sizes come from the free heap.
• serial covers echo, RX buffer overrun, line and byte transfer, and output with goroutines.

A class is skipped when its program does not fit on the target. A test is skipped when the board cannot run it, for example when there is not enough heap. The README explains how to add tests. If tools is not the right place I am happy to move it.

This supersedes #6023. That PR added the serial tests as a standalone tool. They are the serial class here, along with host drivers for the tests that need the host to send data, so #6023 can be closed.

Boards I ran it on:

• Arduino Uno R4 WiFi with machine/ra4m1: add SCI UART driver #6015 passed everything, with one skip for lack of heap.
• Seeed XIAO M0 passed with machine/atsamd21, atsamd51: handle TX complete with RX complete #6024. The conc test fails without it.
• Pico 2 W, Pico 2 and Pico passed when they booted. Some runs fail on these boards because a board stops responding after a flash. See rp2040, rp2350: board stops responding after tinygo flash and ignores the 1200 bps reset #6036.

The programs also build for pico, pico2-w, arduino-uno-r4-wifi, feather-m4 and stm32f4disco. On the arduino-nano, timing, gc and serial build and compute does not fit. I have not run it on a Nano.

[dgryski]

I added three commits to this PR. qbs can now run without a board.

Emulators. -emulator runs each class with tinygo run and talks to the program over stdin and stdout, for example go run ./tools/qbs -target wasip1 -emulator. To make that work on wasip1, the programs now use a small serial layer, dev.Serial. On boards it wraps machine.Serial. On wasip1 it uses stdin and stdout.

Targets that cannot receive input. -noinput is for targets with no UART receive, such as riscv-qemu and cortex-m-qemu. The programs start without the host and the serial class is skipped. The timing tests skip when the clock does not advance or follows a simulated time source, and the gc tests use at most 4 MB of heap.

simavr. simavr holds back its output when it is not a terminal and colours each line, so -pty runs it under a pseudo terminal and cleans the lines. -scheduler and -gc change how the programs are built. The simavr target needs -scheduler none because an explicit runtime.GC() fails with its tasks scheduler.

Runs that never finish. A second @@GO now fails the class as a restart. -class-timeout ends a class that keeps printing, which a program does after it runs out of memory. A class that does not fit on the target shows the linker error.

I ran wasip1, riscv-qemu and cortex-m-qemu with no failures except one gc linked-list growth on wasip1 that I cannot explain. simavr passes timing, skips compute because it does not fit, and fails gc because memory use grows on every run. That also happens in a small program without qbs, so I think it is a problem on AVR. I have not filed an issue for it yet. I ran the emulators before the last change to the gc tests and have not repeated them since.