[rdon-key]

Status: Superseded by #6001

The fix from this PR has been incorporated into #6001, along with additional changes to ensure compatibility with existing flash operations (machine.Flash).

Please refer to #6001 for further development.

---

This PR fixes an ESP32-S3 boot failure that occurs when the DROM segment exceeds 1 MiB.

Previously, DROM and IROM were stored as regular ESP image segments visible to the ROM bootloader. When the image contained a large DROM segment, the ROM rejected the image before the TinyGo startup code could run.

The failure looked like this:

load:0x3c000020,len:0x780114
Invalid image block, can't boot.

Cause

On the ESP32-S3, DROM and IROM are not copied into RAM by the ROM bootloader. They are accessed through the flash cache MMU configured by the startup code.

The previous implementation had the following problems:

• DROM and IROM were included in the ROM-visible image segment table.
• The MMU table was split into fixed IROM and DROM regions.
• The mapping assumed that virtual address pages were identical to physical flash pages.

As a result, the ROM bootloader attempted to process a large DROM segment and rejected the image before entering the TinyGo startup code.

Changes

This PR makes the following changes:

• Excludes ESP32-S3 DROM and IROM from the ROM-loaded image segments.
• Places IROM and DROM after the DRAM/IRAM image, aligned to 64 KiB flash pages.
• Stores the physical flash offsets of IROM and DROM in .data.
• Places IROM at the beginning of the shared MMU table.
• Calculates the IROM/DROM MMU split from the number of pages used by IROM.
• Configures the MMU entries during startup using the physical flash offsets written by the builder.
• Maps DROM immediately after the IROM entries in the shared MMU table.

Hardware verification

The fix was tested on an M5Stamp-S3A with an ESP32-S3 and 8 MiB of flash.

Test data

A 7.5 MiB binary file was embedded in DROM.

The file is 7,864,320 bytes long and consists of the following 256-byte sequence repeated 30,720 times:

00 01 02 03 ... FD FE FF

The data is deterministic and can be reproduced with the following script:

Details

from pathlib import Path
import zlib

size = 15 * 1024 * 1024 // 2
pattern = bytes(range(256))
data = pattern * (size // len(pattern))

path = Path("payload_7_5m.bin")
path.write_bytes(data)

print("file:", path)
print("size:", len(data))
print("crc32:", f"0x{zlib.crc32(data) & 0xffffffff:08x}")

Expected output:

file: payload_7_5m.bin
size: 7864320
crc32: 0x4f758caa

The expected CRC32 values are:

hexadecimal: 0x4f758caa
decimal:     1333103786

Verification program

The following program reads the entire 7.5 MiB payload and calculates its CRC32 five consecutive times:

package main

import (
    _ "embed"
    "time"
)

//go:embed payload_7_5m.bin
var payload string

var crcTable [256]uint32

func initCRC32Table() {
    for i := range crcTable {
        value := uint32(i)

        for bit := 0; bit < 8; bit++ {
            if value&1 != 0 {
                value = value>>1 ^ 0xedb88320
            } else {
                value >>= 1
            }
        }

        crcTable[i] = value
    }
}

func verify(pass int) bool {
    crc := uint32(0xffffffff)

    println("PASS START", pass)

    for i := 0; i < len(payload); i++ {
        crc = crcTable[byte(crc)^payload[i]] ^ crc>>8

        if i != 0 && i%(1024*1024) == 0 {
            println("PROGRESS", pass, i/(1024*1024), "MiB")
        }
    }

    crc = ^crc

    println("SIZE", len(payload))
    println("CRC32", crc)

    if len(payload) != 15*1024*1024/2 {
        println("SIZE MISMATCH", pass)
        return false
    }

    if crc != 0x4f758caa {
        println("CRC32 MISMATCH", pass)
        return false
    }

    println("VERIFY OK", pass)
    return true
}

func main() {
    time.Sleep(2 * time.Second)
    initCRC32Table()

    for pass := 1; pass <= 5; pass++ {
        if !verify(pass) {
            println("TEST FAILED")

            for {
                time.Sleep(time.Hour)
            }
        }

        time.Sleep(time.Second)
    }

    println("TEST DONE")

    for {
        time.Sleep(time.Hour)
    }
}

The program was flashed with:

tinygo flash \
  -target=m5stamp-s3a \
  -port=COM6 \
  -monitor \
  main.go

Before this change

On the previous dev revision, the ROM bootloader attempted to load the 7.5 MiB DROM segment and rejected the image before the TinyGo startup code was reached:

load:0x3c000020,len:0x780114
Invalid image block, can't boot.

After this change

With this change, all five consecutive CRC32 verification passes succeeded:

PASS START 1
PROGRESS 1 1 MiB
...
PROGRESS 1 7 MiB
SIZE 7864320
CRC32 1333103786
VERIFY OK 1

...

PASS START 5
PROGRESS 5 1 MiB
...
PROGRESS 5 7 MiB
SIZE 7864320
CRC32 1333103786
VERIFY OK 5
TEST DONE

The complete flash-and-verification procedure was repeated three times, and all runs succeeded.

This confirms not only that an image containing a large DROM region can boot, but also that the complete 7.5 MiB DROM contents remain correctly accessible across repeated full reads.

Other ESP32 targets

The ESP32-C3 and ESP32-C6 paths are implemented in separate files that this PR does not touch.

The ESP32 shares builder/esp.go with the ESP32-S3, but the new image layout is applied only to the ESP32-S3 branch, and the existing ESP32 code path remains unchanged. The MMU changes are confined to ESP32-S3-specific files.

I do not have ESP32 hardware available, so the ESP32 code path has not been verified on a device.

[digitalentity]

The #5554 now absorbes this change and is verified to boot and pass the test:

PASS START 5
PROGRESS 5 1 MiB
PROGRESS 5 2 MiB
PROGRESS 5 3 MiB
PROGRESS 5 4 MiB
PROGRESS 5 5 MiB
PROGRESS 5 6 MiB
PROGRESS 5 7 MiB
SIZE 7864320
CRC32 1333103786
VERIFY OK 5

[rdon-key]

Since PSRAM support in #5554 may take some time, I've rebased
this PR onto the latest dev branch.

As this is an independent bug fix, I'd like to move it forward
without waiting for PSRAM support.

I've also removed the unnecessary flash padding pointed out in #5554.

Tested on M5Stamp-S3A (8 MB flash):

• Before: 7,995,840 bytes
• After: 7,873,104 bytes
• Reduction: 122,736 bytes
• 7.5 MiB DROM verification: 5/5 passes
• CRC32: 1333103786 (identical across all passes)

The updated image boots successfully and passes all five
DROM verification runs.

The CI smoke test also revealed a 4-byte alignment gap before
DROM. I've adjusted the padding check to allow small alignment
gaps (less than 16 bytes).

All CI checks now pass.

[rdon-key]

The fix from this PR has been incorporated into #6001, following the maintainers' decision.

The integration also includes changes to ensure compatibility with existing flash operations (machine.Flash).

Thanks for carrying this work forward!