Skip to content

Multi-host test mode: role-based launch and a named coordination buffer #1

Description

@moonchan-park

Where we are

Every case under tests/cases is single-host by construction, not by choice of backend.

The peers come from fork(), and the channel the children report through is an anonymous
shared mapping:

::mmap(nullptr, count * sizeof(int), PROT_READ | PROT_WRITE,
       MAP_SHARED | MAP_ANONYMOUS, -1, 0);

That mapping has no name, so nothing but a forked child can reach it. test_admission_race,
test_admission_stress, test_create_race and test_util each carry their own copy of the
helper, and the children never unmap.

So a _dax registration today puts the region on the devdax node and the coordination in
host DRAM. That is fine for what those cases assert, but it means the dax and uc variants
exercise the medium without ever crossing a host boundary.

What a multi-host mode has to decide

Naming a --role (rather than a topology flag) settles five things at once. Single host
stays the default: one process that forks, which is what runs today.

  1. Who creates the peers. Forked children, or processes already running on other hosts.
  2. Where the coordination buffer lives. DRAM inherited through fork, or a named area
    on the substrate. TestMemory::uriFor("scratch") already gives the second one a name.
  3. Who formats the region. With fork the parent is obvious. Across hosts one role has
    to be designated, and the others wait for it.
  4. How the run ends. The parent wait()s today. Across hosts there is nothing to wait
    on, so completion has to be observed: a counter, against a deadline.
  5. Who reports the verdict. This is the sharp edge. runCase assumes one process owns
    TestContext::failures() and its exit code is the answer. Across hosts every process has
    its own. Either one role reads the others' results off the substrate, or an external
    launcher collects the exit codes. The launcher is simpler and is needed anyway.

Two things that do not come for free

Coherency discipline. The coordination code writes with __atomic_store_n and plain
int stores, which assume cache coherence. On uc or dax that assumption is gone, so those
writes need the same fences the library uses (cme::coherency::wmb / rmb). Moving the
allocation is not enough; the coordination code changes with it.

The barrier perturbs what is measured. test_admission_stress uses a spin barrier so
claims land in the narrowest window it can arrange. A barrier that spins on the same dax
window widens exactly that window. Results are read once at the end and can move to the
substrate; a barrier is hit repeatedly right before the contention and should not, so a
multi-host barrier needs redesigning rather than relocating.

First step

Give TestContext a scratch area alongside the region it already owns, so the four
hand-rolled mmap/munmap helpers collapse into one owner and the buffer gets a name. That
is useful on its own -- it fixes the missing unmap in the children -- and it is the piece
every later step needs.

The launcher, the role flag and the barrier redesign follow after.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions