Where we are
Every case under tests/cases is single-host by construction, not by choice of backend.
The peers come from fork(), and the channel the children report through is an anonymous
shared mapping:
::mmap(nullptr, count * sizeof(int), PROT_READ | PROT_WRITE,
MAP_SHARED | MAP_ANONYMOUS, -1, 0);
That mapping has no name, so nothing but a forked child can reach it. test_admission_race,
test_admission_stress, test_create_race and test_util each carry their own copy of the
helper, and the children never unmap.
So a _dax registration today puts the region on the devdax node and the coordination in
host DRAM. That is fine for what those cases assert, but it means the dax and uc variants
exercise the medium without ever crossing a host boundary.
What a multi-host mode has to decide
Naming a --role (rather than a topology flag) settles five things at once. Single host
stays the default: one process that forks, which is what runs today.
- Who creates the peers. Forked children, or processes already running on other hosts.
- Where the coordination buffer lives. DRAM inherited through
fork, or a named area
on the substrate. TestMemory::uriFor("scratch") already gives the second one a name.
- Who formats the region. With
fork the parent is obvious. Across hosts one role has
to be designated, and the others wait for it.
- How the run ends. The parent
wait()s today. Across hosts there is nothing to wait
on, so completion has to be observed: a counter, against a deadline.
- Who reports the verdict. This is the sharp edge.
runCase assumes one process owns
TestContext::failures() and its exit code is the answer. Across hosts every process has
its own. Either one role reads the others' results off the substrate, or an external
launcher collects the exit codes. The launcher is simpler and is needed anyway.
Two things that do not come for free
Coherency discipline. The coordination code writes with __atomic_store_n and plain
int stores, which assume cache coherence. On uc or dax that assumption is gone, so those
writes need the same fences the library uses (cme::coherency::wmb / rmb). Moving the
allocation is not enough; the coordination code changes with it.
The barrier perturbs what is measured. test_admission_stress uses a spin barrier so
claims land in the narrowest window it can arrange. A barrier that spins on the same dax
window widens exactly that window. Results are read once at the end and can move to the
substrate; a barrier is hit repeatedly right before the contention and should not, so a
multi-host barrier needs redesigning rather than relocating.
First step
Give TestContext a scratch area alongside the region it already owns, so the four
hand-rolled mmap/munmap helpers collapse into one owner and the buffer gets a name. That
is useful on its own -- it fixes the missing unmap in the children -- and it is the piece
every later step needs.
The launcher, the role flag and the barrier redesign follow after.
Where we are
Every case under
tests/casesis single-host by construction, not by choice of backend.The peers come from
fork(), and the channel the children report through is an anonymousshared mapping:
That mapping has no name, so nothing but a forked child can reach it.
test_admission_race,test_admission_stress,test_create_raceandtest_utileach carry their own copy of thehelper, and the children never unmap.
So a
_daxregistration today puts the region on the devdax node and the coordination inhost DRAM. That is fine for what those cases assert, but it means the dax and uc variants
exercise the medium without ever crossing a host boundary.
What a multi-host mode has to decide
Naming a
--role(rather than a topology flag) settles five things at once. Single hoststays the default: one process that forks, which is what runs today.
fork, or a named areaon the substrate.
TestMemory::uriFor("scratch")already gives the second one a name.forkthe parent is obvious. Across hosts one role hasto be designated, and the others wait for it.
wait()s today. Across hosts there is nothing to waiton, so completion has to be observed: a counter, against a deadline.
runCaseassumes one process ownsTestContext::failures()and its exit code is the answer. Across hosts every process hasits own. Either one role reads the others' results off the substrate, or an external
launcher collects the exit codes. The launcher is simpler and is needed anyway.
Two things that do not come for free
Coherency discipline. The coordination code writes with
__atomic_store_nand plainintstores, which assume cache coherence. On uc or dax that assumption is gone, so thosewrites need the same fences the library uses (
cme::coherency::wmb/rmb). Moving theallocation is not enough; the coordination code changes with it.
The barrier perturbs what is measured.
test_admission_stressuses a spin barrier soclaims land in the narrowest window it can arrange. A barrier that spins on the same dax
window widens exactly that window. Results are read once at the end and can move to the
substrate; a barrier is hit repeatedly right before the contention and should not, so a
multi-host barrier needs redesigning rather than relocating.
First step
Give
TestContexta scratch area alongside the region it already owns, so the fourhand-rolled
mmap/munmaphelpers collapse into one owner and the buffer gets a name. Thatis useful on its own -- it fixes the missing unmap in the children -- and it is the piece
every later step needs.
The launcher, the role flag and the barrier redesign follow after.