Skip to content

fix(compose): mount named volumes so data survives --force-recreate - #86

Open
itxtoledo wants to merge 1 commit into
us:mainfrom
itxtoledo:fix/compose-mount-named-volumes
Open

fix(compose): mount named volumes so data survives --force-recreate#86
itxtoledo wants to merge 1 commit into
us:mainfrom
itxtoledo:fix/compose-mount-named-volumes

Conversation

@itxtoledo

Copy link
Copy Markdown
Contributor

Problem

Named volumes declared in a docker-compose.yaml are silently skipped by mocker compose up. The volume store directory is created (you can see it under ~/.mocker/volumes/<project>-<name>/_data), but it is never bind-mounted into the container.

The consequence is severe for stateful services:

  • Writes go to the container's ephemeral layer, not to the volume.
  • Any container recreation — compose up --force-recreate, a changed config hash, compose down && compose updiscards the data.
  • MySQL, Postgres, Redis, etc. come back empty with no warning at all.

I hit this in production dev: a MySQL stack lost its entire database on --force-recreate, and every named volume under ~/.mocker/volumes/ was 0 bytes — proof they were never written to.

Root cause

ComposeOrchestrator.resolveVolumeMounts (ComposeOrchestrator.swift:564) intentionally drops named volumes:

// named volumes (bare names without path separators) are skipped — Apple's
// virtiofs doesn't support chown from within containers, which breaks images
// like postgres that chown their data directory on init.

The skip was added in c0934d8 as a revert of a87cf87 (which had the right idea). The virtiofs/chown concern is real but applies equally to regular bind mounts, which compose already supports and which demonstrably work — including for postgres-style images. Silently losing data on every recreation is the worse failure mode, and the volume store directory is owned by the host user (exactly what mocker run -v name:/path already relies on).

Fix

Named volumes are now resolved to their backing directory and bind-mounted, exactly like Docker does internally:

  • name:/container/path-v <volumesPath>/<runtimeName>/_data:/container/path
  • runtimeName applies the project prefix unless the volume declares an explicit name: or is external: (reusing the existing ComposeVolume.runtimeName(projectName:) logic, so compose down --volumes removal stays in sync).

Changes

File Change
Sources/MockerKit/Compose/ComposeOrchestrator.swift resolveVolumeMounts now takes projectName, declaredVolumes and volumesPath; declared named volumes resolve to <volumesPath>/<runtimeName>/_data. Undeclared bare names are still dropped (previous behaviour).
Sources/MockerKit/Volume/VolumeManager.swift Expose mountpointPath (nonisolated) so compose can resolve volume backing dirs.
Tests/MockerKitTests/ComposeOrchestratorTests.swift Replaced the old "named volume skipped" expectation with coverage for declared, custom-named, external, undeclared, and mixed specs.
CHANGELOG.md Unreleased → Bug Fixes entry.

Verification

  • Unit: swift test — 448 tests in 35 suites, all green.
  • Integration (alpine with a named volume):
    services:
      app:
        image: alpine:3.20
        command: ["sh", "-c", "echo hello > /data/hello.txt && sleep 5 && cat /data/hello.txt"]
        volumes:
          - mydata:/data
    volumes:
      mydata:
    1. mocker compose uphello.txt appears in ~/.mocker/volumes/<project>-mydata/_data/ (previously the volume stayed empty).
    2. mocker compose up --force-recreatehello.txt survives (previously the data was lost).
    3. mocker compose down --volumes → volume removed, as expected.

Notes / trade-offs

  • Images that chown their data dir on init (e.g. official postgres) may need the host directory to be writable by the container user — the same constraint regular bind mounts already impose today. If virtiofs chown support is added later, no further changes are needed here.
  • Behavior is now consistent with Docker Compose: named volumes persist across recreation and are removed only by down --volumes.

Named volumes declared in compose files were silently skipped: the
volume store directory was created, but never bind-mounted into the
container, so writes went to the container's ephemeral layer and were
discarded on `compose up --force-recreate` (and on any container
recreation). That made stateful services like MySQL lose their data
whenever the stack was recreated.

Resolve named volumes to their backing directory
(`<volumesPath>/<runtimeName>/_data`, applying the project prefix or an
explicit `name:`/`external:` like Docker does) and bind-mount it, so
the data lives in the volume store and survives recreation. `down
--volumes` continues to remove them as before.

A previous attempt (a87cf87) was reverted (c0934d8) over concerns that
Apple's virtiofs cannot chown from inside the container, breaking
images like postgres that chown their data dir on init. That concern
applies equally to regular bind mounts, which compose already supports
and which work; silently discarding data on every recreation is the
worse failure mode. The volume store directory is owned by the host
user, matching what `mocker run -v name:/path` already does.

Tests: added unit coverage for declared, custom-named, external and
undeclared volume specs, plus an integration check that a file written
to a named volume survives `compose up --force-recreate`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant