Fit a z-stack from QuPath

CITATION.cff

@@ -12,7 +12,7 @@ authors:
     orcid: "https://orcid.org/0000-0003-4605-1682"
 type: software
 license: GPL-3.0-or-later
-version: 0.5.0
+version: 0.6.0
 date-released: 2026-08-21
 doi: 10.5281/zenodo.21954999
 identifiers:

README.md

@@ -109,6 +109,40 @@ POST /v1/datasets   {"path": "/path/R_2_flimkit"}
 
 A region drawn on that canvas is then fitted from the photons in the stitched cube, not from the displayed image.
 
+## Fitting a z-stack from QuPath
+
+`Fit a z-stack...` points FLIMKit's z-stack pipeline at a folder of slices, the same folder you would hand its own z-stack fit. One file per slice, named `region_z1.ptu`, `region_z2.ptu` and so on; `region_t1_s1_z1.ptu` works too, and a folder holding several regions is fitted as several stacks in one run.
+
+```
+POST /v1/zstack/scan   {"ptu_dir": "/path/slices"}
+POST /v1/zstack        {"ptu_dir": "/path/slices", "params": {"n_exp": 2, "z_step_um": 2.0}}
+GET  /v1/zstack/defaults
+```
+
+The scan runs before the settings dialog, so a folder with nothing in it says so, and a folder with something in it says how many stacks and how many slices each before anything is fitted. A directory chooser is easy to miss with, and the fit is long enough that finding out afterwards is expensive.
+
+Each stack is fitted as one FOV, which is the point of doing it this way rather than fitting each slice on its own. The decay is pooled over every slice, the lifetimes are fitted once from that pooled decay and then locked, and each slice gets its own per-pixel fit with only the amplitudes free. A single slice rarely carries the photons to identify two lifetimes; the whole stack usually does. `ref_tau1_ns` and the rest skip the pooled fit and lock lifetimes you already know, and they have to be given for every component or none.
+
+The result comes back as one OME-Zarr store per stack, a `(C, Z, Y, X)` volume with a channel per map: intensity in photons, `tau_mean_int` and `tau_mean_amp` in nanoseconds, an `alpha_N` per component, and the bound fraction and chi-squared maps when they were computed. QuPath opens it as a real z-stack, so the z-slider moves through the slices and an annotation carries the plane it was drawn on. `z_step_um` sets the z spacing in the store; the lateral spacing is read from the first slice's own pixel size.
+
+Zarr because these get large. A 12 slice stack of 512 square maps with eight channels is 100 MB before anything is compressed, and a mosaic of them is worse. The store is chunked per slice per channel, so a viewer reads the plane it is showing and not the volume. It is written at a single resolution deliberately: a pyramid is built by interpolating between neighbours, and a lifetime map is mostly NaN where nothing was fitted, so the coarse levels would smear that NaN over the pixels that did fit.
+
+`zarr` and `ome-zarr` are dependencies of `flimkit-bridge` itself rather than an extra, so `pip install flimkit-bridge` is all this needs and a bridge that cannot write a store does not exist.
+
+`ome-tiff` is the other choice in the dialog, same channels and same axes, for a QuPath or a Fiji that will not read the store. Nothing is lost by picking it beyond the chunking. A run that has already happened can be rewritten in the other format without fitting again:
+
+```
+POST /v1/zstack/export   {"group_dir": "/path/out/RegionA", "format": "ome-tiff", "z_step_um": 2.0}
+```
+
+which is also how to correct a z step that was entered wrong.
+
+The volume reaches QuPath as a path, not as bytes: the bridge writes the store and QuPath opens it off disk, the same way the stitch pipeline hands over its products. Over an SSH-forwarded port that path is meaningless locally, so when QuPath cannot open it the extension falls back to `GET /v1/zstack/volume.ome.tif`, which streams the same volume as one OME-TIFF and stores it beside the project. The notification says which stacks came that way. The slices themselves are never uploaded: `ptu_dir` is a path FLIMKit opens, so a forwarded session has to name a folder the bridge can see, and gets a 404 naming the folder if it does not.
+
+The `.npy` maps FLIMKit writes per slice stay where they are, and so do its `_zseries.csv` and `_zseries.json`, which carry the mean lifetime, the bound fraction and the fitted pixel count per slice. The project entry description names both, alongside the locked lifetimes and the voxel size, so the depth trend is one file away rather than something to reconstruct.
+
+Per-slice PNGs, RGB composites and detail plots are off by default here. FLIMKit writes them for its own review; the bridge is feeding QuPath, and each one costs a second summed fit per slice. `Save per-slice plots` in the advanced settings turns them back on.
+
 ## Where the server lives
 
 The server moved out into [flimkit-bridge](https://github.com/FLIMKit/flimkit-bridge) once there were two clients for it, so a fix reaches this extension and the Fiji one at the same time. This repository keeps the QuPath extension and the catalog.

catalog.json

@@ -8,6 +8,16 @@
             "author": "Alex Hunt",
             "homepage": "https://github.com/FLIMKit/flimkit-qupath-bridge",
             "releases": [
+                {
+                    "name": "v0.6.0",
+                    "main_url": "https://github.com/FLIMKit/flimkit-qupath-bridge/releases/download/v0.6.0/qupath-extension-flimkit-bridge-0.6.0.jar",
+                    "javadoc_urls": [
+                        "https://github.com/FLIMKit/flimkit-qupath-bridge/releases/download/v0.6.0/qupath-extension-flimkit-bridge-0.6.0-javadoc.jar"
+                    ],
+                    "version_range": {
+                        "min": "v0.7.0"
+                    }
+                },
                 {
                     "name": "v0.5.0",
                     "main_url": "https://github.com/FLIMKit/flimkit-qupath-bridge/releases/download/v0.5.0/qupath-extension-flimkit-bridge-0.5.0.jar",

pyproject.toml

@@ -4,7 +4,7 @@ build-backend = 'setuptools.build_meta'
 
 [project]
 name = 'flimkit-qupath-bridge'
-version = '0.5.0'
+version = '0.6.0'
 description = 'Direct image and ROI exchange between FLIMKit and QuPath'
 readme = 'README.md'
 requires-python = '>=3.12'

qupath-extension/build.gradle.kts

@@ -31,7 +31,7 @@ repositories {
 qupathExtension {
     name = "qupath-extension-flimkit-bridge"
     group = "io.github.flimkit"
-    version = "0.5.0"
+    version = "0.6.0"
     description = "Direct image and ROI exchange between FLIMKit and QuPath"
     automaticModule = "io.github.flimkit.bridge"
 }

qupath-extension/src/main/java/io/github/flimkit/bridge/BridgeClient.java

@@ -61,6 +61,71 @@ public String runPipeline(String body) throws IOException, InterruptedException
         return response.body();
     }
 
+    public String zstackDefaults() throws IOException, InterruptedException {
+        var response = client.send(
+                request("/v1/zstack/defaults").GET().build(),
+                HttpResponse.BodyHandlers.ofString(StandardCharsets.UTF_8));
+        if (response.statusCode() != 200)
+            throw new IOException("GET z-stack defaults returned " + response.statusCode()
+                    + ": " + response.body());
+        return response.body();
+    }
+
+    public String scanZstack(String ptuDir) throws IOException, InterruptedException {
+        return postZstack("/v1/zstack/scan", "{\"ptu_dir\":" + quote(ptuDir) + "}");
+    }
+
+    public String runZstack(String body) throws IOException, InterruptedException {
+        return postZstack("/v1/zstack", body);
+    }
+
+    public String exportZstack(String body) throws IOException, InterruptedException {
+        return postZstack("/v1/zstack/export", body);
+    }
+
+    private String postZstack(String path, String body)
+            throws IOException, InterruptedException {
+        var response = client.send(
+                request(path)
+                        .header("Content-Type", "application/json")
+                        .POST(HttpRequest.BodyPublishers.ofString(body, StandardCharsets.UTF_8))
+                        .build(),
+                HttpResponse.BodyHandlers.ofString(StandardCharsets.UTF_8));
+        if (response.statusCode() != 200)
+            throw new IOException("POST " + path + " returned " + response.statusCode()
+                    + ": " + response.body());
+        return response.body();
+    }
+
+    /**
+     * Downloads one fitted z-stack as an OME-TIFF, for a QuPath that cannot
+     * reach the bridge's own disk. Streamed straight to a file: a stack is
+     * routinely larger than the heap.
+     */
+    public FetchedImage fetchZstackVolume(String groupDir, String label,
+                                          double zStepUm, double pixelSizeUm)
+            throws IOException, InterruptedException {
+        String path = "/v1/zstack/volume.ome.tif?group_dir="
+                + URLEncoder.encode(groupDir, StandardCharsets.UTF_8)
+                + "&z_step_um=" + zStepUm;
+        if (label != null && !label.isBlank())
+            path += "&label=" + URLEncoder.encode(label, StandardCharsets.UTF_8);
+        if (pixelSizeUm > 0)
+            path += "&pixel_size_um=" + pixelSizeUm;
+        Path file = Files.createTempFile("flimkit-zstack-", ".ome.tif");
+        var response = client.send(
+                request(path).timeout(Duration.ofHours(1)).GET().build(),
+                HttpResponse.BodyHandlers.ofFile(file));
+        if (response.statusCode() != 200) {
+            String why = Files.exists(file) ? Files.readString(file) : "";
+            Files.deleteIfExists(file);
+            throw new IOException("GET z-stack volume returned "
+                    + response.statusCode() + ": " + why);
+        }
+        return new FetchedImage(label == null ? "z-stack" : label,
+                response.body(), "ns");
+    }
+
     public String jobStatus(String jobId) throws IOException, InterruptedException {
         var response = client.send(
                 request("/v1/jobs/" + jobId).GET().build(),

qupath-extension/src/main/java/io/github/flimkit/bridge/FlimKitBridgeExtension.java

@@ -39,7 +39,7 @@ public class FlimKitBridgeExtension implements QuPathExtension, GitHubProject {
     private String token = "";
     private boolean manual = false;
 
-    static final String EXTENSION_VERSION = "0.5.0";
+    static final String EXTENSION_VERSION = "0.6.0";
 
     static final int PROTOCOL_VERSION = 1;
 
@@ -68,6 +68,7 @@ public void installExtension(QuPathGUI qupath) {
                 menuItem("Add FLIMKit images to project", () -> addImages(qupath)),
                 null,
                 menuItem("Stitch and fit a mosaic...", () -> stitchAndFit(qupath)),
+                menuItem("Fit a z-stack...", () -> fitZstack(qupath)),
                 menuItem("Fit ROI decays...", () -> fitRois(qupath)),
                 menuItem("Fit per-pixel lifetimes...", () -> fitPixels(qupath)),
                 menuItem("Phasor plot...", () -> openPhasor(qupath)),
@@ -509,6 +510,191 @@ private void stitchAndFit(QuPathGUI qupath) {
         }
     }
 
+    private void fitZstack(QuPathGUI qupath) {
+        var chooser = new javafx.stage.DirectoryChooser();
+        chooser.setTitle("Choose the folder holding the z-slices");
+        var folder = chooser.showDialog(null);
+        if (folder == null)
+            return;
+        try {
+            var client = client();
+            var found = JsonParser.parseString(
+                    client.scanZstack(folder.getAbsolutePath())).getAsJsonObject();
+            int stacks = found.get("n_stacks").getAsInt();
+            if (stacks == 0) {
+                Dialogs.showErrorMessage(getName(),
+                        "No z-stack slices in " + folder.getName() + ".\n\n"
+                                + "FLIMKit reads one file per slice, named "
+                                + "region_z1.ptu, region_z2.ptu and so on "
+                                + "(region_t1_s1_z1.ptu also works).");
+                return;
+            }
+            int slices = found.get("n_slices").getAsInt();
+            if (!Dialogs.showConfirmDialog(getName(),
+                    describeStacks(found) + "\n\nEach stack is fitted as one FOV: "
+                            + "the decay is pooled over its slices, the lifetimes "
+                            + "are fitted once and locked, and every slice then "
+                            + "gets its own per-pixel maps."))
+                return;
+            var defaults = JsonParser.parseString(
+                    client.zstackDefaults()).getAsJsonObject();
+            var chosen = new FitDialog(defaults).prompt("Fit a z-stack", "zstack");
+            if (chosen == null)
+                return;
+            var body = new JsonObject();
+            body.addProperty("ptu_dir", folder.getAbsolutePath());
+            body.add("params", chosen);
+            var started = JsonParser.parseString(
+                    client.runZstack(body.toString())).getAsJsonObject();
+            String jobId = started.get("job").getAsString();
+            String outputDir = started.get("output_dir").getAsString();
+            new BridgeJob(client, jobId,
+                    "FLIMKit: " + stacks + " z-stack(s), " + slices + " slices").watch(
+                    status -> importVolumes(qupath, client, jobId, outputDir),
+                    problem -> {
+                        if (problem == null)
+                            Dialogs.showInfoNotification(getName(), "Cancelled");
+                        else
+                            Dialogs.showErrorMessage(getName(),
+                                    "Z-stack fit failed\n\n" + problem);
+                    });
+        } catch (Exception e) {
+            logger.error("Could not start the z-stack fit", e);
+            Dialogs.showErrorMessage(getName(),
+                    "Could not start the z-stack fit\n\n" + e.getMessage());
+        }
+    }
+
+    static String describeStacks(JsonObject found) {
+        var lines = new ArrayList<String>();
+        int stacks = found.get("n_stacks").getAsInt();
+        lines.add(stacks == 1 ? "Found 1 z-stack:" : "Found " + stacks + " z-stacks:");
+        for (var element : found.getAsJsonArray("stacks")) {
+            var stack = element.getAsJsonObject();
+            lines.add("  " + stack.get("label").getAsString() + " - "
+                    + stack.get("n_slices").getAsInt() + " slices (z "
+                    + stack.get("z_first").getAsInt() + " to "
+                    + stack.get("z_last").getAsInt() + ")");
+        }
+        return String.join("\n", lines);
+    }
+
+    private void importVolumes(QuPathGUI qupath, BridgeClient client,
+                               String jobId, String outputDir) {
+        List<ZStackResults.Volume> volumes = List.of();
+        var problems = new ArrayList<String>();
+        try {
+            var result = resultOf(client.jobResult(jobId));
+            volumes = ZStackResults.volumes(result);
+            problems.addAll(ZStackResults.problems(result));
+        } catch (Exception e) {
+            logger.warn("Could not read the z-stack result", e);
+            problems.add("could not read the job result: " + e.getMessage());
+        }
+        var project = qupath.getProject();
+        if (project == null || volumes.isEmpty()) {
+            String why = project == null
+                    ? "\n\nOpen a project and the volumes can be added to it "
+                            + "automatically."
+                    : "";
+            Dialogs.showInfoNotification(getName(),
+                    "Finished. FLIMKit wrote its z-stacks to\n" + outputDir + why
+                            + report(problems));
+            return;
+        }
+        var manifest = ProjectManifest.open(project);
+        var added = new ArrayList<String>();
+        var fetched = new ArrayList<String>();
+        for (var volume : volumes) {
+            String name = volume.label() + " z-stack";
+            Path stored = null;
+            try {
+                var entry = ProjectImporter.addVolume(project,
+                        ZStackResults.openableAs(volume.file(), volume.format()), name);
+                stored = volume.file();
+                describeVolume(entry, volume, outputDir);
+                added.add(entry.getImageName());
+            } catch (Exception e) {
+                logger.info("Could not open {} from disk, fetching it instead: {}",
+                        volume.file(), e.getMessage());
+                try {
+                    stored = fetchVolume(project, client, volume);
+                    var entry = ProjectImporter.addNamed(project, stored, name);
+                    describeVolume(entry, volume, outputDir);
+                    added.add(entry.getImageName());
+                    fetched.add(volume.label());
+                } catch (Exception fetchFailed) {
+                    logger.warn("Could not add {}", volume.label(), fetchFailed);
+                    problems.add(volume.label() + ": " + fetchFailed.getMessage());
+                    continue;
+                }
+            }
+            if (manifest != null)
+                manifest.recordImage(volume.label(), stored.toString(),
+                        String.join(", ", volume.labelled()), outputDir);
+        }
+        try {
+            project.syncChanges();
+        } catch (IOException e) {
+            logger.error("Could not save the project", e);
+        }
+        saveManifest(manifest);
+        qupath.refreshProject();
+        if (added.isEmpty()) {
+            Dialogs.showErrorMessage(getName(),
+                    "Fitted, but nothing could be added from\n" + outputDir
+                            + report(problems)
+                            + "\n\nRun the fit again with Volume format set to "
+                            + "ome-tiff if QuPath cannot read the Zarr store.");
+            return;
+        }
+        String note = fetched.isEmpty() ? ""
+                : "\n\nDownloaded over the bridge as OME-TIFF, since this QuPath "
+                        + "cannot reach FLIMKit's disk: " + String.join(", ", fetched);
+        Dialogs.showInfoNotification(getName(),
+                "Added " + String.join(", ", added) + note + report(problems));
+    }
+
+    private Path fetchVolume(Project<BufferedImage> project, BridgeClient client,
+                             ZStackResults.Volume volume) throws IOException {
+        if (!volume.canBeFetched())
+            throw new IOException("QuPath cannot open " + volume.file()
+                    + " and this bridge is too old to send it over the connection");
+        try {
+            var downloaded = client.fetchZstackVolume(
+                    volume.groupDir(), volume.label(), volume.zStepUm(),
+                    volume.pixelSizeUm());
+            return ProjectImporter.storeBesideProject(
+                    project, volume.label() + "_zstack", downloaded.file(),
+                    ".ome.tif");
+        } catch (InterruptedException e) {
+            Thread.currentThread().interrupt();
+            throw new IOException("interrupted while downloading the volume");
+        }
+    }
+
+    private static void describeVolume(
+            qupath.lib.projects.ProjectImageEntry<BufferedImage> entry,
+            ZStackResults.Volume volume, String outputDir) {
+        entry.setDescription(volume.describe());
+        entry.getMetadata().put("flimkit.source", outputDir);
+        entry.getMetadata().put("flimkit.n_z", String.valueOf(volume.nZ()));
+        if (!volume.tausNs().isEmpty())
+            entry.getMetadata().put("flimkit.taus_ns", joinNumbers(volume.tausNs()));
+    }
+
+    private static String report(List<String> problems) {
+        return problems.isEmpty() ? ""
+                : "\n\nNot added:\n" + String.join("\n", problems);
+    }
+
+    private static String joinNumbers(List<Double> values) {
+        var parts = new ArrayList<String>();
+        for (Double value : values)
+            parts.add(String.format("%.4f", value));
+        return String.join(", ", parts);
+    }
+
     private void fitPixels(QuPathGUI qupath) {
         var imageData = qupath.getImageData();
         if (imageData == null) {