Skip to content

fix(tiles): an export waits for an MBTiles file's metadata as for its tiles - #1807

Closed
Mavrick-F wants to merge 1 commit into
opengeos:mainfrom
Mavrick-F:fix/mbtiles-metadata-in-flight
Closed

Mavrick-F wants to merge 1 commit into
opengeos:mainfrom
Mavrick-F:fix/mbtiles-metadata-in-flight

Conversation

@Mavrick-F

@Mavrick-F Mavrick-F commented Nov 11, 2026 •

Copy link
Copy Markdown
Member

Summary

An MBTiles file's metadata (its credits, and what the layer needs before it can draw) is read on a thread of its own, and that read didn't count as under way in cache::in_flight(). An export's complete render (render::complete) stops waiting once nothing is in flight and nothing more lands, so it could find the metadata still loading, nothing in flight, and give up within milliseconds: the layer came out as a gap ("some tiles were still on their way when the export stopped waiting…").

The read now counts as under way from before its thread starts until what it read is kept, as a PMTiles header read (fetch_local) already does. cache::UnderWay is that count: it ends, and wakes the map, when dropped, so a thread that fails to start or a read that panics can't leave it stuck. It is native only; the browser reads the metadata in place.

Testing

  • New mbtiles::tests::a_metadata_read_counts_as_under_way_while_it_runs looks at in_flight() from inside the metadata worker (through the existing Still check), so it doesn't depend on timing. It fails without the fix ("the worker's own read wasn't counted as under way") and passes with it.
  • metadata_ready_checks_still_outside_the_pending_lock now waits for the read it starts, so the count is back before the next test that holds the cache's turn compares in_flight().
  • cargo test -p rustgis-tiles: 436 passed. fmt, clippy and the wasm build check of rustgis-tiles.
  • The pull request's checks, run on a fork before opening: https://github.com/Mavrick-F/RustGIS/actions/runs/43091103691
  • On Windows ARM64, where the test this fixes failed most (about 1 full rustgis-engine lib run in 8 before), 12 full runs with this fix all passed: https://github.com/Mavrick-F/RustGIS/actions/runs/43084074971 (the Windows x64 job's log was lost by GitHub; it is being run again).

UI evidence

No visible UI change.

Related issue

Fixes #1801. Part of getting the nightly green again (#1776).

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Bug Fixes
    • Map readiness now accounts for metadata reads that are still running, helping ensure the map does not appear ready before those reads finish. Waiting operations are notified after results are cached or the read worker exits. Reads whose workers fail to start are not counted as ongoing. This makes map loading status more accurately reflect outstanding metadata work.

… tiles

An MBTiles file's metadata (its credits, and what the layer needs to draw) is read on a thread of
its own, which didn't count as under way. An export's complete render stops waiting once nothing
is in flight and nothing more lands, so it could find the metadata still loading, nothing in
flight, and give up within milliseconds with the layer missing. The read now counts as under way
from before its thread starts until what it read is kept, as a PMTiles header read already does.

Fixes opengeos#1801.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

Explain Change →

Caution

Review failed

The pull request is closed.

Walkthrough

On desktop, MBTiles metadata reads now contribute to the cache’s in-flight count while their worker runs. The count ends after the worker attempts to retain the result and before pending-read waiters are notified. A test checks that the worker appears in the count.

Changes: MBTiles in-flight accounting

Layer / File(s) Summary
Count metadata worker lifetime
crates/tiles/src/cache.rs, crates/tiles/src/mbtiles.rs
The native-only UnderWay guard increments the in-flight count and decrements it on drop, notifying the map. The metadata worker uses the guard around its work. Tests wait for a pending-read worker to finish and check that a metadata worker appears in the in-flight count.

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Bug fix · Severity of issue fixed: Medium

Possibly related issues

  • opengeos/RustGIS#1761: This PR counts MBTiles metadata reads in flight, addressing the issue’s uncounted-work behavior for that path.

Possibly related PRs

  • opengeos/RustGIS#1805: This PR uses a different drop guard for the same MBTiles metadata in-flight accounting.
  • opengeos/RustGIS#1765: This PR adds similar in-flight accounting to the desktop MBTiles metadata worker.

Suggested reviewers: mgaux-dev, giswqs

Merge Risk: Minimal · up to 5f545

The change makes exports wait for MBTiles metadata reads on desktop, as they already do for tile reads. No actionable merge-blocking risk was identified.

- [ ] Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

A rabbit counts each worker’s flight,
And keeps the waiting map in sight.
When metadata has had its turn,
The count drops as the waiters learn.
Then off the rabbit hops at night.

Comment on lines +333 to 337
// Over once what it read is kept, before its waiters wake: whoever sees it over
// sees the metadata, and a caller that waited sees it no longer counted.
drop(under_way);
pending.lock().unwrap_or_else(|e| e.into_inner()).remove(&worker_key);
ready.notify_all();

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Possible lost wake-up (confidence: moderate, ~40%). drop(under_way) runs finished(), which bumps the generation, decrements IN_FLIGHT and wakes the map, all before pending.remove and ready.notify_all(). The old trailing crate::cache::notify() (which came after the removal) is also gone, so the last wake now comes before the key leaves the pending set.

When remember_info doesn't store the result (the epoch moved, or check_still refused, as the test at mbtiles.rs:475 arranges), a render woken by that bump can run in the gap and see the key still pending. info_ready then returns Ok(None) and nothing wakes the map after the removal. The export loop recovers because its next render re-spawns the read. The interactive map has no such recovery in this path.

Dropping the count last fixes the ordering: anyone who sees in_flight() == 0 also sees the read off the pending set, and the final wake follows the removal.

Suggested change
// Over once what it read is kept, before its waiters wake: whoever sees it over
// sees the metadata, and a caller that waited sees it no longer counted.
drop(under_way);
pending.lock().unwrap_or_else(|e| e.into_inner()).remove(&worker_key);
ready.notify_all();
// Over last: once the read is off the pending set and its waiters are woken, so the
// wake that ends the count follows the removal, and whoever sees it over sees both.
pending.lock().unwrap_or_else(|e| e.into_inner()).remove(&worker_key);
ready.notify_all();
drop(under_way);

@github-actions

Copy link
Copy Markdown
Contributor

Code review

The change is small and sound in its main idea: an MBTiles metadata read now counts as under way, so an export waits for it instead of giving up. I found one ordering concern, posted inline. Prior reviews: none, so no earlier concerns were left alone.

Bugs

  • Wake-up ordering in the metadata worker (crates/tiles/src/mbtiles.rs:333-337, posted inline). drop(under_way) wakes the map before the pending key is removed, and the trailing notify() is gone. If the result isn't stored (epoch change or a refused still), a render in the gap sees the key still pending and nothing wakes it afterwards. The export recovers on its next render; the interactive map has no such recovery. Suggested fix: drop the count last. Confidence: moderate, about 40%.

Security

  • None found. The change touches no input parsing or external data.

Performance

  • None significant. The read already ran on its own thread; the change adds one atomic increment and decrement.
  • Behaviour note: an export whose metadata read stalls now waits out its existing 60-second budget instead of returning early. This is bounded and intended. Confidence: high.

Quality

  • finished()'s doc comment (cache.rs:1773, "A fetch is over") now also covers metadata reads. Left for the release quality pass.
  • The new test a_metadata_read_counts_as_under_way_while_it_runs asserts >= 1 from inside the worker. It doesn't check that the count returns to its baseline after the read ends. Left for the release quality pass.

AGENTS.md

  • No violations found. The change is native-only behind cfg(not(wasm32)), uses web_time-free code, adds no unwrap/expect/panic/unsafe, and has a regression test that fails without the fix.
  • Not verified: the waker runs while the pending lock is held on the spawn-failure path. Whether that can deadlock depends on the app's waker, which isn't in this repository. Confidence in a problem: about 15%.

Change size

Compared with main (merge base b373948 to PR head 5f54546).

Measure Value
Files changed 2 (0 added, 2 modified, 0 removed, 0 renamed)
Size before 336.7 KB
Size after 339.2 KB
Total size change +2.4 KB
Path Before After Change
crates/tiles/src/cache.rs 309.7 KB 310.6 KB +942 B
crates/tiles/src/mbtiles.rs 27.0 KB 28.6 KB +1.5 KB

@Mavrick-F

Copy link
Copy Markdown
Member Author

Superseded by #1765, which merged the same fix for #1761 (and also spawns the metadata thread unlocked, so a waker can't deadlock). Closing this one.

@Mavrick-F Mavrick-F closed this Nov 11, 2026
@Mavrick-F
Mavrick-F deleted the fix/mbtiles-metadata-in-flight branch November 11, 2026 21:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

An export can stop waiting at once for an MBTiles tile that is pending but not being read

1 participant