Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 4 additions & 6 deletions .cargo/config.toml
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,7 @@ pr-check = "run -q -p xtask -- pr-check"
licenses = "run -q -p xtask -- licenses"
xtask = "run -q -p xtask --"

# The app and CLI default to the reference table executor, so plain
# `cargo check -p plotx` and `cargo run -p plotx` already are the fast path and
# need no alias. Shipping builds must opt into DataFusion, and the release
# workflow calls this alias instead of spelling the feature out, so the flag
# lives in one version-controlled place.
release-build = "build --locked --release -p plotx --features datafusion"
# Development and shipping builds use the reference table executor. DataFusion
# remains available through a separate, explicit maintenance build.
release-build = "build --locked --release -p plotx"
datafusion-build = "build --locked --release -p plotx --features datafusion"
13 changes: 1 addition & 12 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -118,12 +118,6 @@ jobs:
steps:
- uses: actions/checkout@v7

# `--all-features` builds plotx-substrait, which generates code with protoc.
- name: Install protoc
uses: arduino/setup-protoc@v3
with:
repo-token: ${{ secrets.GITHUB_TOKEN }}

- name: Update Rust toolchain
run: |
rustup update --no-self-update stable
Expand Down Expand Up @@ -154,11 +148,6 @@ jobs:
steps:
- uses: actions/checkout@v7

- name: Install protoc
uses: arduino/setup-protoc@v3
with:
repo-token: ${{ secrets.GITHUB_TOKEN }}

- name: Update Rust toolchain
run: |
rustup update --no-self-update stable
Expand All @@ -169,7 +158,7 @@ jobs:
save-if: ${{ github.ref == 'refs/heads/main' }}
cache-on-failure: true

- name: Run the test suite in both backend configurations
- name: Run the reference-backend test suite
run: cargo xtask pr-check test

# Validates that the user manual builds. Deployment is separate: Cloudflare
Expand Down
11 changes: 6 additions & 5 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,16 +58,17 @@ These instructions apply to the entire repository.
- `plotx-data` stays engine-free: never add datafusion, datafusion-substrait or
sqlparser to it. Backend adapters reach its internals through the
`#[doc(hidden)] pub` seams; keep that set minimal instead of widening the API.
- The app and CLI default to the reference table executor. DataFusion is an
opt-in `datafusion` feature, and releases are built with `cargo release-build`
so the flag lives in one place. Keep both configurations compiling.
- The app, CLI, and shipping builds use the reference table executor. DataFusion
remains available through the opt-in `datafusion` feature and the
`cargo datafusion-build` maintenance alias, but default PR checks and CI do
not compile or validate DataFusion or Substrait.
- Keep Rust source files below the repository's 800-line limit; prefer cohesive
modules over large files. Extract tests to a sibling `*_tests.rs` referenced
with `#[path]` when a file approaches it.
- Run `cargo pr-check` before completing code changes. It checks formatting,
source sizes, dependency licenses and advisories, a default-configuration
build of both frontends, Clippy with warnings denied, and the test suite in
both backend configurations. It requires `cargo-deny`.
build of both frontends, Clippy with warnings denied, and the reference-backend
test suite. It requires `cargo-deny`.

## Documentation

Expand Down
16 changes: 9 additions & 7 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,23 +20,25 @@ long wait — use the narrowest one that answers your question:
| Day-to-day development: build and run the app | `cargo run -p plotx` |
| Test the crate you changed | `cargo test -p plotx-core` |
| Optimized build for performance work | `cargo build --release -p plotx` |
| Shipping configuration, adds the DataFusion table backend | `cargo release-build` |
| Shipping configuration, uses the reference table executor | `cargo release-build` |
| Explicit DataFusion maintenance build | `cargo datafusion-build` |
| The same checks as CI, before a pull request | `cargo pr-check` |

Two things that are easy to trip over:

- Development builds keep the scientific processing crates optimized so they
stay usable under a debugger, but the UI and renderer are not. Judge frame
rate or rendering performance only with an optimized build.
- The app and CLI default to an in-memory reference table executor.
Shipping builds enable the DataFusion backend via `cargo release-build`;
the About window shows which engine is active.
- The app, CLI, and shipping build use the in-memory reference table executor.
DataFusion and Substrait remain in the workspace for explicit maintenance,
but `cargo pr-check` and CI do not build or validate them.

`cargo pr-check` requires `protoc` and `cargo-deny`
(`cargo install --locked cargo-deny`); nothing else does. CI runs exactly the
`cargo pr-check` requires `cargo-deny`
(`cargo install --locked cargo-deny`). CI runs exactly the
same steps, split into parallel jobs (`cargo xtask pr-check quick|lint|test`).
`cargo licenses` (requires `cargo-about`) regenerates
`dist/THIRD-PARTY-LICENSES.html`, the report of bundled third-party licenses.
`dist/THIRD-PARTY-LICENSES.html` from the reference-only Windows shipping app's
dependency graph.

The user manual is an Astro/Starlight site in `docs/` (`npm run build` to
validate). UI screenshots come from the built-in harness: point `PLOTX_SHOT`
Expand Down
6 changes: 2 additions & 4 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -14,12 +14,10 @@ members = [
"crates/app",
"xtask",
]
# Keep the Substrait exchange crate out of bare root builds: it is the only
# member that requires protoc, prost, and pbjson-types. CI and `cargo pr-check`
# use `--workspace`, so they still cover every crate.
# DataFusion and Substrait remain workspace members for explicit maintenance,
# but bare root builds stay on the lightweight reference executor.
default-members = [
"crates/data",
"crates/datafusion",
"crates/io",
"crates/analysis",
"crates/processing",
Expand Down
11 changes: 4 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,17 +45,14 @@ cd plotx
cargo run --release -p plotx
```

That is the fast development build. It uses the built-in reference table
executor, which keeps whole tables in memory and cannot spill, so large-table
work needs the shipping engine instead:
Development and shipping builds use the built-in reference table executor:

```sh
cargo release-build # the desktop app with the DataFusion backend
cargo release-build # the shipping desktop app
```

Releases are produced with `cargo release-build`. The About window always
names the active table engine, so you can tell which kind of build you are
running.
The dormant DataFusion backend remains available for explicit maintenance with
`cargo datafusion-build`, but it is not part of default builds or releases.

The development workflow and pre-submission checks are described in
[CONTRIBUTING.md](CONTRIBUTING.md).
Expand Down
4 changes: 2 additions & 2 deletions crates/app/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,8 @@ name = "plotx"
path = "src/main.rs"

[features]
# Default builds use the reference executor. Release builds enable DataFusion
# through `cargo release-build`; the About window shows the active executor.
# Development and shipping builds use the reference executor. This feature is
# retained as an explicit maintenance switch for the dormant DataFusion backend.
default = []
datafusion = ["plotx-core/datafusion"]

Expand Down
7 changes: 0 additions & 7 deletions crates/app/src/ui/canvas/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -414,13 +414,6 @@ fn welcome_page(app: &mut PlotxApp, ui: &mut Ui) {
egui::RichText::new("Scientific data analysis and figure preparation")
.color(ui.visuals().text_color()),
);
if !cfg!(feature = "datafusion") {
ui.add_space(8.0);
ui.label(
egui::RichText::new("Development build: large tables may run more slowly.")
.color(Color32::from_rgb(0xE0, 0x6C, 0x22)),
);
}
ui.add_space(22.0);
welcome_start(app, ui);
},
Expand Down
4 changes: 2 additions & 2 deletions crates/cli/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,8 @@ name = "plotx-cli"
path = "src/main.rs"

[features]
# Default builds use the reference executor. Enable this feature for
# spill-capable DataFusion execution on large tables.
# Default builds use the reference executor. This feature is retained as an
# explicit maintenance switch for the dormant DataFusion backend.
default = []
datafusion = ["plotx-core/datafusion"]

Expand Down
2 changes: 1 addition & 1 deletion deny.toml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
# Keep the license allow-list aligned with xtask/about.toml.

[graph]
all-features = true
all-features = false
# Every platform CI builds on. Releases currently ship for 64-bit Windows
# only, but the dependency policy must cover whatever CI compiles.
targets = [
Expand Down
48 changes: 25 additions & 23 deletions xtask/src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ fn pr_check(stage: Option<PrCheckStage>) -> Result<(), String> {
use PrCheckStage::{Lint, Quick, Test};
use PrCheckStep::{Cargo, DependencyPolicy, RustFileSizes};

let steps: [(PrCheckStage, PrCheckStep); 7] = [
let steps: [(PrCheckStage, PrCheckStep); 6] = [
(
Quick,
Cargo {
Expand Down Expand Up @@ -117,8 +117,11 @@ fn pr_check(stage: Option<PrCheckStage>) -> Result<(), String> {
args: &[
"clippy",
"--workspace",
"--exclude",
"plotx-datafusion",
"--exclude",
"plotx-substrait",
"--all-targets",
"--all-features",
"--locked",
"--quiet",
"--",
Expand All @@ -134,26 +137,10 @@ fn pr_check(stage: Option<PrCheckStage>) -> Result<(), String> {
args: &[
"test",
"--workspace",
"--all-features",
"--locked",
"--profile",
"pr-check",
"--quiet",
],
},
),
// `--all-features` always enables plotx-core/datafusion, so the
// reference executor that default builds depend on is never exercised
// above.
(
Test,
Cargo {
name: "test (reference backend)",
args: &[
"test",
"-p",
"plotx-core",
"--no-default-features",
"--exclude",
"plotx-datafusion",
"--exclude",
"plotx-substrait",
"--locked",
"--profile",
"pr-check",
Expand Down Expand Up @@ -207,6 +194,12 @@ fn licenses() -> Result<(), String> {
let about_args = [
"about",
"generate",
"--manifest-path",
"crates/app/Cargo.toml",
"--no-default-features",
"--target",
"x86_64-pc-windows-msvc",
"--locked",
"-c",
"xtask/about.toml",
"xtask/about.hbs",
Expand Down Expand Up @@ -289,7 +282,16 @@ fn cargo_output(repo_root: &Path, args: &[&str]) -> Result<Output, String> {
}

fn run_cargo_deny_step(repo_root: &Path, index: usize, total: usize) -> Result<(), String> {
let args = ["deny", "--locked", "check"];
let args = [
"deny",
"--locked",
"--no-default-features",
"--exclude",
"plotx-datafusion",
"--exclude",
"plotx-substrait",
"check",
];
let started_at = Instant::now();
print_step(index, total, "dependency policy")?;
let output = cargo_output(repo_root, &args)?;
Expand Down
Loading