Skip to content
surefoldPublic

About

Composable computation engine with incremental execution, GPU acceleration and multi-GPU scenario processing. Describe a calculation once; choose how it runs afterwards.

Topics

Resources

Contributing

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

Watershed

CI Licence: Apache 2.0

Describe a calculation once. Choose how it runs afterwards.

Watershed is a computation engine. You write a model as ordinary typed formulas, and the same model then updates in microseconds when an input changes, runs across millions of scenarios on CPU threads or GPUs, and spreads over several GPUs when the batch grows. None of those needs the model rewritten.

Try it in your browser · Get started · Guide

On one model of 19.9 million calculations
58 µs to update after one record changes
10.2 s for 10,000 scenarios on one GPU
70 s for 2,000,000 scenarios on eight GPUs

Try it in your browser

The playground is the engine itself, compiled to WebAssembly. Nothing to install.

The playground on a model of an online order. The price was changed: four of five formulas were recalculated, the update stopped at free_shipping, whose value came out the same, and shipping was never looked at. Below, the same model run as a plan over a million prices in 126 ms.

  • Change an input on four example models and see which formulas are recalculated, where an update stops, and which are never looked at.
  • Ask any value why. Click a node for the tree of values behind it.
  • Run a model as scenarios. The same formulas, compiled to a plan and run over up to a million values of one input. In the screenshot, a million took 126 ms in a browser on one thread.
  • Time it at scale. Build a graph of a million nodes and compare one change against recalculating everything.
  • Watch it live. A crypto book priced from a public feed, every Wikipedia edit as it happens, and a data grid of 50,000 rows where every cell is a formula.

What you can build

Each of these is a working example in the repository, on made-up data, with its checks and timings.

  • A book that changes all day. Amendments, corrections and late trades set on the model, each costing what it touches: 58 µs on a book of a million policies.
  • Risk across thousands of market paths. An insurance book, a loan book and a pension scheme valued along 10,000 paths on one GPU, with stress tests that replace a few assumptions and run again.
  • A search over a large population. A thousand price reviews over five million policies in 5 seconds, with the population loaded onto the GPU once.
  • Results that stay current. After a batch, one input changes and all 10,000 results come up to date in 0.2 s, where a rerun takes 11.2 s.
  • A spreadsheet, ten million times. Formulas read from an .xlsx workbook and run as a plan.
  • Live tables in the browser. A grid whose cells, columns and totals are formulas, updated as edits arrive, as a TypeScript package with components for Solid and React. The cookbook shows it in an app.

The guide has each with its figures.

How it works

You describe the calculation. Watershed records what every formula reads, and everything else follows from that record.

use watershed_core::prelude::*;
use watershed_gpu::Scenarios;

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let mut g = Graph::new();
    let qty = g.input("qty", 10.0)?;
    let price = g.input("price", 2.5)?;
    let tax = g.input("tax_rate", 0.2)?;
    let subtotal = g.define("subtotal", qty * price)?;
    let total = g.define("total", subtotal * (1.0 + tax))?;

    // Change one input. Only what depends on it is recalculated.
    g.set(price, 3.0)?;
    let report = g.stabilize();
    assert_eq!(g.get(total)?, 36.0);
    assert_eq!(report.recalculated, 2); // subtotal and total; qty and tax_rate were not visited
    println!("{}", g.explain(total));

    // The same model for three prices at once, on every core. The graph is not touched.
    let plan = Scenarios::new(&g, &[price.id()], &[total.id()])?;
    assert_eq!(plan.run_cpu(&[2.5, 3.0, 4.0]), [30.0, 36.0, 48.0]);
    assert_eq!(g.get(total)?, 36.0);
    Ok(())
}
total = 36  ← subtotal * (1 + tax_rate)  [changed]
├── tax_rate = 0.2  (input)
└── subtotal = 30  ← qty * price  [changed]
    ├── qty = 10  (input)
    └── price = 3  (input)  [changed]

Expressions are typed: adding a number to a string does not compile, and neither does setting a calculated node.

An update follows the record forwards from a change, lowest level first, and stops at any value that comes out the same. A plan follows it backwards from the outputs you want and compiles what they need for a batch of scenarios. The two are held to the same numbers by tests.

The engine on a small model of an order, captured from the browser build. One input changes; the formulas it reaches are recalculated, an update stops where a value comes out the same, and the rest are not looked at. Then the same model runs as a plan for three prices.

More: internals, how plans work, testing and correctness.

Execution

The same model, run the way the question needs.

  • Incrementally. Work in proportion to the change, with dynamic dependencies and early cutoff, across cores for large updates.
  • Across scenarios. One plan for thousands to millions of input sets, on every CPU core.
  • On a GPU. The plan's kernels are generated for it; you write no GPU code. A plan and its data can stay on the GPU between runs, with results totalled there.
  • Fused. Groups of formulas that feed one another run in one GPU thread, keeping intermediate values out of memory. Experimental.
  • Refreshed. After a batch, only the part of the plan a change reaches is rerun. Experimental.
  • Across several GPUs. A saved plan runs in lots, one worker per GPU, with a lot moved if a worker stops. Experimental; run on eight GPUs in one machine.

You choose where a plan runs. Choosing automatically from measured costs is research in progress.

Status

State Where
Incremental updates, typed values, collections and grouping Built core guide
History, explanation, change sinks, formulas as text Built core guide
Scenario plans on CPU threads and CUDA; data resident on the GPU Built watershed-gpu, how plans work
Python, Kotlin, Scala and WebAssembly interfaces to the incremental engine Built Architecture
Refreshing a batch after a change Experimental watershed-gpu
Fused kernels Experimental, off by default watershed-gpu
Saved plans, and lots across GPUs Experimental; run on eight GPUs in one machine watershed-gpu, watershed-jvm
Scenario plans from Kotlin, and in the browser on one thread Experimental watershed-jvm, grid README
Reading a model from an .xlsx workbook Experimental; one workbook tried watershed-gpu
Choosing CPU or GPU automatically from measured costs Research research note
Workers on several machines; a release on crates.io and PyPI Planned

Performance

Three measurements, one model: lifelib's savings.CashValue_ME_EX2, 20,000 groups of policies projected month by month, 19.9 million calculations, on synthetic data.

What Result On
One record changes 58 µs; 570 calculations rerun Ryzen 9 5950X, median of five runs
10,000 scenarios 10.2 s, including transfers and kernel compilation One RTX 3090, median of five runs
2,000,000 scenarios 18 min 46 s on one GPU, 2 min 43 s on eight, 70 s on eight with fused kernels A100 (40 GB) GPUs in one machine; eight plain twice, the others once

Every result is checked: the model agrees with lifelib to 3.1 × 10⁻¹³, and a plan on CPU threads is bit-identical to the graph.

The full picture, with hardware, method, the cases where CPU threads are the better choice, and what did not pay: benchmark study, eight-GPU study and docs/benchmarks.md.

Get started

Watershed is pre-release. Nothing is published to crates.io, PyPI or Maven yet, so every route builds from source and needs a Rust toolchain (1.85 or later).

Install Covers
Rust A git dependency The whole engine
Python One pip install from the repository Incremental engine, collections, formulas as text
Kotlin, Scala Clone watershed-jvm; needs JDK 25 and sbt Incremental engine; scenario plans in Kotlin, experimental
Browser Nothing: the playground The example models, updates and plans on one thread

Rust. Add the crates and run the example above:

[dependencies]
watershed-core = { git = "https://github.com/surefold/watershed" }
watershed-gpu  = { git = "https://github.com/surefold/watershed" }   # scenario plans; CPU threads by default

For a model of realistic size, in a clone of this repository, with no data files:

cargo run --release -p watershed-gpu --example loans -- 2000 1000     # 2,000 loans, 1,000 scenarios, on CPU threads
cargo run --release -p watershed-gpu --features cuda --example loans  # 20,000 × 10,000 on an NVIDIA GPU

Python. Python 3.9 or later.

pip install "git+https://github.com/surefold/watershed#subdirectory=crates/watershed-py"
import watershed as ws

g = ws.Graph()
qty = g.input("qty", 10.0)
price = g.input("price", 2.5)
total = g.define("total", qty * price * 1.2)

g.set(price, 3.0)
g.stabilize()
print(total.value)       # 36.0
print(g.explain(total))  # the tree behind it

More in the Python guide.

Kotlin and Scala. sbt exampleKotlin/run and sbt exampleScala/run in watershed-jvm run a first program in each.

Browser. The grid package holds rows in a Table and draws them, with components for Solid and React. The cookbook has whole examples that run on the page, in Solid and React: rows from an API, loading as you scroll, saving edits back, WebSocket and SSE streams, charts and a table in a worker. To build the WebAssembly module and the pages yourself, see the grid README.

The guide has the same example in every language, with what each prints.

Documentation

Guide and playground surefold.dev/guides/watershed, surefold.dev/watershed
The engine, feature by feature core guide, scenario plans
Examples loans, liabilities, capital and the rest; cargo run --example tour
API reference cargo doc --open -p watershed-core (or -p watershed-gpu); Python type stubs
How it is built internals, how plans work, testing
Benchmarks and how to reproduce them docs/benchmarks.md
Limits docs/limitations.md
Contributing CONTRIBUTING.md

Architecture

Component Location Role
Core crates/watershed-core The graph, incremental update, collections, decimals, history, formulas. No unsafe.
Scenario plans crates/watershed-gpu Plans on CPU threads; the cuda feature adds the NVIDIA GPU executor. CUDA is found at run time, so the workspace builds without it.
Parquet sink crates/watershed-parquet Writes each update's changes to Parquet files on a background thread.
Python crates/watershed-py A PyO3 package over the core.
WebAssembly crates/watershed-web, web/ The engine behind the playground: updates, and plans on one thread.
Data grid crates/watershed-grid, packages/, web/grid/, web/grid-react/ A browser data grid in which every cell is a graph node: the engine as WebAssembly, the TypeScript packages (a Table, and components for Solid and React), and the pages built with them.
Kotlin and Scala surefold/watershed-jvm The JVM API, and the experimental cluster that runs a plan's lots across GPUs.

Limitations

The ones that decide whether Watershed fits; the rest are in docs/limitations.md.

  • Plans cover numeric formulas. f64 and bool with arithmetic, min, max, abs, exp, ln, sqrt, pow, comparisons, logic and conditionals; collection and group totals (sum, min, max, count); and choose by an option number that is the same in every scenario. Integer arithmetic, decimals and text are incremental only. A plan names the node it cannot take, so a model is never silently run differently.
  • Setup is paid once. Building a graph and compiling a plan each take about a microsecond a node. Watershed earns that back on the second run and on every change after.
  • GPU agreement has documented allowances for NaN payloads, signed zero and rounding in exp, ln and pow.
  • CUDA only on the GPU side.

Building, releases and licence

cargo build --workspace
cargo test --workspace                           # no GPU or CUDA needed
cargo test -p watershed-gpu --features cuda      # GPU tests; needs the CUDA toolkit and an NVIDIA GPU

There are no tagged releases yet: depend on the repository, and expect APIs to change. CONTRIBUTING.md has the checks CI runs.

Licensed under the Apache License 2.0; see LICENSE. Work derived from other projects is listed in THIRD-PARTY-NOTICES.md.

About

Composable computation engine with incremental execution, GPU acceleration and multi-GPU scenario processing. Describe a calculation once; choose how it runs afterwards.

Topics

Resources

Contributing

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages