laterite

A modern AGS4 toolkit for the AGS4 geotechnical data format: validate, read as born-typed data, query, build, fix, diff, certify, and convert ↔ Excel.

One Rust engine, surfaced natively for Python, Node.js, the CLI, DuckDB and the browser — the same rules, the same bytes, the same findings on every one.

Files come back born-typed — a 2DP heading is a float, a DT a datetime, an ID a string — so polars, SQL and the typed graph see real types, not text.

📖 Documentation · 🌐 Browser validator + data explorer · 📓 Cookbook

Pick your surface

Every surface is the same engine — no surface re-implements a rule, and scriptable output is byte-identical across them.

Surface	Get it	Use it for	Docs
Python	pip install laterite	polars frames, the typed graph, the python-ags4 drop-in	guide
Node.js	npm install laterite	apache-arrow tables, server-side validation	guide
CLI — lat	Releases, or pip install laterite / npx laterite	pipelines, CI gates, one-off checks	guide
DuckDB	INSTALL laterite_ags4 FROM community;	SQL straight over .ags files, no conversion step	guide
Browser	npm i @laterite/ags4-wasm	validate + explore in the page, nothing uploaded anywhere	guide

Every surface above is in beta — the engine is tested; what it hasn't had is your files. So is the Rust crate (cargo add laterite), a library for Rust programs rather than a door you pick from this table — see it on crates.io and docs.rs.

The web app is built on the browser package and nothing else — a worked example of that surface rather than a sixth one. Go and use it; there's nothing to install.

Quickstart

import laterite

report = laterite.validate("delivery.ags")   # errors + warnings (FYI is opt-in)
report.is_valid

ags = laterite.read("delivery.ags")          # born-typed polars frames
ags["LOCA"]["LOCA_GL"][0]                    # → 12.3, a float — not "12.30"

Node.js

import { read, validate, buildAgs4 } from "laterite";

const ags = read("delivery.ags");                  // or read(bytes) / read(undefined, { text })
ags.table("LOCA").getChild("LOCA_GL")?.get(0);     // → 12.3, born-typed apache-arrow

validate("delivery.ags").toJson();                 // byte-identical to `lat validate --json`

buildAgs4(new Map([
  ["PROJ", [{ PROJ_ID: "P1", PROJ_NAME: "Demo" }]],
  ["LOCA", [{ LOCA_ID: "BH01", LOCA_GL: 12.3 }]],
])).save("out.ags");

Cross-group SQL (ags.sql(...) / ags.at(...)) needs the optional peer @duckdb/node-api.

CLI — lat

uvx --from laterite lat validate delivery.ags   # try it with no install at all

lat delivery.ags                       # shorthand for `lat validate`
lat validate delivery.ags --json       # machine-readable findings
lat validate delivery.ags --no-warnings   # errors only
lat validate delivery.ags --warnings-as-errors  # fail on warnings too (-Werror)
lat fix delivery.ags                   # repair → sibling .fixed.ags (safe fixes)
lat diff old.ags new.ags               # KEY-aware revision delta
lat certify delivery.ags               # mint delivery.ags.idx if clean
lat rules                              # the AGS4 rule catalogue

Exit codes: 0 passed · 1 failed · 3 unreadable · 4 not AGS4 · 5 bad args · 6 schema. Errors decide the verdict; a warning is reported without failing the run. lat --readme prints the full guide.

DuckDB

INSTALL laterite_ags4 FROM community;
LOAD laterite_ags4;

-- every row carries an `_id` / `_parent_id` (UUIDv8 over the AGS key), so
-- parent↔child joins need no hand-written key list
SELECT l.loca_id, s.samp_ref, s.samp_top
FROM read_ags('delivery.ags', 'SAMP') s
JOIN read_ags('delivery.ags', 'LOCA') l ON s._parent_id = l._id;

Rust

use laterite::ags4;

let mut doc = ags4::read("delivery.ags").run()?;
for group in doc.groups() {
    println!("{} — {} rows", group.code(), group.len());
}

let report = ags4::validate("delivery.ags").warnings(true).run()?;
println!("{} finding(s)", report.findings().len());

doc.set_cell("PROJ", 0, "PROJ_NAME", "Renamed site")?;
ags4::write(&doc).to_path("out.ags")?;

Excel ↔ AGS4 sits behind the excel feature (cargo add laterite --features excel), keeping the spreadsheet dependencies out of builds that don't want them. Full API on docs.rs.

Python — typed graph and the python-ags4 drop-in

# Typed graph: PROJ → LOCA → SAMP → …
from laterite.ags4 import read_typed
for loca in read_typed("delivery.ags").locas:
    print(loca.loca_id, loca.loca_gl)

# python-ags4 drop-in — swap the import, keep your code
from laterite import compat as AGS4
tables, headings = AGS4.AGS4_to_dataframe("delivery.ags")

The drop-in mirrors python-ags4's library API name for name. Its CLI is deliberately not mirrored — see Parity.

The cookbook shows each task side by side across Python, Node, DuckDB and the CLI, with synced tabs.

What it does

The closest open-source tool, python-ags4, focuses on Python validation and pandas I/O — and inspired this project. laterite matches that and adds a cross-surface toolchain on top:

	laterite	python-ags4
Runs on	Rust · Python · Node · CLI · DuckDB · browser	Python
Validate — numbered AGS4 rules	✅	✅
Read → typed data	✅ born-typed	strings; opt-in convert_to_numeric
Build / write AGS4 · Excel ↔ AGS4	✅	✅
Repair engine (fix)	✅	—
SQL across groups · revision diff	✅	—
Validity certificates (.ags.idx)	✅	—
Transport — compress + encrypt	✅	—
Typed PROJ → LOCA → SAMP graph	✅	—

Performance

Synthetic, spec-valid AGS4 from ags4-forge — the wide scaffold: 123 groups, realistic type mix, zero findings. macOS arm64, hot files, mean of 5 warm runs, against python-ags4 1.2.0. Each cell is laterite's time and its speedup.

File (123 groups)	laterite.validate	laterite.read (typed)	laterite.compat (strings)
4.9 MB	41 ms · 36.8×	15 ms · 12.3×	38 ms · 3.7×
25.0 MB	188 ms · 19.5×	74 ms · 10.6×	148 ms · 4.6×
103.0 MB	781 ms · 15.7×	300 ms · 10.7×	589 ms · 4.6×
276.5 MB	2.0 s · 15.9×	780 ms · 11.2×	1.5 s · 4.9×
551.6 MB	4.1 s · 16.4×	1.6 s · 11.1×	3.0 s · 5.1×

Peak memory (same run, one fresh process per cell; each cell is laterite's peak RSS and python-ags4's peak over it, so above 1 laterite holds less — the largest rung is time-only):

File	laterite.validate	laterite.read (typed)	laterite.compat (strings)
4.9 MB	95 MB · 1.80×	95 MB · 1.64×	198 MB · 0.75×
25.0 MB	175 MB · 1.96×	182 MB · 1.73×	388 MB · 0.81×
103.0 MB	470 MB · 2.17×	505 MB · 1.86×	1098 MB · 0.84×
276.5 MB	1121 MB · 2.38×	1266 MB · 1.93×	2556 MB · 0.86×

The compat door's memory ratios sit below 1 on purpose — it holds more than python-ags4, and saying so is the point of publishing the column. The recommended laterite.validate and laterite.read paths hold materially less, on memory as on time.

Reproduce with uv run python tools/bench-vs-python-ags4.py. It generates the rungs, verifies each against a pinned SHA-256 so a change to the generator can't move the numbers unnoticed, and prints these tables. node tools/bench-node.mjs does the same for the Node surface.

Read-typed is the honest comparison for real work: python-ags4 needs AGS4_to_dataframe + convert_to_numeric on every group to get there, and still skips dates — laterite.read is born-typed, dates included.

Parity with python-ags4

121 / 131 of python-ags4 1.2.0's own test suite passes through laterite.compat (92 %). The 10 remaining are deliberate non-closures, documented rule by rule in COMPAT.md and the parity coverage map. A weekly job compares the two public surfaces, so a function added upstream can't quietly go missing here.

The CLI is deliberately not mirrored. laterite ships lat instead, with its own JSON / NDJSON shapes — so a script that shells out to ags4_cli needs porting rather than an import swap. Your Python code is unaffected.

Every validator rule is written from the published AGS4 specification, not adapted from another library's source — which is what lets laterite ship under a permissive MIT licence.

Contributing

See CONTRIBUTING.md. Short version: PRs welcome; CI gates cargo test + pytest + the python-ags4 parity oracle.