Byte sum mode, notes on the gematria channel, light mode and an escape menu

.github/workflows/ci.yml

@@ -0,0 +1,30 @@
+name: ci
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+
+env:
+  CARGO_TERM_COLOR: always
+
+jobs:
+  test:
+    name: test (${{ matrix.os }})
+    runs-on: ${{ matrix.os }}
+    strategy:
+      fail-fast: false
+      matrix:
+        os: [ubuntu-latest, windows-latest, macos-latest]
+    steps:
+      - uses: actions/checkout@v4
+      - name: windowing headers
+        if: runner.os == 'Linux'
+        run: sudo apt-get update && sudo apt-get install -y libxkbcommon-dev libwayland-dev pkg-config
+      - uses: dtolnay/rust-toolchain@stable
+        with:
+          components: clippy
+      - uses: Swatinem/rust-cache@v2
+      - run: cargo build --all-targets --locked
+      - run: cargo test --locked
+      - run: cargo clippy --all-targets --locked -- -D warnings

.gitignore

@@ -22,3 +22,9 @@ rustc-ice-*.txt
 #  and can be added to the global gitignore or merged into this file.  For a more nuclear
 #  option (not recommended) you can uncomment the following to ignore the entire idea folder.
 #.idea/
+
+# memories and exports made while running neum here
+/memory
+/*.memory
+/memory.*.json
+neum-settings.json

CHANGELOG.md

@@ -0,0 +1,43 @@
+# changelog
+
+## 0.2.0
+
+### added
+
+- **b**, the byte sum bucket: a third sum beside ordinal and reversed. The middle letter now cycles
+  `o → r → b`. Every memory is indexed by its byte sum, so the bucket can be searched like the
+  other two.
+- the byte sum story of the input, shown in **b** mode. Input written as bytes (`104,105`) is read
+  as those bytes.
+- notes on the gematria channel, working as lists do on the search channel: Enter notes what was
+  typed on the alt name, Enter on an empty bar shows the alt name's notes, Delete unlinks one.
+- light mode, toggled from the escape menu.
+- the escape menu: the open memory's details, saves (open, new, close, export, import, recent),
+  settings and keys.
+- settings kept between runs (`neum/settings.json` in the config folder, or `$NEUM_CONFIG`).
+- `neum [MEMORY]`, `--help` and `--version`.
+- a status line under the bar instead of messages printed to the terminal.
+- tests: the sums and their AVX2 fast paths against scalar, byte sums, factoring, the memory end
+  to end (ids, indexes, lists, notes, search, backup and restore, repair, migration, saves) and
+  settings. CI on GitHub Actions.
+- `examples/inspect.rs`.
+
+### changed
+
+- the code is split into a library (`gematria`, `store`, `settings`) and the window (`main.rs`).
+- the open memory can be swapped at run time; a memory that can't be opened no longer panics.
+- a number typed on the gematria channel looks up that sum (ctrl+Enter sums it as text).
+- a note now goes *on* the alt name (before, the alt name went on what was typed).
+- entries show in id order, and memories that aren't UTF-8 show as their bytes.
+- forgetting a list lets its members go; a memory can't be a note on itself.
+- dependencies have real version requirements instead of `*`.
+
+### schema
+
+- memories gain a version in `STATE` (v2) and the `MEMORY_BYTE_SUM` index. A v1 memory is brought
+  up to date the first time it opens.
+
+## 0.1.0
+
+- ordinal and reversed gematria over a redb memory: saturate, search by words and sums, forget.
+- lists on the search channel.

CONTRIBUTING.md

@@ -0,0 +1,32 @@
+# contributing
+
+## layout
+
+| file | what it is |
+|---|---|
+| `src/gematria.rs` | the sums (ordinal, reversed, bytes), tokens, factoring, the byte sum story |
+| `src/store.rs` | the memory: tables, ids, indexes, lists, notes, search, backup, repair, the open memory |
+| `src/settings.rs` | settings kept between runs |
+| `src/main.rs` | the window |
+| `tests/` | the library end to end, on temporary files |
+
+## before a pull request
+
+```console
+cargo test
+cargo clippy --all-targets
+```
+
+Both run in CI. On Linux, building needs `libxkbcommon-dev`, `libwayland-dev` and `pkg-config`.
+
+## changing the memory's tables
+
+Bump `SCHEMA` in `src/store.rs`. `Memory::open` runs `repair_in` on anything older, which rebuilds
+every index from the memories themselves; a new index only has to be part of `index`, `unindex`
+and `repair_in`. Add a test that opens a memory written the old way (see
+`a_schema_1_memory_gets_its_byte_bucket` in `tests/store.rs`).
+
+## the fast paths
+
+The AVX2 sums and tokenizer must agree with their scalar versions byte for byte; the tests in
+`src/gematria.rs` check that over many lengths, including the block edges.

Cargo.toml

@@ -1,16 +1,35 @@
 [package]
 name = "neum"
-version = "0.1.0"
+version = "0.2.0"
 edition = "2024"
+rust-version = "1.88"
+description = "ordinal, reversed and byte-sum gematria over a stateful redb memory, with lists and notes"
+license = "MIT"
+readme = "README.md"
+repository = "https://github.com/SaulDoesCode/neum"
+keywords = ["gematria", "redb", "zgui", "notes"]
+categories = ["gui", "text-processing"]
+
+[lib]
+name = "neum"
+path = "src/lib.rs"
+
+[[bin]]
+name = "neum"
+path = "src/main.rs"
 
 [dependencies]
-anyhow = "*"
+anyhow = "1"
 zgui = { git = "https://github.com/zortax/zgui" }
 zgui-ui = { git = "https://github.com/zortax/zgui" }
+zgui-ui-tokens = { git = "https://github.com/zortax/zgui" }
 redb = { version = "4.4.0" }
-rayon = { version = "*"}
-serde = "*"
-serde_json = "*"
+rayon = "1"
+serde = { version = "1", features = ["derive"] }
+serde_json = "1"
+
+[dev-dependencies]
+tempfile = "3"
 
 [profile.release]
 opt-level = 3

README.md

@@ -1,12 +1,111 @@
+# neum
+
 ![neum app screenshot and code in the background](./Screenshot_2026-10-22_16-58-36.png)
 
-ordinal and reversed gematria, in a stateful redb app that uses zgui for a basic interface
+ordinal, reversed and byte-sum gematria, in a stateful [redb](https://github.com/cberner/redb) app
+that uses [zgui](https://github.com/zortax/zgui) for its interface. Everything typed can be
+saturated (saved) into a memory, found again by any of its sums or its words, gathered into lists
+and hung with notes. You can divinate with this so long #ccru
+
+## build and run
+
+```console
+cargo run --release               # opens the last memory, or ./memory
+cargo run --release -- my.memory  # opens (or makes) my.memory
+```
+
+Linux needs the usual windowing headers to build: `libxkbcommon-dev` and `libwayland-dev`
+(and `pkg-config`). Rust 1.88 or newer.
+
+## the bar
+
+Type into the bar and its sums show underneath. The three letters between them are toggles:
+
+``s [o/r/b] s`` — **S**earch, which sum (**O**rdinal / **R**eversed / **B**ytes), **S**aturate.
+
+| letter | on | off |
+|---|---|---|
+| first `s` | the **search channel**: Enter looks memories up by their words | the **gematria channel**: Enter looks memories up by the sum |
+| middle | click to cycle `o → r → b → o` | hidden while searching by words |
+| last `s` | Enter also saves what was typed (once) | Enter only looks |
+
+**o** is a=1 … z=26, **r** is a=26 … z=1, and **b** is the byte sum bucket: every byte counts
+`byte + 1`, so it sums anything, letters or not, text or raw bytes. With **b** on, the byte sum
+story of the input shows above the sums: its length, mean, digit sum and roots, the byte sum's
+prime factors, and its prime-factor reduction chain. Input written as bytes (`104,105`, the
+way keys are shown) is read as those bytes, so `hi` and `104,105` tell the same story.
+
+On the gematria channel a number is a sum to look up (ctrl+Enter sums it as text instead).
+
+### lists and notes
+
+Middle click the bar to move its text into the **alt name** (right click the bar to see it). The
+alt name is a memory too, and what Enter and Delete do with it depends on the channel:
+
+| | search channel (`s` on) | gematria channel (`s` off) |
+|---|---|---|
+| Enter with text | adds what was found to the alt name's **list** | makes it a **note** on the alt name |
+| Enter on an empty bar | shows the list | shows the notes |
+| Delete on a key | takes it off the list | unlinks the note |
+
+Without an alt name, Delete on a key forgets that memory (and puts its text back in the bar).
+
+### the entries
+
+Click an entry to put its text in the bar; double click to put its key there (keys are bytes,
+like `0,1`). Ctrl + right click looks up the mean of the two sums shown. Each entry is underlined
+in a colour from one of its other sums, so entries that agree on that sum share a colour.
+
+## the escape menu
+
+Esc opens it (Esc again, or a click outside, closes it).
+
+- **memory** — the open memory's file, size on disk, how many memories, lists and notes it holds,
+  the last id handed out and how many freed ids wait to be reused. *repair* rebuilds every index
+  and drops links that point at nothing.
+- **saves** — memories are files, kept like game saves. Type a path and *open* it (a new path makes
+  a new memory), *close* the open one, *export* it to JSON (to a typed `.json` path, or next to the
+  memory with the time in its name) or *import* a JSON export into an empty memory. The memories
+  opened before are listed; click one to open it, × to take it off the list.
+- **settings** — light or dark mode, whether saturate starts on, whether a memory is compacted when
+  opened, and how many of a query's words a search match must share (50 / 75 / 90 / 100%).
+- **keys** — the controls above, in short.
+
+Settings live in `neum/settings.json` under your config folder (`$XDG_CONFIG_HOME`, `~/.config`
+or `%APPDATA%`), or wherever `NEUM_CONFIG` points.
+
+## which memory opens
+
+The first argument, else `$NEUM_MEMORY`, else the last memory open, else `./memory`. A memory
+that can't be opened (already open in another neum, say) leaves the window up, and the escape
+menu can open another.
+
+Memories made by 0.1 open as they are; their byte sum bucket is built the first time they open.
+
+## as a library
+
+The window is `src/main.rs`; everything under it is the `neum` library, usable on its own:
+
+```rust
+use neum::{Memory, Mode, gematria::sums};
+
+let m = Memory::open("my.memory", false)?;
+let id = m.saturate(b"ordinary cunt", None)?;
+assert_eq!(sums(b"ordinary cunt"), (162, 162));
+assert!(m.gsearch(162, Mode::Ordinal)?.contains_key(&id));
+```
+
+`cargo run --example inspect -- my.memory 162` prints a memory's details and what sits at a sum.
 
-pressing enter saves things and does gsearch or search depening on the the middle three letters 
+## development
 
-``(s [o/r] s)`` which stands for Search, Orinal/Reversed and Saturate (save)
+```console
+cargo test            # sums, fast paths against scalar, the memory end to end, settings
+cargo clippy --all-targets
+```
 
-double clicking entries puts their key in the bar, pressing delete on the bar deletes that one
+See [CONTRIBUTING.md](CONTRIBUTING.md) and [CHANGELOG.md](CHANGELOG.md).
 
-lists and notes still need ui, but you can divinate with this so long #ccru
+## license
 
+MIT, see [LICENSE](LICENSE).

examples/inspect.rs

@@ -0,0 +1,32 @@
+//! what a memory holds, without the window:
+//!
+//! ```text
+//! cargo run --example inspect -- path/to/memory          # its details
+//! cargo run --example inspect -- path/to/memory 162      # memories at 162, by every sum
+//! ```
+
+use neum::{Memory, Mode, gematria::byte_list, human_bytes};
+
+fn main() -> anyhow::Result<()> {
+    let mut args = std::env::args().skip(1);
+    let path = args.next().unwrap_or_else(|| "./memory".into());
+    let m = Memory::open(&path, false)?;
+    let s = m.stats()?;
+    println!("{}", s.path.display());
+    println!("  {} memories ({}), {} on disk", s.memories, human_bytes(s.stored_bytes), human_bytes(s.file_bytes));
+    println!("  {} lists holding {}, {} notes on {} memories", s.lists, s.listed, s.notes, s.noted);
+    println!("  {} ids to reuse, schema v{}", s.reusable, s.schema);
+    if let Some(sum) = args.next() {
+        let sum: u64 = sum.parse()?;
+        for mode in Mode::ALL {
+            let mut found: Vec<_> = m.gsearch(sum, mode)?.into_iter().collect();
+            found.sort();
+            println!("{} {sum}: {}", mode.name(), found.len());
+            for (k, v) in found {
+                let shown = String::from_utf8(v.clone()).unwrap_or_else(|_| format!("⟨{}⟩", byte_list(&v)));
+                println!("  [{}] {shown}", byte_list(&k));
+            }
+        }
+    }
+    Ok(())
+}