Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions specs/catalog/omni-graph-live-watcher-fidelity-parity.md
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,13 @@ A recursive divide-and-conquer code and database audit of `tools/omni-graph/src/
- **AC-051**: Traversing cyclic filesystem symlinks (e.g. `a/loop -> ..` or mutual circular links) terminates deterministically in $O(1)$ lookup per directory without recursion loop, process hang, or stack overflow.
- **AC-052**: A symlinked file within the workspace is indexed as a dedicated `symlink` node with a `LINKS_TO` edge pointing directly to the canonical target file, producing zero duplicate AST function/symbol declarations and zero duplicate community assignments.
- **AC-053**: The incremental watcher (`IncrementalPipeline`) safely resolves file or directory symlink modifications, deletions, or creations without panic.
- **AC-054**: MATLAB `.m` files with single-return or multi-return functions (`function [y1, y2] = name(x1, x2)`) are parsed into first-class `function` nodes with accurate line boundaries and in-body `CALLS` edges.
- **AC-055**: MATLAB `classdef` definitions with single or multiple inheritance (`classdef Foo < Bar & Baz`) emit `class` nodes and valid `EXTENDS` edges to all base classes.
- **AC-056**: Properties inside MATLAB `properties` blocks emit `property` nodes linked with `CONTAINS` edges from the enclosing class.
- **AC-057**: Class methods emit `method` or `constructor` nodes with `DECLARES` or `CONTAINS` edges from the enclosing class.
- **AC-058**: Method invocations with receiver targets (e.g. `FilterEngine.calculate(x)`) preserve root receiver objects as `REFERENCES` edges.
- **AC-059**: MATLAB `import` statements (e.g. `import control.guidance.NavSystem;` or `import math.filters.*;`) emit `import` nodes and `IMPORTS` edges.
- **AC-060**: Package directory hierarchies prefixed with `+` (e.g. `+control/+guidance/NavSystem.m`) synthesize `package` nodes (`control.guidance`) with `DECLARES` edges to contained classes and functions.

### 2.3 Non-Functional Requirements (NF)

Expand Down
3 changes: 2 additions & 1 deletion tools/omni-graph/CONTEXT.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,8 @@ blockers: []
| Lock-Free Generational Versioning (R-099) | Complete | — | 🟢 Arc<AtomicU64> incremented on all DB mutations; X-Graph-Generation injected on all HTTP responses |
| High-Throughput Parallel Batch Persistence (R-100) | Complete | — | 🟢 tokio::join! on backbone storage and community inference; omni-mcp.py handles HTTP 423 gracefully |
| Filesystem Symlink Cycle Guard & Navigation (R-109) | Complete | — | 🟢 Symlink metadata interception, canonical target resolution, LINKS_TO edge emission, zero duplicate AST clones |
| Test Coverage | 113/113 Unit Tests | 113 | 🟢 100% Verified (0.08s, 0 clippy warnings) |
| MathWorks MATLAB AST Ingestion (R-107) | Complete | — | 🟢 Vendored ABI 14 tree-sitter-matlab; extracted classdef, multiple inheritance EXTENDS, properties, constructors, methods, receiver preservation, import commands, package hierarchy |
| Test Coverage | 118/118 Unit Tests | 118 | 🟢 100% Verified (0.05s, 0 clippy warnings) |
| Live Watcher In-Memory SAIR Acceleration (R-078, R-091) | Complete | — | 🟢 Integrated in-memory SymbolTable into IncrementalPipeline; eliminated 392s subquery bottleneck; live sync completes in <250ms with dedicated 1s quiescent trigger |
| AI-Native Cosmology & Modularity Guard (R-086, R-087) | Complete | — | 🟢 Purged Miller's Law clamp; enforced Affinity-Floor Invariant and delta Q guard |
| Scale-Adaptive Subsystem Sizing (R-088) | Complete | — | 🟢 Sub-linear dynamic ceiling K_adaptive(N) bounding natural domains without forced merges |
Expand Down
1 change: 1 addition & 0 deletions tools/omni-graph/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ tree-sitter-go = "0.21"
tree-sitter-java = "0.21"
tree-sitter-c = "0.21"
tree-sitter-cpp = "0.21"
tree-sitter-matlab = { path = "grammars/tree-sitter-matlab" }
tracing = "0.1"
tracing-subscriber = { version = "0.3", features = ["env-filter"] }
walkdir = "2.5"
Expand Down
32 changes: 32 additions & 0 deletions tools/omni-graph/DEVLOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,35 @@
### 2026-11-06 — MathWorks MATLAB (`.m`) AST Ingestion Engine (R-107)

**Agent/Author**: Antigravity (Google DeepMind) & `@vadde`
**SDLC Phase**: `testing` (AST Parser Expansion & Polyglot Hardening)
**Branch**: `feat/r107-matlab-ast-ingestion`
**Requirement**: `R-107` ([`specs/catalog/omni-graph-live-watcher-fidelity-parity.md`](file:///Users/aparv/Library/CloudStorage/OneDrive-Personal/G-Drive/Interviews/knowledge/PolyTuring/specs/catalog/omni-graph-live-watcher-fidelity-parity.md))
**Duration**: Complete

#### Executive Summary & Key Milestones Completed
1. **Grammar Sub-Crate Engine Setup (`grammars/tree-sitter-matlab/`)**:
- Vendored official ABI 14 `tree-sitter-matlab` parser (`parser.c` and `scanner.c`) in `tools/omni-graph/grammars/tree-sitter-matlab/`.
- Resolved tree-sitter crate version pinning and ABI 15 incompatibilities with `tree-sitter = "0.22"`.
2. **MATLAB Parser Wiring (`src/parser/mod.rs`)**:
- Added `"m"` file extension dispatch in `CodeParser::parse_file` utilizing `tree_sitter_matlab::language()`.
- Extracted standalone and multi-return functions (`function [y1, y2] = name(x1, x2)`) emitting `function` nodes and internal `CALLS` edges.
- Extracted `classdef` OOP classes with single and multiple inheritance (`classdef Foo < Bar & Baz`) emitting `class` nodes and `EXTENDS` edges.
- Extracted class properties inside `properties` blocks emitting `property` nodes linked with `CONTAINS` edges from the enclosing class.
- Extracted class methods and constructors (`parent_name == name`) emitting `constructor` (with `CONTAINS` edge) and `method` (with `DECLARES` edge).
- Preserved AST receivers on method/function invocations (e.g. `FilterEngine.calculate(x)`) emitting `REFERENCES` edges to the root receiver.
- Extracted MATLAB `import` commands (`import control.guidance.NavSystem;` and `import math.filters.*;`) emitting `import` nodes and `IMPORTS` edges.
- Synthesized package namespace nodes for `+pkg1/+pkg2` directory structures (e.g. `+aerospace/+gnc/GuidanceComputer.m` -> `aerospace.gnc`) with `DECLARES` edges to contained classes and functions.
3. **Verification & Testing (`tests/unit_tests.rs`)**:
- Authored 5 dedicated unit tests:
- `test_r107_matlab_function_extraction`: Verifies multi-return functions and internal call graph edges.
- `test_r107_matlab_classdef_and_inheritance`: Verifies multiple inheritance (`EXTENDS`), properties (`CONTAINS`), constructors (`CONTAINS`), and methods (`DECLARES`).
- `test_r107_matlab_receiver_preservation`: Verifies receiver preservation (`REFERENCES` to `SensorClient`, `FilterEngine`).
- `test_r107_matlab_import_commands`: Verifies specific and wildcard import command extraction.
- `test_r107_matlab_package_hierarchy`: Verifies `+pkg` directory package node synthesis.
- All 118 unit tests pass (100% green, 0 clippy warnings).

---

### 2026-11-06 — Filesystem Symlink Cycle Guarding & Alias Navigation (R-109)

**Agent/Author**: Antigravity (Google DeepMind) & `@vadde`
Expand Down
7 changes: 5 additions & 2 deletions tools/omni-graph/STATUS.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,15 +53,16 @@ fidelity_spec: ../../specs/catalog/omni-graph-live-watcher-fidelity-parity.md
| 2026-11-05 | `review` | `review` | Forensic resolution of watcher 392s subquery bottleneck: Integrated in-memory SymbolTable (R-078/R-091) into IncrementalPipeline with parameterized batch edge resolution, dedicated 1s quiescent auto-cluster trigger, and test_r091 (99 unit tests pass). |
| 2026-11-06 | `review` | `in-progress` | Launch v0.7.0 Live Watcher Fidelity Parity & Concurrency Overhaul (ADR-002, SPEC-OMNI-005, R-093 to R-109): Closing 14 forensic loopholes across transactional atomic batch swap, reactive reverse dependencies, unresolved retention pool, deduplication, and non-blocking Louvain actor. |
| 2026-11-06 | `in-progress` | `testing` | Completed Milestones 1 through 6 (R-093 to R-106): SymbolTable hardening, directory backbone synthesis, transactional atomic swap, edge deduplication, dual-tier modularity, non-blocking Louvain actor, convergence barrier (wait_for_sync), nanometer fidelity diagnostics, and lock-free generational versioning. 111 unit tests pass (100% green, 0 clippy warnings). |
| 2026-11-06 | `testing` | `testing` | Completed R-109 (Filesystem Symlink Cycle Guarding & Alias Navigation) and R-107 (MathWorks MATLAB AST Ingestion): Vendored ABI 14 tree-sitter-matlab grammar, extracted classdef, multiple inheritance EXTENDS, properties, constructors, methods, receiver preservation, import commands, and package hierarchy. 118 unit tests pass (100% green, 0 clippy warnings). |

---

## Implementation Progress

| Category | Done | Total | Percentage |
|----------|------|-------|------------|
| Requirements (R-XXX) | 78 | 80 | 97.5% |
| Acceptance Criteria (AC-XXX) | 35 | 37 | 94.6% |
| Requirements (R-XXX) | 80 | 81 | 98.8% |
| Acceptance Criteria (AC-XXX) | 44 | 45 | 97.8% |
| Non-Functional (NF-XXX) | 15 | 15 | 100% |

---
Expand Down Expand Up @@ -174,6 +175,8 @@ fidelity_spec: ../../specs/catalog/omni-graph-live-watcher-fidelity-parity.md
| R-104 | Edge Relationship Deduplication & Call Weighting | `src/db/mod.rs` | `tests/unit_tests.rs:test_r104_edge_relationship_deduplication` | ✅ |
| R-105 | Zero-Delay Community Inference for New Files | `src/analysis/mod.rs`, `src/watcher/pipeline.rs` | `tests/unit_tests.rs:test_r105_zero_delay_community_inference_for_new_files` | ✅ |
| R-106 | Language-Aware SAIR Scoping & Exact Match Fast-Path | `src/ingestion/symbol_table.rs`, `src/db/mod.rs` | `tests/unit_tests.rs:test_r106_language_aware_sair_scoping` | ✅ |
| R-107 | MathWorks MATLAB (`.m`) AST & Classdef Ingestion | `src/parser/mod.rs`, `grammars/tree-sitter-matlab` | `tests/unit_tests.rs:test_r107_matlab_*` | ✅ |
| R-108 | MathWorks Simulink (`.slx` / `.mdl`) Block Diagram MBSE Ingestion | `src/parser/simulink.rs`, `src/ingestion/mod.rs` | Spec/Research spike complete, queued for Phase 3 | ⬜ |
| R-109 | Filesystem Symlink Cycle Guarding & Alias Navigation | `src/ingestion/mod.rs`, `src/watcher/pipeline.rs` | `tests/unit_tests.rs:test_r109_symlink_cycle_guard_and_alias_navigation` | ✅ |

**Status Legend**: ⬜ Not started · 🔨 In progress · ✅ Implemented · 🧪 Tested · ❌ Blocked
Expand Down
15 changes: 15 additions & 0 deletions tools/omni-graph/grammars/tree-sitter-matlab/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
[package]
name = "tree-sitter-matlab"
version = "1.3.3"
edition = "2021"
description = "MATLAB grammar for tree-sitter"
license = "MIT"

[lib]
path = "src/lib.rs"

[dependencies]
tree-sitter = "0.22"

[build-dependencies]
cc = "1.0"
10 changes: 10 additions & 0 deletions tools/omni-graph/grammars/tree-sitter-matlab/build.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
fn main() {
let src_dir = std::path::Path::new("src");
let mut c_config = cc::Build::new();
c_config.std("c11").include(src_dir);
#[cfg(target_env = "msvc")]
c_config.flag("-utf-8");
c_config.file(src_dir.join("parser.c"));
c_config.file(src_dir.join("scanner.c"));
c_config.compile("tree-sitter-matlab");
}
87 changes: 87 additions & 0 deletions tools/omni-graph/grammars/tree-sitter-matlab/src/lib.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
use tree_sitter::Language;

extern "C" {
fn tree_sitter_matlab() -> *const ();
}

/// Returns the tree-sitter Language for MATLAB
pub fn language() -> Language {
unsafe { Language::from_raw(tree_sitter_matlab() as *const _) }
}

#[cfg(test)]
mod tests {
use super::*;

#[test]
fn test_can_load_grammar() {
let mut parser = tree_sitter::Parser::new();
parser
.set_language(&language())
.expect("Error loading Matlab parser");
}

#[test]
fn test_parse_sample_matlab() {
let mut parser = tree_sitter::Parser::new();
parser.set_language(&language()).unwrap();

let code = r#"
import control.guidance.NavSystem;
import math.filters.*;

classdef KalmanFilter < control.BaseFilter & math.Observer
properties (Access = private)
CovarianceMatrix double
StateEstimate = [0; 0]
end
methods
function obj = KalmanFilter(initialCov)
obj.CovarianceMatrix = initialCov;
end
function [predState, predCov] = predict(obj, u)
predState = obj.StateEstimate + u;
predCov = obj.CovarianceMatrix;
helperFunction(predState);
obj.stepMethod(predState);
end
end
end

function [out1, out2] = standaloneFunc(arg1, arg2)
out1 = arg1 * 2;
out2 = arg2 + 10;
end
"#;

let code = r#"
function [filtered, residual] = filterSignals(raw_sig, threshold)
% Multi-return signal filtering function
gain = 2.5;
residual = raw_sig - threshold;
filtered = applyGain(residual, gain);
saturate(filtered);
end

function y = singleOutput(x)
y = x * 10;
end
"#;
let tree = parser.parse(code, None).unwrap();
println!("AST:\n{}", tree.root_node().to_sexp());

let root = tree.root_node();
let mut cursor = root.walk();
for child in root.children(&mut cursor) {
if child.kind() == "function_definition" {
println!("Top-level function_definition:");
println!(" by_field('name'): {:?}", child.child_by_field_name("name").map(|n| &code[n.byte_range()]));
for c in 0..child.child_count() {
let sub = child.child(c).unwrap();
println!(" child {}: kind='{}', field_name={:?}, text='{}'",
c, sub.kind(), child.field_name_for_child(c as u32), &code[sub.byte_range()]);
}
}
}
}
}
Loading
Loading