Skip to content

Add AMem memory policies with evaluations - #15

Open
Jacky-020 wants to merge 1 commit into
mainfrom
a-mem
Open

Jacky-020 wants to merge 1 commit into
mainfrom
a-mem

Conversation

@Jacky-020

@Jacky-020 Jacky-020 commented Oct 3, 2026 •

Copy link
Copy Markdown
Collaborator

It is the reproduction according to the source repo https://github.com/WujiangXu/A-mem-sys . The retreival of neighbor note (through note links) is self implemented as not well described in the paper and the source repo. For the interpretation of the policy, please refer to the comments in policy.py.

Summary by CodeRabbit

  • New Features
    • Added A-Mem, a memory option that evolves and links notes to support retrieval.
    • Added a runnable demo that stores sample conversations, checks retrieval, and verifies state recovery.
    • Made A-Mem available through the package’s public interface.
  • Documentation
    • Updated the documented project structure to include the A-Mem demo.

It is the reproduction according to the source repo https://github.com/WujiangXu/A-mem-sys . The retreival of neighbor note (through note links) is self implemented as not well described in the paper and the source repo. For the interpretation of the policy, please refer to the comments.
@Jacky-020
Jacky-020 requested a review from SleepyLGod as a code owner October 3, 2026 08:05
@coderabbitai

coderabbitai Bot commented Oct 3, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: bc1e091c-38e8-4afd-b1bc-e52fbbe19b1e
📥 Commits

Reviewing files that changed from the base of the PR and between 93d98d1 and a6e11c6.

📒 Files selected for processing (10)
  • README.md
  • examples/a_mem/e2e_demo.py
  • src/agent_memory/__init__.py
  • src/agent_memory/memories/__init__.py
  • src/agent_memory/memories/a_mem/__init__.py
  • src/agent_memory/memories/a_mem/policy.py
  • src/agent_memory/memories/a_mem/prompts.py
  • src/agent_memory/memories/a_mem/storage.py
  • tests/test_a_mem_e2e_example.py
  • tests/test_a_mem_policy.py

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.


📝 Walkthrough

Walkthrough

This pull request adds the A-Mem memory policy, a Neo4j storage profile, public package exports, and a LOCOMO end-to-end demonstration. The demonstration checks retrieval and restored state, and writes run metrics and reproducibility artifacts.

Changes

A-Mem memory policy

Layer / File(s) Summary
Note analysis, evolution, and retrieval
src/agent_memory/memories/a_mem/policy.py, src/agent_memory/memories/a_mem/prompts.py, tests/test_a_mem_policy.py
Adds prompts and policy logic for note analysis, suggested links, neighbor updates, note consolidation, embedding text, and retrieval with depth-one neighbor expansion. Tests cover policy behavior, prompt contracts, and state aggregation.
Neo4j storage and public exports
src/agent_memory/memories/a_mem/storage.py, src/agent_memory/memories/a_mem/__init__.py, src/agent_memory/memories/__init__.py, src/agent_memory/__init__.py, tests/test_a_mem_policy.py
Adds the A-Mem Neo4j embedding, note and relationship mappings, indexes, and insert statements. Exports AMem and storage constants. Tests check public exports and retrieval planning.
LOCOMO end-to-end demonstration
examples/a_mem/e2e_demo.py, tests/test_a_mem_e2e_example.py, README.md
Adds a five-row LOCOMO run that adds turns, checks retrieval, saves and restores state through a fresh Neo4j connection, and writes reports and reproducibility artifacts. Tests cover input handling, validation, cleanup, and run outputs. README lists the example.

Priority: ➖ Normal

Estimated code review effort: 4 (Complex) | ~45 minutes

Change: Feature

Sequence Diagram(s)

sequenceDiagram
  participant Demo as e2e_demo.py main()
  participant Memory as AMem
  participant Storage as Neo4j storage
  Demo->>Memory: Add selected LOCOMO turns
  Memory->>Storage: Store note state and links
  Demo->>Memory: Run retrieval query
  Memory-->>Demo: Return retrieval results
  Demo->>Storage: Checkpoint runtime state
  Demo->>Storage: Restore state through a fresh connection
Loading

Merge Risk: ⚪ Minimal · up to a6e11

No demonstrated A-Mem storage issue remains that would block merging after normal checks.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 68.54% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 89 functions across 9 files. (1 skipped: … Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly identifies the main change: adding the AMem memory policy. The evaluation wording also relates to the added tests and end-to-end demo.
Full details: Docstring Coverage

Explanation

Docstring coverage is 68.54% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 89 functions across 9 files. (1 skipped: 1 unsupported.)

  • Fix all pre-merge checks with AI
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@SleepyLGod SleepyLGod self-assigned this Oct 4, 2026

@SleepyLGod SleepyLGod left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review

From Von

This is a good starting point, but it is not yet a Native-aligned A-MEM implementation. Please address the following before merging later.

1. Choose One Reference Implementation

Both repositories are official, but serve different purposes:

For our research comparison, please use a pinned commit of A-mem as the baseline. Keep prompts, constants, storage configuration, and retrieval behavior consistent with that version.

2. Issues In This PR

  • Array parsing: The final multi-output sem_agg expects an array-valued tags, but the production parser only accepts scalar fields. The fake adapter bypasses this limitation. Use the existing single-JSON-column → unnest pattern and test the real parsing path.
  • Update semantics: The new consolidation prompt preserves historical context/tags. Native replaces them with the evolution output. This changes the memory semantics; aggregation should represent the intended final state, not introduce another summarization policy.
  • Evolution inputs: Later decisions read frozen initial analyses rather than evolved notes. Please verify the proposed view and generated maintenance query against the selected Native implementation.
  • Candidate selection: All earlier notes enter evolution, whereas Native uses vector top-5 neighbors. This needs an explicit, Native-aligned physical candidate configuration.
  • Embedding assembly: sem_map should not concatenate strings. It adds unnecessary LLM cost and cannot guarantee exact formatting.
  • Retrieval: Native already implements outgoing-link expansion in find_related_memories_raw(). Follow that implementation rather than guessing. Preserve the returned metadata and verify expansion limits. Neo4j/BGE is also a different physical configuration from the research baseline.

3. What Our Repository Still Needs

We lack small deterministic expressions for string concatenation and joining array elements into text. assign() exists, but these expressions do not.

Native-aligned vector-index integration and maintenance candidate lookup also need additional work. Please flag these gaps rather than replace them with LLM operations or undocumented behavior.

4. Recommended Next Step

First pin the research repository and write a short Native-to-policy mapping. Then fix parsing, update semantics, and retrieval; add only the missing general-purpose capabilities. Validate both the production parser and generated maintenance behavior. Passing fake-adapter tests alone does not establish Native parity.

Feel free to use this for your local testing: https://build.nvidia.com/models

@SleepyLGod SleepyLGod changed the title [feat-a-mem] Add a-mem policies with its test Add AMem memory policies with tests Oct 4, 2026
@SleepyLGod SleepyLGod changed the title Add AMem memory policies with tests Add AMem memory policies with evaluations Oct 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants