Home / Compare
Win the bake-off on the numbers.
Recall quality on LOCOMO judged by mem0's own harness, plus five regulated invariants scored across six products - all reproducible from the open repo.
Recall quality · LOCOMO · run 2026-07-09
Their benchmark. Their harness. Our number.
Recall quality · LongMemEval-S · run 2026-07-12
LongMemEval: they lead, we publish it anyway.
Regulated-memory eval · head-to-head
Five invariants. One winner.
| Regulated invariant | Lians | Zep / Graphiti | Letta | Hindsight | Supermemory | mem0 |
|---|---|---|---|---|---|---|
| Stale revision suppressed | pass | partial | partial | partial | partial | fail |
| Point-in-time (as-of) recall | pass | partial | absent | partial | absent | absent |
| Provable erasure (crypto-shred + cert) | pass | partial | partial | absent | partial | partial |
| Lookahead / backtest guard | pass | absent | absent | absent | absent | absent |
| Audit-state snapshot at T | pass | partial | absent | absent | absent | absent |
| Score (pass = 1, partial = ½) | 5.0 / 5 | 2.0 / 5 | 1.0 / 5 | 1.0 / 5 | 1.0 / 5 | 0.5 / 5 |
Why audit matters
Storage is not the moat.
Current reported top-200 LoCoMo results are 92.9 for Lians and 92.5 for Mem0; the separate regulated-memory evaluation covers stale suppression, as-of recall, and audit evidence.
vs Zep / GraphitiStrong knowledge graph without access control, multi-tenancy, or audit by design.
vs LettaExcellent self-editing context, but edits are silent overwrites with no validity windows.
vs HindsightTemporal-aware episodic banks without validity-gated supersession or provable erasure.
vs SupermemoryFast ingestion and retrieval, but accumulate-and-rank, with no point-in-time reconstruction.
vs Heartwood MemoryStrong policy, provenance, and erasure controls. Lians adds bitemporal state, deterministic supersession, and point-in-time reconstruction.