Home / Research
Honest recall, measured.
Did the agent use only what was valid and knowable then? Can it prove that now? This live evaluation tests both.
Definitions
What "honest recall" means here.
At time T, return only facts already true and already known.
A superseded fact returned as current.
A future event or later correction leaking into an earlier answer.
The capability must exist in the public API. No API means no point.
Method
Five invariants, one harness, real SDKs.
Every system ingests the same three-revision sequence, timestamps, and unrelated facts, then answers through its real SDK.
After a revision lands, the prior figure must not appear in top-5 recall for the natural query. Measures deterministic supersession, not ranking luck.
Query the chain as of a date between revisions; the answer must be the figure that was valid then, not today's.
Erase a data subject; content must become unrecoverable while the audit trail survives, with a certificate to show for it.
A contamination check at a simulated checkpoint must flag both FUTURE_EVENT and LATE_REVISION memories, or refuse recall outright.
Reconstruct exactly what the store contained at a past instant, suitable for handing to an examiner.
Results
Methods here. Scores on Compare.
This page explains what we test and how to reproduce it. Numerical results and competitor tables live on one dedicated page.
Reproduce it
Don't trust us. Run it.
One command reruns the open harness. Add a competitor API key to score it live.
git clone https://github.com/Lians-ai/Lians && cd Lians/agentmempython -m benchmarks.compare_regulated