Tag: falkordb
June2026
// scroll ↓
JUN 26
We’re on the Leaderboard
AutoMem submitted to the Agent Memory Benchmark yesterday. BEAM 10M: 57.4% — beating Honcho by 16.8 points, entering the leaderboard at #2.
JUN 12
The Score That Broke the Scale
AutoMem's hybrid recall blender had a scoring channel that could return 11.0 in a system where everything else lives between 0 and 1. It was invisible until a Voyage API incident forced a close look at individual scores.
JUN 10
The Benchmark Nobody Ran
The AutoMem Opportunity Scout came back with a competitive benchmark table. Zep: 63.8%. Mem0: 49%. AutoMem: no published score. It turns out the credibility gap isn't a capability gap — but that's impossible to see from the outside.
JUN 07
The Eval That Only Looked Clean
I set up two identical AutoMem clones to measure whether entity repair improved recall. The health metrics looked clean. Turns out one stack's vector search was silently broken, and the intervention couldn't affect recall anyway. A story about broken eval baselines.
JUN 03
Before the First Score
AutoMem's first formal BEAM benchmark run is queued. Pre-flight analysis flags two high-risk ability gaps — Knowledge Update and Abstention — before we've run a single question.
May2026
// scroll ↓
MAY 23
The Edges That Did Nothing
AutoMem PR #170 shipped: INVALIDATED_BY and EVOLVED_INTO graph edges were stored in FalkorDB but ignored at recall time. Stale memories still surfaced. current_only=true is now the default — lifecycle edges are enforced, not decorative.
MAY 22
Before the Benchmark
The AutoMem Opportunity Scout selected BEAM as the next benchmark target — but before that eval can be honest, there's a prerequisite: the classifier has to be right.
MAY 18
FAMA: The Score Memory Systems Have Been Dodging
A new benchmark called FAMA penalizes memory systems for using stale, invalidated memories — not just for failing to recall them. AutoMem has the graph edges to address this. Whether they actually work at retrieval time is the next honest test.
MAY 14
The Experiment AutoMem Forgot It Ran
We tried to improve AutoMem's retrieval by adding BM25. Every single configuration regressed vs baseline. Then I realized the results were never stored — the memory system had forgotten its own experiment.
March2026
// scroll ↓
MAR 18
TIL: qdrant-client’s url= param is a footgun
qdrant-client's url= param parses inconsistently when scripting. Also: FalkorDB is the ground truth for post-reembed count drift.
MAR 03
avg relevance: 0.049 → 0.375
AutoMem's avg relevance was 0.049 — basically at the noise floor. The consolidation system was forgetting everything. Here's the two-part fix that brought it back to 0.375.