Era Brain: Evaluating a Permission-Aware Business Memory on Retrieval, Multi-hop Records and Access Control

Memory layers for language-model agents are usually evaluated on long chat histories: one user talks to an assistant for weeks, and the system must recall what was said [1, 2]. A business asks different questions. The answer to “has the invoice for this customer's last order been paid?” sits in three synced records joined by relations, may have changed since yesterday, and must not reach a sales assistant who is not allowed to see invoices. The language is often Turkish, whose suffixes defeat plain keyword search.

Public benchmarks for this setting are scarce. Agent benchmarks over CRM data [3, 4] and a 2026 benchmark for access control in shared agent memory [5] exist, but none measures a memory layer's retrieval over synced business records under per-person permissions. We therefore report two kinds of evidence: standard retrieval and answer-accuracy numbers on public conversational benchmarks, so that Era Brain can be placed next to published systems, and self-built business and permission tests, whose construction and limits are described in Sections 3 and 6.

This report makes four contributions:

1. a description of Era Brain's write, store and read paths, with each claim traced to source code (Appendix A);
2. retrieval and answer-accuracy results on LongMemEval-S and LoCoMo, reported with the benchmarks' own question counts and metric names;
3. a business-memory evaluation in Turkish and English: question types, multi-hop record joins at 100,000 graph rows, latest-value questions and context size;
4. a permission test with 865 planted secrets run through every member-facing read path, and a capability comparison with six memory systems as documented on 22 October 2026.

Era Brain has a write path, a store and a read path (Figure 1). The write path turns synced records and conversations into facts, entities and typed relations. The store is one PostgreSQL database. The read path answers a query without calling a language model. Background jobs refine the graph; this evaluation did not exercise them.

2.1 Architecture

Records from connected apps are written without a language model. Each record is tagged with the module it came from (sales, finance, HR, …), versioned from its own date, and written once even when a sync is retried: the memory, its links, edges, version and sync ledger share one locked transaction, and entities are upserted idempotently before it. Conversations go through LLM fact extraction. Extracted facts link to people, companies, products and topics, not to synced records; an order number mentioned in a chat is matched to the record only at query time, through text.

Two components exist in code but receive no production data yet: the causal decision graph and its hash-chained log¹. Nightly consolidation also has a known gap².

2.2 Read path

A query is answered in six steps (Figure 2), inside one read transaction:

1. 1Anchors. Names in the question are matched exactly first, with Turkish case endings removed (“Ağaoğlunun” → Ağaoğlu), then by word containment and trigram similarity, then by meaning (entities behind the nearest memories). Kind words (“order”, “fatura”, “talebindeki” → talep, with consonant softening) set the kind of node to reach.
2. 2Hybrid recall. A full-text query, with English and Turkish stop words removed, and a vector query (HNSW, cosine) run side by side; their results are fused by reciprocal rank.
3. 3Path search. From the anchors, a walk over typed relations goes up to three hops. At each level the 48 most promising nodes walk on (at most 600 in total). A relation that fans out to more than 50 neighbours is not followed unless the question names it. The walk stops once the asked relations and kinds are covered.
4. 4Rank. Graph facts are scored by vector distance to the question and by how many of its word stems they contain; period words (“geçen ay”, “last week”) favour facts from that period. When the same attribute is stated twice, the newest comes first and the older one is marked as earlier.
5. 5Visibility. Every statement carries the asker's visibility rule (Section 2.3): a node the person may not see is neither returned nor walked through.
6. 6Compact. Anchors, answer paths with their evidence, and the best remaining facts are written under a token budget (about 1,000 tokens in chat, estimated at three characters per token).

2.3 Visibility rule

Access is enforced in two layers (Figure 4). PostgreSQL row-level security binds every connection to one organization and workspace. On top of it, one shared visibility rule is added to every read query: facts from a module the member cannot read (resolved from the platform's role grants), other people's private or personal facts, and superseded facts are excluded. An entity named only by hidden facts is not a node, and an edge is followed only if its module is readable and both ends are visible. If the member's grants cannot be resolved, the rule falls back to reading no module. A product caller with no person, such as a workflow or a research job, reads only the workspace's shared facts. Module and privacy are query predicates, not row-level-security policies³.

2.4 What one search call touches

Figure 5 counts the services one search call touches. Era Brain runs the query embedding in process and sends about ten SQL statements to one PostgreSQL database in one transaction. Several current systems also avoid a language model on their default read path, among them Mem0 v3, Graphiti and Zep, and Supermemory [6, 7, 8, 9]. The difference in Figure 5 is therefore the number of services and network hops, not the absence of an LLM alone. In production the studio reaches the embedding model over one internal HTTP call; the benchmark ran it in process.

Table 2 lists the datasets. All runs used the code of commit 3aba0bbfb unless a footnote says otherwise.

3.1 Metrics

Evidence in top 10 (any / all).

Whether at least one, or every, gold evidence item is among the 10 returned items: LongMemEval sessions or rounds, LoCoMo dialogue turns. LongMemEval's official retrieval metric is recall_all [1]; we report both. The 30 abstention questions have no evidence and are excluded, as in LongMemEval's own retrieval scoring. Four LoCoMo questions have no evidence label and count as misses.

Answer in context.

On the business sets: the gold answer string appears in the returned context (case-insensitive, Turkish locale). It does not prove that the answer came from the right evidence; on the 100k three-hop questions the source facts for every hop are present in 5/40.

QA accuracy.

A reader model answers each LoCoMo question from the returned context, and an LLM judge grades the answer against the gold answer, following [10]. Reader: claude-haiku-5-5; judge: claude-haiku-5-5.

Tokens.

cl100k_base count of the context text as returned; memory lists are rendered one line per memory.

Latency.

Median (p50) wall time of one recall call in the benchmark process, including the local query embedding and all queries to PostgreSQL in Docker on the same machine (Windows 11, 24 cores); one client, warm, first query excluded, no remote network.

3.2 What was not exercised

On LongMemEval and LoCoMo the conversations were stored as raw text without fact extraction, so only hybrid recall ran and the graph was not used. LLM extraction, relation judging, consolidation and relationship discovery were switched off to keep model cost at zero; their effect is not measured. In the business sets, records and relations were generated directly in the writer's input format rather than produced by a live app's connector mapping. Repeated runs differ by about one question per set, and latencies vary between runs⁴.

4.1 Answer accuracy

Figure 6 sets Era Brain's LoCoMo answer accuracy next to a same-model baseline that reads the full conversation, and next to published figures. Published figures use different reader and judge models and were not re-run by us; they are grouped by date. Mem0's and Zep's 2026 results [11, 8] use newer readers than their 2025 papers [10, 12]. The 2025 Zep figure of 66.0% was produced by Mem0's team and is disputed by Zep, whose own run reports 75.1% [13]. LoCoMo's labels contain errors that affect every judge-based score⁵.

4.2 Retrieval

Figure 7 reports retrieval on the public benchmarks. On LongMemEval-S, hybrid recall puts at least one evidence session in the top 10 for 98.3% of questions, and every evidence session for 90.2%. A plain vector top-10 already reaches 97.9% on the “any” measure, so the benchmark leaves little room there; the “all” measures separate methods more. On LoCoMo, hybrid recall finds at least one evidence turn for 72.4% against 62.6% for plain vector, with about 596 tokens of context.

4.3 Business memory

Figure 8 breaks the held-out set down by question type. Direct lookups, two-hop questions (person → employer → tax number, IBAN or website; ticket → assigned staff phone; product → supplier phone), period questions and latest-value questions are answered almost completely. Two types remain weak: questions that join a record with a conversation (an order is named, the answer is in the customer's WhatsApp or e-mail) at 37/60, and the complaint → order → invoice → payment chain at 19/30. Turkish questions score 91.8%, English ones 80.0%.

Latest value wins. When a record is re-synced with a new value, or a conversation corrects an earlier statement, the newer fact must come first. This held for 20/20 record updates and 24/25 conversation updates.

Figure 9 isolates multi-hop questions on the 100,000-row scale set. Path search answers every direct, two-hop and three-hop question. A plain vector top-10 over the same memories answers 0/60 two-hop and 1/40 three-hop questions; its 35% overall score (70/200) comes almost entirely from direct lookups. The same pattern is reported for vector RAG on public multi-hop queries [14].

4.4 Context size

Figure 10 compares context tokens per query. On LoCoMo, hybrid recall returns ten raw turns, about 596 tokens. On business questions, path search returns 1,067 tokens and an Era chat turn hands the model 1,245. The 2025 figures are what Mem0, Mem0g and Zep used for their QA scores; the 2026 systems use larger contexts at much higher accuracy. Fewer tokens at lower recall is not cheaper like-for-like, and tokenizers differ, so these numbers are indicative.

4.5 Latency

Figure 11 shows median search time. Era Brain's figures are measured in process without a network hop; vendor figures are measured through their hosted services or as stated in their publications, so the comparison indicates an order of magnitude, not a ranking. Against the fastest published hosted retrieval latency (Zep, 87 ms on LoCoMo), Era Brain's 29 ms is about 3× lower. Graph recall on business questions takes longer, 53 ms, because it walks up to three levels and applies the visibility rule at each⁶.

4.6 Permissions

In the permission test a sales member (no finance, invoices or HR) asks every question, and a finance member (no HR) asks every targeted permission question. 865 secrets are planted in set A (820 invoice amounts, 20 salaries, 15 credit limits, 10 private notes) and 790 in set H. A leak is any secret the asker may not see appearing in the returned context. Seven member read paths were swept: memory and graph recall in the agent service, recall in the studio, Era's brain-search and brain-graph tools, the Era chat-turn context and the builder copilot context. Eight callers with no person (research, two workflow blocks, internal recall and list, copilot and graph context, meetings brief) were checked separately. Figure 12 shows the result: no leak in any answer.

4.7 What changed the numbers

Table 3 lists the changes that moved the results most. Each row compares the engine before and after the named change on the same data and machine; most rows bundle several related edits, so a row does not isolate a single technique.

Table 4 compares capabilities as documented by each project on 22 October 2026, from vendor documentation, papers and source code [6, 7, 15, 9, 16, 17, 18, 19]. We mark only what is documented; “?” means we searched and found nothing. Two findings cut against Era Brain. First, a read path without a language model is no longer distinctive: Mem0 v3, Graphiti, Zep and Supermemory skip the LLM by default. Second, fine-grained access exists elsewhere: Zep Enterprise offers policy- and attribute-based rules on ingestion metadata [15], Supermemory restricts members to namespaces, and Cognee grants permissions per dataset. What we did not find elsewhere is a per-fact rule derived from the business's own role and module grants and applied at every graph step, or a published permission-leak test.

Others lead in several areas: published end-to-end QA (Mem0, Zep, Supermemory); bi-temporal edges with invalidation (Zep and Graphiti; Era's edges carry no validity interval and graph walks read current facts only); managed extraction and custom ontologies (Zep, Mem0); hosted products, SDKs and integrations; documented compliance (Zep: SOC 2 Type II, HIPAA; Supermemory: SOC 2, HIPAA); and open evaluation harnesses (Mem0, Supermemory). On LoCoMo, a plain file-search agent (Letta, gpt-4o-mini) scored 74.0% [17], above Mem0's 2025 score, so a files-and-search memory such as a wiki is a strong baseline for chat recall. Our comparison concerns business records and permissions.

• Answer accuracy is pending in this draft. Every other accuracy number is retrieval: whether the evidence reached the model, not whether a correct answer was produced.
• The business sets are synthetic and self-built. H2 is a reworded variant of a set used during tuning; validation on real customer data is still needed.
• Records and relations in the business sets were generated in the writer's input format. Today's connector mapping for a generic app emits generic relations (bought, paid by, related to), so a live graph may be sparser than the one walked here.
• “Answer in context” is a string match. On the 100k three-hop questions the source facts for every hop are present in 5/40; the answer arrives through the path line.
• The 100k set was bulk-loaded by SQL rather than written through the product, and only the owner asked.
• Latency is one in-process call with a single client on a local database; LoCoMo and LongMemEval latency predate a final speed-only change to recall.
• LLM layers (extraction, relation judging, consolidation, relationship discovery) were not run. Facts derived by consolidation are not yet tagged with module, workspace or privacy, so the zero-leak result does not cover them.
• As-of (point-in-time) recall was not tested, and graph edges carry no validity interval.
• Competitor figures are taken from their publications, not re-run; they use different models, tokenizers and setups.
• LoCoMo is licensed CC BY-NC 4.0; it was used for evaluation only and is not redistributed. The harness is not yet public.

Era Brain answers business questions from one PostgreSQL database with no language model on the read path. On the tests we built for that setting (multi-hop record joins, latest values, Turkish phrasing and per-member permissions) it found the answer for 88.9% of held-out questions and leaked no secret in 12,560 answers, with about 1,245 tokens of context per chat turn. On public conversational benchmarks its retrieval improves on a plain vector baseline, and its LoCoMo answer accuracy is 76.1%. Next steps: validation on real customer data, a public release of the harness, a run of a public access-control benchmark [5], and tagging consolidation-derived facts with module, workspace and privacy.

Repository commit 3aba0bbfb. Paths are relative to the repository root; brain-paths = packages/db/brain-paths.ts, brain-viewer = packages/db/brain-viewer.ts.