125 million tokens in memory. The score didn't move.
BEAM is the public benchmark for AI memory: 200 questions about 10 million tokens of conversation. We added 115 million tokens of unrelated libraries and law to the same memory, and asked again.
Written by Claude Fable 5.1, an AI model, from the run notes and grading output. Reviewed by James Youngquist, Mad Science Software. Published 22 October 2026, updated 26 October 2026.
17,000 → 19,000 tokens
at 10M and 125M in memory
2–3 cents
list price
Level with Mem0
| Ability | Chiron 10M | Chiron 125M | Mem0 10M | |
|---|---|---|---|---|
| 2 | Where Chiron leads | |||
| 3 | Says when it doesn't knowAsked about something that was never said, it says so instead of inventing an answer. | 85.0 | 90.0 | 40.0 |
| 4 | Resolves contradictionsTwo things were said that can't both be true. It works out which one stands. | 48.1 | 50.0 | 32.5 |
| 5 | Keeps up with updatesA fact changed later in the conversation. It answers with the newer one. | 78.3 | 77.5 | 75.0 |
| 6 | Pulls out detailsNames, numbers and specifics from anywhere in the history. | 62.5 | 65.0 | 56.3 |
| 7 | Puts events in orderWhich happened first, and which came after. | 26.6 | 28.4 | 20.2 |
| 8 | About even | |||
| 9 | SummarizesThe whole of a long thread in a few lines. | 47.9 | 52.3 | 46.9 |
| 10 | Reasons across sessionsAn answer that needs pieces from several conversations. | 25.8 | 23.0 | 26.1 |
| 11 | Reasons about timeHow long ago, how long between, and what was true when. | 16.2 | 20.0 | 16.3 |
| 12 | Where Mem0 leads | |||
| 13 | Follows preferencesRemembers how you like things done, and does them that way. | 58.3 | 57.1 | 90.4 |
| 14 | Follows instructionsA rule set earlier in the conversation still applies later. | 55.8 | 50.0 | 82.5 |
| 15 | BEAM scoreMean of the ten abilities | 50.5 | 51.3 | 48.6 |
Scores out of 100. 10M is the mean of three passes, 125M one pass. Under two points is inside our own pass-to-pass spread.
Standing rules belong in a file the assistant reads every time, such as CLAUDE.md, not in recall. Chiron teams keep them there.
How we ran it
BEAM: Beyond a Million Tokens (Tavakoli and others, ICLR 2026). Ten conversations of about 10 million tokens, 20 questions each, two per ability.
One Chiron memory per conversation, cues by Claude Haiku 5.5. The 125M run added nine team libraries, the full United States Code and a quarter of the Code of Federal Regulations.
Claude Sonnet 5.5 with the Chiron plugin, a fresh container per question, BEAM's answering prompt.
BEAM's method: GPT-4.1-mini at temperature zero scores each point of the reference answer 0, ½ or 1; event ordering by rank correlation. The BEAM score is the mean of the ten abilities.
10M: 51.4, 49.6 and 50.4. 125M: 51.3.
Chiron 0.7.1 at 10M; at 125M, a development build with the search index from Chiron 0.9.1.
As published in May 2026. Mem0 does not name its answering or judging model. Its per-ability figures are the only others we found.
Internal results, October 2026. Ask us for the run notes and grading scripts.
Questions
What is BEAM?
BEAM (Beyond a Million Tokens) is a public benchmark for long-term memory in AI assistants, published at ICLR 2026 by Tavakoli and colleagues. It holds 100 generated conversations of up to 10 million tokens each, and 2,000 questions that test ten memory abilities, from recalling a fact to putting events in order.
How does Chiron score on BEAM?
At the 10-million-token size, Chiron scores 50.5 out of 100, the mean of three complete passes, against 48.6 for Mem0's published result. A difference of under two points is inside the spread between our own passes, so we call the two level.
Does more memory lower the score?
No. With 115 million tokens of unrelated libraries and law added, 125 million tokens in all, Chiron scored 51.3 against 50.5. A question reads 17,000 to 19,000 tokens either way, so answers don't get slower or costlier as the memory grows.
Which models answered and graded?
Claude Sonnet 5.5 with the Chiron plugin answered, in a fresh container for every question, with BEAM's own answering prompt. Grading used BEAM's method: GPT-4.1-mini at temperature zero. Mem0 did not publish which models answered or graded its run.