Neotoma vs file-based memory
File-based memory stores state in Markdown, JSON, or similar artifacts. Three production-scale agent platforms, Manus, Claude Code, and OpenClaw, independently converged on this pattern. It is portable, human-editable, and cache-friendly, but provides no schema enforcement, conflict detection, or auditable provenance unless layered on manually. Neotoma stores structured observations and composes entity state via deterministic reducers with formal guarantees.
Markdown and JSON files are the most common memory store for AI agents in production. Manus ($100M ARR), Claude Code ($2.5B run-rate revenue), and OpenClaw (310K GitHub stars) all independently converged on file-based memory, a convergent evolution driven by LLM economics and simplicity. They work well for single-writer workflows. The question is what happens when requirements grow beyond what files can guarantee.
Why not just use markdown files for agent memory?
File-based memory
File-based memory stores facts as plain text artifacts on disk. Agents read and write files directly, or commit them to git. The format is maximally portable, human-readable, and friendly to LLM KV-cache economics. Production patterns include todo.md for attention shaping (Manus), hierarchical CLAUDE.md files for progressive context loading (Claude Code), and MEMORY.md with optional sqlite-vec vector indexing (OpenClaw). Versioning comes from git, but conflict detection, schema validation, and provenance are not built in.
Neotoma
Neotoma stores append-only observations about entities. Deterministic reducers compose all observations into a single entity snapshot. Every field traces to its source observation, schema validation rejects malformed writes, and conflicting facts are resolved deterministically. MCP-based retrieval of entity snapshots is equally cache-friendly to LLMs, structured text injected into context with the same KV-cache benefits as files, but with state integrity guarantees.
Guarantee comparison
When to use which
Use File-based memory when
You have a single agent, simple state, and value being able to open memory in a text editor. Git-based versioning is sufficient for your audit needs, and you do not need conflict resolution or schema enforcement.
Use Neotoma when
Multiple agents or tools write to the same entities. You need schema validation, conflict detection, reproducible state reconstruction, or auditable provenance that goes beyond git commit history. You have outgrown what files can guarantee but do not want to build and maintain memory infrastructure yourself.
Common questions
If Manus and Claude Code use markdown files, why would I need Neotoma?
Those systems document the same failure modes Neotoma solves: concurrent writes corrupt state, there is no conflict detection, no entity resolution across files, no schema validation, and no provenance. Claude Code caps always-loaded memory at 200 lines because files get bloated and contradictory. OpenClaw implements automatic memory flush when context limits approach, losing information. Neotoma's observation model and deterministic reducers are the engineering answers to these documented limitations.
Can git replace versioned history?
Git versions file snapshots, not entity observations. It can tell you what a file looked like at a commit, but not which observation changed which field, or how conflicting writes from different agents were resolved. Git versions the output; Neotoma versions the inputs.
What about structured formats like JSON or YAML?
Structured file formats give you parseable data but not memory guarantees. A malformed edit, an accidental deletion, or two agents writing different values to the same key are all accepted silently. Neotoma validates writes against schemas and rejects invalid data at store time.
Can I use file-based memory alongside Neotoma?
Yes. You can use files for human-editable configuration and notes while using Neotoma for entity state that needs formal guarantees. They address different layers of the problem, and this layered approach mirrors what production systems like OpenClaw already do by indexing files with sqlite-vec for retrieval.