Architecture
How GEM works inside.
The overview shows what GEM does. This page is for engineers: the path every write takes, how dependencies are stored, the decision layer that keeps it cheap, what that saves, and where to plug in your own model.
The write path.
Two passes on every write, and a cascade only when something changed. Reads are plain similarity search over facts that are still valid.
The memory graph.
Facts are nodes. Edges are typed, and only one type carries a change.
Where edges come from. Pass derived_from=[ids] when you store a fact and the edge is exact. Leave it out and GEM infers the links from similar facts, with about 85–88% recall and 75–84% precision in our tests. Pin the ones you can't afford to miss.
What a fact can become. Active, possibly rewritten in place when its new value follows from the change. Stale with needs_review when its value is now unknown. Superseded when it was directly contradicted or forgotten. Nothing is deleted, so every change can be traced with why.
Closing the loop. A stale fact waits for the user. When the agent saves the answer, GEM matches it to the stale fact (or the agent names it with resolves): the answer replaces it, inherits its parents and takes over its dependents, so the next change still reaches it. If the user says the old value holds, confirm restores it, along with the facts that went stale only because it was in doubt.
Where it lives. In memory, in a JSON file per memory space (safe across processes: writes take a file lock), or in FalkorDB.
The decision layer.
Most of GEM's work is judgment calls. A small typed-decision model answers them first; the LLM only does what needs one.
Every write raises a series of small questions. Does this new fact conflict with that stored one? Is this dependent affected by this specific change? Can its new value be worked out? Asking a large language model each question is accurate but slow and costly, and most answers are easy.
So GEM puts a decision layer in front: Jev, from TypeSafe, a small model that answers typed questions (yes/no, multiple choice, 0–3 scores) with probabilities instead of text. GEM batches every question in a step into one request and routes each answer:
- 1Stopping needs 90% confidence. A wrong “unaffected” would hide a stale fact, so anything doubtful costs an LLM call instead.
- 2The decision layer never supersedes a dependent on its own. It can stop the cascade or mark a fact stale for review; a destructive change needs the LLM.
- 3If it's unavailable, nothing is skipped. Every decision falls back to the LLM, and the fallback is counted so a run can be checked.
Same accuracy, fewer LLM calls.
Measured on the repository's evaluation suites. Each chart names its run files.
LLM calls to run the same 12 scenarios
All three answered 37 of 37 facts correctly
Accuracy on 19 held-out scenarios, five runs each
Share of facts handled correctly; the axis starts at 90%
LLM calls per write on the held-out set (about 25 for 19 writes)
fewer LLM calls than the original engine on the 12-scenario suite (58 → 12)
decision-layer failures in the measured runs: no decision fell back to the LLM
Not yet measured: the LLM-only call count on the held-out set, so the saving there is not quantified; and a sweep over all 139 generated scenarios. Details in PRODUCTIONIZATION.md.
Bring your own model.
Everything is set in one .env file. Copy .env.example and pick a backend.
| Backend | Settings |
|---|---|
| Ollama, local or cloud | GEM_LLM=ollama · GEM_MODEL · OLLAMA_HOST |
| Groq | GEM_LLM=groq · GROQ_MODEL · GROQ_API_KEY |
| Anything else | A class with chat() and chat_json(), returned from make_llm() in gem/llm.py |
| Decision layer | Optional: pip install -e ".[jev]", then GEM_DECIDER=jev · TYPESAFE_API_KEY |
class MyClient:
def chat(self, system, user, *, json_mode=False, temperature=None) -> str:
... # call your provider, return the reply text
def chat_json(self, system, user, *, retries=1, temperature=None) -> dict:
return _salvage_json(self.chat(system, user, json_mode=True,
temperature=temperature))
def make_llm():
return MyClient()The published numbers use gpt-oss-120b and gpt-oss-20b. Smaller models work but over-invalidate more; pair them with Memory(conservative=True), which flags facts for review instead of superseding them.
Code map.
Where each part lives.
- gem/engine.pyThe write path: conflict scan, linking, the cascade and its stopping rules.
- gem/decide.pyThe decision layer: batched typed questions and the routing thresholds.
- gem/classify.pyThe LLM questions: conflict labels and inferred links, with retries and safe defaults.
- gem/llm.pyModel clients and make_llm(), the one place to plug in your own.
- gem/memory.pyThe Python API: add, forget, search, profile, why.
- gem/tools.py · mcp_server.pyTool definitions and the MCP server.
- gem/json_store.pyFile storage with cross-process locking; FalkorDB lives in falkor_store.py.