GEMarchitectureby Revanth Darshan D R ↗

Architecture

How GEM works inside.

The overview shows what GEM does. This page is for engineers: the path every write takes, how dependencies are stored, the decision layer that keeps it cheap, what that saves, and where to plug in your own model.

The write path.

Two passes on every write, and a cascade only when something changed. Reads are plain similarity search over facts that are still valid.

01New factadd_memory(…) 02Neighbourssimilarity top-k 03Conflict scanchanges a stored fact? 04Store + linkderived-from edges 05Cascadeone hop at a time 06Resultupdated · stale · kept one batched questionone call per hop Decision layertyped questions answered with probabilities, every item in one requestsettles: no conflict · fully replaced · stop here · mark stale LLMwrites correctedfacts, settlesunsure cases only ifneeded
Stages 03 and 05 ask the decision layer; only rewrites and doubtful cases reach the LLM. Without the decision layer configured, the LLM answers every question: same pipeline, more calls.

The memory graph.

Facts are nodes. Edges are typed, and only one type carries a change.

I moved to Mumbai I live in Bangalore Commute is 45 minutes Visited Bangalore Palace contradictsaudit: what replaced it derived fromthe cascade follows this associatedsimilar text, never followed
A fact that only mentions the same place, name or number is associated, not derived, so a change never travels to it.

Where edges come from. Pass derived_from=[ids] when you store a fact and the edge is exact. Leave it out and GEM infers the links from similar facts, with about 85–88% recall and 75–84% precision in our tests. Pin the ones you can't afford to miss.

What a fact can become. Active, possibly rewritten in place when its new value follows from the change. Stale with needs_review when its value is now unknown. Superseded when it was directly contradicted or forgotten. Nothing is deleted, so every change can be traced with why.

Closing the loop. A stale fact waits for the user. When the agent saves the answer, GEM matches it to the stale fact (or the agent names it with resolves): the answer replaces it, inherits its parents and takes over its dependents, so the next change still reaches it. If the user says the old value holds, confirm restores it, along with the facts that went stale only because it was in doubt.

Where it lives. In memory, in a JSON file per memory space (safe across processes: writes take a file lock), or in FalkorDB.

The decision layer.

Most of GEM's work is judgment calls. A small typed-decision model answers them first; the LLM only does what needs one.

Every write raises a series of small questions. Does this new fact conflict with that stored one? Is this dependent affected by this specific change? Can its new value be worked out? Asking a large language model each question is accurate but slow and costly, and most answers are easy.

So GEM puts a decision layer in front: Jev, from TypeSafe, a small model that answers typed questions (yes/no, multiple choice, 0–3 scores) with probabilities instead of text. GEM batches every question in a step into one request and routes each answer:

A dependent fact+ the change upstream Decision layerP(unaffected)P(new value follows)one batched call per hop P(unaffected) ≥ 0.9Stop here: the fact is kept, nothing below is checked 0.35 < P(unaffected) < 0.9Unsure: the LLM decides affected · new value followsThe LLM writes the corrected fact affected · new value unknownMarked stale and flagged for review, no LLM
One decision in the cascade. The conflict scan works the same way: a confident “no conflict” settles it, a fact the new one fully replaces is rewritten, and anything else goes to the LLM.

Same accuracy, fewer LLM calls.

Measured on the repository's evaluation suites. Each chart names its run files.

LLM calls to run the same 12 scenarios

All three answered 37 of 37 facts correctly

The first two bars are the LLM making every decision; the second skips conflict checks on facts the cascade already reached. Measured with the expected answers supplied, so the decisions are identical and only the calls differ. With a real model (gpt-oss-20b), the decision layer used 20 LLM calls and still scored 37/37. Source: gem/eval_diverse.py, runs/eval_diverse_groq20b_jev.txt.

Accuracy on 19 held-out scenarios, five runs each

Share of facts handled correctly; the axis starts at 90%

LLM decides everythingwith the decision layer
Scenarios not used to set the thresholds. With the decision layer, all five runs scored 100% and no scenario flipped between runs; the LLM-only runs averaged 99.0% with two flips. Same model (gpt-oss-120b). Source: runs/determinism_120b.txt, runs/determinism_jev_groq120b.txt.
≈1.3

LLM calls per write on the held-out set (about 25 for 19 writes)

79%

fewer LLM calls than the original engine on the 12-scenario suite (58 → 12)

0

decision-layer failures in the measured runs: no decision fell back to the LLM

Not yet measured: the LLM-only call count on the held-out set, so the saving there is not quantified; and a sweep over all 139 generated scenarios. Details in PRODUCTIONIZATION.md.

Bring your own model.

Everything is set in one .env file. Copy .env.example and pick a backend.

BackendSettings
Ollama, local or cloudGEM_LLM=ollama · GEM_MODEL · OLLAMA_HOST
GroqGEM_LLM=groq · GROQ_MODEL · GROQ_API_KEY
Anything elseA class with chat() and chat_json(), returned from make_llm() in gem/llm.py
Decision layerOptional: pip install -e ".[jev]", then GEM_DECIDER=jev · TYPESAFE_API_KEY
gem/llm.py
class MyClient:
    def chat(self, system, user, *, json_mode=False, temperature=None) -> str:
        ...  # call your provider, return the reply text

    def chat_json(self, system, user, *, retries=1, temperature=None) -> dict:
        return _salvage_json(self.chat(system, user, json_mode=True,
                                       temperature=temperature))

def make_llm():
    return MyClient()

The published numbers use gpt-oss-120b and gpt-oss-20b. Smaller models work but over-invalidate more; pair them with Memory(conservative=True), which flags facts for review instead of superseding them.

Code map.

Where each part lives.