officialCodeWork · officialCodeWork · May 26, 2026 · May 26, 2026
diff --git a/TRACKER.md b/TRACKER.md
@@ -8,7 +8,7 @@
 > Use `Taskfile.yml` for task targets (cross-platform), `Makefile` for Unix convenience.
 > CI matrix always includes `ubuntu-22.04`, `macos-14`, and `windows-latest`.
 
-**Last updated:** 2026-05-26 (Step 2.4 — Graph retrieval)
+**Last updated:** 2026-05-26 (tracker sync — PR #84)
 **Current phase:** Phase 2 — Retrieval Engine
 **Next action:** Phase 2 Step 2.5 — Hybrid RRF fusion: combine dense vector + BM25 + graph retrieval into a single ranked list via Reciprocal Rank Fusion with configurable per-backend weights; this is the first step that *consumes* the three storage backends Steps 2.1–2.4 have shipped. Step 2.4 added the graph triple (`Neo4jGraphStore` + `MemgraphGraphStore` + `NetworkXGraphStore`) with `expand()` as the canonical multi-hop traversal primitive on the `GraphRetrievalBackend` SPI + `FilterExpr` push-down on traversal edges.
 
@@ -95,7 +95,7 @@
 | 2.1 | Knowledge store read layer | ✅ | `build/phase-2/step-2.1-knowledge-store-read-layer` | [#74](https://github.com/officialCodeWork/AgentContextOS/pull/74) | `FilterExpr` AST + in-memory `evaluate()` reference semantics relocated from `rag-policy` into `rag_core.filter` so SPI signatures can take it directly without inverting the dependency graph (`rag_policy.filter` keeps re-exporting every symbol for back-compat). `VectorRetrievalBackend.retrieve_ids` switched from `dict[str, Any]` to typed `FilterExpr \| None`; `KeywordRetrievalBackend.retrieve_ids` gains the same parameter. New SPI signature gate `test_retrieve_ids_accepts_filter_expr` enforces the shape on every retrieval ABC. Noop stores (`NoopVectorStore`, `NoopKeywordStore`) apply the predicate via `evaluate()` against a per-chunk attribute view — the canonical oracle every translator must agree with. `PgVectorStore`: new `_filter_sql.translate()` produces parameterised `WHERE` fragments, `acl_labels TEXT[]` column + GIN index added (idempotent `ADD COLUMN IF NOT EXISTS` migration on `initialize()`), `bulk_index` persists ACL labels from `Embedding.acl_labels`. `QdrantVectorStore`: new `_filter_qdrant.translate()` builds `Filter(must / must_not / should)` merged with existing tenant/corpus `must` clauses, payload gains `acl_labels` on every write. Unsupported fields raise `RetrievalError` (no silent push-down loss). 75 new tests (16 `evaluate` units, 16 SQL translator units, 11 Qdrant translator units, 5 vector contract push-down, 3 keyword contract push-down, 1 signature gate, schema regen). New docs: [architecture/retrieval-read-layer.md](docs/architecture/retrieval-read-layer.md); `reference/rag-core.md` + `reference/rag-policy.md` updated. `rag-core` 0.13.0 → 0.14.0. Schemas regenerated (`Eq.json`, `AnyIn.json`, `And.json`, `Or.json`, `Not.json`, `TrueExpr.json`). |
 | 2.2 | Vector retrieval backends | ✅ | `build/phase-2/step-2.2-vector-retrieval-backends` | [#76](https://github.com/officialCodeWork/AgentContextOS/pull/76) | Three new vector backends behind their own optional extras: `WeaviateVectorStore` (`[weaviate]`, weaviate-client v4 async; single `RagEmbeddings` collection w/ `tenant_id` filter + UUID5 `(tenant, chunk)` IDs; `text[]` `acl_labels` property pushed down via `_filter_weaviate.translate()` that lazy-imports the SDK and composes `Filter.by_property(...).equal/contains_any` with `&`/`\|`/`~`), `PineconeVectorStore` (`[pinecone]`, PineconeAsyncio v5+; namespace=tenant primary isolation primitive plus belt-and-braces `tenant_id` filter; metadata: `tenant_id`/`chunk_id`/`corpus_id`/`model`/`acl_labels`; `_filter_pinecone.translate()` emits MongoDB-style `$eq`/`$in`/`$ne`/`$nin`/`$and`/`$or` and rewrites `Not` via De Morgan since Pinecone has no `$not`), `ElasticsearchDenseVectorStore` (`[elasticsearch]`, AsyncElasticsearch 8.x w/ `dense_vector` + native kNN; single index, `tenant_id` keyword filter, composite `_id="<tenant>::<chunk>"`; `_filter_elasticsearch.translate()` emits query-DSL `bool.filter`/`terms`/`match_all`/`match_none`). All five backends now consume `IndexHint` via the new shared `_index_hint.select_index_variant()` helper that centralises the ADR-0009 §1 size/recall → variant table — pgvector picks ivfflat vs HNSW DDL at `initialize()` time (with a new `_index_ddl_for` helper; PQ tier falls back to HNSW w/ larger `m` since pgvector has no PQ), Qdrant configures `HnswConfigDiff` + scalar `ScalarQuantization(INT8)` for the PQ tier (new `_qdrant_index_config` helper), Weaviate picks `Configure.VectorIndex.{flat,hnsw,hnsw+pq}` via a `_vector_index_config` helper, ES picks `index=false` (flat) vs HNSW `index_options.{m,ef_construction}` per tier, Pinecone records the advisory variant in logs (managed indexes). Each SDK is imported lazily inside `__init__` so importing the module without the extra still succeeds (constructing the class raises a clear `ImportError` pointing at the extra). 53 new tests: 11 `_index_hint` parametrised tier tests, 12 `_filter_pinecone` translator units, 11 `_filter_elasticsearch` translator units, 13 `_filter_weaviate` translator units (SDK stubbed at `sys.modules` so they run without the extra) + 6 integration suites per backend gated on `pytest.importorskip` + a service-reachability fixture (Weaviate via `/v1/.well-known/ready`, ES via `_cluster/health`, Pinecone via `TEST_PINECONE_API_KEY`). Policy-engine coverage allowlist extended for the three new files (the call site, not the engine, consults `PolicyEngine`). New extras + workspace wiring in `packages/backends/pyproject.toml` (`weaviate`/`pinecone`/`elasticsearch`) and mypy overrides in root `pyproject.toml` so CI without the extras stays green. Cross-platform: all three SDKs ship pure-Python or wheel distributions for ubuntu/macos/windows. Deliberately deferred: `ragctl ann tune` (Step 2.6 — query understanding owns the planner-side tuning surface) and per-corpus `dtype: int8` knob in `rag.yaml` (Step 3.5 — corpus router lands rag.yaml → backend wiring). New docs: [reference/backends.md](docs/reference/backends.md) (Weaviate / Pinecone / ES sections + IndexHint mapping tables per backend), [architecture/vector-backends.md](docs/architecture/vector-backends.md) (IndexHint → variant table, FilterExpr translator contracts, tenant-isolation patterns, extension points). |
 | 2.3 | BM25 / keyword retrieval | ✅ | `build/phase-2/step-2.3-bm25-keyword-retrieval` | [#82](https://github.com/officialCodeWork/AgentContextOS/pull/82) | Two new `KeywordStore` backends behind their own optional extras, on top of the Step 2.1 read-layer contract — no new SPI surface, per the Step 2.3 constraint. `ElasticsearchKeywordStore` (`[elasticsearch]`, AsyncElasticsearch 8.x; single `rag-keyword` index for all tenants w/ `tenant_id` keyword filter + composite `_id="<tenant>::<chunk>"`; BM25 over operator-selected analyzer on `body`/`title`/`section_path`; `acl_labels`/`trust_level`/`document_id`/`corpus_id` keyword-filterable; `metadata` stored as `object` w/ `enabled: false` so the mapping doesn't explode; new `rag_backends.keyword._filter_es.translate()` emits ES query-DSL `bool.filter`/`terms`/`match_all`/`match_none` with the broader keyword-side field set vs. the vector translator). `TantivyKeywordStore` (`[tantivy]`, tantivy-py 0.22+; RAM or persistent index; sync API wrapped in `asyncio.to_thread`; composite `_doc_id` field enables idempotent re-indexing via `delete_documents` before `add_document`; `_filter_tantivy.translate()` composes `boolean_query` w/ `term_query` + pairs `MustNot` with `all_query` so `Not(x)` composes stand-alone; metadata stashed as JSON blob for hydrate round-trip). **Per-field boosting is a backend configuration knob, not a per-call SPI parameter** (operator passes `field_boosts={"title": 3.0, "section_path": 2.0, "body": 1.0}` at construction time; ES applies via `multi_match.fields` with `type=best_fields`, tantivy applies via parser's `field_boosts` arg). Same indexed-field schema across both backends so swapping is a config change. SDKs imported lazily so `import rag_backends.keyword.{elasticsearch,tantivy}` without the extra still succeeds. 47 new tests: 13 ES filter-translator units + 14 tantivy filter-translator units (tantivy module stubbed at `sys.modules`) + 9 ES backend units (full SDK stubbed — initialize/bulk_index/search/mget/delete) + 11 tantivy backend units (full SDK stubbed — including idempotency + boost ranking). Policy-engine coverage allowlist extended for the two new files (call site, not the backend, consults `PolicyEngine`). `tantivy>=0.22` added to root `pyproject.toml` mypy overrides. Cross-platform: tantivy-py ships pre-built wheels for ubuntu/macos/windows (no Rust toolchain required at install time); ES SDK pure-Python. Deliberately deferred: rag.yaml → backend wiring (Step 3.5 corpus router), real-time index lag metrics (Step 5.1), custom highlighter / snippet generation (Step 2.8 context packer owns snippet extraction). New docs: [reference/backends.md](docs/reference/backends.md) (ES BM25 + tantivy sections), [architecture/keyword-backends.md](docs/architecture/keyword-backends.md) (per-field boosting as a backend config knob, indexed-field schema, tenant isolation pattern, FilterExpr translator contracts, tantivy `Not` quirk, when to pick which). |
-| 2.4 | Graph retrieval | ✅ | `build/phase-2/step-2.4-graph-retrieval` | — | Three new `GraphStore` backends + a structured multi-hop traversal primitive layered onto the existing SPI surface. New `NeighborResult` Pydantic frozen model in `rag_core.types` (node_id / labels / properties / distance / path). New `expand(ctx, seed_ids, *, hops, rel_types, direction, node_filter, edge_filter, limit) → list[NeighborResult]` method on `GraphRetrievalBackend` — added as a **non-abstract default method** that raises `NotImplementedError`, so the abstract SPI surface is unchanged and existing ABC consumers (NoopGraphStore aside, which we explicitly upgraded) keep compiling. `NoopGraphStore` upgraded with a reference BFS implementation supporting `FilterExpr` node/edge filters — the conformance oracle every concrete backend must agree with. **`FilterExpr` push-down on edges + nodes uses the existing AST — no new types, no new parameter shape.** `edge_filter` applies to **every relationship** along the traversal path (the Step 2.4 headline ACL use case); `node_filter` applies to the **final neighbor only**, mirroring Cypher `WHERE neighbor.foo = X` placed at the path endpoint rather than every intermediate node (intermediate-node filtering would silently drop reachability — almost always wrong). Three backends: `Neo4jGraphStore` (`[neo4j]`, neo4j async driver via Bolt + Cypher), `MemgraphGraphStore` (`[memgraph]`, same neo4j driver — Memgraph speaks the same protocol), `NetworkXGraphStore` (`[networkx]`, pure-Python in-process MultiDiGraph). Neo4j + Memgraph share `BoltCypherStore` so the Cypher composition + parameter handling + tenant isolation + relationship-type validation are written once. Schema convention shared across all three: nodes MERGE'd by `(id, tenant_id)`, labels stored as a property (avoids APOC / dynamic labels), every relationship carries `tenant_id` so multi-hop traversal blocks cross-tenant leaks at every edge, relationship types validated against `^[A-Za-z_][A-Za-z0-9_]*$` + backtick-quoted (Cypher has no relationship-type parameter — interpolation is unavoidable, but injection is not). New `rag_backends.graph._filter_cypher.translate(expr, *, variable, array_fields, param_prefix)` emits parameterised Cypher `WHERE` fragments + a parameter dict; `param_prefix` lets a caller compose multiple translator outputs into one Cypher statement without parameter collisions (used by `expand()` to combine edge + node filters). 65+ new tests: 13 conformance tests for `expand()` on `NoopGraphStore` (the oracle — one-hop / two-hop / seed exclusion / path recording / rel_types / direction / edge & node filters / limit / tenant isolation / hops≤0 / missing seed), 16 Cypher translator units, 21 Neo4j + Memgraph unit tests (neo4j driver stubbed at `sys.modules` — every Cypher statement captured + asserted on for shape + parameter scoping + injection guard), 14 NetworkX backend tests (real networkx pulled via the extra; results compared element-wise against the Noop reference oracle for portability). Policy-engine coverage allowlist extended for the four new backend files (call site, not the backend, consults `PolicyEngine`). `neo4j>=5.20` / `networkx>=3.2` added to root `pyproject.toml` mypy overrides. Schemas regenerated (`NeighborResult.json`). `query()` on `NetworkXGraphStore` deliberately raises `RetrievalError` — NetworkX has no native query language, callers should use `expand()` or reach into the exposed `.graph` MultiDiGraph property for direct NetworkX algorithm calls. Cross-platform: neo4j driver is pure-Python; networkx is pure-Python (~2 MB, no system deps). Deliberately deferred: GraphRAG / community detection (Step 2.9), per-edge weight scoring (Step 2.9), `rag.yaml` → backend wiring (Step 3.5 corpus router), per-tenant separate databases (Step 6.2 physical tenancy). New docs: [reference/backends.md](docs/reference/backends.md) (Neo4j / Memgraph / NetworkX sections including `expand()` Cypher composition + schema convention + extension points), [architecture/graph-backends.md](docs/architecture/graph-backends.md) (why `expand()` is the canonical primitive, tenant-isolation pattern, FilterExpr push-down on edges vs. node, NoopGraphStore as the conformance oracle, when to pick which backend). |
+| 2.4 | Graph retrieval | ✅ | `build/phase-2/step-2.4-graph-retrieval` | [#84](https://github.com/officialCodeWork/AgentContextOS/pull/84) | Three new `GraphStore` backends + a structured multi-hop traversal primitive layered onto the existing SPI surface. New `NeighborResult` Pydantic frozen model in `rag_core.types` (node_id / labels / properties / distance / path). New `expand(ctx, seed_ids, *, hops, rel_types, direction, node_filter, edge_filter, limit) → list[NeighborResult]` method on `GraphRetrievalBackend` — added as a **non-abstract default method** that raises `NotImplementedError`, so the abstract SPI surface is unchanged and existing ABC consumers (NoopGraphStore aside, which we explicitly upgraded) keep compiling. `NoopGraphStore` upgraded with a reference BFS implementation supporting `FilterExpr` node/edge filters — the conformance oracle every concrete backend must agree with. **`FilterExpr` push-down on edges + nodes uses the existing AST — no new types, no new parameter shape.** `edge_filter` applies to **every relationship** along the traversal path (the Step 2.4 headline ACL use case); `node_filter` applies to the **final neighbor only**, mirroring Cypher `WHERE neighbor.foo = X` placed at the path endpoint rather than every intermediate node (intermediate-node filtering would silently drop reachability — almost always wrong). Three backends: `Neo4jGraphStore` (`[neo4j]`, neo4j async driver via Bolt + Cypher), `MemgraphGraphStore` (`[memgraph]`, same neo4j driver — Memgraph speaks the same protocol), `NetworkXGraphStore` (`[networkx]`, pure-Python in-process MultiDiGraph). Neo4j + Memgraph share `BoltCypherStore` so the Cypher composition + parameter handling + tenant isolation + relationship-type validation are written once. Schema convention shared across all three: nodes MERGE'd by `(id, tenant_id)`, labels stored as a property (avoids APOC / dynamic labels), every relationship carries `tenant_id` so multi-hop traversal blocks cross-tenant leaks at every edge, relationship types validated against `^[A-Za-z_][A-Za-z0-9_]*$` + backtick-quoted (Cypher has no relationship-type parameter — interpolation is unavoidable, but injection is not). New `rag_backends.graph._filter_cypher.translate(expr, *, variable, array_fields, param_prefix)` emits parameterised Cypher `WHERE` fragments + a parameter dict; `param_prefix` lets a caller compose multiple translator outputs into one Cypher statement without parameter collisions (used by `expand()` to combine edge + node filters). 65+ new tests: 13 conformance tests for `expand()` on `NoopGraphStore` (the oracle — one-hop / two-hop / seed exclusion / path recording / rel_types / direction / edge & node filters / limit / tenant isolation / hops≤0 / missing seed), 16 Cypher translator units, 21 Neo4j + Memgraph unit tests (neo4j driver stubbed at `sys.modules` — every Cypher statement captured + asserted on for shape + parameter scoping + injection guard), 14 NetworkX backend tests (real networkx pulled via the extra; results compared element-wise against the Noop reference oracle for portability). Policy-engine coverage allowlist extended for the four new backend files (call site, not the backend, consults `PolicyEngine`). `neo4j>=5.20` / `networkx>=3.2` added to root `pyproject.toml` mypy overrides. Schemas regenerated (`NeighborResult.json`). `query()` on `NetworkXGraphStore` deliberately raises `RetrievalError` — NetworkX has no native query language, callers should use `expand()` or reach into the exposed `.graph` MultiDiGraph property for direct NetworkX algorithm calls. Cross-platform: neo4j driver is pure-Python; networkx is pure-Python (~2 MB, no system deps). Deliberately deferred: GraphRAG / community detection (Step 2.9), per-edge weight scoring (Step 2.9), `rag.yaml` → backend wiring (Step 3.5 corpus router), per-tenant separate databases (Step 6.2 physical tenancy). New docs: [reference/backends.md](docs/reference/backends.md) (Neo4j / Memgraph / NetworkX sections including `expand()` Cypher composition + schema convention + extension points), [architecture/graph-backends.md](docs/architecture/graph-backends.md) (why `expand()` is the canonical primitive, tenant-isolation pattern, FilterExpr push-down on edges vs. node, NoopGraphStore as the conformance oracle, when to pick which backend). |
 | 2.5 | Hybrid RRF fusion | ⏳ | — | — | Reciprocal Rank Fusion across dense+sparse+graph; configurable weights |
 | 2.6 | Query understanding | ⏳ | — | — | Rewriter, decomposer, HyDE, synonym/glossary expansion; `query.understand` span |
 | 2.7 | Cross-encoder reranker | ⏳ | — | — | Cross-encoder + MMR diversity; `rerank` span with top_k_in/out |
@@ -248,6 +248,8 @@
 | [#76](https://github.com/officialCodeWork/AgentContextOS/pull/76) | feat(backends): vector retrieval backends — Weaviate / Pinecone / Elasticsearch + IndexHint wiring (Step 2.2) | `build/phase-2/step-2.2-vector-retrieval-backends` | ✅ Merged | 2026-05-26 |
 | [#77](https://github.com/officialCodeWork/AgentContextOS/pull/77) | chore(tracker): sync Step 2.2 → log PR #76 + history rows #75, #76 | `chore/tracker-sync-pr76` | ✅ Merged | 2026-05-26 |
 | [#82](https://github.com/officialCodeWork/AgentContextOS/pull/82) | feat(backends): BM25 keyword retrieval — Elasticsearch + tantivy (Step 2.3) | `build/phase-2/step-2.3-bm25-keyword-retrieval` | ✅ Merged | 2026-05-26 |
+| [#83](https://github.com/officialCodeWork/AgentContextOS/pull/83) | chore(tracker): sync Step 2.3 → log PR #82 + history rows #77, #82 | `chore/tracker-sync-pr82` | ✅ Merged | 2026-05-26 |
+| [#84](https://github.com/officialCodeWork/AgentContextOS/pull/84) | feat(core,backends): graph retrieval — Neo4j + Memgraph + NetworkX + expand() (Step 2.4) | `build/phase-2/step-2.4-graph-retrieval` | ✅ Merged | 2026-05-26 |
 
 ---