You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Two retrieval patterns are now recognized as first-class routing cases:
71
+
Gate 1 now provides powerful, deterministic knowledge base routing with flexible syntax that supports hierarchical category navigation, text search, and LLM-assisted filtering.
72
72
73
-
***Deep slash-path KB prompts** such as `sop:/a/b/c` or `a/b/c/d` are treated as focused KB retrieval requests. The system tries direct category-path lookup first, which is faster and more deterministic than falling back to a generic discovery loop.
74
-
***Explicit post-retrieval synthesis markers** use the form `:LLM <instruction>`, for example `a/b/c:LLM extract Apple company from the matches`. This suffix is reserved for cases where the user wants the model to summarize, filter, or synthesize after the KB results are already retrieved.
73
+
##### Routing Syntax Patterns
75
74
76
-
In other words, pure path-style lookup stays grounded in KB retrieval, while `:LLM ...` is the opt-in switch for higher-level reasoning over the matched items.
75
+
**1. Root Category Display**
76
+
77
+
Query just the KB name to explore available root categories:
78
+
79
+
```
80
+
omni:<KB> # Display root categories
81
+
```
82
+
83
+
**Example:**
84
+
85
+
```
86
+
Query: omni:sop
87
+
88
+
Response:
89
+
Available Categories:
90
+
91
+
• Language (150 items, 5 subcategories)
92
+
Programming language guides and tutorials
93
+
Navigate: omni:sop:language
94
+
95
+
• Architecture (89 items, 3 subcategories)
96
+
System design and architecture patterns
97
+
Navigate: omni:sop:architecture
98
+
99
+
• Operations (203 items, 7 subcategories)
100
+
DevOps, deployment, and operational guides
101
+
Navigate: omni:sop:operations
102
+
```
103
+
104
+
This provides directory-style exploration without needing to know category names upfront.
105
+
106
+
**Pagination:** Category displays show 20 items per page. Navigation (supports both `:` and `/` separators):
107
+
```
108
+
omni:sop # Page 1 (default)
109
+
omni:sop:page:2 # Page 2
110
+
omni:sop/page/3 # Page 3 (slash separator)
111
+
112
+
omni:sop:language:page:2 # Page 2 of subcategories under 'language'
113
+
omni:sop/language/page/2 # Same, using slash separator
114
+
```
115
+
116
+
When multiple pages exist, the response shows:
117
+
```
118
+
Available Categories: (Page 2 of 5, showing 21-40 of 87)
119
+
...
120
+
Previous: omni:sop:page:1 | Next: omni:sop:page:3
121
+
```
122
+
123
+
**2. Hierarchical Category Path Routing**
124
+
125
+
Any-depth category hierarchies are supported with colon-separated paths:
126
+
127
+
```
128
+
omni:<KB>:cat1 # Single level
129
+
omni:<KB>:cat1:subcat1.1 # Two levels
130
+
omni:<KB>:cat1:subcat1.1:subsubcat1.1.1 # Deep hierarchy
131
+
omni:sop:operations:performance:caching # Real-world example
132
+
```
133
+
134
+
The system performs intelligent category resolution:
135
+
-**Direct lookup** via `CategoriesByPath` B-Tree (O(1) when exact match exists)
136
+
-**Semantic fallback** using category embeddings when lexical match fails
137
+
-**Text-based discovery** for natural language category queries
138
+
139
+
**3. The `:llm <instruction>` Meta-Token**
140
+
141
+
**3. The `:llm <instruction>` Meta-Token**
142
+
143
+
Add `:llm <instruction>` after any routing query to have the LLM process the retrieved results:
144
+
145
+
```
146
+
omni:sop:operations:performance:llm summarize
147
+
omni:sop:language bindings:c#:llm explain with code examples
148
+
omni:myapp:cat1:subcat1.1:llm extract top 5 by relevance
149
+
```
150
+
151
+
**How it works:**
152
+
- The `:llm <instruction>` portion is **stripped from the query** before KB search
153
+
- KB retrieval proceeds normally using the clean category path
154
+
- Results are passed to the LLM along with the instruction as meta-guidance
155
+
- The LLM processes, filters, or synthesizes the matches according to the instruction
156
+
157
+
**Example flow:**
158
+
```
159
+
Input: omni:sop:operations:performance:caching:llm summarize the top 3
✅ omni:sop:architecture:patterns:microservices:llm compare with monolith
246
+
```
247
+
248
+
The routing system automatically:
249
+
- Normalizes colon separators to forward slashes for internal paths
250
+
- Preserves the full hierarchical context for LLM enrichment
251
+
- Strips only the `:llm` meta-token, not the category structure
77
252
78
253
***Clarification First When Needed**: Before routing and execution, Gate 0 can now keep the interaction in a clarification-first mode. If the assistant asks a focused clarification question, the next user reply is rewritten back onto the original target ask and the normal execution path resumes.
79
254
***Macro Then Micro**: Routing gates prepare the Ask frame first. The inner native ReAct loop then executes inside that frame without re-running the gates on every retry.
- LLM filtering suggestion for large sets (>40 categories)
16
+
-**`:llm <instruction>` Meta-Token**: Added support for explicit LLM post-processing instructions using `:llm` suffix (e.g., `omni:sop:operations:performance:llm summarize top 3`).
17
+
-**Clean Query Separation**: The `:llm` meta-token is automatically stripped from the KB search query and treated as post-retrieval guidance.
18
+
-**TaskContextClassification Fields**: Added `CleanQuery` and `LLMInstruction` fields to properly separate user intent from meta-commands.
19
+
-**Three-Way Routing**: Intelligent decision-making based on result count and `:llm` presence:
20
+
-`:llm` present → LLM processes with instruction (highest priority)
21
+
- 1-5 matches → Direct display (no LLM)
22
+
- 6+ matches → Automatic LLM summarization
23
+
-**Flexible Hierarchy Support**: Full support for any-depth category paths (e.g., `omni:kb:cat1:subcat1.1:subsubcat1.1.1:...`).
24
+
-**Subcategory Navigation**: When a category path has no direct items (and no `:llm` instruction), automatically returns child categories with item counts and descriptions as navigation hints.
25
+
-**Enhanced Parsing**: New `stripLLMInstruction()` function ensures consistent meta-token extraction across all query patterns.
26
+
-**Architecture Improvements**:
27
+
-`getSubcategories()` function for root and path-level category display
28
+
-`buildKBEnrichedQuery()` now uses clean queries without meta-tokens for proper LLM context
29
+
-`trySpecializedFocusedRouting()` handles root navigation, flexible hierarchy, and meta-token parsing
30
+
- Comprehensive test coverage for all routing patterns and hierarchy depths
31
+
-**Roadmap - Quoted Text Search**: Proposed support for combined category + text queries (e.g., `omni:sop:language bindings "java tutorial"`).
32
+
-**Documentation Updates**: Updated `AI_COPILOT.md`, `AI_COPILOT_USAGE.md`, and `IMPLEMENTATION.md` with comprehensive routing guides including root category navigation.
33
+
3
34
## SOP V2 build 53 (Upcoming)
4
35
-**Schema Format Enhancement**: Introduced flat schema format for better LLM understanding and correlation with Store Relations.
5
36
-**New Fields**: Added `FlatSchema`, `KeyFields`, and `ValueFields` to `StoreInfo` for improved schema representation.
LLMInstructionstring// Extracted instruction from :llm suffix
285
+
KBSearchResultsstring// Retrieved KB matches
286
+
KBMatchCountint// Number of matches found
287
+
DirectDisplaybool// Whether to bypass LLM (Case 1)
288
+
}
289
+
```
248
290
249
-
- Input shape: explicit namespace such as omni:stores:users, plus deep slash-path KB prompts such as `sop:/a/b/c` or `a/b/c/d`.
250
-
- Action: parse hard constraints and classify only the missing parts (mainly layers and CRUD intent). Deep path-style KB prompts now short-circuit into focused KB retrieval rather than drifting into the generic discovery loop.
251
-
- Result: deterministic route with low token overhead, and a clean path for direct category-path answers.
291
+
**Implementation reference:**
292
+
-`trySpecializedFocusedRouting()` in `ai/agent/classifier.go`: Main routing logic
293
+
-`stripLLMInstruction()` in `ai/agent/copilottools.search.go`: Meta-token parsing
294
+
-`searchKnowledgeBase()` in `ai/agent/copilottools.search.go`: KB search orchestration
295
+
-`buildKBEnrichedQuery()` in `ai/agent/copilot.go`: LLM context assembly
296
+
297
+
**Example query flow:**
298
+
```
299
+
Input: omni:sop:operations:performance:caching:llm summarize top 3
**Action:** Parse hard constraints, extract meta-tokens, and classify missing parts (layers, CRUD intent). Deep path-style KB prompts now short-circuit into focused KB retrieval with proper meta-token handling.
254
322
255
-
- If the user adds an explicit `:LLM <instruction>` suffix, for example `a/b/c:LLM extract Apple company from the matches`, the ask remains grounded in KB retrieval but the model is invited to synthesize or narrow the returned candidates after the path lookup.
323
+
**Result:** Deterministic route with low token overhead, clean separation of query vs. instruction, and intelligent LLM delegation based on result count.
0 commit comments