Appearance
AI Engine
TLDR — The AI module (
service/api/src/modules/ai/) hosts two subsystems under one namespace: AI App Studio (current, mounted at/ai/*) — build and run AI apps as flow graphs over knowledge bases, LLM integrations and MCP tool servers; and a legacy assistant-config + RAG ingest surface (/assistant/*) that survives for per-collection ingest mapping and backfill into KAOS. Everything is gated by the app-moduleai(view/manage). The old grounded/assistant/chatorchestrator (gate → retrieve → synthesize →enforceGrounding) has been retired — chat now runs through the flow executor.
Overview
The module registers under app-module ai and exposes a single Fastify router (routes.ts) that aggregates eight controllers:
| Controller | Prefix | Purpose |
|---|---|---|
apps.controller | /ai/apps | App CRUD + publish/clone/monitoring, flow graph, logs, annotations, runtime chat |
knowledge.controller | /ai/knowledge | Knowledge-base CRUD + documents + retrieval-test query |
integrations.controller | /ai/integrations, /ai/models | LLM/model-provider CRUD + connection test + model discovery |
mcpServers.controller | /ai/mcp-servers | MCP tool-server CRUD + connection test + tool listing |
sources.controller | /ai/sources | Collection/field picker for collection-type KBs |
ingest.controller | /assistant/* | Legacy assistant-config CRUD + RAG ingest mapping/backfill |
skills.controller | /ai/skills, /ai/skill-categories | Agent Skills, see AI Skills |
agentic.controller | /ai/agentic/* | Agentic UI settings, prepare and the model gateway, see Agentic UI |
Two data-model families coexist:
ai_*collections (uuid PK, registered viaai-schema.tsas a setup-wizardModuleSchema) — the App Studio store. These are real collections (visible in the schema, RBAC-capable) but most are markedhiddenin their meta.odp_ai_*system tables (int PK, migrations066/067) — the legacy assistant-config store, inSYSTEM_TABLES(hidden from/items).
The module is named
ai(schema groupai), notkaos— KAOS is one knowledge-base kind, not the contract. The retrieval/LLM ports live inports.ts; KB kinds dispatch throughproviders/kb/.
AI App Studio
An AI app (ai_apps row) has a type that decides how it runs:
type | Runtime shape |
|---|---|
chatbot | Synthetic graph: start → [knowledge-retrieval] → llm → answer (retrieval node added only when the app config lists knowledge_ids). |
text-generation | Synthetic graph: start → llm → answer. |
chatflow | Author-defined graph loaded from ai_flow_nodes / ai_flow_edges. |
agent | Author-defined graph (same loader). |
workflow | Author-defined graph (same loader). |
runChat (runtime/chat.ts) dispatches on type: chatflow|agent|workflow load the persisted graph; chatbot|text-generation build a synthetic graph from app.config. Both then run the same executor (runtime/executor.ts).
Output cap (max_tokens)
app.config.max_output_tokens of a chatbot or text-generation app is the most output tokens one model call of its answer may produce (runtime/max-tokens.ts). It goes to the provider as max_tokens. The synthetic graph copies it into the llm node, and the node sends it on every call: complete, stream, each tool-loop round and the forced final answer, on an integration or on the ASSISTANT_LLM_* env provider. On the Agentic path the handoff carries it to the gateway (see Agentic UI).
- Valid: an integer from 1 to 128000.
nullor no key means unset: no field is sent and the provider default applies. Studio shows the field empty by default and removes the key when it is cleared. POSTandPATCH /ai/appsrefuse any other value with 400 (max_output_tokens must be an integer from 1 to 128000, or null).- A stored value that is not valid, written through
/items/ai_appsor before the check, counts as unset. - The older key
config.max_tokensis never read. Studio saved it (default 1024) before any model call used it, so its values carry no intent.POSTandPATCH /ai/appsdrop it without an error. - The engine knows no per-model limit. A value under 128000 but above the model's own limit is refused by the provider: a failed run on the engine path,
upstream_400on the gateway. - Flow apps (
chatflow,agent,workflow) ignore the app value; Studio does not show the field for them. Studio has no node-level field either; anllmnode givenmax_output_tokensthroughPUT /ai/apps/:id/graphis read by the same rule, with no write check. - The executor's own calls (query rewrite, classifiers, the skill picker, conversation summary) send none, except on a native Anthropic row, where every call without a cap sends 4096 (the Messages API requires one).
- The field name on the wire is
max_tokenson an OpenAI-compatible row. A provider that accepts onlymax_completion_tokens(OpenAI's reasoning models) refuses such a call there; on a native OpenAI row (provider: openaionapi.openai.com) the client sendsmax_completion_tokensfor those models. - Which client a row uses is in Integrations. The webapp BFF (
agent-model.ts) has its own native Anthropic and Google providers, used when Agentic has no app selected: no app, so no cap applies there.
Flow graph (nodes + edges)
Graphs are stored normalized (ai_flow_nodes + ai_flow_edges) and edited through GET/PUT /ai/apps/:id/graph, which converts to/from { nodes, edges }. A node has node_key, type, label, position, config; an edge has source_node, target_node, source_handle (branch label), condition, order. The executor walks the graph trigger-driven and branch-aware: a node runs once all its incoming edges' sources have resolved and at least one incoming edge is active (branch taken); nodes whose incoming branches are all dead are marked skipped.
An edge out of a branching node names its branch in sourceHandle: the class label of a question-classifier, the branch key (or fallback) of a smart-switch, true / false of an if-else. An edge with no sourceHandle is taken whatever the branch, so an if-else saved by the studio before 2026-11 (it wrote no handle) ran both branches. Such an app fails its next publish with handle_missing until its edges are reconnected from the true / false handles; a published app keeps running as before.
Write a graph only through the AI routes or tools. Use
POST /ai/apps,PATCH /ai/apps/:idandPUT /ai/apps/:id/graph, or theai_*MCP tools (odp_guide({topic: "ai"})). Do not writeai_apps,ai_flow_nodesorai_flow_edgeswith/items, an import or the MCPitems_*tools: a raw row skips the start node, the transaction and the graph check below. The MCPitems_*tools refuse these three collections;/itemsand imports do not yet.
Node catalog and graph check
runtime/node-catalog.ts describes each node type once: its data schema, defaults, branch handles, the ids it references, and an example. The executor reads node data through it (nodeData), so the defaults the check documents are the values a node runs with. A number field set to '' (the studio writes that when a field is cleared) counts as not set and takes the default; before 2026-11 it ran as 0. GET /ai/node-types serves the catalog.
services/graph-validator.ts checks a graph against the catalog and the rows that exist now (knowledge bases, integrations and their models, MCP servers, skills, skill categories, built-in tools, AI_ENABLE_CODE_NODE). Each problem is {code, severity, node?, edge?, field?, message, fix}.
| Code | Severity | Means |
|---|---|---|
graph_shape, node_id_invalid, node_type_invalid | error | The body cannot be stored: nodes / edges not arrays, an id that is blank or longer than 64, a node with no type. |
duplicate_node_id, edge_endpoint_missing, duplicate_edge | error | Two nodes share an id; an edge points to no node; the same edge twice. |
unknown_node_type | error | The type is not in the catalog (the old mcp node included). |
data_invalid | error | Node data breaks its schema (required field, enum, range, branch key, two branches with one name). |
reference_missing | error | A referenced id does not exist, or an llm node's integration cannot serve its model. |
node_disabled | error | A code node while AI_ENABLE_CODE_NODE is off. |
handle_missing, handle_invalid | error | An edge out of a branching node has no handle, or a handle the node does not have (a renamed branch). |
start_missing, start_duplicate, start_has_input, cycle, unreachable_node, no_output | error | The flow has no single start, loops, has a node start cannot reach, or reaches no answer and no terminal llm. |
branch_unwired | warning | A branch has no edge: the flow ends there when it is taken. |
Where each level applies:
| Path | Refused | Saved and returned |
|---|---|---|
PUT /ai/apps/:id/graph (studio) | graph_shape, node_id_invalid, node_type_invalid | everything else. An edge with a missing end and a duplicate edge are dropped and reported as edge_dropped warnings. |
MCP graph writes (ai_flow_add / _update / _remove) | a problem of the "wrong value" codes (graph_shape to node_disabled above, and removing start) that the call brings in | the structure codes: a flow is built one call at a time |
POST /ai/apps/:id/publish (flow apps) | any error | warnings |
A refusal is a 422 with code AI_GRAPH_INVALID (a graph write) or AI_APP_NOT_PUBLISHABLE (publish), body {errors:[{message, extensions:{code, problems, total, next}}]}. problems holds at most 50 of total; the message lists the first 20, one line each with its fix. next is the MCP call that goes on (odp_execute_read({name:"ai_flow_validate", ...})). An app published before this check stays published; the check runs on its next publish.
Node types (runtime/registry.ts, described in runtime/node-catalog.ts)
A handler is { type, execute(node, ctx) }; registering a node adds a type, the executor never changes.
| Node type | Role |
|---|---|
start | Emits { query }. |
question-classifier | LLM classifies the query into exactly one configured class (+ optional entity extraction); sets the branch handle. Loose-JSON parse + normalized label match + default_class fallback. |
knowledge-retrieval | Searches one or more KBs (knowledge_ids) via the KB registry. Text chunks → chunks lane + citations; structured rows → items lane (no citation). Supports top_k, score_threshold, and expand: 'full-doc' (refetch each doc's full text). |
llm | Builds context from upstream chunks/items + history, calls the LLM. Streams tokens when it is the terminal answer LLM; runs a guarded MCP tool loop when the app has MCP servers attached and the provider supports tool-calling. Can bind its own integration, see Integrations. |
template | Renders a {{path}} template string. |
variable-aggregator | Pass-through join point ({}). |
if-else | Evaluates a condition → `handle: 'true' |
http-request | SSRF-guarded outbound HTTP (safeFetch). |
code | Runs a JS snippet via Function() — disabled by default (AI_ENABLE_CODE_NODE=1; not a sandbox). |
tool | Invokes one built-in tool from the allow-list registry (runtime/tools.ts: http_get, math, current_datetime). |
mcp-agent | LLM acts as a data-fetch planner over an MCP server's tools; raw tool results flow into the items lane un-summarized. |
answer | Terminal node — emits the final answer text (or falls back to the most recent LLM output); maps config.meta refs into response meta. |
Knowledge bases (ai_knowledge_bases)
A KB is a pluggable grounding source dispatched by kind through KbProviderManager (providers/kb/manager.ts), modeled on the storage-driver pattern. Two kinds ship:
kaos(legacytype: 'rag') — wraps the KAOS RAG backend (providers/kb/kaos.provider.ts+kaos-client.ts). Capabilities:read + write + cites + ingestConfig.source_configcarriesbase_url/api_key/space(empty →KAOS_*env). ODP forwards only small chunk knobs (chunk_config.size/overlap); KAOS owns the RAG strategy.collection— live ODP-collection rows viaItemsService(collection.provider.ts). Capabilities:readonly,cites: false, no ingest. RBAC applies through the requester's accountability.
KAOS searches a shared index and receives no user identity, so providers/kb/readable-rows.ts re-checks each hit as the requester, on every search path (app chat, Agentic, /ai/knowledge/:id/query, the MCP ai_knowledge_search tool). A hit mirrored from an ODP row is kept only if the requester can read that row and all its stored fields; a partly readable row is dropped because the chunk can hold any mirrored field. A hit that cannot be traced to a live row is dropped too. A run with no accountability and an admin are not filtered. Fewer hits than top_k is normal after filtering.
Two limits apply. An uploaded document has no row and no per-user access, so everyone who can chat with the app receives its chunks. The filter also recognizes a mirrored hit by its address (buildFileKey in services/ingest-mapper.ts: odp__<collection>__<pk>). It was tested against a stubbed KAOS only: a real KAOS that returned another address shape would pass the hit as an uploaded document. Check one mirrored row on a real KAOS before you rely on this for a restricted collection.
Capability ∩ policy: each kind has intrinsic capabilities (the ceiling); each KB row carries allow_read / allow_write flags that can only narrow them (never widen). Enforced centrally in KbProviderManager (canRead/canWrite, search degrades silently, ingest/delete fail loudly). GET /ai/kb-kinds returns the catalog so admin pickers can filter.
Integrations (ai_integrations) — LLM/model providers only
An integration is one model provider (provider, config.base_url, config.api_key, config.models[], capabilities, enabled). The api_key is masked on read and preserved on a blank PATCH. Endpoints test the connection (GET {base_url}/models), discover the full model catalogue, and derive the allow-listed model list. An app binds a model to its provider via config.integration_id (disambiguates two rows of the same provider type).
The engine calls models through the Vercel AI SDK v7 (providers/llm/ai-sdk.ts). The package is picked from provider AND the host of base_url; both must name the vendor:
provider | Host of base_url | Called as |
|---|---|---|
openai | api.openai.com | OpenAI Chat Completions (@ai-sdk/openai, .chat()) |
anthropic | api.anthropic.com | Anthropic Messages API (@ai-sdk/anthropic) |
google, gemini | generativelanguage.googleapis.com | Gemini API (@ai-sdk/google); a trailing /openai of the base URL is dropped for this client |
anything else (openai-compatible, 9router, vllm, free text, or the 3 above on another host) | any | OpenAI-compatible POST {base_url}/chat/completions (@ai-sdk/openai-compatible) |
Rules:
- Every request goes through
safeFetch(SSRF guard). A loopback or internal model server must be inSSRF_ALLOWED_HOSTS. - A native row needs its own API key; the engine never reads
OPENAI_API_KEY,ANTHROPIC_API_KEYorGOOGLE_GENERATIVE_AI_API_KEYfrom the environment.ASSISTANT_LLM_API_KEYgoes only toASSISTANT_LLM_BASE_URL. - Anthropic requires an output cap: an app with no
max_output_tokenssends 4096 on an Anthropic row. Other providers get no cap field, as before. - No retries (
maxRetries: 0). A call with no signal of its own stops after 120 s. - The tool loop keeps the leniency of the old adapter for OpenAI-compatible servers: a tool call id repeated in a later round gets a suffix, tool calls with no
finish_reasonstill run, and a stream that ends with[DONE]and no finish reason ends normally. A finish reason the server did send (for examplelength) is kept, and the loop stops there. - The Agentic gateway (
/ai/agentic/v1/chat/completions) is not part of this layer: it stays an OpenAI-wire proxy to{base_url}/chat/completions, sobase_urlkeeps the vendor's OpenAI-compatible URL.
An llm node can bind its own provider with config.integration_id + config.model, so one flow can send each branch to a different provider. A node without integration_id inherits the app, exactly as before: resolveLLM and its fallbacks apply, and any model name is sent as written.
A node binding has no fallback. The node fails with a clear error and calls no provider when its integration is deleted, disabled, decision-only, or has no base URL or API key, or when the model is outside the integration's llm allow-list. A blank node model takes that integration's first allow-listed llm model, never app.config.model, which belongs to another provider. Other nodes keep running after a failure, so the run can end as partial with an earlier llm's text; on the Agentic path that is a 503. Studio's NodeInspector shows Inherit from app by default.
MCP servers (ai_mcp_servers) — tool servers the app consumes
An MCP server row (label, url, transport = streamable-http|sse, headers, allowed_tools, timeout_ms) is a remote tool server an app attaches (via config.mcp_server_ids) so its llm / mcp-agent nodes can function-call. headers values are secrets — AES-256-GCM encrypted at rest (via SECRET), masked on read, decrypted only for the outbound request. See Runtime for the client contract.
Data model
ai_* collections (App Studio — uuid PK, group ai)
Declared in collections/ai-schema.ts (ModuleSchema, version: 6). All carry created_at/updated_at audit; several also carry created_by/updated_by.
| Collection | Key fields |
|---|---|
ai_apps | name, description, type, icon, tags, status (draft|published), config (json), starred, is_template, category, version, published_at |
ai_flow_nodes | app_id, node_key, type, label, position (json), config (json) |
ai_flow_edges | app_id, source_node, target_node, source_handle, condition, label, order |
ai_app_logs | app_id, conversation_id, query, answer, status (running|success|partial|error), reason, error, trace (json), steps (json), mcp_calls (json), citations (json), retrieved_chunks, tokens, latency_ms, user_identifier, source (studio|agentic, null on rows written before schema v6), started_at, finished_at |
ai_conversations | app_id, title, source, external_ref, summary, message_count |
ai_messages | conversation_id, role (user|assistant|system|summary), content, meta (json) |
ai_annotations | app_id, question, answer, hit_count |
ai_knowledge_bases | name, description, type (rag|collection), source_config (json), embedding_model, chunk_config (json), retrieval_config (json), classification_default, scopes (json), allow_read, allow_write, doc_count, chunk_count |
ai_documents | knowledge_id, source_type, source_config (json), filename, content (raw text), content_hash (sha256 → dedup), status, chunk_count, word_count, classification, error_message |
ai_integrations | provider, label, config (json), capabilities (json), enabled, install_count, description |
ai_mcp_servers | label, url, transport, headers (json, secret), enabled, allowed_tools (json), timeout_ms, description |
Relations: nodes/edges/logs/conversations/annotations → ai_apps; edges' source_node/target_node → ai_flow_nodes; ai_messages → ai_conversations; ai_documents → ai_knowledge_bases.
The schema seed provides a minimal openai integration (required, OpenAI-compatible — point base_url at any gateway) plus demo KBs and app templates (sample, loaded from plans/ai-engine/app-templates/*.json by the seed script).
odp_ai_* system tables (legacy assistant-config — int PK)
Created by migrations 066-ai-assistant-config and 067-ai-ingest-maps; listed in SYSTEM_TABLES (hidden from /items). Secrets are stored as env-ref names (api_key_ref = the name of an env var, never the raw key).
| Table | Purpose | Notable fields |
|---|---|---|
odp_ai_llm_providers | OpenAI-compatible LLM endpoints | name, provider_type, base_url, api_key_ref, model, is_default |
odp_ai_knowledge_backends | RAG backends | name, backend_type, base_url, api_key_ref, config, is_default |
odp_ai_prompts | Domain context + system prompt | name, domain_context, system_prompt, refusal_message, variables |
odp_ai_agents | Binds prompt + provider + backend | name, llm_provider_id, knowledge_backend_id, prompt_id, temperature, enabled |
odp_ai_ingest_maps | Per-collection ingest field mapping | collection (unique), enabled, version_field, title_field, field_map, template |
Two more system tables serve Agentic UI. They are not legacy config, and their rows are written with raw Knex, so they produce no activity or revisions.
| Table | Migration | Purpose | Notable fields |
|---|---|---|---|
odp_ai_agentic_settings | 102-ai-agentic-settings | Singleton (id = 1), no seeded row | enabled, app_id, updated_at, updated_by |
odp_ai_agentic_turns | 103-ai-agentic-turns | One row per turn: the gateway's guard, shared by all instances | user_id, app_id, turn_id (unique together), query_hash, kind, model, content, closed, calls, expires_at |
Endpoints
Every route is guarded by app-module ai, except the Agentic UI runtime routes: reads use action view, writes use manage (controllers/shared.ts → gate(request, action) → validateAppAccess(acc, 'ai', action, …)). Data operations run ItemsService without accountability (already gated at the app-module layer), mirroring the core admin routes. CRUD for App-Studio collections is emitted by the generic registerCrud helper.
/ai/apps
| Method + path | Action | Notes |
|---|---|---|
CRUD /ai/apps, /ai/apps/:id | view/manage | name required. config.max_output_tokens is checked and the legacy config.max_tokens dropped on POST and PATCH (Output cap). DELETE removes the app with its nodes and edges in one transaction. |
POST /ai/apps/:id/publish | manage | Publish the app. A chatflow / agent / workflow app with a graph error is refused with 422 AI_APP_NOT_PUBLISHABLE (graph check). |
POST /ai/apps/:id/clone | manage | Duplicate the app with its graph as a new draft. |
GET /ai/apps/:id/monitoring | view | Aggregated run metrics. |
GET/PUT /ai/apps/:id/graph | view/manage | Read/write the normalized flow graph as { nodes, edges }. PUT returns {nodes, edges, problems, total} (counts stored, problems of the stored graph). |
POST /ai/apps/:id/validate | view | {valid, publishable, problems, total} for the stored graph, or for a {nodes, edges} body (nothing is saved). 404 AI_APP_NOT_FOUND for an unknown app. |
GET /ai/node-types | view | The node catalog: type, family, title, summary, data_schema, defaults, handles, references, example. |
GET /ai/apps/:id/logs | view | ai_app_logs scoped to the app. Takes the standard query; the Logs tab filters on source (studio also matches null). |
GET/POST /ai/apps/:id/annotations, DELETE /ai/annotations/:id | view/manage | Q/A annotations. |
POST /ai/apps/:id/chat | view | Runtime chat. One handler; stream flag (body.stream:true or ?stream=1) switches between a single JSON FlowResult and an SSE stream of ChatEvent frames. |
/ai/knowledge
| Method + path | Action | Notes |
|---|---|---|
CRUD /ai/knowledge, /ai/knowledge/:id | view/manage | source_config.api_key masked on read; blank PATCH keeps the secret. |
GET /ai/kb-kinds | view | Registered kinds + intrinsic capabilities. |
GET /ai/knowledge/stats | view | Per-KB document counts by status. |
GET /ai/knowledge/:id/documents | view | Paginated docs + filter_count + status stats. |
POST /ai/knowledge/:id/documents | manage | Upload a document (write-gated by capability ∩ policy); size-guarded; dedup by (kb, filename) + content hash; enqueues background ingest. |
DELETE /ai/documents/:id | manage | Purges the doc's units from KAOS (best-effort) then deletes the row. |
POST /ai/knowledge/:id/reindex | manage | Re-queues this KB's error documents. |
POST /ai/knowledge/:id/query | view | Retrieval test — runs a query through the KB provider registry. |
/ai/integrations and /ai/models
| Method + path | Action | Notes |
|---|---|---|
CRUD /ai/integrations, /ai/integrations/:id | view/manage | config.api_key masked; blank PATCH keeps it. provider required. |
POST /ai/integrations/:id/test | manage | Probe GET {base_url}/models; never throws (ok:false + reason). |
GET /ai/integrations/:id/available-models | manage | Full provider model catalogue for the allow-list picker. |
GET /ai/models?capability= | view | Models derived from enabled integrations (each carries integration_id). |
/ai/mcp-servers
| Method + path | Action | Notes |
|---|---|---|
CRUD /ai/mcp-servers, /ai/mcp-servers/:id | view/manage | headers encrypted on write, masked on read; blank PATCH keeps secrets. label required. |
POST /ai/mcp-servers/test | manage | Test-before-save with ad-hoc form values. |
POST /ai/mcp-servers/:id/test | manage | Test a saved server → { ok, tools[] }. |
GET /ai/mcp-servers/:id/tools | view | allowed_tools-filtered tool list for the attach picker. |
/ai/sources
| Method + path | Action | Notes |
|---|---|---|
GET /ai/sources/collections | view | Non-system collections for collection-KB config. |
GET /ai/sources/fields?collection= | view | Fields of a collection. |
/assistant/* (legacy assistant-config + RAG ingest)
All gated ai / manage. CRUD (GET list, POST, GET/PATCH/DELETE /:id) over the odp_ai_* tables:
| Resource | Path |
|---|---|
| LLM providers | /assistant/providers |
| Knowledge backends | /assistant/knowledge-backends |
| Prompts | /assistant/prompts |
| Agents | /assistant/agents |
| Ingest maps | /assistant/ingest-maps |
Plus the ingest tooling (controllers/ingest.controller.ts):
GET /assistant/ingest/collections— non-system collections for the mapping dropdown.GET /assistant/ingest/fields?collection=— a collection's fields with parsedspecial(relations visible).POST /assistant/ingest/backfill— bulk-load a collection into KAOS: paginate rows (read as the requestingai.manageuser, so RBAC applies) → map each via the ingest map → write through the KB registry (an ephemeral KAOS KB from the resolved agent-config). Returns{ ingested, skipped, units }.
The retired
/assistant/chat,/assistant/chat/streamand/assistant/healthendpoints (the gate → retrieve → synthesize →enforceGroundingorchestrator) no longer exist. Chat is served only byPOST /ai/apps/:id/chat.
Runtime
The executor (runtime/executor.ts) walks the graph, keeping a variable pool (vars[node_key] = output), a citation list and a meta bag.
- Provider resolution (
runtime/providers.ts):resolveLLMprefers the app's boundintegration_id(base_url/key), else the first enabled integration, elseASSISTANT_LLM_*env, viacreateOpenAICompatibleLLM.resolveDefaultModelresolvesapp.config.model→ integration's firstllmmodel → env.loadKbloads a KB row by uuid. Anllmnode with its ownintegration_idusesresolveNodeLLMinstead, which has no fallback. - API keys:
ASSISTANT_LLM_API_KEYgoes only toASSISTANT_LLM_BASE_URL. A custombase_urlsends the integration's own key, or noAuthorizationheader when it has none. - Query rewrite: when a conversation history exists, the executor rewrites a follow-up into a standalone question (
query_resolved) before classify/retrieval/mcp-agent read it. No history → no LLM call → byte-for-byte identical to the stateless path. - KB retrieval: dispatched through
KbProviderManager.search, which enforces capability ∩ policy (a read-blocked KB yields[]and the flow degrades rather than erroring). - MCP client (
runtime/mcp-client.ts): every connection is short-lived (connect → op → close), tool listings cached per(id, updated_at). The outbound URL is vetted through the core SSRF guard (assertUrlAllowed) before connecting;allowed_toolsfilters both listing and invocation; the guarded tool loop is bounded toMAX_TOOL_ITERATIONS = 5(then forces a final tool-less answer). - Result:
FlowResult { status: 'answered' | 'partial' | 'error', answer, reason, citations, meta, trace }. When a node fails and no answer exists,reasonnames the node and its error:Node error: final (<err>) — no answer produced.The SSE layer (runtime/chat-stream.ts) maps this to the wireAnswerStatus(answered→answered,partial→degraded, elsedeclined) and encodesChatEventframes (conversation→status→step* →answerdeltas →citations→done).
MCP tools (group ai)
The MCP server (docs/api/modules/mcp.md) serves these tools from modules/ai/mcp-tools.ts. Each runs the same gate as its REST route: ai.view to read and to test-run, ai.manage to write. Its scope names the tables it writes and the tables it only reads, so the MCP allowlist and read-only list bind them.
| Tool | Does |
|---|---|
ai_apps_list | List apps (no config, no graph). |
ai_apps_read | One app with its config, graph and current problems. Secret-looking values in node data and config are masked. |
ai_flow_node_types | The node catalog; with {type}, one type's data schema, defaults and example. |
ai_flow_validate | Problems of the stored graph, or of a proposed {nodes, edges} (nothing is saved). |
ai_models_list | Enabled integrations and their models, without URL or key. |
ai_mcp_servers_list | MCP servers with the tool names stored on the row, without URL or headers. |
ai_skills_list | Skills with their category, without the body. |
ai_knowledge_list, ai_knowledge_search | Knowledge bases, and a retrieval test on one. |
ai_apps_create | Create a draft app; a flow app gets its start node. |
ai_apps_update | Change name, description, icon or config. Config keys are merged; a key set to null is removed. |
ai_flow_add, ai_flow_update | Add, or change, nodes and edges. Refused when the call brings in a wrong value (graph check); an unfinished flow is saved. A code node is taken from an admin only. |
ai_flow_remove | Remove nodes (with their edges) and edges. Never start. Hidden when MCP deletes are off. |
ai_apps_publish | Publish after the strict check. |
ai_apps_clone | Copy an app with its graph as a new draft. An app with a code node is copied for an admin only. |
ai_apps_delete | Delete an app and its graph in one transaction, as REST DELETE does. Hidden when MCP deletes are off. |
ai_apps_test_run | One question through the app, answer and node trace. See below. |
The build loop an agent follows (the ai guide topic has the full version):
ai_apps_createachatflow(it starts with the nodestart).ai_flow_addthe nodes and edges, copied fromai_flow_examplesand the node topics.ai_flow_validate, and fix each problem withai_flow_add,ai_flow_updateorai_flow_remove.ai_apps_test_runwith one question for each branch.ai_apps_publish.
A write keeps a half-built flow (an unreachable node or an unwired branch is saved) and returns its problems; only a wrong value is refused. test/modules/ai/mcp-flow-build-e2e.test.ts runs this loop through the registry-mode root tools (odp_search, odp_execute_read, odp_execute_write, odp_guide), with the LLM and KAOS replaced by a stub on 127.0.0.1.
A refused call carries extensions {problems, total, next}. In registry mode the problems come back as error.details.problems, and an odp_* value of next becomes the gateway next.
ai_apps_test_run runs the stored graph (draft or published) once, with persist: false: no conversation, no ai_app_logs row, one audit log line (user, app, status, ms). It refuses a graph with an error before it calls a model, charges one point of ASSISTANT_RATE_LIMIT_PER_MIN (the bucket Agentic chat uses), and stops after 45 s with status: "timeout" and the trace so far: the walk stops before its next node, a node already running finishes in the background. The answer is cut at 4 000 characters, each node output at 1 500, the trace at 50 steps, and secret-looking values in node outputs are masked.
A tool that checks a graph (ai_apps_read, ai_flow_validate) reads only the reference tables the graph uses: a graph with a knowledge-retrieval node needs ai_knowledge_bases in the allowlist, one without it does not.
odp_guide serves the topics of modules/ai/mcp-guides.ts: ai (read first: app types, build loop, the rule not to write the AI tables with items_*), ai_flow_nodes_llm, ai_flow_nodes_routing, ai_flow_nodes_tools (generated from the node catalog), ai_flow_edges, ai_flow_refs and ai_flow_examples. The MCP items_* tools refuse writes to ai_apps, ai_flow_nodes and ai_flow_edges and point to these tools. Those three collections carry a note that says the same (migration 106 sets it on existing installs; an admin note is kept).
Programmatic trigger (other extensions / AI workflows)
Besides the HTTP route, another extension — or an AI workflow that needs to call an app as a sub-step — can trigger an AI app by ID in-process, without self-calling /ai/apps/:id/chat. The core AI module publishes a façade on the cross-extension service registry (see Extension Guardrails §6), under the ref id ai.engine. It runs the exact same flow as the HTTP route (conversation memory + ai_app_logs + streaming), so an app behaves identically whether a browser or an extension started it.
ts
// endpoint / hook that runs AFTER 'extensions.register' (never at a hook's top level — registry not sealed yet)
export default defineEndpoint((router, ctx) => {
router.post('/run-ai', async (req, reply) => {
// consume by string id — no need to import the ref (keeps you on @odp/api/types only)
const ai = ctx.registry.tryConsume('ai.engine');
if (!ai) return reply.status(503).send({ error: 'AI engine not available' });
const result = await ai.run('<APP_ID>', {
query: 'câu hỏi của user',
variables: { orderId: 123, locale: 'vi' }, // structured params → the app variable pool ({{app.variables.*}})
}, {
persist: false, // throwaway run — don't touch channel history / log timeline (default true)
source: 'my-ext', // channel tag on the conversation + logs (default 'extension')
externalRef: 'wf-inst-9', // idempotent conversation mapping (e.g. a workflow instance id)
accountability: req.accountability, // scope the flow's own data access (default null = system)
onStep: (ev) => {}, // live node/tool progress (optional)
onAnswerToken: (d) => {}, // live answer tokens (optional)
});
return reply.send({ data: result }); // FlowResult
});
});run(appId, input, options?) → Promise<FlowResult>— resolves (or creates) the conversation, runs the graph, resolves after the flow completes. Throws ifappIddoes not exist.input:{ query, conversation_id?, variables? }.variablesis the params channel forworkflow/agentapps — merged into the variable pool so nodes read{{ app.variables.* }}.options(all optional, defaults mirror the HTTP route):persist,source,externalRef,accountability,onAnswerToken,onStep.- Contract lives at
modules/ai/engine-ref.ts(aiEngineRef,AiEngine,AiEngineInput,AiEngineOptions); provided at boot insrc/index.tsbefore the registry is sealed. Consume by the typedaiEngineRefor the plain'ai.engine'id.
Agentic UI
Agentic UI is the in-app assistant. An admin can select an AI app to serve it, so provider, model, skills and knowledge are managed in Studio. The webapp BFF keeps its streamText loop, because tools run in the browser and each tool round is a new request. The engine therefore stops a flow at its last llm and hands that one model call back to the BFF, which makes it through a gateway on this API. The provider key never leaves the API.
One turn goes like this:
- The BFF reads the public
statusto learn whether the engine path applies. - On each round it calls
POST /ai/agentic/prepare. The first call runs the flow as the user and stops before the final llm. Later calls of the turn return the stored result. - If the flow answered by itself (
kind: 'answer'), the BFF streams that text and the turn is done. - Otherwise (
kind: 'llm') the BFF calls the gateway. The gateway checks the turn, adds the app prompt and knowledge, and passes the SSE through.
Settings
odp_ai_agentic_settings holds { enabled, app_id } and the chat history retention (history_retention_days, see Chat history). Provider, model and skills live in the app; with no drawn flow, use a chatbot app. The widget state follows this table. No row means nothing changes on upgrade.
| Setting | Result |
|---|---|
ODP_AGENT_ENABLED=false on the webapp | Off, whatever is saved (kill switch, applied by the BFF) |
No row, or /ai/agentic/status answers 404 | Env path (ODP_AGENT_*) |
enabled: false | Off |
enabled: true, no app_id | On, env path |
enabled: true with app_id | Engine path; every chat fails closed while the app is not ready |
A save applies on the next read. The BFF caches status for 15 seconds, and a page reload shows or hides the widget.
Endpoints
| Method + path | Gate | Notes |
|---|---|---|
GET /ai/agentic/status | None (public) | { state: 'unset' | 'on' | 'off', engine, ready } and nothing else. ready is false only when engine is true and the app is not ready. It is cached 5 seconds, so a graph edit can take that long to show; a save applies at once. |
GET /ai/agentic/config | ai.view | { state, enabled, app_id, app_name, ready, issues[], history_retention_days }, always fresh. |
GET /ai/agentic/apps | ai.view | Selectable apps, templates left out: { id, name, type, status, ready, issues[] }. A not-ready app stays listed. |
PATCH /ai/agentic/settings | ai.manage | Body { enabled?, app_id?, history_retention_days? } (days 1–3650, null = 90), other keys are 400. The first save must carry enabled, because the first row turns the assistant on or off. An unknown app is 400; a not-ready app is saved and reported in issues. |
POST /ai/agentic/prepare | User with app access | Body { turn_id, query, history?, purpose? }, 4 MB limit. purpose is chat (default) or summary, see Summary turns. |
POST /ai/agentic/v1/chat/completions | User with app access | OpenAI chat completions, 20 MB limit. |
Who may chat is app access, not the ai module, so the assistant reaches more users than Studio does.
turn_id must match ^[A-Za-z0-9_-]{16,128}$. The client sets it, and it stays the same through every tool round and approval of one turn. prepare answers one of:
json
{ "data": { "kind": "llm", "turn_id": "...", "model": "<integration_id>:<model_id>", "integration_id": "...", "skills_loaded": [], "ui": false, "citations": [] } } // "ui" only when the node sets it
{ "data": { "kind": "answer", "turn_id": "...", "text": "...", "citations": [] } }The app's system prompt and knowledge are not in this response, so a user token cannot read them.
Branch flag ui. A kind: "llm" response carries "ui": false (or true) when the final llm node of the branch sets config.agentic.ui (node data agentic.ui, boolean; set it with PUT /ai/apps/:id/graph or the MCP flow tools, Studio has no field for it yet). It tells the caller whether this branch needs a generative-UI section and ui.* tools in its own prompt: false means no. A node that does not set it, or sets anything but a boolean, adds no ui key, and the caller keeps its default (UI on). The flag is a hint to the caller, not a permission: it widens nothing the caller could read, and the gateway ignores it. The response already lists skills_loaded (names of the skills attached to that node); a caller can use those names to avoid adding a skill of its own that the engine already loaded.
The gateway takes a standard OpenAI body with stream: true, the model string prepare returned, a user token, and the header X-ODP-Turn-Id. Every other field passes through unchanged in both directions, including tool_calls[].extra_content, which the Gemini shim needs. The one exception is the output cap: when the app sets max_output_tokens, it is a ceiling. The caller may ask for less, never more; a value it sends that is not an integer from 1 to 128000 is replaced by the app's. max_tokens and max_completion_tokens are each capped if sent, and a body with neither gets max_tokens. With no app value the body is not touched. A provider error keeps its status but loses its body, which can name accounts or internal hosts.
The gateway merges one system message, because some compatible providers reject a second: the caller's text (BASE_SYSTEM_PROMPT leads it), then the app's system prompt, then knowledge in a <knowledge> tag marked as data. < is escaped inside the tag. A later system message or a developer message is 400.
| Status | prepare code | Gateway code | Meaning |
|---|---|---|---|
| 400 | Schema validation | invalid_payload | Malformed body. A refused gateway body does not use up a call. |
| 401 | INVALID_CREDENTIALS | invalid_credentials | No user. |
| 403 | FORBIDDEN, AGENTIC_TURN_INVALID | forbidden, agentic_turn_invalid, agentic_model_not_allowed | No app access; a forged, foreign, closed, expired or exhausted turn; a model the turn was not prepared for. |
| 404 | AGENTIC_NOT_ENGINE | agentic_not_engine | Off, or no app selected. |
| 409 | AGENTIC_NOT_READY, AGENTIC_TURN_CONFLICT | agentic_not_ready | App not ready; same turn_id with another query. |
| 429 | REQUESTS_EXCEEDED | requests_exceeded | Quota spent, with Retry-After. A provider 429 passes through as upstream_429. |
| 502 | upstream_unreachable | Provider unreachable. | |
| 503 | AGENTIC_PREPARE_FAILED, AGENTIC_TURN_UNREADABLE | agentic_turn_unreadable | Failed run; content unreadable, for example after a SECRET change. |
prepare errors use { errors: [{ message, extensions: { code } }] }, except a schema validation failure, which keeps Fastify's default shape. Gateway errors use the OpenAI shape { error: { message, type, code } }, so @ai-sdk/openai can read them.
A failed run is a 503, stored nowhere. Any node error before the handoff counts, so a flow that lost its retrieval does not answer on empty context, and a draft from an earlier llm is never served as the answer.
Turns
A row keeps query_hash and never the user's query or messages. content is the app system prompt and knowledge, encrypted with AES-256-GCM (utils/encryption.ts) under a key derived from SECRET for this purpose. It is decrypted only after the turn guard passes for the user the turn was issued to. Neither value goes to logs or activity. The turn's ai_app_logs row is separate (see Logs).
A turn lives 30 minutes from its last prepare or gateway call, and the gateway enforces that expiry. It allows 24 gateway calls, and it closes when the provider finishes with no tool calls; closing empties content. finish_reason alone does not decide this, because Gemini reports stop on a response that calls tools. The gateway holds the finishing frame until the provider stream ends and closes the turn before it sends that frame, so a client that holds the whole answer cannot replay the turn. A stream that breaks mid-way leaves the turn open. A retry from the client carries a new turn_id, so it reruns the flow.
A schedule (*/5 * * * *, ai-agentic-turn-prune) deletes expired rows with their content, so content outlives a turn by five minutes at most, even with no new chat. Several instances can run it safely.
Quota
ASSISTANT_RATE_LIMIT_PER_MIN (default 20) limits each user. Each gateway call costs one point, and so does a prepare that runs the flow; a prepare that returns the stored result costs nothing. A value below 1 means no limit. RATE_LIMITER_ENABLED does not affect it.
The count is shared across API instances only with RATE_LIMITER_STORE=redis and REDIS_ENABLED. Without Redis each instance counts alone, so the real limit is the value times the number of instances. Turn state is in the database and stays correct either way. One long turn can use 13 points, and a second one straight after can hit the limit.
Logs
Each Agentic turn writes one ai_app_logs row with source: 'agentic', so it shows in the Studio Logs tab next to the Studio rows (badge "Agentic", source filter). Studio chat rows carry source: 'studio'.
preparecreates the row after the input check, soqueryis the masked user text, as on the Studio chat. A refused input logs[guardrail: <rule>]as the query and no user text.- A turn that
prepareanswers (refusal, flow answer) is finished at once. A handoff turn staysrunningand keeps the trace, citations and chunk count of the flow. - The gateway round that closes the turn, or
turns/end, finishes the row:status: 'success', the shown answer (or the tool-limit note),latency_msfromprepare,finished_at, and the tool names of each round inmcp_calls(no arguments, no results). - A turn that never closes (client gone, stream broken, expired) stays
running. A resumedprepareof the sameturn_idadds no row. A failedprepare, and the run that loses a concurrentprepare, end aserror. - A guard finding is in
reasonasguardrail: <rule> <action> (<in|out>), on both paths. A gateway round that replaces or notes the answer sets it; otherwise the input verdict does. The guard audit line is unchanged (contract §8): it carries no text. tokensstays 0: the gateway does not read usage from the provider stream.- The history fold that
prepareruns (summary of older turns) writes no row of its own.
On an install below schema v6 the rows are still written without source, and the source filter of the Logs tab fails: run the upgrade in /ai/setup (see Upgrade notes).
Chat history
The API stores the Agentic chats the client chose to keep, in odp_ai_agentic_conversations and odp_ai_agentic_messages (migration 105). Every query is limited to the caller's own rows, admin included; prepare and the gateway never read them. Routes under /ai/agentic/conversations (list, create, read, update, delete; messages read, upsert, truncate) and POST /ai/agentic/history/purge for admins. Retention is history_retention_days in the settings (default 90), pruned every hour.
Row lifecycle of the engine's own writes (AI guardrails contract §16): prepare only reserves the id of a new conversation, and the conversation row is inserted with the first pair of messages (end of the turn, turns/end, a canned answer, or a stopped turn), so no turn leaves an empty conversation. An id with no row that nobody else holds is a new conversation; an id held by another user is treated the same way and nothing is ever written into that conversation. When the client closes the stream, the engine stores the text that reached it plus the finished tool rounds as an unchecked pair; nothing reached it, nothing is written; a provider failure writes nothing. prepare with regenerate_message_id equal to the last stored answer replaces the last pair at write time instead of appending.
Full guide, with examples, caps, errors and the rules for the client: AI Engine — Agentic chat history.
Summary turns
POST /ai/agentic/prepare with purpose: 'summary' prepares a turn that runs no flow, on the app's model. On that turn the gateway refuses tools, puts its fixed summarizer prompt in place of the app's, folds the messages into an escaped <conversation> data block, and caps the output at min(app max_output_tokens if set, 2048) tokens; a smaller caller value is kept. Streamed only, engine path only. Details: Agentic chat history §Summary turns.
Readiness
runtime/agentic-readiness.ts checks an app the way it would run. error makes the app not ready, and every chat then fails closed. warning is shown and the app stays ready. config, apps, status, prepare and the gateway all use this check. Graph rules apply to chatflow, agent and workflow; a chatbot has no stored graph, so only its config is checked.
| Code | Severity | Meaning | Fix |
|---|---|---|---|
app_missing | error | The selected app no longer exists. | Select another app. |
app_not_published | error | The app is a draft. There is no snapshot: once published, edits apply at once. | Publish it. |
app_has_mcp_servers | error | The app has mcp_server_ids, not supported in v1. | Remove them, or select another app. |
app_type_unsupported | error | text-generation (a one-shot form, not a conversation) or an unknown type. | Use a chatbot or a flow app. |
no_llm_node | error | The flow has no llm to hand off. | Add one. |
node_after_handoff | error | A node other than answer follows the final llm. It would never run. | Move the node before the llm. |
parallel_handoff | error | Two final llms can run in one turn. Two llms on exclusive handles of question-classifier, smart-switch or if-else are routing and pass. | Put them on exclusive branches. An llm with no incoming edge runs as a root and counts. |
llm_integration_missing | error | An llm has no usable integration: deleted, disabled, decision-only, no base URL, or no API key for a node binding. | Fix the integration or the binding. |
llm_model_missing | error | An llm has no model, or its node binding names one outside the allow-list. | Pick an allow-listed model. |
app_llm_fallback | error | Query rewrite, classifier and skill router run on the app-level provider whatever the llm nodes bind. They would reach a provider nobody picked: the app's integration_id points to a deleted row, which moves it to the first enabled integration, or the integration it resolves to has no base URL, or none exists, which means the env provider. Not repeated when llm_integration_missing is already reported. | Bind the app to an existing integration that has a base URL. |
mcp_agent_node | warning | An mcp-agent node runs its tools on the server, without Agentic approval. | Accept it, or remove the node. |
answer_template_ignored | warning | An answer after the final llm has text other than {{ llm.text }}. The BFF streams the llm itself, so that text is not applied. | Move the text into the prompt. |
app_endpoint_not_openai_wire | warning | The base URL of an integration the app reaches (app level or an llm node) looks like a vendor's native API: path ends in /messages or :generateContent/:streamGenerateContent, or it is generativelanguage.googleapis.com or /v1beta/models outside /openai. The gateway appends /chat/completions, so the call fails at run time. Unknown gateways are not flagged. | Use https://api.anthropic.com/v1/ for Claude, https://generativelanguage.googleapis.com/v1beta/openai for Gemini. |
app_integration_no_key | warning | The app's integration has no API key. Chat fails unless the provider needs none, such as a local one. | Save a key. |
Readiness is stricter than the chat path on purpose. A disabled or deleted bound integration is an error here, although Studio still runs on it through resolveLLM. The runtime is unchanged, so the same app can work in Studio and be not ready for Agentic.
Limits in v1
- Anyone who can chat with the app receives chunks from uploaded KB documents (see Knowledge bases).
- The KAOS filter depends on the address format and has not run against a real KAOS.
- Audit is one log line per model call, with user, turn, integration, model and status. Each turn also writes one
ai_app_logsrow (Logs) but noai_conversationsrows; history is in Chat history. - No model picker in the widget and no provider fallback.
Upgrade notes
- AI schema v6 adds
ai_app_logs.source. No migration: an admin opens/ai/setupand runs the upgrade ("Upgrade available: v5→v6"). It adds only what is missing, so an install that also lacks v5 items (for exampleai_conversations,ai_messages,ai_skill_categories,ai_skills) gets them in the same run. Existing rows are kept withsourcenull, which the Logs tab counts as Studio. Restarting the API does not add the column; until the upgrade runs, logs are written withoutsourceand the source filter fails. - Migrations
102-ai-agentic-settingsand103-ai-agentic-turnsadd the two system tables. No row is seeded and existing deployments keep working as before until an admin saves a setting. - Migration
105-ai-agentic-historyaddsodp_ai_agentic_conversations,odp_ai_agentic_messagesand thehistory_retention_dayscolumn. Nothing is stored until the BFF saves a chat. From then on, conversations idle for 90 days are deleted every hour; sethistory_retention_daysto keep them longer. ASSISTANT_LLM_API_KEYnow goes only toASSISTANT_LLM_BASE_URL. Before, an integration with a custombase_urland no key sent the env key to that host, so anyone withai.managecould collect it. Such an integration now sends no key and fails on a provider that needs one: save a key on it.- KAOS search is filtered per user on all paths, so a non-admin can get fewer chunks than before from a KB that mirrors a restricted collection, in Studio and through MCP too. Tool names and schemas did not change.
reasonof a failed run now carries the error (Node error: final (<err>) — no answer produced.).- The output cap of chatbot and text-generation apps now reaches the provider, from the new key
config.max_output_tokens. Values Studio saved before (config.max_tokens, 1024 by default) are not carried over: they are ignored and removed when the app is next saved through/ai/apps. Every existing app therefore keeps the provider default. Set Max tokens again in Studio for an app that needs a cap.
Guardrails custom rules
An admin can add global yes/no rules to the guard, next to the built-ins I1..I5 and O1..O5. Each rule is one System One question, asked in the decision call the guard already makes. Code: guardrails/custom-rules.ts (model, limits, wording hash, hit counters), services/custom-rules.service.ts (CRUD, order, quick test, gate), guardrails/guard.ts (run time), controllers/guardrails.controller.ts (routes). Migration 110.
Rule model
| Field | Value |
|---|---|
id | C1..C9999, given by a counter (odp_ai_guardrail_settings.custom_rule_seq). A deleted id is never given again, so an old audit line or ledger row never points to a new rule. Past C9999: 400 AI_GUARDRAIL_RULE_IDS_USED. |
name | 1..80 characters. Admin pages and Studio debug only: an end user never sees it. |
direction | in or out. Fixed after create. |
question, yes_means | 10..600 and 5..200 characters. The question may name state fields in backticks: message, history_tail (in); answer, question, evidence (out). |
block, warn | 0..1, warn < block. |
action | in: refuse, flag; out: replace, note, flag. |
texts | { vi, en }, required unless the action is flag. |
mode | shadow or enforce. A new rule is always shadow; create ignores mode. |
scope | { apps: 'all' } or { apps: [uuid] } (at most 500). An app cannot name, loosen or turn on a custom rule in config.guardrails. |
needs_evidence | out only: the question gets the retrieved evidence. |
examples | at most 30: { text, expect, lang, answer_context? } (answer_context.question / evidence on out). |
severity | 0..100, stored in reputation.severities[id], default 0. Deleting the rule removes it. |
position | order inside a direction. Set with POST /rules/order. |
Run time
- The guard adds every enabled rule whose scope holds the app to the same call. Above 16 keys (
GUARD_BATCH_KEYS) it splits the questions into parallel calls under the same bound; the built-in keys andintentstay in the first call. A failed call with only custom keys fails those rules open (unavailable), not the built-ins. shadow: the verdict does not change, nothing is shown, no point is taken. The finding hasaction: 'shadow',band: 'block' | 'warn'andshadow: true; the audit line hasaction: 'shadow'and no band. Alert 07 counts onlyrefuse|replace, so shadow hits never trigger it.enforce: the rule acts as a built-in. Refusal text order, input: self-harm, refusal (I1, I2), customrefuseinpositionorder, secret input, out of scope. Output: built-in replace, customreplacein order, ungrounded, mask, then notes (customnotetexts in order, each on its own paragraph).meta.guardrailof a non-debug turn never holds a shadow finding or a rule name. In debug (Studio test run)meta.guardrail.findingscarriesrule_namefor custom rules.- The Agentic gateway runs the same checks.
Hit counters
Each finding at the warn or block band of a custom rule (shadow or enforce) adds 1 to hits_warn / hits_block and sets last_hit_at. Counts are kept in memory and flushed every 30 s (HIT_FLUSH_MS), and on shutdown. A change of the wording or the thresholds resets them and sets counted_since. Shadow to enforce resets nothing (Q-C3).
Quick test and enforce gate
POST /rules/testtests a draft;POST /rules/:id/testtests a saved rule and stores the result inlast_test. One call per example, with only that rule's question, concurrency 4. Body{ full_direction: true }also asks every enabled question of the direction, to measure the real latency.- The result lists
p, band, pass andmsper example, and the summary (caught, false blocks, median and maxms). It writes no audit line, no ledger row and no log. - No decision integration: 400
AI_GUARDRAILS_NO_DECISION. - Gate: at least 5 positive and 5 negative examples, at least 90% of the positives at
p >= block, no negative atp >= block.last_testholds a hash ofdirection + question + yes_means + block + warn. mode: 'enforce'is refused with 400AI_GUARDRAIL_RULE_TEST_REQUIREDunlesslast_test.passedis true and its hash matches. A rule view hasenforce_allowed.- A PATCH that changes the question,
yes_meansor a threshold puts the rule back toshadowin the same write, and resets its counters.
Reputation
An enforce input rule with a severity above 0 deducts as a built-in (severity x intent factor x p) and asks the intent question. An enforce output rule deducts only with a severity above 0, with the input intent of the turn. A shadow rule never deducts. The ledger keeps the rule id after a delete; the admin page shows "deleted rule (C7)".
Routes
| Route | Gate |
|---|---|
GET /ai/guardrails/rules | ai view. Every rule, both directions, with counters and enforce_allowed. |
POST /ai/guardrails/rules | admin, not a bounded grant. Creates in shadow. |
PATCH /ai/guardrails/rules/:id | same. Partial body; mode passes the gate. |
DELETE /ai/guardrails/rules/:id | same. Removes the severity too. |
POST /ai/guardrails/rules/order | same. Body { direction, ids }: every id of the direction once. |
POST /ai/guardrails/rules/test, /rules/:id/test | same. |
GET /ai/guardrails/settings?app_id= | ai view. app.effective.custom lists the enabled rules that apply to the app (id, name, direction, mode, action). |
Unknown id: 404 AI_GUARDRAIL_RULE_NOT_FOUND. A body the checks refuse: 400 INVALID_PAYLOAD with every problem. odp_ai_guardrail_rules is a system table (SYSTEM_TABLES.AI_GUARDRAIL_RULES), written by these routes only.
Admin UI
Guardrails page, Settings tab: each direction shows its custom rules under the built-ins (mode badge, action, thresholds, severity, scope, counters). The switch and the up/down buttons save at once. The drawer edits the rule, its examples and its severity, runs the quick test, shows the cost estimate (keys and calls of the direction, a warning above 16 keys or when a full-direction test took more than 1000 ms) and holds the mode switch: it opens only when the last test passed on the current wording. Delete asks in a modal. A user without admin sees the rows read only. The webapp BFF forwards each route in apps/webapp/modules/ai/server/routes.ts; a new API path needs a line there. The Studio badge shows C1 · name from the debug findings. The Reputation users table shows the rule name, or "deleted rule (id)".
Guardrails reputation
Each counted user has a balance of points. A refused or flagged input with a severity costs points; at 0 the user is blocked for block_hours and every counted surface refuses them. Points come back each day. Code: guardrails/reputation-settings.ts (settings), services/reputation.service.ts (scoring, ledger, lifecycle), controllers/reputation.controller.ts (routes). Reputation runs only when the guard is enabled: it reads the guard's findings and adds an intent question to the same decision call.
Modes and settings
- The
reputationblock ofGET/PATCH /ai/guardrails/settings(columnodp_ai_guardrail_settings.reputation, migration 109). Null reads as the defaults withmode: 'shadow'. mode:offcounts nothing.shadowcounts and writes the ledger, but never refuses and never notifies; only admins see balances (would_blockmarks a deduction that would start a block).enforcerefuses a blocked user.enforceneeds a passing eval record (eval: { ran_at, integration_id, model, passed: true }, written byeval.ts --record), else 400AI_REPUTATION_EVAL_REQUIRED. The gate: accidental read as deliberate < 2%, deliberate recall >= 85%, simulated false block = 0. The switch toenforceresets every subject to its max.severities(0..100) are keyed by a string rule id, not a fixed union, so a custom guardrail rule carries its own severity (see custom rules). A PATCH may only name a rule that exists today (I1..I5,O1..O5and the custom ids). Defaults:I110,I220, every other rule 0.I2_self_harmis fixed at 0 (400AI_REPUTATION_SELF_HARM): a person at risk never loses access.intent_factors(0..1): deliberate 1, unclear 0.3, accidental 0. With probabilities the expected factor is used.- Numbers:
default_max100,regen_per_day10,warn_balance30 (clamped to max - 1),block_hours24,excerpt_days90.
Scoring
- One deduction per rule and turn:
severity x intent factor x p, rounded to 2 decimals; the highestpof a rule wins. - Input findings with action
refuse,maskorwarncount. Output findings count only when their rule has a severity (all 0 by default); they reuse the input intent of the turn. - Regen is lazy (computed on read, no scheduler). The first drop to
warn_balancesends a warn notice once; reaching 0 starts a block, emitsAI_USER_BLOCKED(security event) and notifies every user withaimanage.
Who is counted
| Caller | Counted |
|---|---|
| Public chat, engine call with no user | never |
| Logged-in user (own token or session), admins included | yes, subject = user |
Shared token (shared: true on the service user) with end_user | yes, subject = user + end_user |
Shared token without end_user | no (warn log, throttled) |
Studio test run of a user with ai manage | no (Q-R2) |
end_user is a body field of POST /ai/apps/:id/chat and POST /ai/agentic/prepare (option endUser of aiEngineRef.run): 1..128 characters, ^[A-Za-z0-9_.:@-]+$, else 400. It is ignored for a user that is not shared. The gateway reads the subject from the turn row (odp_ai_agentic_turns.end_user).
Where it runs
AiChatService.chat(route chat, MCPai_apps_test_run, engine ref): a blocked subject gets thereputation_blockedtext with statusanswered,meta.guardrail.reason: 'reputation', log reasonreputation: blocked. A counted turn inenforcecarriesmeta.reputation { notice, balance, max, blocked_until?, text }when it warns or blocks.- Agentic
prepare: same refusal, before the quota is charged; the response hasreputation. The gateway checks the block on every round, so a block mid-turn ends the turn. - Texts
reputation_blockedandreputation_warnare editable VI/EN with the other guard texts.
Routes
| Route | Gate |
|---|---|
GET /ai/reputation/me | a user who can run a counted turn. In enforce: { counted, mode, balance, max, blocked_until, warn_balance, can_self_reset }; otherwise { counted, mode }. ?end_user= for a shared caller. |
POST /ai/reputation/me/reset | ai manage, not a sub-token or impersonation; enforce only (else 400). Notifies the other ai managers. |
GET /ai/reputation/users | ai manage. Query: search, blocked, warned, role, app, limit, offset. Meta { total, mode }. |
GET /ai/reputation/users/:id/events | ai manage. Ledger, newest first, with masked excerpts. |
POST /ai/reputation/users/:id/reset, /unblock | ai manage, not a sub-token or impersonation |
PATCH /ai/reputation/users/:id | same; body { max?, shared? } (max 1..10000) |
GET /ai/reputation/roles | ai manage |
PUT /ai/reputation/roles/:role | admin only; sets the role max and pushes it to every user of the role |
Without the v7 tables the admin routes answer 409 AI_REPUTATION_NOT_INSTALLED and reputation reads as off.
Storage
- AI schema v7 (re-run
/ai/setup; boot does not apply it):ai_reputation_events(append-only ledger),ai_reputation_users(one row per subject),ai_reputation_roles(role max). /items/ai_reputation_*is read-only for everyone, admins included; writes go through the routes above only.- Retention: the ledger is kept 365 days, the masked excerpt (240 characters)
excerpt_days. A deleted user's rows are deleted. Purge runs lazily and in a daily synchronized job. - Migration 109 also adds
end_user,intent,p_intenttoodp_ai_agentic_turns, and seeds three in-app notify templates (ai_reputation_warn,ai_reputation_blocked,ai_reputation_self_reset) when a primary in-app provider exists. Without one the notifies are no-ops.
Admin UI
The Guardrails page has three tabs for a user with ai manage: Settings (with the Reputation section), Reputation users (table, ledger drawer, reset / unblock / edit max / mark shared, each confirmed in a modal) and Role max (admin saves). The Studio test chat shows a banner in enforce when the caller is warned or blocked; an ai manager can reset their own points from it.
Ingest (RAG) — two paths
Both paths write only through the KB registry (KbProviders), never a raw backend client, so capability ∩ policy is always enforced.
- Studio document upload (
ingest-queue.ts) —POST /ai/knowledge/:id/documentswrites anai_documentsrow (status: 'queued') and enqueues a BullMQ / Redis job (Redis is mandatory — no memory fallback). The single-concurrency worker reads the row, ingests to the KB (KAOS), and markscompleted/error. Transient failures (KAOS 5xx/429/network/timeout) retry with backoff; permanent 4xx fail fast. A row deleted during ingest has its orphan units purged. - Event-driven collection sync (
ingest-listener.ts) — whenASSISTANT_INGEST_COLLECTIONSlists a collection,items.create/items.update/items.deleteactions map the row (via itsodp_ai_ingest_mapsentry, or a generic scalar dump) and upsert/delete it into KAOS. The base row is read WITH the triggering user's accountability (RBAC applies to what is ingested), and a failure is logged and never rethrown — an ingest error must not break the underlying CRUD write.
An extension can intervene on the built doc through the ai-engine.ingest.remap filter (fires in both the backfill and listener paths): return the (possibly modified) IngestDoc to ingest it, or null/undefined to skip the row.
Invariants & gotchas
- Actor split. Control-plane
/ai/*CRUD runsItemsServicewithout accountability — access is already gated at the app-module layer (aiview/manage), mirroring core admin routes. Ingest (both paths) runs with accountability so collection/field RBAC decides what reaches the KB. - Raw Knex only off the actor path. Direct
knex(...)use is limited to runtime/config reads (graph load, integration/KB/MCP resolution, aggregate stats with fixed status literals) and the legacyodp_ai_*config services — never a per-user data mutation. - Secret masking everywhere. Integration
config.api_key, KBsource_config.api_key, and MCPheadersare masked on read and preserved on a blank PATCH; MCP headers are additionally AES-256-GCM encrypted at rest and only decrypted for the outbound request. Legacyodp_ai_*rows store secrets as env-ref names (api_key_ref), never raw. codenode disabled by default — requiresAI_ENABLE_CODE_NODE=1in a trusted deployment (Function()eval, not a sandbox).AI_MOCK_RETRIEVALis dev-only — synthetic chunks let a RAG flow complete without real KAOS data; hard-disabled in production (a fabricated "grounded" answer must never ship). Mocked responses carrymeta.mock = trueandmock://citations.- Redis is mandatory for ingest — with
REDIS_ENABLEDoff, documents queue but do not ingest. - KAOS address format is part of the access check. A change to
buildFileKeymust changeproviders/kb/readable-rows.tswith it, or mirrored rows stop being filtered.
Conversation memory & logging status
These recently-landed pieces are present and wired at HEAD, but are the most actively evolving part of the module — treat them as maturing:
- Streaming chat (
runtime/chat-stream.ts+ the SSE branch ofPOST /ai/apps/:id/chat). Real per-token streaming happens for the terminal answer LLM node (viaonAnswerToken); other cases buffer-then-chunk the completed answer. ANOTEinchat-stream.tsstill frames full per-token streaming as a forward-compatible target. - Multi-turn conversation persistence (
services/conversation.service.ts) — channel-agnostic memory overai_conversations+ai_messages: reuse-or-create a conversation, load a token-budgeted history (AI_HISTORY_TOKEN_BUDGET), append each turn, and roll older turns into a summary once history crosses ~50% of the budget (folded rows are kept, markedmeta.folded). All I/O goes throughItemsService, never raw Knex. - Lifecycle logging (
services/log.service.ts) —ai_app_logsuses a create-at-running→ throttled step-timeline updates → finalize-in-finallylifecycle, so even a mid-flight crash leaves a durable trace instead of no log.
Environment
| Var | Default | Purpose |
|---|---|---|
ASSISTANT_LLM_PROVIDER | openai-compatible | Provider of the env path: openai-compatible, openai, anthropic or google (native only on the vendor host); any other value fails the call. |
ASSISTANT_LLM_BASE_URL | http://localhost:20128/v1 | Base URL of the env path. |
ASSISTANT_LLM_MODEL | gpt-4o-mini | Fallback model id. |
ASSISTANT_LLM_API_KEY | — | Fallback key, sent only to ASSISTANT_LLM_BASE_URL (prefer per-integration config.api_key). |
ASSISTANT_INGEST_COLLECTIONS | `` (off) | Comma list of collections auto-synced to KAOS on item change. |
ASSISTANT_RATE_LIMIT_PER_MIN | 20 | Per-user Agentic quota per minute; below 1 means no limit. |
KAOS_BASE_URL | http://localhost:7071 | KAOS RAG API base URL. |
KAOS_API_KEY | — | KAOS bearer token. |
KAOS_IDENTITY_ID / KAOS_IDENTITY_KIND / KAOS_IDENTITY_SCOPES | odp / service / knowledge:jobs:write,knowledge:read | KAOS identity used for ingest/read. |
AI_INGEST_MAX_CONTENT_BYTES | 15 MB | Per-document size guard (below KAOS's body limit). |
AI_HISTORY_TOKEN_BUDGET | 3000 | Conversation-memory history budget (tokens). |
AI_MOCK_RETRIEVAL | false | Dev-only synthetic retrieval; hard-disabled in production. |
AI_ENABLE_CODE_NODE | false | Enable the code flow node (Function() eval — not a sandbox). |
REDIS_ENABLED | — | Required for the background ingest queue. |