Skip to content

AI Engine ​

TLDR — The AI module (service/api/src/modules/ai/) hosts two subsystems under one namespace: AI App Studio (current, mounted at /ai/*) — build and run AI apps as flow graphs over knowledge bases, LLM integrations and MCP tool servers; and a legacy assistant-config + RAG ingest surface (/assistant/*) that survives for per-collection ingest mapping and backfill into KAOS. Everything is gated by the app-module ai (view / manage). The old grounded /assistant/chat orchestrator (gate → retrieve → synthesize → enforceGrounding) has been retired — chat now runs through the flow executor.


Overview ​

The module registers under app-module ai and exposes a single Fastify router (routes.ts) that aggregates eight controllers:

ControllerPrefixPurpose
apps.controller/ai/appsApp CRUD + publish/clone/monitoring, flow graph, logs, annotations, runtime chat
knowledge.controller/ai/knowledgeKnowledge-base CRUD + documents + retrieval-test query
integrations.controller/ai/integrations, /ai/modelsLLM/model-provider CRUD + connection test + model discovery
mcpServers.controller/ai/mcp-serversMCP tool-server CRUD + connection test + tool listing
sources.controller/ai/sourcesCollection/field picker for collection-type KBs
ingest.controller/assistant/*Legacy assistant-config CRUD + RAG ingest mapping/backfill
skills.controller/ai/skills, /ai/skill-categoriesAgent Skills, see AI Skills
agentic.controller/ai/agentic/*Agentic UI settings, prepare and the model gateway, see Agentic UI

Two data-model families coexist:

  • ai_* collections (uuid PK, registered via ai-schema.ts as a setup-wizard ModuleSchema) — the App Studio store. These are real collections (visible in the schema, RBAC-capable) but most are marked hidden in their meta.
  • odp_ai_* system tables (int PK, migrations 066/067) — the legacy assistant-config store, in SYSTEM_TABLES (hidden from /items).

The module is named ai (schema group ai), not kaos — KAOS is one knowledge-base kind, not the contract. The retrieval/LLM ports live in ports.ts; KB kinds dispatch through providers/kb/.


AI App Studio ​

An AI app (ai_apps row) has a type that decides how it runs:

typeRuntime shape
chatbotSynthetic graph: start → [knowledge-retrieval] → llm → answer (retrieval node added only when the app config lists knowledge_ids).
text-generationSynthetic graph: start → llm → answer.
chatflowAuthor-defined graph loaded from ai_flow_nodes / ai_flow_edges.
agentAuthor-defined graph (same loader).
workflowAuthor-defined graph (same loader).

runChat (runtime/chat.ts) dispatches on type: chatflow|agent|workflow load the persisted graph; chatbot|text-generation build a synthetic graph from app.config. Both then run the same executor (runtime/executor.ts).

Output cap (max_tokens) ​

app.config.max_output_tokens of a chatbot or text-generation app is the most output tokens one model call of its answer may produce (runtime/max-tokens.ts). It goes to the provider as max_tokens. The synthetic graph copies it into the llm node, and the node sends it on every call: complete, stream, each tool-loop round and the forced final answer, on an integration or on the ASSISTANT_LLM_* env provider. On the Agentic path the handoff carries it to the gateway (see Agentic UI).

  • Valid: an integer from 1 to 128000. null or no key means unset: no field is sent and the provider default applies. Studio shows the field empty by default and removes the key when it is cleared.
  • POST and PATCH /ai/apps refuse any other value with 400 (max_output_tokens must be an integer from 1 to 128000, or null).
  • A stored value that is not valid, written through /items/ai_apps or before the check, counts as unset.
  • The older key config.max_tokens is never read. Studio saved it (default 1024) before any model call used it, so its values carry no intent. POST and PATCH /ai/apps drop it without an error.
  • The engine knows no per-model limit. A value under 128000 but above the model's own limit is refused by the provider: a failed run on the engine path, upstream_400 on the gateway.
  • Flow apps (chatflow, agent, workflow) ignore the app value; Studio does not show the field for them. Studio has no node-level field either; an llm node given max_output_tokens through PUT /ai/apps/:id/graph is read by the same rule, with no write check.
  • The executor's own calls (query rewrite, classifiers, the skill picker, conversation summary) send none, except on a native Anthropic row, where every call without a cap sends 4096 (the Messages API requires one).
  • The field name on the wire is max_tokens on an OpenAI-compatible row. A provider that accepts only max_completion_tokens (OpenAI's reasoning models) refuses such a call there; on a native OpenAI row (provider: openai on api.openai.com) the client sends max_completion_tokens for those models.
  • Which client a row uses is in Integrations. The webapp BFF (agent-model.ts) has its own native Anthropic and Google providers, used when Agentic has no app selected: no app, so no cap applies there.

Flow graph (nodes + edges) ​

Graphs are stored normalized (ai_flow_nodes + ai_flow_edges) and edited through GET/PUT /ai/apps/:id/graph, which converts to/from { nodes, edges }. A node has node_key, type, label, position, config; an edge has source_node, target_node, source_handle (branch label), condition, order. The executor walks the graph trigger-driven and branch-aware: a node runs once all its incoming edges' sources have resolved and at least one incoming edge is active (branch taken); nodes whose incoming branches are all dead are marked skipped.

An edge out of a branching node names its branch in sourceHandle: the class label of a question-classifier, the branch key (or fallback) of a smart-switch, true / false of an if-else. An edge with no sourceHandle is taken whatever the branch, so an if-else saved by the studio before 2026-11 (it wrote no handle) ran both branches. Such an app fails its next publish with handle_missing until its edges are reconnected from the true / false handles; a published app keeps running as before.

Write a graph only through the AI routes or tools. Use POST /ai/apps, PATCH /ai/apps/:id and PUT /ai/apps/:id/graph, or the ai_* MCP tools (odp_guide({topic: "ai"})). Do not write ai_apps, ai_flow_nodes or ai_flow_edges with /items, an import or the MCP items_* tools: a raw row skips the start node, the transaction and the graph check below. The MCP items_* tools refuse these three collections; /items and imports do not yet.

Node catalog and graph check ​

runtime/node-catalog.ts describes each node type once: its data schema, defaults, branch handles, the ids it references, and an example. The executor reads node data through it (nodeData), so the defaults the check documents are the values a node runs with. A number field set to '' (the studio writes that when a field is cleared) counts as not set and takes the default; before 2026-11 it ran as 0. GET /ai/node-types serves the catalog.

services/graph-validator.ts checks a graph against the catalog and the rows that exist now (knowledge bases, integrations and their models, MCP servers, skills, skill categories, built-in tools, AI_ENABLE_CODE_NODE). Each problem is {code, severity, node?, edge?, field?, message, fix}.

CodeSeverityMeans
graph_shape, node_id_invalid, node_type_invaliderrorThe body cannot be stored: nodes / edges not arrays, an id that is blank or longer than 64, a node with no type.
duplicate_node_id, edge_endpoint_missing, duplicate_edgeerrorTwo nodes share an id; an edge points to no node; the same edge twice.
unknown_node_typeerrorThe type is not in the catalog (the old mcp node included).
data_invaliderrorNode data breaks its schema (required field, enum, range, branch key, two branches with one name).
reference_missingerrorA referenced id does not exist, or an llm node's integration cannot serve its model.
node_disablederrorA code node while AI_ENABLE_CODE_NODE is off.
handle_missing, handle_invaliderrorAn edge out of a branching node has no handle, or a handle the node does not have (a renamed branch).
start_missing, start_duplicate, start_has_input, cycle, unreachable_node, no_outputerrorThe flow has no single start, loops, has a node start cannot reach, or reaches no answer and no terminal llm.
branch_unwiredwarningA branch has no edge: the flow ends there when it is taken.

Where each level applies:

PathRefusedSaved and returned
PUT /ai/apps/:id/graph (studio)graph_shape, node_id_invalid, node_type_invalideverything else. An edge with a missing end and a duplicate edge are dropped and reported as edge_dropped warnings.
MCP graph writes (ai_flow_add / _update / _remove)a problem of the "wrong value" codes (graph_shape to node_disabled above, and removing start) that the call brings inthe structure codes: a flow is built one call at a time
POST /ai/apps/:id/publish (flow apps)any errorwarnings

A refusal is a 422 with code AI_GRAPH_INVALID (a graph write) or AI_APP_NOT_PUBLISHABLE (publish), body {errors:[{message, extensions:{code, problems, total, next}}]}. problems holds at most 50 of total; the message lists the first 20, one line each with its fix. next is the MCP call that goes on (odp_execute_read({name:"ai_flow_validate", ...})). An app published before this check stays published; the check runs on its next publish.

Node types (runtime/registry.ts, described in runtime/node-catalog.ts) ​

A handler is { type, execute(node, ctx) }; registering a node adds a type, the executor never changes.

Node typeRole
startEmits { query }.
question-classifierLLM classifies the query into exactly one configured class (+ optional entity extraction); sets the branch handle. Loose-JSON parse + normalized label match + default_class fallback.
knowledge-retrievalSearches one or more KBs (knowledge_ids) via the KB registry. Text chunks → chunks lane + citations; structured rows → items lane (no citation). Supports top_k, score_threshold, and expand: 'full-doc' (refetch each doc's full text).
llmBuilds context from upstream chunks/items + history, calls the LLM. Streams tokens when it is the terminal answer LLM; runs a guarded MCP tool loop when the app has MCP servers attached and the provider supports tool-calling. Can bind its own integration, see Integrations.
templateRenders a {{path}} template string.
variable-aggregatorPass-through join point ({}).
if-elseEvaluates a condition → `handle: 'true'
http-requestSSRF-guarded outbound HTTP (safeFetch).
codeRuns a JS snippet via Function() — disabled by default (AI_ENABLE_CODE_NODE=1; not a sandbox).
toolInvokes one built-in tool from the allow-list registry (runtime/tools.ts: http_get, math, current_datetime).
mcp-agentLLM acts as a data-fetch planner over an MCP server's tools; raw tool results flow into the items lane un-summarized.
answerTerminal node — emits the final answer text (or falls back to the most recent LLM output); maps config.meta refs into response meta.

Knowledge bases (ai_knowledge_bases) ​

A KB is a pluggable grounding source dispatched by kind through KbProviderManager (providers/kb/manager.ts), modeled on the storage-driver pattern. Two kinds ship:

  • kaos (legacy type: 'rag') — wraps the KAOS RAG backend (providers/kb/kaos.provider.ts + kaos-client.ts). Capabilities: read + write + cites + ingestConfig. source_config carries base_url / api_key / space (empty → KAOS_* env). ODP forwards only small chunk knobs (chunk_config.size/overlap); KAOS owns the RAG strategy.
  • collection — live ODP-collection rows via ItemsService (collection.provider.ts). Capabilities: read only, cites: false, no ingest. RBAC applies through the requester's accountability.

KAOS searches a shared index and receives no user identity, so providers/kb/readable-rows.ts re-checks each hit as the requester, on every search path (app chat, Agentic, /ai/knowledge/:id/query, the MCP ai_knowledge_search tool). A hit mirrored from an ODP row is kept only if the requester can read that row and all its stored fields; a partly readable row is dropped because the chunk can hold any mirrored field. A hit that cannot be traced to a live row is dropped too. A run with no accountability and an admin are not filtered. Fewer hits than top_k is normal after filtering.

Two limits apply. An uploaded document has no row and no per-user access, so everyone who can chat with the app receives its chunks. The filter also recognizes a mirrored hit by its address (buildFileKey in services/ingest-mapper.ts: odp__<collection>__<pk>). It was tested against a stubbed KAOS only: a real KAOS that returned another address shape would pass the hit as an uploaded document. Check one mirrored row on a real KAOS before you rely on this for a restricted collection.

Capability ∩ policy: each kind has intrinsic capabilities (the ceiling); each KB row carries allow_read / allow_write flags that can only narrow them (never widen). Enforced centrally in KbProviderManager (canRead/canWrite, search degrades silently, ingest/delete fail loudly). GET /ai/kb-kinds returns the catalog so admin pickers can filter.

Integrations (ai_integrations) — LLM/model providers only ​

An integration is one model provider (provider, config.base_url, config.api_key, config.models[], capabilities, enabled). The api_key is masked on read and preserved on a blank PATCH. Endpoints test the connection (GET {base_url}/models), discover the full model catalogue, and derive the allow-listed model list. An app binds a model to its provider via config.integration_id (disambiguates two rows of the same provider type).

The engine calls models through the Vercel AI SDK v7 (providers/llm/ai-sdk.ts). The package is picked from provider AND the host of base_url; both must name the vendor:

providerHost of base_urlCalled as
openaiapi.openai.comOpenAI Chat Completions (@ai-sdk/openai, .chat())
anthropicapi.anthropic.comAnthropic Messages API (@ai-sdk/anthropic)
google, geminigenerativelanguage.googleapis.comGemini API (@ai-sdk/google); a trailing /openai of the base URL is dropped for this client
anything else (openai-compatible, 9router, vllm, free text, or the 3 above on another host)anyOpenAI-compatible POST {base_url}/chat/completions (@ai-sdk/openai-compatible)

Rules:

  • Every request goes through safeFetch (SSRF guard). A loopback or internal model server must be in SSRF_ALLOWED_HOSTS.
  • A native row needs its own API key; the engine never reads OPENAI_API_KEY, ANTHROPIC_API_KEY or GOOGLE_GENERATIVE_AI_API_KEY from the environment. ASSISTANT_LLM_API_KEY goes only to ASSISTANT_LLM_BASE_URL.
  • Anthropic requires an output cap: an app with no max_output_tokens sends 4096 on an Anthropic row. Other providers get no cap field, as before.
  • No retries (maxRetries: 0). A call with no signal of its own stops after 120 s.
  • The tool loop keeps the leniency of the old adapter for OpenAI-compatible servers: a tool call id repeated in a later round gets a suffix, tool calls with no finish_reason still run, and a stream that ends with [DONE] and no finish reason ends normally. A finish reason the server did send (for example length) is kept, and the loop stops there.
  • The Agentic gateway (/ai/agentic/v1/chat/completions) is not part of this layer: it stays an OpenAI-wire proxy to {base_url}/chat/completions, so base_url keeps the vendor's OpenAI-compatible URL.

An llm node can bind its own provider with config.integration_id + config.model, so one flow can send each branch to a different provider. A node without integration_id inherits the app, exactly as before: resolveLLM and its fallbacks apply, and any model name is sent as written.

A node binding has no fallback. The node fails with a clear error and calls no provider when its integration is deleted, disabled, decision-only, or has no base URL or API key, or when the model is outside the integration's llm allow-list. A blank node model takes that integration's first allow-listed llm model, never app.config.model, which belongs to another provider. Other nodes keep running after a failure, so the run can end as partial with an earlier llm's text; on the Agentic path that is a 503. Studio's NodeInspector shows Inherit from app by default.

MCP servers (ai_mcp_servers) — tool servers the app consumes ​

An MCP server row (label, url, transport = streamable-http|sse, headers, allowed_tools, timeout_ms) is a remote tool server an app attaches (via config.mcp_server_ids) so its llm / mcp-agent nodes can function-call. headers values are secrets — AES-256-GCM encrypted at rest (via SECRET), masked on read, decrypted only for the outbound request. See Runtime for the client contract.


Data model ​

ai_* collections (App Studio — uuid PK, group ai) ​

Declared in collections/ai-schema.ts (ModuleSchema, version: 6). All carry created_at/updated_at audit; several also carry created_by/updated_by.

CollectionKey fields
ai_appsname, description, type, icon, tags, status (draft|published), config (json), starred, is_template, category, version, published_at
ai_flow_nodesapp_id, node_key, type, label, position (json), config (json)
ai_flow_edgesapp_id, source_node, target_node, source_handle, condition, label, order
ai_app_logsapp_id, conversation_id, query, answer, status (running|success|partial|error), reason, error, trace (json), steps (json), mcp_calls (json), citations (json), retrieved_chunks, tokens, latency_ms, user_identifier, source (studio|agentic, null on rows written before schema v6), started_at, finished_at
ai_conversationsapp_id, title, source, external_ref, summary, message_count
ai_messagesconversation_id, role (user|assistant|system|summary), content, meta (json)
ai_annotationsapp_id, question, answer, hit_count
ai_knowledge_basesname, description, type (rag|collection), source_config (json), embedding_model, chunk_config (json), retrieval_config (json), classification_default, scopes (json), allow_read, allow_write, doc_count, chunk_count
ai_documentsknowledge_id, source_type, source_config (json), filename, content (raw text), content_hash (sha256 → dedup), status, chunk_count, word_count, classification, error_message
ai_integrationsprovider, label, config (json), capabilities (json), enabled, install_count, description
ai_mcp_serverslabel, url, transport, headers (json, secret), enabled, allowed_tools (json), timeout_ms, description

Relations: nodes/edges/logs/conversations/annotations → ai_apps; edges' source_node/target_node → ai_flow_nodes; ai_messages → ai_conversations; ai_documents → ai_knowledge_bases.

The schema seed provides a minimal openai integration (required, OpenAI-compatible — point base_url at any gateway) plus demo KBs and app templates (sample, loaded from plans/ai-engine/app-templates/*.json by the seed script).

odp_ai_* system tables (legacy assistant-config — int PK) ​

Created by migrations 066-ai-assistant-config and 067-ai-ingest-maps; listed in SYSTEM_TABLES (hidden from /items). Secrets are stored as env-ref names (api_key_ref = the name of an env var, never the raw key).

TablePurposeNotable fields
odp_ai_llm_providersOpenAI-compatible LLM endpointsname, provider_type, base_url, api_key_ref, model, is_default
odp_ai_knowledge_backendsRAG backendsname, backend_type, base_url, api_key_ref, config, is_default
odp_ai_promptsDomain context + system promptname, domain_context, system_prompt, refusal_message, variables
odp_ai_agentsBinds prompt + provider + backendname, llm_provider_id, knowledge_backend_id, prompt_id, temperature, enabled
odp_ai_ingest_mapsPer-collection ingest field mappingcollection (unique), enabled, version_field, title_field, field_map, template

Two more system tables serve Agentic UI. They are not legacy config, and their rows are written with raw Knex, so they produce no activity or revisions.

TableMigrationPurposeNotable fields
odp_ai_agentic_settings102-ai-agentic-settingsSingleton (id = 1), no seeded rowenabled, app_id, updated_at, updated_by
odp_ai_agentic_turns103-ai-agentic-turnsOne row per turn: the gateway's guard, shared by all instancesuser_id, app_id, turn_id (unique together), query_hash, kind, model, content, closed, calls, expires_at

Endpoints ​

Every route is guarded by app-module ai, except the Agentic UI runtime routes: reads use action view, writes use manage (controllers/shared.ts → gate(request, action) → validateAppAccess(acc, 'ai', action, …)). Data operations run ItemsService without accountability (already gated at the app-module layer), mirroring the core admin routes. CRUD for App-Studio collections is emitted by the generic registerCrud helper.

/ai/apps ​

Method + pathActionNotes
CRUD /ai/apps, /ai/apps/:idview/managename required. config.max_output_tokens is checked and the legacy config.max_tokens dropped on POST and PATCH (Output cap). DELETE removes the app with its nodes and edges in one transaction.
POST /ai/apps/:id/publishmanagePublish the app. A chatflow / agent / workflow app with a graph error is refused with 422 AI_APP_NOT_PUBLISHABLE (graph check).
POST /ai/apps/:id/clonemanageDuplicate the app with its graph as a new draft.
GET /ai/apps/:id/monitoringviewAggregated run metrics.
GET/PUT /ai/apps/:id/graphview/manageRead/write the normalized flow graph as { nodes, edges }. PUT returns {nodes, edges, problems, total} (counts stored, problems of the stored graph).
POST /ai/apps/:id/validateview{valid, publishable, problems, total} for the stored graph, or for a {nodes, edges} body (nothing is saved). 404 AI_APP_NOT_FOUND for an unknown app.
GET /ai/node-typesviewThe node catalog: type, family, title, summary, data_schema, defaults, handles, references, example.
GET /ai/apps/:id/logsviewai_app_logs scoped to the app. Takes the standard query; the Logs tab filters on source (studio also matches null).
GET/POST /ai/apps/:id/annotations, DELETE /ai/annotations/:idview/manageQ/A annotations.
POST /ai/apps/:id/chatviewRuntime chat. One handler; stream flag (body.stream:true or ?stream=1) switches between a single JSON FlowResult and an SSE stream of ChatEvent frames.

/ai/knowledge ​

Method + pathActionNotes
CRUD /ai/knowledge, /ai/knowledge/:idview/managesource_config.api_key masked on read; blank PATCH keeps the secret.
GET /ai/kb-kindsviewRegistered kinds + intrinsic capabilities.
GET /ai/knowledge/statsviewPer-KB document counts by status.
GET /ai/knowledge/:id/documentsviewPaginated docs + filter_count + status stats.
POST /ai/knowledge/:id/documentsmanageUpload a document (write-gated by capability ∩ policy); size-guarded; dedup by (kb, filename) + content hash; enqueues background ingest.
DELETE /ai/documents/:idmanagePurges the doc's units from KAOS (best-effort) then deletes the row.
POST /ai/knowledge/:id/reindexmanageRe-queues this KB's error documents.
POST /ai/knowledge/:id/queryviewRetrieval test — runs a query through the KB provider registry.

/ai/integrations and /ai/models ​

Method + pathActionNotes
CRUD /ai/integrations, /ai/integrations/:idview/manageconfig.api_key masked; blank PATCH keeps it. provider required.
POST /ai/integrations/:id/testmanageProbe GET {base_url}/models; never throws (ok:false + reason).
GET /ai/integrations/:id/available-modelsmanageFull provider model catalogue for the allow-list picker.
GET /ai/models?capability=viewModels derived from enabled integrations (each carries integration_id).

/ai/mcp-servers ​

Method + pathActionNotes
CRUD /ai/mcp-servers, /ai/mcp-servers/:idview/manageheaders encrypted on write, masked on read; blank PATCH keeps secrets. label required.
POST /ai/mcp-servers/testmanageTest-before-save with ad-hoc form values.
POST /ai/mcp-servers/:id/testmanageTest a saved server → { ok, tools[] }.
GET /ai/mcp-servers/:id/toolsviewallowed_tools-filtered tool list for the attach picker.

/ai/sources ​

Method + pathActionNotes
GET /ai/sources/collectionsviewNon-system collections for collection-KB config.
GET /ai/sources/fields?collection=viewFields of a collection.

/assistant/* (legacy assistant-config + RAG ingest) ​

All gated ai / manage. CRUD (GET list, POST, GET/PATCH/DELETE /:id) over the odp_ai_* tables:

ResourcePath
LLM providers/assistant/providers
Knowledge backends/assistant/knowledge-backends
Prompts/assistant/prompts
Agents/assistant/agents
Ingest maps/assistant/ingest-maps

Plus the ingest tooling (controllers/ingest.controller.ts):

  • GET /assistant/ingest/collections — non-system collections for the mapping dropdown.
  • GET /assistant/ingest/fields?collection= — a collection's fields with parsed special (relations visible).
  • POST /assistant/ingest/backfill — bulk-load a collection into KAOS: paginate rows (read as the requesting ai.manage user, so RBAC applies) → map each via the ingest map → write through the KB registry (an ephemeral KAOS KB from the resolved agent-config). Returns { ingested, skipped, units }.

The retired /assistant/chat, /assistant/chat/stream and /assistant/health endpoints (the gate → retrieve → synthesize → enforceGrounding orchestrator) no longer exist. Chat is served only by POST /ai/apps/:id/chat.


Runtime ​

The executor (runtime/executor.ts) walks the graph, keeping a variable pool (vars[node_key] = output), a citation list and a meta bag.

  • Provider resolution (runtime/providers.ts): resolveLLM prefers the app's bound integration_id (base_url/key), else the first enabled integration, else ASSISTANT_LLM_* env, via createOpenAICompatibleLLM. resolveDefaultModel resolves app.config.model → integration's first llm model → env. loadKb loads a KB row by uuid. An llm node with its own integration_id uses resolveNodeLLM instead, which has no fallback.
  • API keys: ASSISTANT_LLM_API_KEY goes only to ASSISTANT_LLM_BASE_URL. A custom base_url sends the integration's own key, or no Authorization header when it has none.
  • Query rewrite: when a conversation history exists, the executor rewrites a follow-up into a standalone question (query_resolved) before classify/retrieval/mcp-agent read it. No history → no LLM call → byte-for-byte identical to the stateless path.
  • KB retrieval: dispatched through KbProviderManager.search, which enforces capability ∩ policy (a read-blocked KB yields [] and the flow degrades rather than erroring).
  • MCP client (runtime/mcp-client.ts): every connection is short-lived (connect → op → close), tool listings cached per (id, updated_at). The outbound URL is vetted through the core SSRF guard (assertUrlAllowed) before connecting; allowed_tools filters both listing and invocation; the guarded tool loop is bounded to MAX_TOOL_ITERATIONS = 5 (then forces a final tool-less answer).
  • Result: FlowResult { status: 'answered' | 'partial' | 'error', answer, reason, citations, meta, trace }. When a node fails and no answer exists, reason names the node and its error: Node error: final (<err>) — no answer produced. The SSE layer (runtime/chat-stream.ts) maps this to the wire AnswerStatus (answered → answered, partial → degraded, else declined) and encodes ChatEvent frames (conversation → status → step* → answer deltas → citations → done).

MCP tools (group ai) ​

The MCP server (docs/api/modules/mcp.md) serves these tools from modules/ai/mcp-tools.ts. Each runs the same gate as its REST route: ai.view to read and to test-run, ai.manage to write. Its scope names the tables it writes and the tables it only reads, so the MCP allowlist and read-only list bind them.

ToolDoes
ai_apps_listList apps (no config, no graph).
ai_apps_readOne app with its config, graph and current problems. Secret-looking values in node data and config are masked.
ai_flow_node_typesThe node catalog; with {type}, one type's data schema, defaults and example.
ai_flow_validateProblems of the stored graph, or of a proposed {nodes, edges} (nothing is saved).
ai_models_listEnabled integrations and their models, without URL or key.
ai_mcp_servers_listMCP servers with the tool names stored on the row, without URL or headers.
ai_skills_listSkills with their category, without the body.
ai_knowledge_list, ai_knowledge_searchKnowledge bases, and a retrieval test on one.
ai_apps_createCreate a draft app; a flow app gets its start node.
ai_apps_updateChange name, description, icon or config. Config keys are merged; a key set to null is removed.
ai_flow_add, ai_flow_updateAdd, or change, nodes and edges. Refused when the call brings in a wrong value (graph check); an unfinished flow is saved. A code node is taken from an admin only.
ai_flow_removeRemove nodes (with their edges) and edges. Never start. Hidden when MCP deletes are off.
ai_apps_publishPublish after the strict check.
ai_apps_cloneCopy an app with its graph as a new draft. An app with a code node is copied for an admin only.
ai_apps_deleteDelete an app and its graph in one transaction, as REST DELETE does. Hidden when MCP deletes are off.
ai_apps_test_runOne question through the app, answer and node trace. See below.

The build loop an agent follows (the ai guide topic has the full version):

  1. ai_apps_create a chatflow (it starts with the node start).
  2. ai_flow_add the nodes and edges, copied from ai_flow_examples and the node topics.
  3. ai_flow_validate, and fix each problem with ai_flow_add, ai_flow_update or ai_flow_remove.
  4. ai_apps_test_run with one question for each branch.
  5. ai_apps_publish.

A write keeps a half-built flow (an unreachable node or an unwired branch is saved) and returns its problems; only a wrong value is refused. test/modules/ai/mcp-flow-build-e2e.test.ts runs this loop through the registry-mode root tools (odp_search, odp_execute_read, odp_execute_write, odp_guide), with the LLM and KAOS replaced by a stub on 127.0.0.1.

A refused call carries extensions {problems, total, next}. In registry mode the problems come back as error.details.problems, and an odp_* value of next becomes the gateway next.

ai_apps_test_run runs the stored graph (draft or published) once, with persist: false: no conversation, no ai_app_logs row, one audit log line (user, app, status, ms). It refuses a graph with an error before it calls a model, charges one point of ASSISTANT_RATE_LIMIT_PER_MIN (the bucket Agentic chat uses), and stops after 45 s with status: "timeout" and the trace so far: the walk stops before its next node, a node already running finishes in the background. The answer is cut at 4 000 characters, each node output at 1 500, the trace at 50 steps, and secret-looking values in node outputs are masked.

A tool that checks a graph (ai_apps_read, ai_flow_validate) reads only the reference tables the graph uses: a graph with a knowledge-retrieval node needs ai_knowledge_bases in the allowlist, one without it does not.

odp_guide serves the topics of modules/ai/mcp-guides.ts: ai (read first: app types, build loop, the rule not to write the AI tables with items_*), ai_flow_nodes_llm, ai_flow_nodes_routing, ai_flow_nodes_tools (generated from the node catalog), ai_flow_edges, ai_flow_refs and ai_flow_examples. The MCP items_* tools refuse writes to ai_apps, ai_flow_nodes and ai_flow_edges and point to these tools. Those three collections carry a note that says the same (migration 106 sets it on existing installs; an admin note is kept).

Programmatic trigger (other extensions / AI workflows) ​

Besides the HTTP route, another extension — or an AI workflow that needs to call an app as a sub-step — can trigger an AI app by ID in-process, without self-calling /ai/apps/:id/chat. The core AI module publishes a façade on the cross-extension service registry (see Extension Guardrails §6), under the ref id ai.engine. It runs the exact same flow as the HTTP route (conversation memory + ai_app_logs + streaming), so an app behaves identically whether a browser or an extension started it.

ts
// endpoint / hook that runs AFTER 'extensions.register' (never at a hook's top level — registry not sealed yet)
export default defineEndpoint((router, ctx) => {
  router.post('/run-ai', async (req, reply) => {
    // consume by string id — no need to import the ref (keeps you on @odp/api/types only)
    const ai = ctx.registry.tryConsume('ai.engine');
    if (!ai) return reply.status(503).send({ error: 'AI engine not available' });

    const result = await ai.run('<APP_ID>', {
      query: 'câu hỏi của user',
      variables: { orderId: 123, locale: 'vi' }, // structured params → the app variable pool ({{app.variables.*}})
    }, {
      persist: false,           // throwaway run — don't touch channel history / log timeline (default true)
      source: 'my-ext',         // channel tag on the conversation + logs (default 'extension')
      externalRef: 'wf-inst-9', // idempotent conversation mapping (e.g. a workflow instance id)
      accountability: req.accountability, // scope the flow's own data access (default null = system)
      onStep: (ev) => {},       // live node/tool progress (optional)
      onAnswerToken: (d) => {}, // live answer tokens (optional)
    });
    return reply.send({ data: result }); // FlowResult
  });
});
  • run(appId, input, options?) → Promise<FlowResult> — resolves (or creates) the conversation, runs the graph, resolves after the flow completes. Throws if appId does not exist.
  • input: { query, conversation_id?, variables? }. variables is the params channel for workflow/agent apps — merged into the variable pool so nodes read {{ app.variables.* }}.
  • options (all optional, defaults mirror the HTTP route): persist, source, externalRef, accountability, onAnswerToken, onStep.
  • Contract lives at modules/ai/engine-ref.ts (aiEngineRef, AiEngine, AiEngineInput, AiEngineOptions); provided at boot in src/index.ts before the registry is sealed. Consume by the typed aiEngineRef or the plain 'ai.engine' id.

Agentic UI ​

Agentic UI is the in-app assistant. An admin can select an AI app to serve it, so provider, model, skills and knowledge are managed in Studio. The webapp BFF keeps its streamText loop, because tools run in the browser and each tool round is a new request. The engine therefore stops a flow at its last llm and hands that one model call back to the BFF, which makes it through a gateway on this API. The provider key never leaves the API.

One turn goes like this:

  1. The BFF reads the public status to learn whether the engine path applies.
  2. On each round it calls POST /ai/agentic/prepare. The first call runs the flow as the user and stops before the final llm. Later calls of the turn return the stored result.
  3. If the flow answered by itself (kind: 'answer'), the BFF streams that text and the turn is done.
  4. Otherwise (kind: 'llm') the BFF calls the gateway. The gateway checks the turn, adds the app prompt and knowledge, and passes the SSE through.

Settings ​

odp_ai_agentic_settings holds { enabled, app_id } and the chat history retention (history_retention_days, see Chat history). Provider, model and skills live in the app; with no drawn flow, use a chatbot app. The widget state follows this table. No row means nothing changes on upgrade.

SettingResult
ODP_AGENT_ENABLED=false on the webappOff, whatever is saved (kill switch, applied by the BFF)
No row, or /ai/agentic/status answers 404Env path (ODP_AGENT_*)
enabled: falseOff
enabled: true, no app_idOn, env path
enabled: true with app_idEngine path; every chat fails closed while the app is not ready

A save applies on the next read. The BFF caches status for 15 seconds, and a page reload shows or hides the widget.

Endpoints ​

Method + pathGateNotes
GET /ai/agentic/statusNone (public){ state: 'unset' | 'on' | 'off', engine, ready } and nothing else. ready is false only when engine is true and the app is not ready. It is cached 5 seconds, so a graph edit can take that long to show; a save applies at once.
GET /ai/agentic/configai.view{ state, enabled, app_id, app_name, ready, issues[], history_retention_days }, always fresh.
GET /ai/agentic/appsai.viewSelectable apps, templates left out: { id, name, type, status, ready, issues[] }. A not-ready app stays listed.
PATCH /ai/agentic/settingsai.manageBody { enabled?, app_id?, history_retention_days? } (days 1–3650, null = 90), other keys are 400. The first save must carry enabled, because the first row turns the assistant on or off. An unknown app is 400; a not-ready app is saved and reported in issues.
POST /ai/agentic/prepareUser with app accessBody { turn_id, query, history?, purpose? }, 4 MB limit. purpose is chat (default) or summary, see Summary turns.
POST /ai/agentic/v1/chat/completionsUser with app accessOpenAI chat completions, 20 MB limit.

Who may chat is app access, not the ai module, so the assistant reaches more users than Studio does.

turn_id must match ^[A-Za-z0-9_-]{16,128}$. The client sets it, and it stays the same through every tool round and approval of one turn. prepare answers one of:

json
{ "data": { "kind": "llm", "turn_id": "...", "model": "<integration_id>:<model_id>", "integration_id": "...", "skills_loaded": [], "ui": false, "citations": [] } }   // "ui" only when the node sets it
{ "data": { "kind": "answer", "turn_id": "...", "text": "...", "citations": [] } }

The app's system prompt and knowledge are not in this response, so a user token cannot read them.

Branch flag ui. A kind: "llm" response carries "ui": false (or true) when the final llm node of the branch sets config.agentic.ui (node data agentic.ui, boolean; set it with PUT /ai/apps/:id/graph or the MCP flow tools, Studio has no field for it yet). It tells the caller whether this branch needs a generative-UI section and ui.* tools in its own prompt: false means no. A node that does not set it, or sets anything but a boolean, adds no ui key, and the caller keeps its default (UI on). The flag is a hint to the caller, not a permission: it widens nothing the caller could read, and the gateway ignores it. The response already lists skills_loaded (names of the skills attached to that node); a caller can use those names to avoid adding a skill of its own that the engine already loaded.

The gateway takes a standard OpenAI body with stream: true, the model string prepare returned, a user token, and the header X-ODP-Turn-Id. Every other field passes through unchanged in both directions, including tool_calls[].extra_content, which the Gemini shim needs. The one exception is the output cap: when the app sets max_output_tokens, it is a ceiling. The caller may ask for less, never more; a value it sends that is not an integer from 1 to 128000 is replaced by the app's. max_tokens and max_completion_tokens are each capped if sent, and a body with neither gets max_tokens. With no app value the body is not touched. A provider error keeps its status but loses its body, which can name accounts or internal hosts.

The gateway merges one system message, because some compatible providers reject a second: the caller's text (BASE_SYSTEM_PROMPT leads it), then the app's system prompt, then knowledge in a <knowledge> tag marked as data. < is escaped inside the tag. A later system message or a developer message is 400.

Statusprepare codeGateway codeMeaning
400Schema validationinvalid_payloadMalformed body. A refused gateway body does not use up a call.
401INVALID_CREDENTIALSinvalid_credentialsNo user.
403FORBIDDEN, AGENTIC_TURN_INVALIDforbidden, agentic_turn_invalid, agentic_model_not_allowedNo app access; a forged, foreign, closed, expired or exhausted turn; a model the turn was not prepared for.
404AGENTIC_NOT_ENGINEagentic_not_engineOff, or no app selected.
409AGENTIC_NOT_READY, AGENTIC_TURN_CONFLICTagentic_not_readyApp not ready; same turn_id with another query.
429REQUESTS_EXCEEDEDrequests_exceededQuota spent, with Retry-After. A provider 429 passes through as upstream_429.
502upstream_unreachableProvider unreachable.
503AGENTIC_PREPARE_FAILED, AGENTIC_TURN_UNREADABLEagentic_turn_unreadableFailed run; content unreadable, for example after a SECRET change.

prepare errors use { errors: [{ message, extensions: { code } }] }, except a schema validation failure, which keeps Fastify's default shape. Gateway errors use the OpenAI shape { error: { message, type, code } }, so @ai-sdk/openai can read them.

A failed run is a 503, stored nowhere. Any node error before the handoff counts, so a flow that lost its retrieval does not answer on empty context, and a draft from an earlier llm is never served as the answer.

Turns ​

A row keeps query_hash and never the user's query or messages. content is the app system prompt and knowledge, encrypted with AES-256-GCM (utils/encryption.ts) under a key derived from SECRET for this purpose. It is decrypted only after the turn guard passes for the user the turn was issued to. Neither value goes to logs or activity. The turn's ai_app_logs row is separate (see Logs).

A turn lives 30 minutes from its last prepare or gateway call, and the gateway enforces that expiry. It allows 24 gateway calls, and it closes when the provider finishes with no tool calls; closing empties content. finish_reason alone does not decide this, because Gemini reports stop on a response that calls tools. The gateway holds the finishing frame until the provider stream ends and closes the turn before it sends that frame, so a client that holds the whole answer cannot replay the turn. A stream that breaks mid-way leaves the turn open. A retry from the client carries a new turn_id, so it reruns the flow.

A schedule (*/5 * * * *, ai-agentic-turn-prune) deletes expired rows with their content, so content outlives a turn by five minutes at most, even with no new chat. Several instances can run it safely.

Quota ​

ASSISTANT_RATE_LIMIT_PER_MIN (default 20) limits each user. Each gateway call costs one point, and so does a prepare that runs the flow; a prepare that returns the stored result costs nothing. A value below 1 means no limit. RATE_LIMITER_ENABLED does not affect it.

The count is shared across API instances only with RATE_LIMITER_STORE=redis and REDIS_ENABLED. Without Redis each instance counts alone, so the real limit is the value times the number of instances. Turn state is in the database and stays correct either way. One long turn can use 13 points, and a second one straight after can hit the limit.

Logs ​

Each Agentic turn writes one ai_app_logs row with source: 'agentic', so it shows in the Studio Logs tab next to the Studio rows (badge "Agentic", source filter). Studio chat rows carry source: 'studio'.

  • prepare creates the row after the input check, so query is the masked user text, as on the Studio chat. A refused input logs [guardrail: <rule>] as the query and no user text.
  • A turn that prepare answers (refusal, flow answer) is finished at once. A handoff turn stays running and keeps the trace, citations and chunk count of the flow.
  • The gateway round that closes the turn, or turns/end, finishes the row: status: 'success', the shown answer (or the tool-limit note), latency_ms from prepare, finished_at, and the tool names of each round in mcp_calls (no arguments, no results).
  • A turn that never closes (client gone, stream broken, expired) stays running. A resumed prepare of the same turn_id adds no row. A failed prepare, and the run that loses a concurrent prepare, end as error.
  • A guard finding is in reason as guardrail: <rule> <action> (<in|out>), on both paths. A gateway round that replaces or notes the answer sets it; otherwise the input verdict does. The guard audit line is unchanged (contract §8): it carries no text.
  • tokens stays 0: the gateway does not read usage from the provider stream.
  • The history fold that prepare runs (summary of older turns) writes no row of its own.

On an install below schema v6 the rows are still written without source, and the source filter of the Logs tab fails: run the upgrade in /ai/setup (see Upgrade notes).

Chat history ​

The API stores the Agentic chats the client chose to keep, in odp_ai_agentic_conversations and odp_ai_agentic_messages (migration 105). Every query is limited to the caller's own rows, admin included; prepare and the gateway never read them. Routes under /ai/agentic/conversations (list, create, read, update, delete; messages read, upsert, truncate) and POST /ai/agentic/history/purge for admins. Retention is history_retention_days in the settings (default 90), pruned every hour.

Row lifecycle of the engine's own writes (AI guardrails contract §16): prepare only reserves the id of a new conversation, and the conversation row is inserted with the first pair of messages (end of the turn, turns/end, a canned answer, or a stopped turn), so no turn leaves an empty conversation. An id with no row that nobody else holds is a new conversation; an id held by another user is treated the same way and nothing is ever written into that conversation. When the client closes the stream, the engine stores the text that reached it plus the finished tool rounds as an unchecked pair; nothing reached it, nothing is written; a provider failure writes nothing. prepare with regenerate_message_id equal to the last stored answer replaces the last pair at write time instead of appending.

Full guide, with examples, caps, errors and the rules for the client: AI Engine — Agentic chat history.

Summary turns ​

POST /ai/agentic/prepare with purpose: 'summary' prepares a turn that runs no flow, on the app's model. On that turn the gateway refuses tools, puts its fixed summarizer prompt in place of the app's, folds the messages into an escaped <conversation> data block, and caps the output at min(app max_output_tokens if set, 2048) tokens; a smaller caller value is kept. Streamed only, engine path only. Details: Agentic chat history §Summary turns.

Readiness ​

runtime/agentic-readiness.ts checks an app the way it would run. error makes the app not ready, and every chat then fails closed. warning is shown and the app stays ready. config, apps, status, prepare and the gateway all use this check. Graph rules apply to chatflow, agent and workflow; a chatbot has no stored graph, so only its config is checked.

CodeSeverityMeaningFix
app_missingerrorThe selected app no longer exists.Select another app.
app_not_publishederrorThe app is a draft. There is no snapshot: once published, edits apply at once.Publish it.
app_has_mcp_serverserrorThe app has mcp_server_ids, not supported in v1.Remove them, or select another app.
app_type_unsupportederrortext-generation (a one-shot form, not a conversation) or an unknown type.Use a chatbot or a flow app.
no_llm_nodeerrorThe flow has no llm to hand off.Add one.
node_after_handofferrorA node other than answer follows the final llm. It would never run.Move the node before the llm.
parallel_handofferrorTwo final llms can run in one turn. Two llms on exclusive handles of question-classifier, smart-switch or if-else are routing and pass.Put them on exclusive branches. An llm with no incoming edge runs as a root and counts.
llm_integration_missingerrorAn llm has no usable integration: deleted, disabled, decision-only, no base URL, or no API key for a node binding.Fix the integration or the binding.
llm_model_missingerrorAn llm has no model, or its node binding names one outside the allow-list.Pick an allow-listed model.
app_llm_fallbackerrorQuery rewrite, classifier and skill router run on the app-level provider whatever the llm nodes bind. They would reach a provider nobody picked: the app's integration_id points to a deleted row, which moves it to the first enabled integration, or the integration it resolves to has no base URL, or none exists, which means the env provider. Not repeated when llm_integration_missing is already reported.Bind the app to an existing integration that has a base URL.
mcp_agent_nodewarningAn mcp-agent node runs its tools on the server, without Agentic approval.Accept it, or remove the node.
answer_template_ignoredwarningAn answer after the final llm has text other than {{ llm.text }}. The BFF streams the llm itself, so that text is not applied.Move the text into the prompt.
app_endpoint_not_openai_wirewarningThe base URL of an integration the app reaches (app level or an llm node) looks like a vendor's native API: path ends in /messages or :generateContent/:streamGenerateContent, or it is generativelanguage.googleapis.com or /v1beta/models outside /openai. The gateway appends /chat/completions, so the call fails at run time. Unknown gateways are not flagged.Use https://api.anthropic.com/v1/ for Claude, https://generativelanguage.googleapis.com/v1beta/openai for Gemini.
app_integration_no_keywarningThe app's integration has no API key. Chat fails unless the provider needs none, such as a local one.Save a key.

Readiness is stricter than the chat path on purpose. A disabled or deleted bound integration is an error here, although Studio still runs on it through resolveLLM. The runtime is unchanged, so the same app can work in Studio and be not ready for Agentic.

Limits in v1 ​

  • Anyone who can chat with the app receives chunks from uploaded KB documents (see Knowledge bases).
  • The KAOS filter depends on the address format and has not run against a real KAOS.
  • Audit is one log line per model call, with user, turn, integration, model and status. Each turn also writes one ai_app_logs row (Logs) but no ai_conversations rows; history is in Chat history.
  • No model picker in the widget and no provider fallback.

Upgrade notes ​

  • AI schema v6 adds ai_app_logs.source. No migration: an admin opens /ai/setup and runs the upgrade ("Upgrade available: v5→v6"). It adds only what is missing, so an install that also lacks v5 items (for example ai_conversations, ai_messages, ai_skill_categories, ai_skills) gets them in the same run. Existing rows are kept with source null, which the Logs tab counts as Studio. Restarting the API does not add the column; until the upgrade runs, logs are written without source and the source filter fails.
  • Migrations 102-ai-agentic-settings and 103-ai-agentic-turns add the two system tables. No row is seeded and existing deployments keep working as before until an admin saves a setting.
  • Migration 105-ai-agentic-history adds odp_ai_agentic_conversations, odp_ai_agentic_messages and the history_retention_days column. Nothing is stored until the BFF saves a chat. From then on, conversations idle for 90 days are deleted every hour; set history_retention_days to keep them longer.
  • ASSISTANT_LLM_API_KEY now goes only to ASSISTANT_LLM_BASE_URL. Before, an integration with a custom base_url and no key sent the env key to that host, so anyone with ai.manage could collect it. Such an integration now sends no key and fails on a provider that needs one: save a key on it.
  • KAOS search is filtered per user on all paths, so a non-admin can get fewer chunks than before from a KB that mirrors a restricted collection, in Studio and through MCP too. Tool names and schemas did not change.
  • reason of a failed run now carries the error (Node error: final (<err>) — no answer produced.).
  • The output cap of chatbot and text-generation apps now reaches the provider, from the new key config.max_output_tokens. Values Studio saved before (config.max_tokens, 1024 by default) are not carried over: they are ignored and removed when the app is next saved through /ai/apps. Every existing app therefore keeps the provider default. Set Max tokens again in Studio for an app that needs a cap.

Guardrails custom rules ​

An admin can add global yes/no rules to the guard, next to the built-ins I1..I5 and O1..O5. Each rule is one System One question, asked in the decision call the guard already makes. Code: guardrails/custom-rules.ts (model, limits, wording hash, hit counters), services/custom-rules.service.ts (CRUD, order, quick test, gate), guardrails/guard.ts (run time), controllers/guardrails.controller.ts (routes). Migration 110.

Rule model ​

FieldValue
idC1..C9999, given by a counter (odp_ai_guardrail_settings.custom_rule_seq). A deleted id is never given again, so an old audit line or ledger row never points to a new rule. Past C9999: 400 AI_GUARDRAIL_RULE_IDS_USED.
name1..80 characters. Admin pages and Studio debug only: an end user never sees it.
directionin or out. Fixed after create.
question, yes_means10..600 and 5..200 characters. The question may name state fields in backticks: message, history_tail (in); answer, question, evidence (out).
block, warn0..1, warn < block.
actionin: refuse, flag; out: replace, note, flag.
texts{ vi, en }, required unless the action is flag.
modeshadow or enforce. A new rule is always shadow; create ignores mode.
scope{ apps: 'all' } or { apps: [uuid] } (at most 500). An app cannot name, loosen or turn on a custom rule in config.guardrails.
needs_evidenceout only: the question gets the retrieved evidence.
examplesat most 30: { text, expect, lang, answer_context? } (answer_context.question / evidence on out).
severity0..100, stored in reputation.severities[id], default 0. Deleting the rule removes it.
positionorder inside a direction. Set with POST /rules/order.

Run time ​

  • The guard adds every enabled rule whose scope holds the app to the same call. Above 16 keys (GUARD_BATCH_KEYS) it splits the questions into parallel calls under the same bound; the built-in keys and intent stay in the first call. A failed call with only custom keys fails those rules open (unavailable), not the built-ins.
  • shadow: the verdict does not change, nothing is shown, no point is taken. The finding has action: 'shadow', band: 'block' | 'warn' and shadow: true; the audit line has action: 'shadow' and no band. Alert 07 counts only refuse|replace, so shadow hits never trigger it.
  • enforce: the rule acts as a built-in. Refusal text order, input: self-harm, refusal (I1, I2), custom refuse in position order, secret input, out of scope. Output: built-in replace, custom replace in order, ungrounded, mask, then notes (custom note texts in order, each on its own paragraph).
  • meta.guardrail of a non-debug turn never holds a shadow finding or a rule name. In debug (Studio test run) meta.guardrail.findings carries rule_name for custom rules.
  • The Agentic gateway runs the same checks.

Hit counters ​

Each finding at the warn or block band of a custom rule (shadow or enforce) adds 1 to hits_warn / hits_block and sets last_hit_at. Counts are kept in memory and flushed every 30 s (HIT_FLUSH_MS), and on shutdown. A change of the wording or the thresholds resets them and sets counted_since. Shadow to enforce resets nothing (Q-C3).

Quick test and enforce gate ​

  • POST /rules/test tests a draft; POST /rules/:id/test tests a saved rule and stores the result in last_test. One call per example, with only that rule's question, concurrency 4. Body { full_direction: true } also asks every enabled question of the direction, to measure the real latency.
  • The result lists p, band, pass and ms per example, and the summary (caught, false blocks, median and max ms). It writes no audit line, no ledger row and no log.
  • No decision integration: 400 AI_GUARDRAILS_NO_DECISION.
  • Gate: at least 5 positive and 5 negative examples, at least 90% of the positives at p >= block, no negative at p >= block. last_test holds a hash of direction + question + yes_means + block + warn.
  • mode: 'enforce' is refused with 400 AI_GUARDRAIL_RULE_TEST_REQUIRED unless last_test.passed is true and its hash matches. A rule view has enforce_allowed.
  • A PATCH that changes the question, yes_means or a threshold puts the rule back to shadow in the same write, and resets its counters.

Reputation ​

An enforce input rule with a severity above 0 deducts as a built-in (severity x intent factor x p) and asks the intent question. An enforce output rule deducts only with a severity above 0, with the input intent of the turn. A shadow rule never deducts. The ledger keeps the rule id after a delete; the admin page shows "deleted rule (C7)".

Routes ​

RouteGate
GET /ai/guardrails/rulesai view. Every rule, both directions, with counters and enforce_allowed.
POST /ai/guardrails/rulesadmin, not a bounded grant. Creates in shadow.
PATCH /ai/guardrails/rules/:idsame. Partial body; mode passes the gate.
DELETE /ai/guardrails/rules/:idsame. Removes the severity too.
POST /ai/guardrails/rules/ordersame. Body { direction, ids }: every id of the direction once.
POST /ai/guardrails/rules/test, /rules/:id/testsame.
GET /ai/guardrails/settings?app_id=ai view. app.effective.custom lists the enabled rules that apply to the app (id, name, direction, mode, action).

Unknown id: 404 AI_GUARDRAIL_RULE_NOT_FOUND. A body the checks refuse: 400 INVALID_PAYLOAD with every problem. odp_ai_guardrail_rules is a system table (SYSTEM_TABLES.AI_GUARDRAIL_RULES), written by these routes only.

Admin UI ​

Guardrails page, Settings tab: each direction shows its custom rules under the built-ins (mode badge, action, thresholds, severity, scope, counters). The switch and the up/down buttons save at once. The drawer edits the rule, its examples and its severity, runs the quick test, shows the cost estimate (keys and calls of the direction, a warning above 16 keys or when a full-direction test took more than 1000 ms) and holds the mode switch: it opens only when the last test passed on the current wording. Delete asks in a modal. A user without admin sees the rows read only. The webapp BFF forwards each route in apps/webapp/modules/ai/server/routes.ts; a new API path needs a line there. The Studio badge shows C1 · name from the debug findings. The Reputation users table shows the rule name, or "deleted rule (id)".

Guardrails reputation ​

Each counted user has a balance of points. A refused or flagged input with a severity costs points; at 0 the user is blocked for block_hours and every counted surface refuses them. Points come back each day. Code: guardrails/reputation-settings.ts (settings), services/reputation.service.ts (scoring, ledger, lifecycle), controllers/reputation.controller.ts (routes). Reputation runs only when the guard is enabled: it reads the guard's findings and adds an intent question to the same decision call.

Modes and settings ​

  • The reputation block of GET/PATCH /ai/guardrails/settings (column odp_ai_guardrail_settings.reputation, migration 109). Null reads as the defaults with mode: 'shadow'.
  • mode: off counts nothing. shadow counts and writes the ledger, but never refuses and never notifies; only admins see balances (would_block marks a deduction that would start a block). enforce refuses a blocked user.
  • enforce needs a passing eval record (eval: { ran_at, integration_id, model, passed: true }, written by eval.ts --record), else 400 AI_REPUTATION_EVAL_REQUIRED. The gate: accidental read as deliberate < 2%, deliberate recall >= 85%, simulated false block = 0. The switch to enforce resets every subject to its max.
  • severities (0..100) are keyed by a string rule id, not a fixed union, so a custom guardrail rule carries its own severity (see custom rules). A PATCH may only name a rule that exists today (I1..I5, O1..O5 and the custom ids). Defaults: I1 10, I2 20, every other rule 0. I2_self_harm is fixed at 0 (400 AI_REPUTATION_SELF_HARM): a person at risk never loses access.
  • intent_factors (0..1): deliberate 1, unclear 0.3, accidental 0. With probabilities the expected factor is used.
  • Numbers: default_max 100, regen_per_day 10, warn_balance 30 (clamped to max - 1), block_hours 24, excerpt_days 90.

Scoring ​

  • One deduction per rule and turn: severity x intent factor x p, rounded to 2 decimals; the highest p of a rule wins.
  • Input findings with action refuse, mask or warn count. Output findings count only when their rule has a severity (all 0 by default); they reuse the input intent of the turn.
  • Regen is lazy (computed on read, no scheduler). The first drop to warn_balance sends a warn notice once; reaching 0 starts a block, emits AI_USER_BLOCKED (security event) and notifies every user with ai manage.

Who is counted ​

CallerCounted
Public chat, engine call with no usernever
Logged-in user (own token or session), admins includedyes, subject = user
Shared token (shared: true on the service user) with end_useryes, subject = user + end_user
Shared token without end_userno (warn log, throttled)
Studio test run of a user with ai manageno (Q-R2)

end_user is a body field of POST /ai/apps/:id/chat and POST /ai/agentic/prepare (option endUser of aiEngineRef.run): 1..128 characters, ^[A-Za-z0-9_.:@-]+$, else 400. It is ignored for a user that is not shared. The gateway reads the subject from the turn row (odp_ai_agentic_turns.end_user).

Where it runs ​

  • AiChatService.chat (route chat, MCP ai_apps_test_run, engine ref): a blocked subject gets the reputation_blocked text with status answered, meta.guardrail.reason: 'reputation', log reason reputation: blocked. A counted turn in enforce carries meta.reputation { notice, balance, max, blocked_until?, text } when it warns or blocks.
  • Agentic prepare: same refusal, before the quota is charged; the response has reputation. The gateway checks the block on every round, so a block mid-turn ends the turn.
  • Texts reputation_blocked and reputation_warn are editable VI/EN with the other guard texts.

Routes ​

RouteGate
GET /ai/reputation/mea user who can run a counted turn. In enforce: { counted, mode, balance, max, blocked_until, warn_balance, can_self_reset }; otherwise { counted, mode }. ?end_user= for a shared caller.
POST /ai/reputation/me/resetai manage, not a sub-token or impersonation; enforce only (else 400). Notifies the other ai managers.
GET /ai/reputation/usersai manage. Query: search, blocked, warned, role, app, limit, offset. Meta { total, mode }.
GET /ai/reputation/users/:id/eventsai manage. Ledger, newest first, with masked excerpts.
POST /ai/reputation/users/:id/reset, /unblockai manage, not a sub-token or impersonation
PATCH /ai/reputation/users/:idsame; body { max?, shared? } (max 1..10000)
GET /ai/reputation/rolesai manage
PUT /ai/reputation/roles/:roleadmin only; sets the role max and pushes it to every user of the role

Without the v7 tables the admin routes answer 409 AI_REPUTATION_NOT_INSTALLED and reputation reads as off.

Storage ​

  • AI schema v7 (re-run /ai/setup; boot does not apply it): ai_reputation_events (append-only ledger), ai_reputation_users (one row per subject), ai_reputation_roles (role max).
  • /items/ai_reputation_* is read-only for everyone, admins included; writes go through the routes above only.
  • Retention: the ledger is kept 365 days, the masked excerpt (240 characters) excerpt_days. A deleted user's rows are deleted. Purge runs lazily and in a daily synchronized job.
  • Migration 109 also adds end_user, intent, p_intent to odp_ai_agentic_turns, and seeds three in-app notify templates (ai_reputation_warn, ai_reputation_blocked, ai_reputation_self_reset) when a primary in-app provider exists. Without one the notifies are no-ops.

Admin UI ​

The Guardrails page has three tabs for a user with ai manage: Settings (with the Reputation section), Reputation users (table, ledger drawer, reset / unblock / edit max / mark shared, each confirmed in a modal) and Role max (admin saves). The Studio test chat shows a banner in enforce when the caller is warned or blocked; an ai manager can reset their own points from it.


Ingest (RAG) — two paths ​

Both paths write only through the KB registry (KbProviders), never a raw backend client, so capability ∩ policy is always enforced.

  1. Studio document upload (ingest-queue.ts) — POST /ai/knowledge/:id/documents writes an ai_documents row (status: 'queued') and enqueues a BullMQ / Redis job (Redis is mandatory — no memory fallback). The single-concurrency worker reads the row, ingests to the KB (KAOS), and marks completed / error. Transient failures (KAOS 5xx/429/network/timeout) retry with backoff; permanent 4xx fail fast. A row deleted during ingest has its orphan units purged.
  2. Event-driven collection sync (ingest-listener.ts) — when ASSISTANT_INGEST_COLLECTIONS lists a collection, items.create / items.update / items.delete actions map the row (via its odp_ai_ingest_maps entry, or a generic scalar dump) and upsert/delete it into KAOS. The base row is read WITH the triggering user's accountability (RBAC applies to what is ingested), and a failure is logged and never rethrown — an ingest error must not break the underlying CRUD write.

An extension can intervene on the built doc through the ai-engine.ingest.remap filter (fires in both the backfill and listener paths): return the (possibly modified) IngestDoc to ingest it, or null/undefined to skip the row.


Invariants & gotchas ​

  • Actor split. Control-plane /ai/* CRUD runs ItemsService without accountability — access is already gated at the app-module layer (ai view/manage), mirroring core admin routes. Ingest (both paths) runs with accountability so collection/field RBAC decides what reaches the KB.
  • Raw Knex only off the actor path. Direct knex(...) use is limited to runtime/config reads (graph load, integration/KB/MCP resolution, aggregate stats with fixed status literals) and the legacy odp_ai_* config services — never a per-user data mutation.
  • Secret masking everywhere. Integration config.api_key, KB source_config.api_key, and MCP headers are masked on read and preserved on a blank PATCH; MCP headers are additionally AES-256-GCM encrypted at rest and only decrypted for the outbound request. Legacy odp_ai_* rows store secrets as env-ref names (api_key_ref), never raw.
  • code node disabled by default — requires AI_ENABLE_CODE_NODE=1 in a trusted deployment (Function() eval, not a sandbox).
  • AI_MOCK_RETRIEVAL is dev-only — synthetic chunks let a RAG flow complete without real KAOS data; hard-disabled in production (a fabricated "grounded" answer must never ship). Mocked responses carry meta.mock = true and mock:// citations.
  • Redis is mandatory for ingest — with REDIS_ENABLED off, documents queue but do not ingest.
  • KAOS address format is part of the access check. A change to buildFileKey must change providers/kb/readable-rows.ts with it, or mirrored rows stop being filtered.

Conversation memory & logging status ​

These recently-landed pieces are present and wired at HEAD, but are the most actively evolving part of the module — treat them as maturing:

  • Streaming chat (runtime/chat-stream.ts + the SSE branch of POST /ai/apps/:id/chat). Real per-token streaming happens for the terminal answer LLM node (via onAnswerToken); other cases buffer-then-chunk the completed answer. A NOTE in chat-stream.ts still frames full per-token streaming as a forward-compatible target.
  • Multi-turn conversation persistence (services/conversation.service.ts) — channel-agnostic memory over ai_conversations + ai_messages: reuse-or-create a conversation, load a token-budgeted history (AI_HISTORY_TOKEN_BUDGET), append each turn, and roll older turns into a summary once history crosses ~50% of the budget (folded rows are kept, marked meta.folded). All I/O goes through ItemsService, never raw Knex.
  • Lifecycle logging (services/log.service.ts) — ai_app_logs uses a create-at-running → throttled step-timeline updates → finalize-in-finally lifecycle, so even a mid-flight crash leaves a durable trace instead of no log.

Environment ​

VarDefaultPurpose
ASSISTANT_LLM_PROVIDERopenai-compatibleProvider of the env path: openai-compatible, openai, anthropic or google (native only on the vendor host); any other value fails the call.
ASSISTANT_LLM_BASE_URLhttp://localhost:20128/v1Base URL of the env path.
ASSISTANT_LLM_MODELgpt-4o-miniFallback model id.
ASSISTANT_LLM_API_KEY—Fallback key, sent only to ASSISTANT_LLM_BASE_URL (prefer per-integration config.api_key).
ASSISTANT_INGEST_COLLECTIONS`` (off)Comma list of collections auto-synced to KAOS on item change.
ASSISTANT_RATE_LIMIT_PER_MIN20Per-user Agentic quota per minute; below 1 means no limit.
KAOS_BASE_URLhttp://localhost:7071KAOS RAG API base URL.
KAOS_API_KEY—KAOS bearer token.
KAOS_IDENTITY_ID / KAOS_IDENTITY_KIND / KAOS_IDENTITY_SCOPESodp / service / knowledge:jobs:write,knowledge:readKAOS identity used for ingest/read.
AI_INGEST_MAX_CONTENT_BYTES15 MBPer-document size guard (below KAOS's body limit).
AI_HISTORY_TOKEN_BUDGET3000Conversation-memory history budget (tokens).
AI_MOCK_RETRIEVALfalseDev-only synthetic retrieval; hard-disabled in production.
AI_ENABLE_CODE_NODEfalseEnable the code flow node (Function() eval — not a sandbox).
REDIS_ENABLED—Required for the background ingest queue.

ODP Internal API Documentation