Specification · spot repository

Spot

The spot is an always-on teammate for one workspace of the Triage core. A supervisor wakes a Go agent loop in turns; the loop asks a model and runs MCP tools of the core, ego and connectors. It has no shell, no filesystem tools and no cluster RBAC, and the core has no Kubernetes client.

Model API Anthropic · Gemini · Vertex AI POD · TRIA-SPOT · ONE REPLICA OVER MCP Supervisor wakes, schedule, ceilings, heartbeat Agent loop empty conversation per wake Tools MCP, schedule_next, finish /state/supervisor usage, cursors, status Spans OTLP/HTTP, per turn Core /mcp issues, memory, rooms, workspace chat Connectors MCP servers, own identity ego MCP reads OTLP spans requests and answers trigger stream and list_open_issues Turn: model requests and tool calls MCP calls and the trigger stream Spans over OTLP/HTTP
Figure 1. Architecture. The model API is outside the pod; the core, ego and connectors are reached over MCP and hold their own credentials. Only the supervisor reads the core between turns.
01

The loop

A wake is why the supervisor starts a turn. Every wake is an empty conversation: nothing is resumed, so a model call never pays again for earlier tool output. Continuity is what the spot reads at the start of a wake and the core’s memory (upsert_memory_item, search_memory). A pod with an empty /state loses only the day’s token count and the room cursors. A workspace is one tenant of the core (ADR-095 lives there, ADR-098 in docs/adr); ego scores agents from their telemetry; a connector is an MCP server holding its own credentials.

TermMeaning
wakeReason: startup, tick (the agent scheduled it), edge (the core changed) or human (someone wrote). The wake prompt is the first user message: reason, facts, token budget left.
turnOne run of the agent loop: core tools first, then connector reads; verify before any write; record on the issue.
Wake prompt reason, facts, budget Model request cached prefix tool_use model asks for tools Tool results one user message schedule_next {after, why} finish {summary}, optional answer without tool calls next request LIMITS PER TURN 20m --turn-timeout 40 --max-steps 16000 max_tokens 60s --tool-timeout 2m to 2h schedule clamp optional finish summary WHEN IT GOES WRONG cut off timeout: next wake 2 retries 429 and 5xx, SDK continue once max_tokens stop 32768 B result cap, cut = min-wake no valid schedule tool error not a failure
Figure 2. One turn. All results of one answer go back in one user message. The turn ends when the model answers without tools, calls finish, or hits a cap; the schedule is the last call.
A turn endsResult
end_turnThe model ends its answer.
finish{"summary": "<one line>"}; the line shows in the status file. Optional.
schedule_next{"after": "15m", "why": "check the rollout"}, last in the turn, clamped to [--min-wake, --max-wake]. Never called, or not a duration: the next wake is --min-wake and the supervisor logs why.
40 requests--max-steps. Not a failure: stepsExhausted.
20m--turn-timeout. Failed; reported as cut off on the next wake.
SIGTERMCancels the turn, counts the tokens seen, marks it cut off, flushes the spans, writes status.json, exits.
failureAn API error after the SDK’s two retries, a refusal, an answer cut at the output cap twice, a timeout, no core tools. The next wake backs off from --min-wake, doubling to --max-wake; edges wait too. A 401 or 403 sets authFailed. A failing tool is not a failed turn.

Between turns

No model is used. With --core-api-url the core pushes content-free frames over a websocket (/v2/messenger/spot/ws); the supervisor reads the core after a 1.5 s pause, so a burst is read once. While the stream is down, or without the flag, it polls every --poll-interval; status.json triggers says stream or poll.

Tick the agent’s schedule_next Trigger stream frames, else poll Issue edges new, reopened, severity Room messages named, person, machine Held edges at most 20 machine ones go first Wake gate cooldown, hourly cap, backoff, ceiling Turn wake prompt, then the agent loop failed before the model saw it: put back, dropped after 3 CEILINGS AND CADENCE 2000000 daily tokens, UTC 12 / hour early wakes 1m edge cooldown 10m chat batch, 0 = now 30s poll, stream down 5 s live message gap AFTER EVERY SUCCESSFUL TURN heartbeat memory note, stale 2x sleep status.json state, lastTurn, nextWake turn frame tokens to the core ledger stats frame also every 2 minutes
Figure 3. Between turns. The supervisor uses no model. A wake whose turn failed before its first model request goes back into the queue; an edge put back three times is dropped, with a log line naming its kind.
EdgeRule
issue changeNew issue, reopen, severity change or new comment (list_open_issues). Wakes after --edge-cooldown. Changes during a turn are held; comment counts are re-baselined, since the agent comments itself.
message naming the spot@<name> or the name. Wakes 5 s after the last turn.
person’s other messageWaits up to --chat-batch (10m).
machine messageNever wakes alone; a fact in the next prompt. Hop count 2 or more is dropped.
held edgesAt most 20; machine messages go first, then the oldest. Past the hourly cap, or inside the backoff, edges wait for the scheduled wake.
room cursorsSaved in /state/supervisor; a message sent while the pod was down is news on start. A room’s first read only sets the cursor.
failed wakeFailed before its first model request (no core tools, rejected credential, timeout): its facts are put back. An edge put back three times is dropped.
heartbeatupsert_memory_item, kind note, stale_after twice the next sleep (at least 5m), refreshed after every successful turn. A failed turn leaves it, so reflection resumes.
core downNo core connection or tool list fails the turn (coreDown); the model never starts without its tools.

A message while a turn runs

With the stream up, a person’s message is handed to the running turn: the model sees it with the next tool results, or before the turn would end. Other messages, and one the model never saw, wait for the next wake.

02

The model

All model traffic goes through internal/llm; this section is the Anthropic provider. The request is built for a prefix cache: sorted tools, then the system prompt with a one-hour cache_control breakpoint on its last block, and a short breakpoint on the latest message.

Tools sorted by name, schemas as written System prompt last block: 1 h cache breakpoint First user message time, wake, budget, steps Latest message short breakpoint CACHED PREFIX · SAME FOR WAKES UP TO 1 H APART VOLATILE STEPS OF A TURN
Figure 4. Request layout. Nothing volatile is in the prefix, so wakes up to an hour apart read it from the cache (the schedule is 2m to 2h). Cache reads and writes are recorded in the usage and in the spans. Gemini caches the same prefix implicitly.
SettingRule
--modelclaude-opus-5-5 when empty; on Vertex AI the same id, no prefix.
--effortlow, medium (default), high, xhigh, max; sent as output_config.effort.
thinkingNot configured: the model thinks adaptively, and its thinking blocks go back with the tool results unchanged.
tool_choiceNot sent (auto).
max_tokens16000 per request, not streamed. A max_tokens stop is answered once with “continue” (a tool call cut in the middle is dropped); a second one fails the turn.
refusalEnds the turn as failed with the policy category in the error.
pause_turnResent once.
429, 5xxRetried twice by the SDK, then the turn fails.
03

Model providers

The loop talks to llm.Client, with two implementations: Anthropic (internal/llm/anthropic.go) and Gemini (internal/llm/gemini.go, Google GenAI Go SDK). --llm-provider (chart llm.provider) picks one; the default is anthropic.

anthropicgemini
Auth modes (--llm-auth)key, federation, vertexkey, vertex; federation is refused
Vertex AI location--vertex-region--vertex-region (one flag for both; global by default)
RetriesSDK, twice (429, 5xx)SDK, three tries (408, 429, 5xx)
CachingExplicit breakpointsImplicit on 2.5 and later; explicit CachedContent is a follow-up
GeminiRule
keyA file passed to the SDK explicitly. An empty or missing file stops the start, so GOOGLE_API_KEY and GEMINI_API_KEY are never read. No variable supplies the key, project, location or host (GOOGLE_GEMINI_BASE_URL cannot redirect).
Vertex AIApplication Default Credentials, the pod’s Workload Identity; no key file.
requestOne system instruction; one Tool with a function declaration per tool, sorted, schema unchanged (parametersJsonSchema); tool choice automatic; results as function responses ({"result": ...} or {"error": ...}).
thought signatureGemini 3 returns one on function calls and needs it back unchanged; kept on the block. A call with no id gets a call_<n> id, never sent back.
stopsA function call is a tool use. STOP ends the turn; MAX_TOKENS is the cut-off case. SAFETY, RECITATION, BLOCKLIST, PROHIBITED_CONTENT, SPII and a blocked prompt are a refusal with the reason as category.
modelsGemini 2.0 and earlier reject a thinking config; not every Gemini 3 model accepts every level.
chartllm.provider; llm.model (empty = the provider’s default, then no --model is passed); secrets and Vertex values as in section 16. Egress networkPolicy.google is required for gemini in any mode and for anthropic with vertex.
--effortgemini-3* (thinking level)other models (thinking budget, tokens)
lowLOW1024
mediumMEDIUM4096
highHIGH8192
xhigh, maxHIGH-1 (dynamic)
04

Tools

The model is offered tools named <server>__<tool>, sorted, with the server’s JSON input schema passed through as written. The core’s token is read from --core-token-file on every request.

GroupToolsRule
core, allowed
(tools.CoreTools)
get_triage_instructions get_tenant_situation get_tenant_health get_tenant_goals list_open_issues list_issues get_issue_detail get_issue_graph get_recent_events search_memory search_corpus get_memory get_memory_context get_memory_items get_memory_cell fetch_memory get_world_context get_service_analytics get_service_prediction inspect_service list_services list_applications list_pattern_proposals analyze_issue verify_hypothesis comment_on_issue upsert_memory_item note_observation say_in_workspace read_workspace_room read_workspace_file set_workspace_status set_spot_statusAllowed by exact name. --core-tools all offers every tool the core offers: an operator’s choice, bounded by the core’s Role (core.verbs).
core, denied
(tools.CoreToolsDenied)
resolve_issue forget_issue update_issue_severity link_issues set_tenant_goals run_rsi emit_event classify_event decide_event guardrail_event test_notify test_push approve_pattern link_pattern recognize_pattern expire_memory_item refute_memory_item supersede_memory_item consolidate_memory build_memory build_knowledge build_rcaRefused by name, even if also in the allow list.
ego
(tools.EgoTools)
list_metrics list_sessions summarize_session explain_session list_runs get_run_results get_usage get_quality explain_regression get_findings list_correctionsRead-only: sessions, usage, quality, findings. Offered when --ego-mcp-url is set.
built-inschedule_next finishThe spot’s own; no prefix.
connectorsthe names in the connector’s toolsBy exact name only.
Tool callRule
timeout--tool-timeout (60s).
resultThe text blocks joined; an image or a resource becomes a one-line placeholder. Past --tool-result-max-bytes (32768) it is cut at a character boundary and ends with a line saying how many bytes were dropped.
errorThe server’s isError, a transport error, a timeout or an unknown tool goes back to the model as an error result and never aborts the turn.
connectionOpened at the start of each turn. A core that cannot be reached fails the turn (coreDown). A connector or ego that cannot be reached is logged and left out of that turn, and the wake prompt names it.
05

Names

NameRule
--nameThe deployment’s technical name: Deployment, volume, ServiceAccount, memory-note tags, ego thread. People never see it.
spotNameSet on the workspace in the core; empty means <workspace name>-spot. Read with get_spot_identity at start, when the trigger stream reconnects, at most every five minutes while it works, and when a person writes an @ it does not recognise.
--chat-nameFallback for a core that cannot say.
wake promptSays “In the messenger you appear as ...”. A message naming that name, the technical name or --chat-name wakes the spot at once.
06

Live conversation

BehaviourRule
typingDuring a turn the supervisor shows “typing...” through set_workspace_typing (renewed every 20 s, cleared at the end): in the team room for a turn a person started, else in the agents room. Without the tool only the indicator is lost.
answer firstThe prompt tells the spot to reply before it investigates, to slice long work (report, note the rest, wake itself in two minutes), and to report when done.
fast wakesA message that wakes the spot (it names it, or --chat-batch has passed; 0s = every message) waits only 5 s after the last turn. Lower --poll-interval to notice sooner.
filesA message that carries a file is marked in the wake; the spot looks at it with read_workspace_file (an image comes back as an image).
07

Flags

Flags only. No credential is a flag value or an environment variable. tria-spot version prints the build.

Core and tools

FlagDefaultMeaning
--tenantrequiredtenant this spot works for
--name<tenant>-spotmessenger and machine identity
--chat-nameemptyname shown until the core says
--core-mcp-urlrequiredthe core’s /mcp (streamable HTTP)
--core-api-urlemptythe core’s HTTP API; enables the trigger websocket
--core-token-filerequiredcore bearer token file
--core-toolscuratedcurated or all (section 4)
--connectors-filenoneJSON list of connectors (section 17)
--ego-mcp-urlemptyego’s MCP endpoint; uses the --otel-token-file token

Model

FlagDefaultMeaning
--llm-provideranthropicanthropic or gemini (section 3)
--llm-authkeykey, federation (anthropic only) or vertex (section 8)
--anthropic-key-filerequired with anthropic and keyAnthropic API key file; rejected with the others
--gemini-key-filerequired with gemini and keyGemini API key file; rejected with the others
--anthropic-identity-token-filerequired with federationprojected identity token (JWT)
--anthropic-federation-rule-idrequired with federationfederation rule id (fdrl_...)
--anthropic-org-idrequired with federationAnthropic organization id
--anthropic-service-account-idrequired with federationAnthropic service account id (svac_...)
--anthropic-workspace-idemptyworkspace id (wrkspc_...), optional
--vertex-projectrequired with vertexGoogle Cloud project serving the model
--vertex-regionglobalVertex AI region, either provider
--modelemptyempty = claude-opus-5-5 or gemini-3-flash-preview
--effortmediumlow, medium, high, xhigh or max

Turns and supervisor

FlagDefaultMeaning
--max-steps40model requests per wake
--turn-timeout20mwall-clock cap per wake
--tool-timeout60scap on one tool call
--tool-result-max-bytes32768cap on one tool result the model sees
--state-dir/statethe supervisor’s files: usage, cursors, status
--min-wake / --max-wake2m / 2hclamp for the agent’s schedule
--daily-token-ceiling2000000tokens per UTC day; 0 = none
--poll-interval30spoll interval, while the stream is down
--edge-cooldown1mgap between a turn and an edge wake
--chat-batch10mwait of a message that does not name the spot; 0 = at once
--max-edge-wakes-per-hour12early wakes per hour; 0 = no cap
--heartbeat-idsystem2-heartbeat-<tenant>memory cell id of the heartbeat

Telemetry and logs

FlagDefaultMeaning
--otel-endpointemptyOTLP/HTTP endpoint; empty = off
--otel-token-fileemptysender’s bearer token file
--otel-tenantthe spot’s nametenant label in ego
--otel-tenant-attrtriage.tenantattribute ego reads it from
--otel-tenant-namethe labelname ego shows
--otel-contenttruecontent on the spans; =false = names, timings, counts
--log-levelinfodebug, info, warn, error
--log-contentoffdebug logs carry message, issue, tool input, model text
08

Credentials

--llm-auth picks the model auth mode; federation is Anthropic only. Federation ids are not secrets, but they are flags.

SOURCE FILE ON DISK READER Secret llm.keySecret · llm.geminiKeySecret key file --anthropic-key-file · --gemini-key-file Model client API key, re-read per request Projected token audience https://api.anthropic.com identity token file /var/run/secrets/anthropic.com/token Anthropic SDK exchanges it, short-lived token ServiceAccount annotation iam.gke.io/gcp-service-account no file Application Default Credentials Model client Vertex AI, Workload Identity Projected token or Secret audience triage-core · core.identity --core-token-file kubelet-rotated · Secret mode 0440 Core client MCP and trigger stream Secret telemetry.tokenSecret --otel-token-file read on every export OTLP exporter and ego sender’s bearer token Connector credentials held by the connector, not the spot none in the pod no file, no env, no argument Connector its own identity and limits NO CREDENTIAL IS A FLAG VALUE, AN ARGUMENT OR AN ENVIRONMENT VARIABLE
Figure 5. Credentials. Every secret is a file read again on every request, so a rotated Secret needs no restart. Vertex AI mounts nothing: the pod’s Workload Identity answers.
CredentialHow it flows
Key (key, default)A Secret file (--anthropic-key-file or --gemini-key-file for Gemini), re-read on every request. The SDK is built without its environment defaults, so a stray ANTHROPIC_* variable cannot replace the key.
Federation (federation)No Anthropic secret. A projected token (audience https://api.anthropic.com, at /var/run/secrets/anthropic.com/token) is exchanged by the SDK for a short-lived one. The SDK reads federation from the environment, so the process sets ANTHROPIC_FEDERATION_RULE_ID, ANTHROPIC_ORGANIZATION_ID, ANTHROPIC_SERVICE_ACCOUNT_ID, ANTHROPIC_IDENTITY_TOKEN_FILE and, if set, ANTHROPIC_WORKSPACE_ID from its flags: ids and a path, never a credential. ANTHROPIC_API_KEY, ANTHROPIC_AUTH_TOKEN, ANTHROPIC_IDENTITY_TOKEN and ANTHROPIC_PROFILE are unset, since each outranks federation. --anthropic-key-file is refused.
Vertex AI (vertex)Application Default Credentials: the pod’s Workload Identity. Annotate the ServiceAccount (serviceAccount.annotations, iam.gke.io/gcp-service-account) with a Google service account allowed on Vertex AI in --vertex-project. The model id has no prefix.
core tokenA file read on every request: the pod’s projected token, audience triage-core (core.identity: serviceAccount), or a Secret (secret).
pod tokenThe default ServiceAccount token is not mounted (automountServiceAccountToken: false).

The identity token must be fresh at every exchange

The projected token carries a jti claim, and Anthropic rejects one it has seen (jti_reused in the Console’s authentication history). The kubelet rewrites the file at about 80% of tokenExpirationSeconds (600 s, the minimum) and the SDK reads it on every exchange, so a second exchange in the same token’s life can send a used token. Whether the SDK’s cached access token lasts long enough is not tested against the real API, and not worked around.

SignalContent
log and status.json lastErrorOn an authentication-looking failure in federation mode: “Anthropic authentication failed in federation mode; check the authentication history in the Claude Console (jti_reused / match_*)”.
the 401Authentication failed gives no reason; only the Console history does: jti_reused, match_subject_prefix and the other match_* rules, workspace_id_required.
09

Boundary

The spot holds no cluster RBAC; a cluster is reached through a connector, whose own identity is the boundary. The model has no web tool, no subagents, no shell and no file tool, because none exists in internal/tools. The model cannot read a mounted token. The supervisor’s counters are held in memory.

THE SPOT HAS THE SPOT NEVER HAS Core MCP tools curated by exact name; destructive names refused Shell no tool runs a command; the image has none ego MCP tools read-only: sessions, usage, quality, findings Filesystem tools none in internal/tools Connector tools only the names listed in the connector Web tool, subagents none exists in internal/tools The model Anthropic, Gemini or Vertex AI Cluster RBAC a cluster is reached only through a connector Spans OTLP to ego, content optional Kubernetes client in the core the core gets none schedule_next and finish the spot’s own two tools Secret rules, ClusterRole the chart renders neither WHAT BOUNDS IT r + w, one tenant core Role, core.verbs spend limit on the model credential NetworkPolicy model, core, connectors, OTLP connector identity its RBAC or IAM, network
Figure 6. Boundary. Only offered tools exist; a call to any other name is answered with an error and never sent. The prompt is best effort; the lists and the connector hold.

Identity towards the core

core.identityBehaviour
serviceAccount (default)The spot’s own ServiceAccount: a projected token, audience triage-core, no core Secret, validated by TokenReview. The chart renders a Role and RoleBinding <spot name>-core, labelled triageagent.dev/managed: "true", granting core.verbs (default get, list, create, update) on the one tenant; core.rbac.create: false if the grant is made elsewhere. The core accepts only ServiceAccounts from its own namespace, so the chart is installed there.
secretA bearer token from the Secret core.tokenSecret, usable from any namespace.
nextA per-spot minted token (ADR-098).
10

Kill switches

Any one is enough.

SwitchEffect
spot.enabled=falseThe chart renders nothing.
delete the DeploymentThe pod stops.
delete the connector’s egress, or the connectorThe spot can read and write nothing in the cluster.
revoke the core tokenThe spot can neither read nor speak.
spend limit on the model credentialStops the model regardless of the pod.
11

Status

/state/supervisor/status.json is world-readable and holds no secrets.

FieldContent
statestarting, running, idle, paused-ceiling, stopping, stopped; when it started and was last updated.
lastTurnWhen, why, seconds, billable tokens, steps (model requests), toolCalls, model, the finish summary, contextTokens (conversation size at the last request), and the flags isError, stepsExhausted, timedOut, authFailed, coreDown.
nextWakeTime, why, source (agent, agent-clamped, default-missing, default-garbled, ceiling) and a note when it was changed.
tokensTodayAgainst the ceiling, with cache reads and the turn count.
lastError, consecutiveFailures, coreErrorThe failure state.
otherHeartbeat time and id, edges held, early wakes in the last hour, how the core is watched (stream or poll).
12

Image

ItemValue
buildko only; no Dockerfile and no Docker.
basegcr.io/distroless/static pinned by digest in .ko.yaml: CA certificates only, no shell, no package manager.
commandsmake image writes an OCI layout to bin/image and pushes nothing; make push REGISTRY=<registry> pushes.
runtimeuid 10001, read-only root filesystem, the runtime’s default seccomp profile; every write goes to /state or /tmp.
13

Traces

With telemetry.enabled (--otel-endpoint) the spot exports OpenTelemetry spans over OTLP/HTTP, in batches. The bearer token is a Secret file (--otel-token-file), read on every export. The tenant in ego is <sender>/<label>, the sender coming from the token.

ONE TURN invoke_agent <name> gen_ai.conversation.id: a fresh UUID per turn · tria.wake.reason chat <model> one per model request execute_tool one per tool call execute_tool one per tool call chat <model> one per model request execute_tool one per tool call chat <model> one per model request ATTRIBUTE LIMITS 64 KiB cap per attribute value oldest first message arrays lose oldest messages --otel-content=false names, timings, tokens only
Figure 7. Spans of one turn, exported over OTLP/HTTP and flushed at the end of each turn and at shutdown.

Resource attributes

AttributeValue
service.name, gen_ai.agent.name, ego.threadThe spot’s name; every wake is a segment of one thread.
--otel-tenant-attr (triage.tenant, ego’s --usage-tenant-attr)The tenant label, with the value --otel-tenant (default the spot’s name, so each spot is its own tenant in ego).
ego.tenant.name--otel-tenant-name (default the label). Ego keeps the first name it sees and drops an invalid one silently, so the spot refuses a name over 80 characters, or with control characters, commas or equals signs.

Spans

SpanAttributesWith content
invoke_agent <name>gen_ai.operation.name, gen_ai.conversation.id (fresh UUID per turn), gen_ai.agent.name, tria.wake.reasongen_ai.input.messages (the wake prompt) and gen_ai.output.messages (the final text).
chat <model>gen_ai.provider.name (anthropic or gemini), request and response model, gen_ai.usage.input_tokens (whole prompt: uncached input plus cache reads plus cache writes, as ego counts a call), output_tokens, cache_read.input_tokens and cache_creation.input_tokens (the split ego prices with), gen_ai.response.finish_reasonsgen_ai.input.messages: the request’s messages as a JSON array in the GenAI shape (role, parts of text, tool_call, tool_call_response; thinking left out); gen_ai.output.messages. The first chat of a turn also has gen_ai.system_instructions.
execute_tool <name>gen_ai.tool.name, gen_ai.tool.call.id, an error status when the tool erroredgen_ai.tool.call.arguments (a JSON object string) and gen_ai.tool.call.result ({"result": <text>, "isError": <bool>} as a string).

Content. By default (telemetry.content: true, --otel-content) the spans carry prompts, answers and every tool call’s arguments and result, because ego cannot evaluate a turn without them. With telemetry.content: false only names, timings and token counts go out and ego reports a turn it cannot score; use it when the workspace’s data must not leave the pod. A long string is cut in the middle with a marker. --log-content governs only the process log. Traces carry no spend control.

14

Spend

ControlRule
--daily-token-ceilingInput plus output plus cache-write tokens per UTC day (cache reads not counted), held in memory and written through to usage.json, read once at start. At the ceiling no wake starts, edges are held, the loop sleeps to UTC midnight. A turn in flight can overshoot by one turn (bounded by --max-steps, --turn-timeout).
contextEvery wake starts empty; status.json shows each turn’s contextTokens.
the core’s capPolicy budgets.spotTenantDailyTokens over budgets.spotDailyDefaultTokens: new input plus output per workspace per UTC day. After every turn a turn frame goes up the trigger stream (model, input incl. cache reads and writes, cached, output, duration, failed); the core books it as usage kind spot (TUI usage panel) and answers with the budget. Over the cap it refuses say_in_workspace and set_workspace_status, and the supervisor pauses all wakes until the budget lifts or UTC midnight. Frames sent while the stream is down wait (the last 100 are kept).
stats frameState, turns and wakes by reason today, failures in a row, context size, own token counts, next wake. After each turn, on a trigger-mode change and every two minutes, for the TUI’s workspace panel. Counts and labels only.
dollarsNo ceiling: the spot has no price table. The spend limit on the Anthropic workspace or Google project that serves the model is the real stop.
15

Logs

Logs carry what happened, never what was said.

KindRule
alwaysCounts, kinds, ids, tool names, token numbers and timings, at every level: an edges line says kinds="room-message=1" human=true, not the message. An error from the model API or a tool is operational and is logged, because it diagnoses a failed login.
contentA person’s message or issue text, tool input and the model’s words reach the log only with --log-content (chart agent.logContent) at debug level.
level--log-level: debug, info (default), warn, error.
model textstatus.json nextWake.why and lastTurn.summary are written by the model; they stay in the pod’s volume.
16

Chart

charts/tria-spot renders nothing unless spot.enabled=true. The chart fails to render when a required value is missing.

PartContent
workloadDeployment, 1 replica, Recreate, read-only root, non-root, all capabilities dropped, runtime seccomp; a ServiceAccount; 100m/128Mi requested, 500m/512Mi limit.
storageA small PVC for /state; an emptyDir for /tmp.
core identityA projected token with audience triage-core plus the <name>-core Role and RoleBinding, or a Secret mounted as a file, mode 0440 with fsGroup (section 9).
llm.auth: key (default)The Secret from llm.keySecret (anthropic) or llm.geminiKeySecret (gemini, at /secrets/gemini/<key>), as a file.
llm.auth: vertexNothing mounted. llm.vertex.project required, region defaults to global; the binding goes in serviceAccount.annotations.
llm.auth: federationA projected token volume at /var/run/secrets/anthropic.com (tokenExpirationSeconds 600; no Secret). llm.federation.ruleId, orgId and serviceAccountId are required, workspaceId is optional. Refused with gemini.
model valuesllm.provider (anthropic default, gemini), llm.model (empty = the provider’s default, no --model passed), llm.effort, llm.maxSteps, agent.toolTimeout, agent.toolResultMaxBytes.

NetworkPolicy

Egress only; nothing in. It allows cluster DNS, the model API, the core, the connectors and, with telemetry.enabled, the OTLP endpoint from networkPolicy.telemetry. The model API is addressed by CIDR.

DestinationAddressed by
anthropic with key or federationThe CIDRs Anthropic publishes (160.79.104.0/23, 2607:6bc0::/48, platform.claude.com/docs/en/api/ip-addresses).
gemini in any mode; vertex with either providerThe Google API ranges you name in networkPolicy.google.cidrs (required; for example the Private Google Access range). With vertex it adds the node’s metadata server for the Workload Identity token (networkPolicy.google.metadata, GKE’s addresses by default).
coreNamespace and pod selector, or CIDR.

Lint

make chart-lint runs helm lint and helm template with the flag off (must render nothing) and with every values file in charts/tria-spot/ci (placeholders only): key, federation, Vertex AI, Gemini with a key and on Vertex AI, telemetry, ServiceAccount identity, connectors. It checks that:

  • the pod has no env, no rule names Secrets, no ClusterRole renders, and the ServiceAccount token is not mounted;
  • nothing in the pod is loosened for a shell, and each mode mounts and opens only its own;
  • a missing required value or an unknown mode fails to render.
17

Connectors

A connector is one more MCP server beside the core, reached by URL, holding its own credentials and limits: a Kubernetes tool server, a code host, a metrics API. The spot has no code for any of them and no credential for them.

connectors:
  - name: example-kube
    url: http://tools.example.svc:8084/mcp
    tools: [k8s_get_resources, k8s_describe_resource, k8s_get_pod_logs]
    hint: The workspace's cluster, read-only.
    egress:
      namespaceSelector: {matchLabels: {kubernetes.io/metadata.name: example}}
      podSelector: {matchLabels: {app.kubernetes.io/name: tools}}
      ports: [8084]
FieldRule
listThe chart renders it into a ConfigMap read by --connectors-file: names, URLs, tool names and hints; no credential.
toolsOffered to the model by exact name as <name>__<tool>. The file is refused when a connector names none, uses core or ego, repeats a name, or lists a tool that is not a plain name. A tool the connector offers and tools does not name is not offered, and a call to it is refused.
hintGoes into the prompt’s Connectors section, which also lists the named tools, so the spot knows a tool it was not given is not “missing”. What a connector returns is data, not instructions.
egressOpens the NetworkPolicy to that connector and nothing else; the chart fails to render without it.
trustThe connector’s own business: its identity, its RBAC or IAM, and the network. The list is never the boundary; the connector’s own limits are. A connector is the only way to any cluster.
18

Run it locally

Source: github.com/triageagent-dev/spot. You need a core you can reach and your own credentials in files. Nothing below is a real name.

make build
mkdir -p /tmp/spot/state && umask 077
printf '%s' "$CORE_TOKEN" > /tmp/spot/core-token          # a token for a tenant identity
printf '%s' "$ANTHROPIC_KEY" > /tmp/spot/anthropic-key
./bin/tria-spot --tenant <tenant> \
  --core-mcp-url https://<core-host>/mcp \
  --core-token-file /tmp/spot/core-token --anthropic-key-file /tmp/spot/anthropic-key \
  --state-dir /tmp/spot/state \
  --min-wake 2m --daily-token-ceiling 200000 --log-level debug

The spot reaches no cluster unless --connectors-file names a connector. Ctrl-C stops it gracefully. Check /tmp/spot/state/supervisor/status.json.

19

Layout

DirectoryPurpose
cmd/tria-spotEntrypoint: flags, wiring, version
internal/configFlags, validation, connectors file
internal/llmThe model client (Anthropic SDK: key, federation, Vertex; Google GenAI SDK: key, Vertex), a scripted fake
internal/toolsThe model’s tools: core, ego, connectors over MCP, schedule_next, finish
internal/mcptoolsThe MCP session the tools use (raw tools/list and tools/call)
internal/agentOne turn: the loop, steps, injection, results
internal/telemetrySpans to ego over OTLP/HTTP
internal/promptThe system prompt (instruction.md, embedded) and turn prompts
internal/supervisorThe wake loop, edges, schedule, ceilings, heartbeat, status
internal/scheduleThe schedule_next request, clamped
internal/triggersThe core’s websocket trigger stream
internal/corecliThe supervisor’s own MCP client (list_open_issues, heartbeat, rooms)
internal/edgesEdge detection between issue snapshots
internal/ledger/state/supervisor: daily usage (in memory, written through), room cursors, status
charts/tria-spotHelm chart
docs/adrThe decision records of this repo
20

Not verified

No model API credential was used while this runtime was written. The SDK calls are tested against a local HTTP server and the loop against a scripted fake.

AreaNot verified
full turnAgainst the real API: adaptive thinking blocks round-tripping with tool results, one-hour prefix cache hits across wakes, real usage numbers.
federationThe exchange from a pod with a valid rule, service account and token; how long the access token lives and whether it is reused (this decides whether jti_reused shows up); the SDK’s error on a rejection. The hint matches Authentication failed, authentication_error, jti_reused, token exchange and federation in the error text.
Vertex AIThe Workload Identity binding, the egress ranges and the model id on a real project.
GeminiTested against a local HTTP server only: a full Gemini 3 turn (thought signatures with parallel calls, accepted levels), implicit cache hits across wakes, Gemini on Vertex AI with Workload Identity.
egoHow it maps the span attributes (GenAI conventions are followed).
read_workspace_roomThe reply shape is read loosely (messages object or array; id, sender|from|author, text|body|message, ts|time); re-check when the tool changes.
heartbeatThe ADR says system2-heartbeat-<tenant>; the core looks only for system2-heartbeat, so until it reads the per-tenant id this heartbeat does not change reflection. --heartbeat-id sets either.
commentslist_open_issues gives a comment count, not the author (all MCP comments are mcp-client), so every new comment is a human wake and the agent reads the issue to see who. Comments made during a turn are absorbed by the post-turn baseline.
hop countsHop counts and caps are enforced by the core; the prompt only tells the agent to pass and respect them.
toolingRunning the image, helm install, kubeconform, and golangci-lint (make lint runs it when present).