Getting Started

Architecture

Ultron is a Next.js application deployed on a containerized cloud runtime, alongside a set of scheduled workers running at the edge. The application runtime orchestrates a 57-skill registry through a tool-calling chat loop. The edge workers handle scheduled work, the marketing swarm, and an inference relay that backs the sandbox's specialist model tier.

Updated today

Overview

At a glance

Frontend: Next.js 14 App Router, output: standalone
Host: A containerized cloud runtime
Build: A build pipeline that ships a new container on every release (multi-stage Docker)
Chat handler: src/app/api/chat/v2/route.ts
Skill registry: 57 skills in src/lib/skills/registry.ts
Skill engine: src/lib/skills/engine.ts
Models: Four tiers — Minuet, Allegro, Forte, Plinth — each mapped to a distinct role
Sandbox: An isolated per-conversation sandbox on managed Kubernetes, with a live remote desktop view
Postgres: Supabase with pgvector for embeddings
Edge: Scheduled workers running alongside the app: cron, swarm, and an inference relay

Ultron runs two coordinated planes. The application plane is a single Next.js container deployed on a containerized cloud runtime; everything a user can touch from the chat surface routes through it. The edge plane is a set of scheduled workers that pick up work the application plane is not the right place for: scheduled jobs, the marketing swarm renderer, and the inference relay used by the sandbox.

Both planes share state. Supabase Postgres is the canonical store for users, conversations, messages, skills metadata, memory, billing, and pause points. Object storage holds binary assets: brain file uploads, carousel exemplar PNGs, the carousel component index. A small database stores the carousel engine's own catalog. A vector index stores the 768-dimension brief embeddings the swarm uses for exemplar retrieval.

Runtimes

Three execution environments, each picked for a specific reason.

Runtime	Where	What runs there	Why
Application runtime	Containerized cloud runtime	Next.js app, chat handler, every interactive API, cron endpoints	Heavy Node runtime with playwright-core, the model SDK, Supabase service-role calls. Needs more headroom than an edge worker's tight memory limit and short execution budget.
Edge worker	Edge	Scheduled cron, marketing swarm pipeline, inference relay	Cheap, globally distributed, native scheduled triggers, native bindings to storage, database, and vector-index services.
Sandbox	Per user, on demand	Bundled engineering skills (CLI coding agent, playwright, ffmpeg)	Untrusted code execution with a real Linux desktop the user can watch over a live remote desktop view.

Note

Cron schedules live in cf-workers/ultron-cron/wrangler.toml, not in the application runtime. The worker fires on schedule, then POSTs to the application runtime with the shared WEBHOOK_SECRET. This keeps the schedule discoverable in one place and isolates a misbehaving cron from the rest of the app.

Deploy stack

What ships when you push to the production branch.

Container build pipeline

A push to the production branch triggers a build pipeline defined in deploy-pipeline.yaml. The build produces a container image tagged with the commit SHA, pushes it to the image registry, and rolls it out as the latest revision.

deploy-pipeline.yamlyaml

1# Build pipeline configuration for Ultron's container image.

2# NEXT_PUBLIC_* env vars are passed as --build-arg because webpack bakes

3# them into the client bundle at build time.

4steps:

5 - name: build-and-push

6 args:

7 - build

8 - --tag

9 - ${_IMAGE_REPO}/${_SERVICE_NAME}:${COMMIT_SHA}

10 - --build-arg

11 - NEXT_PUBLIC_SUPABASE_URL=${_NEXT_PUBLIC_SUPABASE_URL}

12 - .

Dockerfiledockerfile

1# Multi-stage Dockerfile for the container build.

2# - deps: install npm dependencies (cached layer)

3# - build: compile Next.js with output: standalone

4# - run: minimal runtime with the standalone server + static assets

5#

6# Playwright system libs are installed because src/lib/reddit.ts

7# dynamic-imports playwright-core for the residential-proxy fallback.

9FROM node:20-slim AS deps

10WORKDIR /app

11RUN apt-get update && apt-get install -y --no-install-recommends \

12 fonts-liberation libnss3 libatk-bridge2.0-0 libdrm2 libxkbcommon0 \

13 libxcomposite1 libxdamage1 libxfixes3 libxrandr2 libgbm1

Worker deployment

Each worker has its own wrangler.toml. Deployment is wrangler deploy from the worker's directory; secrets are set with wrangler secret put.

cf-workers/ultron-cron/wrangler.tomltoml

1name = "ultron-cron"

2main = "src/index.ts"

3compatibility_date = "2026-04-01"

4compatibility_flags = ["nodejs_compat"]

6[triggers]

7crons = [

8 "0 9 * * *", # daily lifecycle emails

9 "* * * * *", # every minute - job reconciler safety net

10]

12[vars]

13ULTRON_BASE_URL = "https://app.51ultron.com"

Request path

A user message travels through five layers before the assistant responds. Each layer is one file you can open and read.

src/app/api/chat/v2/route.tsts

97// 1. Authenticate and load the conversation

98const { supabase, userId } = await getAuth(req)

99const conversation = await loadConversation(conversationId, userId)

100

101// 2. Pick the model tier based on the message + skill hint

102const { model, tier } = await routeModel(message, conversation)

103

104// 3. Build the tool catalog for this turn

105const tools = await buildToolCatalog({ supabase, userId, conversation })

106

107// 4. Run the tool-calling loop until it stops

108const result = await runTurn({ message, conversation, model, tools })

109

110// 5. Persist the final assistant message and any side effects

111await persistTurn(result)

The handler is structured so each step is its own function. You can open src/app/api/chat/v2/route.ts, scroll to the body of POST, and read top to bottom for the exact path. Tool execution lives in src/lib/skills/engine.ts for skill runs and src/lib/chat-tools.ts for the chat-level native tools.

Where to read more

Each step has a dedicated page. Turn lifecycle covers the loop. Resume and approvals covers HITL. Execution lifecycle covers the skill engine specifically.

Skills

A skill is the unit of work. Skills are loaded from a flat registry; each declares a manifest with a model tier, a turn budget, and a tool allow list.

src/lib/skills/types.tsts

14export interface SkillMeta {

15 id: string // kebab-case, unique

16 name: string // human label

17 category: SkillCategory // one of 8 categories

18 tier: "minuet" | "allegro" | "forte"

19 maxTurns: number // hard cap on the tool-call loop

20 tools: string[] // allow-list of tool names

21 system?: string // skill-specific system prompt

22}

The engine reads the tier and resolves it to a concrete model id at request time. The current mapping is hard-coded in src/lib/skills/engine.ts:

src/lib/skills/engine.tsts

34const MODEL_MAP: Record<string, string> = {

35 minuet: MINUET_MODEL_ID,

36 allegro: ALLEGRO_MODEL_ID,

37 forte: FORTE_MODEL_ID,

38}

40// Engine constants (see module header for the full list)

41const CHAT_COMPLETIONS_URL = resolveEndpointForTier(tier)

42const MAX_CONCURRENT_SKILLS = 2

43const REQUEST_TIMEOUT_MS = 180_000 // 3 min for deep research / multi-canvas

Category	Count	Examples
research	8	`competitive-analysis, market-sizing`
lead-gen	7	`vc-prospector, lead-enrichment`
sales	9	`cold-outreach, pipeline-review`
content	11	`content-pulse, ad-copy`
ops	7	`morning-briefing, churn-save`
legal	5	`contract-draft, clause-library`
design	5	`brand-system, asset-batch`
paid-ads	4	`audience-builder, creative-test`

Tip

Personas like cortex, specter, striker, pulse, sentinel are UI grouping labels that bundle multiple skills under a single slash command. They are not separate runtime entities.

Tools

Three families of tools are exposed to the model on every turn. The registry dispatches in that order on every tool call.

Provider modules

Each module exports a ToolModule. The dispatcher checks modules before falling through to the legacy switch in chat-tools.ts. Two modules are wired today: hitl.ts (approvals) and stripe.ts (billing).

src/lib/tools/types.tsts

50export interface ToolModule {

51 provider: string

52 schemas: OpenAIFunctionSchema[]

53 executors: Record<string, ToolExecutor>

54 statusLabels?: Record<string, StatusLabelFn>

55}

Native catalog

The native catalog at src/lib/chat-tools.ts exposes 91 tools across web research, CRM, email, calendar, content, integrations, code execution, and brain access. Each tool is a tool-call function schema paired with an async executor that receives the tool context.

External MCP servers

Servers expose tools over JSON-RPC 2.0. Ultron treats them as a pluggable layer: each server is registered with a name prefix, an auth header, and an optional per-user scoping. Tools discovered from external servers are cached for ten minutes per user.

src/lib/mcp-client.tsts

120// Per-user scoping example: the URL embeds the workspaceCode so each

121// user sees a different tool catalog from the same MCP endpoint.

122const url = config.url.replace("{workspaceCode}", encodeURIComponent(workspaceCode))

123const headers: HeadersInit = { "content-type": "application/json" }

124if (config.auth?.type === "bearer") {

125 headers["authorization"] = `Bearer ${env[config.auth.envVar]}`

126} else if (config.auth?.type === "header") {

127 headers[config.auth.headerName] = env[config.auth.envVar]

128}

Warning

Two users running the same skill may see different tool catalogs depending on which providers they connected.

Memory

Ultron persists state across three storage surfaces, plus a nightly consolidation pass that reviews and prunes what's stored. Retrieval is hybrid: meaning-based search combined with keyword search, then a small step narrows to what's relevant for the turn.

Surface	Backing store	What it holds	Read path
Notes & auto-summaries	Postgres + pgvector	User-written notes, pinned conversation snapshots, automatic end-of-turn summaries	Hybrid retrieval (meaning + keyword), then a relevance pass
Uploaded files	Postgres + Storage	Uploaded documents, chunked and indexed, read-only once stored	Hybrid retrieval over chunks
Artifacts & agreements	Postgres	Tracked artifacts and agreements	Counted in a per-turn inventory so the model knows the corpus exists without reading it all

src/lib/agentMemory.tsts

233// Selection over a manifest of recent rows, narrowed by a lightweight

234// model pass. Avoids a full vector search when embeddings are sparse or absent.

235async function selectRelevantMemories(ctx, query, limit = 5) {

236 const manifest = await fetchManifest(ctx, 80) // id + agent + tags + short preview

237 const selection = await judgeSelect({

238 system: SELECT_SYSTEM,

239 user: JSON.stringify({ query, manifest }),

240 schema: { ids: { type: "array", items: { type: "string" } } },

241 })

242 return hydrateById(ctx, selection.ids.slice(0, limit))

243}

Note

Retrieval never relies on keyword matching alone, or on embedding similarity alone — it combines meaning-based search with keyword search, then a lightweight step picks the handful most relevant to the current turn. A separate nightly consolidation pass also runs automatically, reviewing, merging, and pruning stored memory so it stays useful over time. This is distinct from the runtime compression cascade applied to the conversation buffer before every turn — see Conversation compression for that mechanism.

Workspace

When a skill needs to run code, browse a site, or render a file, Ultron provisions an isolated per-conversation sandbox on managed Kubernetes.

The sandbox comes prebaked with a virtual display, a window manager, and a remote-desktop server. The user opens the workspace at /dashboard/workspace and sees the sandbox through an embedded live remote desktop view.

The sandbox runs a CLI coding agent for the lite tier. To keep cost down, the CLI is pointed at an inference relay (see Scheduled workers below), which translates the CLI's tool-calling protocol into a call against the Plinth tier — a free, specialist model tuned for sandbox and background-agent work. The CLI speaks the same protocol regardless of which tier answers it.

src/app/api/sandbox/start/route.tsts

1// Sandbox states observed by the UI

2type SandboxState =

3 | "no_sandbox"

4 | "booting"

5 | "port_not_listening"

6 | "connected"

7 | "sandbox_dead"

Canvas

Canvas is the rendering surface for skill output. A canvas is a JSON spec that gets compiled at runtime into one of 30+ block types.

The block catalog at src/components/canvas/CanvasBlock.tsx registers blocks like comparison_table, kanban, score_card, code_diff, and structured_doc. The canvas exporter can save the rendered block as PNG, PDF, or push it into the brain as a memory.

Scheduled workers

A small set of scheduled workers run alongside the application runtime. Each is small, isolated, and has its own deployment config.

ultron-cron

Fires scheduled triggers and POSTs to the application runtime with the shared WEBHOOK_SECRET. Two schedules ship today: a daily lifecycle email run at 09:00 UTC and a one-minute reconciler that fails any stuck job.

cf-workers/ultron-cron/src/index.tsts

40const SCHEDULES: ScheduledRoute[] = [

41 {

42 cron: "0 9 * * *",

43 path: "/api/cron/lifecycle-emails",

44 label: "lifecycle-emails (daily 09:00 UTC)",

45 },

46 {

47 // Every minute. Reconciles in-flight jobs (apify_run, sandbox)

48 // against upstream truth so dropped webhooks / Realtime drops

49 // don't leave jobs stuck in 'running' forever.

50 cron: "* * * * *",

51 path: "/api/cron/reconcile-jobs",

52 label: "reconcile-jobs (every minute)",

53 },

54]

marketing-swarm

The carousel engine. A user posts a brief to POST /carousel, the worker retrieves exemplars from a vector index, plans a deck with a lightweight planning model, validates against per-slide budgets, renders to PNG in-worker via Satori plus Resvg WASM, scores each slide with a vision-scoring model, and promotes the run back to the catalog database and vector index if it scores well.

cf-workers/marketing-swarm/wrangler.tomltoml

1name = "ultron-marketing-swarm"

2main = "src/index.ts"

3compatibility_date = "2026-04-01"

5[ai]

6binding = "AI"

8[[r2_buckets]]

9binding = "INDEX"

10bucket_name = "carousel-engine-index"

12[[r2_buckets]]

13binding = "EXEMPLARS"

14bucket_name = "carousel-engine-exemplars"

16[[d1_databases]]

17binding = "DB"

18database_name = "carousel-engine"

20[[vectorize]]

21binding = "BRIEFS"

22index_name = "carousel-briefs"

Note

The renderer used to run as a separate service on the containerized cloud runtime. That service is still in the repo at services/carousel-renderer/ as a cold fallback but is not called in production. The active path is in-worker render-v2.

inference relay

Exposes a POST /v1/messages-shaped tool-calling surface and translates each request into a call against the Plinth tier. The sandbox's CLI coding agent is pointed at the relay via a base-URL override, so it keeps speaking the same tool-calling protocol while the inference is routed to the cheaper specialist tier.

cf-workers/inference-relay/src/index.tsts

1// Auth: the CLI sends x-api-key: <RELAY_SECRET>

2if (req.headers.get("x-api-key") !== env.RELAY_SECRET) {

3 return new Response("unauthorized", { status: 401 })

4}

5// Streaming: emit the same SSE sequence the CLI already expects

6// message_start → content_block_start → content_block_delta…

7// → message_delta → message_stop

8const stream = env.AI.run(PLINTH_MODEL_ID, { messages, stream: true })

Background jobs

Cron is split between the ultron-cron worker (canonical) and a small set of cron endpoints on the application runtime, called from the worker.

Job	Cadence	Trigger source	Purpose
`reconcile-jobs`	Every 60s	ultron-cron worker	Probe sandbox + apify health, fail stuck jobs
`lifecycle-emails`	Daily 09:00 UTC	ultron-cron worker	Onboarding, winback, churn-save sequences
`workspace-janitor`	Daily 03:30 UTC	scheduler → application runtime	Evict idle sandboxes (idle > 30 days)
`publish`	Per minute	scheduler → application runtime	Drain the content publish queue
`automations`	Every 5 min	scheduler → application runtime	Run user-defined automations
`run-scheduled`	Every 15 min	scheduler → application runtime	Kick scheduled skill runs
`sequences / pipeline / workflows / dispatch / task-update`	Variable	scheduler → application runtime	CRM and content workflow advancement
`marketing-swarm`	On demand	User → marketing-swarm worker	Plan, render, and score carousels
`apify-webhook`	Push	Apify → application runtime	Ingest maps and jobs scraper results

Warning

The full list of cron route handlers lives under src/app/api/cron/. Adding a new schedule means adding the entry to SCHEDULES in cf-workers/ultron-cron/src/index.ts AND to the [triggers].crons array in its wrangler.toml. Forgetting one half is a silent failure.

Repository map

The shape of the codebase. Folders you will visit on every page in this section.

src/app

api

chat

v2/route.tschat handler

resume/route.tsapproval resume

brain

notes/route.ts

files/route.ts

sandbox

start/route.ts

reset/route.ts

cron

lifecycle-emails/route.ts

reconcile-jobs/route.ts

workspace-janitor/route.ts

publish/route.ts

automations/route.ts

run-scheduled/route.ts

sequences/route.ts

pipeline/route.ts

workflows/route.ts

dispatch/route.ts

task-update/route.ts

apify-webhook/route.ts

dashboard

chat/page.tsx

brain/page.tsx

workspace/page.tsx

src/lib

skills

registry.ts57 skills

types.tsSkillMeta interface

engine.tsexecution loop

memory.tscascade compression

tools

types.ts

registry.ts

providers

hitl.ts

stripe.ts

chat-tools.ts91 native tools

agentMemory.tshybrid retrieval

brain-tools.ts

mcp-client.ts

cf-workers

ultron-cron

wrangler.toml

src/index.tsscheduled fanout

marketing-swarm

wrangler.toml

src

index.ts

carousel

plan.ts

validate.ts

render-v2/index.ts

qa.ts

promote.ts

inference-relay

wrangler.toml

src/index.tstool-calling protocol ↔ Plinth tier

services

carousel-renderer

Dockerfilecold fallback

deploy-pipeline.yaml

apify-actors

gmaps/

jobs/

email-finder/

deploy-pipeline.yamlcontainer build pipeline

DockerfileNext.js standalone build

Glossary

Skill: A registered unit of work with a manifest, a tool allow list, a model tier, and a turn budget. 57 skills ship in the registry.
Tool: A function exposed to the model during a turn. Tools come from three families: provider modules, native handlers, and external MCP servers.
Turn: One iteration of the tool-calling loop. A skill run usually spans several turns.
Persona: A UI grouping label that bundles related skills under one slash command. Cortex, Specter, Striker, Pulse, and Sentinel are personas.
Pause point: A row written by the wait_for_approval tool. It blocks the turn until the user approves through the UI.
Canvas: A JSON spec rendered into one of 30+ block types. Canvases are the visual output of a skill run.
Sandbox: An isolated per-conversation sandbox on managed Kubernetes where untrusted code, browsers, and renderers execute.
Application runtime: The containerized cloud runtime that hosts Ultron's Next.js app; a build pipeline ships a new container image on every release.
Worker: A scheduled worker running alongside the app. Ultron ships several, including ultron-cron and marketing-swarm.
Inference relay: The worker that exposes a tool-calling surface for the sandbox's CLI coding agent but routes the actual inference to the Plinth tier.