LLM Node

The llm node is the only node type that calls a language model. Every other node reacts to text, routes on a value, or does I/O — llm is where a workflow actually thinks: answering a question, classifying an intent, extracting a value from free text, or drafting a reply that a downstream response node will send.

See Workflow Overview for the node/edge envelope this fits into.

Config fields

Field	Type	Required	Description
promptTemplate	string	Yes	The Handlebars-templated prompt sent to the model. Can reference any variable in the execution context, e.g. {{userMessage}} or an outputVar written by an earlier node.
outputVar	string	Yes	The context variable the model’s response is written to. Must not collide with a reserved context key (sessionId, tenantId, agentId, interruptGeneration, clientExternalId, clientTurnId, userMessage, conversationHistory, turnCount, agentTimezone, session, attachment, channel, attachmentUrl, attachmentMime, attachmentFilename) or any name starting with _ — the workflow fails validation at save time if it does.
knowledgeBaseId	number	No	Attaches a knowledge base for retrieval-augmented generation. When set, relevant chunks are retrieved for the current userMessage and prepended to the effective system prompt before the model call. If retrieval itself fails (a transient vector-store error), the node fails open — it proceeds without grounding rather than failing the whole node.
systemPrompt	string	No	A static, not Handlebars-rendered system prompt. When set, it’s used verbatim — the rendered promptTemplate is used for token-cost accounting only and is not sent to the model in any form. Leave this unset in almost every case; see the callout below.
modelOverride	string	No	Run this node on a specific model instead of the agent/tenant default, e.g. a cheap model for classification, or a Gemini model for one image node inside an agent that otherwise runs on Haiku. The provider follows the model: naming a model another provider serves runs just this node on that provider. A model no provider serves fails the node. See the callout below.
streamOutput	boolean	No	Controls whether this node’s output streams to the client as it’s generated. Defaults to true (streaming) — you must explicitly set false to disable it. See the callout below.
publishSentences	boolean	No	Added 2026-10-27 (in production). When true, each finished sentence of the reply is sent to the caller or chat while the model is still writing (see Speaking sentences as they are written). Default false. Refused together with outputSchema and with streamOutput: false.
applyVoiceFormatting	boolean	No	Appends spoken-medium formatting rules (no markdown, no bullet points, numbers spelled out, 1–3 sentences) to the system prompt. Intended only for nodes whose output flows to a voice-facing response node. See the callout below — never set this on a node feeding a condition or router.
extractAlphanumericId	boolean	No	When true, the executor discards the model’s raw text and instead stores the first contiguous alphanumeric run of 8 to 20 characters that contains at least one digit (a tracking number, a booking reference) in outputVar, or the literal string NONE when there is none. The digit requirement is deliberate: an ordinary long word is never mistaken for an id. Use it on a node that extracts an id which then feeds an outbound call, and gate the call on != "NONE". Like extractFirstInteger, it needs streamOutput: false.
extractFirstInteger	boolean	No	When true, the executor discards the model’s raw text and instead regex-extracts the first integer found anywhere in the response, storing that as a number (since 2026-09-11 — previously a string) in outputVar — falling back to 0 if no digit is found. See the callout below.
excludeHistory	boolean	No	When true, this node’s underlying model call receives an empty conversation history instead of the session’s real conversationHistory. See the callout below.
allowGeneralKnowledgeFallback	boolean	No	Only relevant when knowledgeBaseId is also set. Softens the retrieval-augmentation directive from “answer using ONLY this reference material” to “use it if relevant, otherwise answer from general knowledge and say so.” See the callout below.
matchAnyOf	string[]	No	For a classification node feeding a router or condition. When set, the executor scans the model’s raw response for the earliest-position candidate label from this list and stores that in outputVar instead of the raw text — falling back to the raw text unchanged if none of the candidates appear anywhere in it. See the callout below.
imageVars	string[]	No	Added 2026-09-17. A list of context variable names whose values are attached to this call as images (vision input), on top of promptTemplate’s text. See the callout below — read it before using this on a node whose provider might not support images.
imageUrlsVar	string	No	Added 2026-10-06. The name of a context variable holding image URLs; the node fetches them itself and attaches them as images. See Fetching images by URL.
imageMarkerHistory	number	No	Added 2026-10-06. Attach the images named by the last N [image:<url>] markers in the user’s messages (1 to 40; up to 40 images per node by default). See Fetching images by URL.
imageUrlsFailPolicy	string	No	Added 2026-10-06. "skip" (default) or "fail": what happens when an image URL requested this turn cannot be used. See Fetching images by URL.
outputSchema	object	No	Added 2026-10-23 (in production since 2026-10-27). A JSON Schema (a supported subset) the reply must match. The node asks the model for a JSON object, checks it, and stores the parsed object in outputVar. See Structured output.
flatten	boolean or string	No	Added 2026-10-23. Only with outputSchema. Also writes each top-level property to its own variable: true gives <outputVar>_<name>, a string gives <prefix><name>. See Structured output.
structuredMode	string	No	Added 2026-10-23. Only with outputSchema. "auto" (default) or "prompt". See How the model is asked.

Gotcha: setting systemPrompt silently drops everything in promptTemplate

Read this before using systemPrompt at all

promptTemplate is required and is where {{...}} data injection happens — variables from earlier nodes, {{userMessage}}, RAG context, all of it. systemPrompt is not rendered through Handlebars and is used instead of, not alongside, the rendered promptTemplate — when both are set, the rendered promptTemplate is computed only for token-cost accounting and never reaches the model in any form, not even as a user message.

In practice: almost every workflow should leave systemPrompt unset entirely. Put your persona/instructions AND any {{ctxVar}}/{{jsonString ctxVar}} data injection together in promptTemplate alone. A node that sets systemPrompt sees none of promptTemplate’s data — only the literal, static systemPrompt string plus {{userMessage}} (passed separately by the runner, not through promptTemplate) and conversation history.

A workflow that queries a tool/action for data and asks an llm node to answer using it is the case most likely to hit this: set systemPrompt to hold grounding instructions and promptTemplate to hold {{jsonString someActionOutput}} plus the question, and the model receives the instructions with zero actual data — it doesn’t error, it just answers from whatever it can infer from the conversation, which for a data-lookup question means a confident, plausible-sounding, entirely fabricated answer. Nothing in the workflow run fails; nodeTrace shows every node COMPLETED. The only way to catch it is checking whether the reply is actually grounded in the real data (see Best Practices).

systemPrompt exists for the rare, narrow case where a node’s instructions must stay completely static across every invocation and genuinely need zero context-variable injection that turn — not as a general “split my persona from my data” mechanism.

Gotcha: streamOutput defaults to true

Unlike most optional booleans in this schema, streamOutput is not off-by-default. The workflow runner dispatches every llm node to its streaming executor unless streamOutput is explicitly set to false. This matches how a direct (non-workflow) chat turn behaves, so a session-facing reply streams sentence-by-sentence to the client by default.

Set streamOutput: false for any llm node whose output isn’t session-facing — a background classification, an extraction step, an evaluation/scoring call. Nothing downstream cares whether that text arrived in one shot or in fragments, and skipping streaming avoids unnecessary per-sentence overhead.

Gotcha: applyVoiceFormatting conflicts with classification prompts

applyVoiceFormatting rewrites the system prompt to say things like “speak naturally, keep it to 1–3 sentences, no lists.” That’s exactly wrong for a node whose job is “respond with ONLY one word: billing, technical, or general” — the two instructions actively fight each other, and you’ll see the model wrap its classification in a sentence instead of returning the bare token your router or condition node needs.

Only set applyVoiceFormatting: true on nodes whose outputVar feeds directly into a response node meant to be read aloud (e.g. a TTS-backed voice agent) — never on a classification or extraction node.

Gotcha: extractFirstInteger is a structural fix, not a prompt trick

Any node used to match a human-given value to an internal numeric ID — a room code to a roomId, a product name to a productId — feeds an LLM’s raw output into a downstream tool or action node’s argsTemplate as a bare, unquoted number. No prompt can guarantee 100% compliance across every model and every call: a well-meaning but off-task reply like “I couldn’t find that room, could you clarify?” breaks argsTemplate’s JSON parsing with a confusing “did not render to valid JSON” error, even after the prompt has been hardened repeatedly.

Setting extractFirstInteger: true sidesteps this: the executor regex-extracts the first integer in the raw response (falling back to 0 — the conventional “no match” sentinel every affected prompt should also document in its own logic) instead of trusting the model to emit only digits. If your workflow needs this pattern, use the flag — don’t keep layering prompt patches on top of it.

Since 2026-09-11, outputVar holds a real JS number, not a digit string. It used to be a string, and that silently broke every graph gating on the sentinel with a condition expression like myVar != 0 or myVar > 0: condition expressions have no type coercion, so "0" != 0 evaluated true and the “no match” sentinel routed straight down the found branch — a live incident fired two outbound writes with an id of 0 before either was rejected by the partner API. If you wrote myVar != "0" or relied on {{#if myVar}} treating the sentinel as falsy, update it to a plain myVar != 0 / myVar > 0 — that’s now what it means. This has no effect on how the value renders in a template: Handlebars renders a number identically to its digit string, so {{myVar}} (bare, for a JSON number position) and "{{myVar}}" (quoted, for an ID!-typed argument) are unchanged.

Gotcha: excludeHistory is for narrow extraction nodes only

Every llm node’s underlying model call receives the session’s full conversationHistory by default, even when its own promptTemplate never references {{conversationHistory}} — the node just doesn’t happen to use what it’s given. That’s usually harmless. It stops being harmless once a session has a few tool-backed turns behind it: if your response nodes render structured envelopes (e.g. a JSON-shaped reply with a type/tool_output field), that JSON becomes part of the history every later llm node sees — including narrow “extract one short value from {{userMessage}}” nodes with no legitimate use for that context. Having real JSON sitting right there in the model’s input measurably increases how often it echoes that shape back instead of the bare value its own prompt asked for.

Set excludeHistory: true on any node whose job is “pull one value out of the current message, nothing else” — a search term, a title, a code. Never set it on a node whose prompt genuinely references {{conversationHistory}} for legitimate multi-turn slot-filling (e.g. resolving a room code or a date the user gave two turns ago) — that node needs the history to do its job.

Gotcha: allowGeneralKnowledgeFallback for RAG nodes with an explicit fallback instruction

The default retrieval-augmentation wrapper tells the model to answer using ONLY the retrieved reference material, and to say so explicitly if the material doesn’t contain the answer — correct for grounded-fact Q&A, where an ungrounded guess would be actively wrong. It fires whenever retrieval returns any chunk, even a poor match — retrieval has no relevance/similarity threshold, it always returns its nearest neighbours.

That’s the wrong default for a node whose own promptTemplate already has an explicit general-knowledge fallback instruction (e.g. a study-kit generator: “if no reference material is available, answer from general knowledge but say so”). Without this flag, the strict wrapper’s “ONLY” directive overrides that instruction the moment retrieval returns even one loosely-related chunk — producing a hard refusal instead of the fallback answer the node’s own prompt asked for.

Set allowGeneralKnowledgeFallback: true on nodes like this. Leave it unset for grounded-fact Q&A nodes where you genuinely want a refusal rather than a guess.

Gotcha: matchAnyOf is a structural fix for classification nodes, same philosophy as extractFirstInteger

A classification node’s prompt typically says something like “respond with ONLY one of these words: billing, technical, general.” No model — especially a smaller or faster one chosen for cost/latency — can guarantee 100% compliance with that instruction on every call. When it’s ignored, the model answers the user’s actual question in prose instead of emitting the bare classification word. A downstream router node does an exact-string match against that output; a runaway prose reply matches nothing and silently falls through to the router’s defaultTarget, producing a generic fallback reply with no error anywhere to point at why.

Setting matchAnyOf to the full list of valid classification labels sidesteps this: the executor scans the raw response for whichever candidate label appears earliest in the text, and uses that instead of the raw text — so even a model that ignores the “one word only” instruction and answers in full sentences still routes correctly, as long as the correct label happens to appear somewhere in that answer. If no candidate is found at all, the raw text is stored unchanged, so a workflow’s existing defaultTarget/fallback behavior is preserved exactly as if matchAnyOf weren’t set.

Set this on any llm node whose outputVar feeds a router or condition based on a fixed, known set of labels. It has no effect on nodes with free-form or numeric output — use extractFirstInteger for the latter.

matchAnyOf replaces the node output with the matched label, so it cannot be combined with a node that must output JSON. Use it only on classification nodes whose output is a single label. For a node that must return several fields, use outputSchema instead (once it is available on your environment).

Models often wrap JSON in a markdown code fence. On Haiku this happened on every call in a test. Without outputSchema, tell the model Output raw JSON only, no markdown code fence in the prompt, and also strip a leading and trailing fence (```json … ```) in whatever parses the output. With outputSchema the node strips a fence for you.

Gotcha: modelOverride picks the model, and the provider follows it

Before 2026-09-27, modelOverride had no effect at all. It was saved with the workflow, but the executor never read it, so every llm node silently ran on the agent’s or tenant’s model. If you set it earlier and wondered why nothing changed, that is why. It is now honoured, and the model a node runs on resolves in this order (most specific wins):

1. The node’s own modelOverride
2. The agent’s llmModelOverride
3. The tenant’s llmModelOverride
4. The resolved provider’s own default model

modelOverride selects the model, and the provider follows the model. Nodes that name no model use the provider from the agent/tenant llmProviderOverride (or the platform default). A node that names a model a different provider serves is run by that provider, for this node only. The author named the model explicitly, so nothing is substituted silently. Added 2026-10-01 (before that, naming another provider’s model failed the node).

A common use is one node on a different provider inside an agent, for example a Gemini model for a single node that must read images with the best accuracy, while every other node stays on the agent’s provider (Haiku reads images too since 2026-10-01, see below, so this is now a choice, not a requirement):

{
  "id": "read_receipt",
  "type": "llm",
  "config": {
    "modelOverride": "gemini-3.5-flash",
    "imageVars": ["receiptImage"],
    "promptTemplate": "Read the total from this receipt image.",
    "outputVar": "receiptTotal"
  }
}

A model nobody serves fails the node

Before calling the model, the node checks its modelOverride against the model catalog (the list the dashboard’s model picker shows, also returned by the llmModelCatalog query). If the model isn’t in it, the node fails with an error that says why. Some provider models are switched on per environment, so a model that appears in the catalog on one environment may not on another. This is deliberate: otherwise the call would be quietly answered by a different model than the one you picked. Unless the node has a failure edge, this fails the entire workflow run. Check the model id before you publish.

If the tenant has a restricted model list, a modelOverride outside it also fails the node, and publishing a workflow that contains one is rejected. A refusal at run time (for example, the list changed after publishing, or the API key that started the session allows fewer models) takes the node’s failure edge when it has one. See Restricting which models a tenant can use.

An empty or whitespace-only modelOverride (for example, a field cleared in the editor) counts as unset and falls through to the agent/tenant choice. The agent-level llmModelOverride is unchanged by this: it still degrades gracefully to the provider default when its model is unavailable. Only the node-level override is validated strictly.

Gotcha: imageVars fails the entire run, not just the node, on an incapable provider

Added 2026-09-17. imageVars names up to 20 context variables (extras are silently dropped, with a logged warning, and the combined size of all images on one call is capped at 18 MB) whose values get attached as images to this node’s model call, alongside promptTemplate’s text. Each named variable must hold a string that’s either a full data URL (data:image/png;base64,...) or a bare base64-encoded image — a value that’s missing/empty/null on a given turn is a benign skip (not every turn has an attachment for every configured slot), but a present-but-unusable value (wrong type, unparseable, or over the ~5MB per-image cap) throws and fails the node.

This can fail the WHOLE workflow run, not just this node

Before calling the model, the executor checks whether the resolved LLM provider actually supports image input, and refuses the node outright if it doesn’t — deliberately, so the model never silently answers a question about an image it was never actually sent. A turn with an image attached, on a provider that doesn’t support images, fails the entire workflow run, not just this one node, unless the node has a failure edge (since 2026-10-15), in which case the run continues down that edge.

Since 2026-10-01 the platform’s default provider (Claude Haiku) and the other Claude models it serves where enabled for your environment (Sonnet 4.6, Opus 4.6) read images, so a Haiku agent can use imageVars with no provider change. One image over 5 MB (base64) is refused by the Claude path and that call is then answered by Gemini. On an agent pinned to a provider that cannot read images, set the agent’s (or tenant’s) llmProviderOverride to a vision-capable provider. A node-level modelOverride can’t do this by itself, because it selects a model within the already-resolved provider and never switches providers (see the modelOverride callout above). With BEDROCK_MANTLE, you can then use modelOverride to pick one of its vision-capable models for just this node.

Image bytes are sniffed by their own magic-byte signature (JPEG/PNG/GIF/WEBP), never by the ctx variable’s name or any mime type you might separately track — an unrecognized signature fails the node the same way an over-size image does.

Fetching images by URL: imageUrlsVar and imageMarkerHistory

Added 2026-10-06. With imageVars, every image first has to be fetched into a context variable by an earlier action node (fetchAsBase64Var), one fetch step per image. With these options the llm node fetches the URLs itself and sends the bytes straight to the model. The images are never stored in the workflow context or the run record.

If the images arrive as structured attachments on sdkSendMessage or sendMessage, set imageUrlsVar to attachmentImageUrls: the runner fills it with the URLs of the IMAGE attachments of the current message, so no extra node is needed. See Sending files and context.

Option	Type	Default	What it does
imageUrlsVar	string	unset	Name of a context variable holding the URLs: an array of strings, a JSON array string, or a newline- or comma-separated string. Missing or empty means no URL images this turn. Any other type (a number, an object) fails the node.
imageMarkerHistory	number (1 to 40)	unset	Attach the images named by the last N [image:<url>] markers in the user’s messages and the IMAGE attachments of earlier and current user messages (structured attachments, since 2026-10-08), newest first. Checked when the workflow is saved.
imageUrlsFailPolicy	"skip" or "fail"	"skip"	"skip" leaves out an image that cannot be loaded and tells the model; "fail" fails the node (and so the run) if any image requested this turn cannot be loaded.

Both URL options can be set on one node, and both work alongside imageVars. They work only on models that can read images, with exactly the same check as imageVars (see the callout above): the check runs before anything is fetched. A node that sets neither option behaves exactly as before.

Example: URLs from an earlier node

An earlier node (or your own integration) puts the URLs in a context variable, here photoUrls:

{
  "id": "describe_photos",
  "type": "llm",
  "config": {
    "imageUrlsVar": "photoUrls",
    "promptTemplate": "Describe the damage visible in each photo. Only describe what you can see.",
    "outputVar": "damageSummary",
    "streamOutput": false
  }
}

photoUrls may hold ["https://cdn.example.com/a.jpg", "https://cdn.example.com/b.jpg"], the same list as a JSON string, or "https://cdn.example.com/a.jpg\nhttps://cdn.example.com/b.jpg". Duplicate URLs are fetched once.

If you set the variable through runWorkflowTask, note that its variables map must be flat scalars, so send the list as a JSON string (or a newline-separated string), not as an array. Invalid image options are refused when you call updateWorkflow or publishWorkflow, with extensions.code = BAD_USER_INPUT and a message naming the node id, the field and the allowed range, for example LLM node "look" has an invalid imageMarkerHistory — must be an integer from 1 to 40 or LLM node "look" has an invalid imageUrlsFailPolicy — must be "skip" or "fail". The rules: imageMarkerHistory must be an integer from 1 to 40 (0, 99, 1.5 and the string "5" are refused); imageUrlsVar must be a non-empty string; imageUrlsFailPolicy must be "skip" or "fail"; imageVars must be an array of non-empty strings (a list longer than the cap is still accepted and the extras are dropped at run time). These are validation errors, so do not retry them unchanged. Before this change the same mistakes answered INTERNAL_SERVER_ERROR.

Example: images named in the conversation

If your channel writes a marker into the message text for each picture the user sends, the node can collect them from the conversation:

Here are the two photos of the parcel [image:https://cdn.example.com/u/123/front.jpg] [image:https://cdn.example.com/u/123/label.jpg]

{
  "id": "reply",
  "type": "llm",
  "config": {
    "imageMarkerHistory": 5,
    "promptTemplate": "The user said: \"{{userMessage}}\". Answer using what you can see in the attached photos.",
    "outputVar": "replyText"
  }
}

How markers are read:

• The syntax is [image:<url>], any number per message, anywhere in the text. The URL runs to the closing ] and cannot contain spaces.
• Only the user’s messages are read: earlier user turns in the conversation history, plus the current message. The agent’s own replies are never read, so a model reply cannot name a URL to fetch. This holds even when excludeHistory is set (that option only changes what text the model sees).
• Markers are taken oldest first and de-duplicated; a picture the user sends again counts at its latest position. imageMarkerHistory: 5 keeps the newest 5. Markers and structured attachments share the same count, the same caps and the same de-duplication (a URL that is both in imageUrlsVar and in the history is fetched once).
• The marker text stays in the message as written; the node does not remove it.

Keep meaning out of the URLs, and tell the model to answer only from the images

The model also sees the marker text, URL included, because the node does not remove it from the message. A model can therefore “read” a word that is only written in the URL (for example ?text=INVOICE or a descriptive file name), and it can do so for an image that failed to load. The note line [note: X of N images could not be loaded] is still added, but a model may answer around it. Use opaque file names, and add an instruction to the promptTemplate such as: “Describe only what is visible in the attached images. Never infer anything from file names or URLs. If an image could not be loaded, say so.” In testing, a plain prompt let the model read the URL text, and this wording stopped it.

The platform stores the message text exactly as you send it, so the node can fetch the image on this and later turns. When that text is read back through the API (session messages, workflowRuns context, the portal), the marker is shown as [image:https://host/path?[redacted]]: scheme, host and path are kept; userinfo, query and fragment are removed (harmless query parameters such as ?w=800&q=80 go too). The raw URL is not shown to dashboard users or API-key readers, but it is still used for fetching. Text outside an [image: marker, such as a bare URL in normal prose, is not rewritten, and the outbound conversation-sync integration a tenant configures still receives the original text. The node never writes the URL to logs or the node trace (only the host and a reason code). Prefer short-lived signatures anyway.

What is fetched, and what is refused

• Only https:// and s3:// URLs. Anything else (including http://) is skipped with reason not-https; a value that is not a URL, or a URL with a space in it, is skipped with invalid-url.
• An s3://bucket/key URI needs the bucket to be granted to your tenant first, the same rule as the webhook node and action node fetches. An HTTPS URL follows your tenant’s media host allowlist, if one is configured.
• Private and internal addresses are refused, and redirects are not followed: point the URL at the final address.
• Each file must really be an image (JPEG, PNG, GIF or WEBP), checked by its bytes, not its name or declared type.
• Originals up to 20 MB are fetched; an original over 20 MB is skipped with reason too-large. An image with more than 36 megapixels is skipped with reason too-many-pixels (a 48 MP phone original is not accepted today; ask support if you need the ceiling raised).
• Images are shrunk for you. Each image is decoded once, rotated upright using its EXIF orientation, and shrunk to a long edge of about 1568 px, then re-encoded as JPEG (a PNG is kept as PNG when it is already small or has transparency and still fits). A shrunk image is typically 200 to 600 KB. An image already within the limits is passed through untouched, metadata included. Sending a resized copy is still welcome because it is faster, but it is no longer required.
• Limits that apply after shrinking: at most 5 MB per image, or about 3.9 MB (3,932,160 bytes) when the node runs on a Claude model through the platform’s default provider; and at most 18 MB in total per call, or 24 MB when the node pins a Gemini model with modelOverride. Because shrunk images are small, 40 photos normally total 10 to 15 MB (measured: 40 phone photos with 253 MB of originals became 12.6 MB). An image that would take the node over the total is skipped with reason total-size-cap.
• Per node: at most 40 images from the URL options by default, and imageVars keeps its own cap of 20. When there are more URLs than fit, imageUrlsVar entries are kept first and then the newest markers; the rest are dropped without being fetched.
• Time: the fetch-and-shrink phase has a hard 90-second deadline for the whole step; images not finished by then are skipped with reason deadline. Each server shrinks three images at a time, with a per-tenant fairness share. Measured with the same 40 large photos:

Run (40 large phone photos, about 253 MB of originals, 12.6 MB after shrinking)	Images attached	End to end
Alone	40 of 40	about 54 s
2 overlapping runs	40 and 40	about 113 s each
4 overlapping runs	38 to 40	109 to 129 s each

The remaining losses under heavy overlap are images skipped with reason deadline.

Every user message fetches the last N marker images again, so keep imageMarkerHistory as small as the question needs.

These limits are platform defaults and may be tuned; check with your Wetel contact if a number matters to your design.

When an image cannot be loaded

By default (imageUrlsFailPolicy: "skip") the node carries on with the images it could load and adds one line to the message the model sees, for example:

[note: 2 of 7 images could not be loaded]

so the model does not describe fewer pictures than the user sent as if that were all of them. The note counts only images requested this turn (imageUrlsVar entries and markers in the current message), including any dropped by the image-count limit. An image from an older message that no longer loads (a signed URL that has expired, typically) is recorded in the trace but adds no note and never fails the node. Your stored userMessage is not changed.

The note gives only a count, not which pictures failed, and a model can name the wrong picture afterwards. Do not rely on image position after a skip; if the order matters, read the imageFetch entry in the trace, which lists the host and reason of each skipped image.

With imageUrlsFailPolicy: "fail", any image requested this turn that cannot be loaded fails the node, which fails the whole run unless the node has a failure edge. Images dropped by the image-count limit never cause a failure.

What the trace records

Signed URLs work like passwords, so the node never records them. The node’s nodeTrace entry gets an imageFetch object with counts and sizes only: how many URLs were requested (in total and this turn), attached, skipped (and how many of those came from older messages), dropped by the count limit, the total bytes attached, the per-image limit applied, the hosts of the attached images, and for each skipped image its host and a reason code (not-https, invalid-url, private-address, dns-failed, host-not-allowed, http-404 and other http-<status> codes, timeout, too-large, too-many-pixels, not-an-image, total-size-cap, deadline, fetch-failed). It never contains a URL path or query string.

Speaking sentences as they are written: publishSentences

Added 2026-10-27. In production. Normally a reply reaches the client as one message after the model has finished. With publishSentences: true on the LLM node that writes the reply, each completed sentence is sent as soon as it is written, as an AI_RESPONSE event with isFinal: false and turnComplete: false. The response node after it saves the whole reply once and sends one last event with empty text, isFinal: true and turnComplete set as usual. This is the same event shape the direct chat path already uses, so a client that renders sentence events and ends on isFinal or turnComplete needs no change; a client that only reads the text of the final event would see an empty text for this node, so leave it off for such clients.

• Off by default, set per node. Nothing changes for a workflow that does not set it.
• Plain text only. It is refused together with outputSchema (a structured object cannot be streamed) and with streamOutput: false. If you need a structured value as well (for example a routing decision), compute it in a second node after the reply.
• The text must reach the user unchanged. Point the response node at the same output variable ({{reply}}). If its template changes the text, the client receives the sentences and then the full message.
• Barge-in works per sentence. If the caller interrupts, no further sentences are sent.
• What it does not do. The model’s first token is still the main wait for a short answer, so a 40-word reply starts about as early with or without it. It pays for longer answers.
• Guardrails. Checks that need the whole reply (for example the faithfulness check) run on the full text after the sentences are already out. Keep it off for nodes where nothing wrong may be said.

Structured output: outputSchema

Added 2026-10-23. In production since 2026-10-27. Do not publish a graph that relies on it to an environment that does not have it yet: older code neither checks nor uses the field, so the node would store plain text.

Set outputSchema when a node must hand several typed values to later nodes: an order id, an intent and a sentiment, say. The node asks the model for one JSON object, reads it out of the reply, checks it against your schema, and stores the parsed object in outputVar. Numbers stay numbers, enum values are checked, and extra properties are refused. It is the general replacement for extractFirstInteger, matchAnyOf and extractAlphanumericId, which each fix one shape of answer; those stay for existing graphs.

A node without outputSchema behaves exactly as before.

The supported schema subset

The schema is checked when you call updateWorkflow or publishWorkflow. Only this subset is accepted, because every model provider has to accept it:

• The root is { "type": "object", ... }.
• Types: object, string, number, integer, boolean and array. An array’s items may be a scalar or an object, not another array.
• Every object needs properties (1 to 20 per object), "additionalProperties": false, and a required list that names every property exactly once.
• An optional value is nullable instead: write "type": ["string", "null"] (any scalar type with "null"). The model then answers null when it does not know. Only scalars can be nullable, not objects or arrays.
• Nesting: at most three levels of objects, counting the root (root, then an object property, then an object inside that).
• Allowed keywords: enum (1 to 50 unique values of the field’s type), pattern (up to 200 characters), minLength and maxLength on strings; minimum and maximum on numbers and integers; minItems, maxItems and items on arrays; description and title anywhere. description is a good place to tell the model what a field means.
• Size: at most 4 KB (4,096 bytes) as JSON.

Refused, with the reason in the error:

• format (for example "format": "email"), anyOf, oneOf, $ref, const, nullable: true and any other keyword not listed above. Use pattern instead of format.
• enum on a nullable field. Add a sentinel value such as "unknown" to the enum instead.
• A pattern that can backtrack badly: a repeated group that itself contains a quantifier or | ((a+)+, (a|ab)*), and backreferences (\1). Simple id patterns such as ^ORD-[0-9]{6}$ or ^[A-Z]{2}\d+$ are fine.
• A missing "additionalProperties": false, or a required list that leaves a property out.

These refusals answer BAD_USER_INPUT and name the node and the path inside the schema, for example LLM node "read_request" has an invalid outputSchema — outputSchema.orderId: "enum" cannot be combined with a nullable type; add a sentinel value such as "unknown" to the enum instead. They are validation errors, so do not retry them unchanged.

Cannot be combined with streamOutput: true, extractFirstInteger, matchAnyOf or extractAlphanumericId: saving such a node is refused (... sets outputSchema together with matchAnyOf — remove it ...). A structured node never streams, whatever streamOutput says, so you can leave streamOutput unset; excludeHistory, knowledgeBaseId, modelOverride and the image options work as usual. flatten or structuredMode without outputSchema is refused too.

What the node stores

outputVar holds the parsed object, for example:

{ "orderId": "ORD-482913", "intent": "cancel_order", "sentiment": "negative" }

How later nodes read it:

• Templates: {{request.orderId}} reads one field. {{request}} on its own renders [object Object]; use {{jsonString request}} to put the whole object into a prompt or a request body.
• Router: a variable such as request.intent reads into the object. The ROUTER_DOTTED_VARIABLE_OF_TEXT warning is not raised for a structured node.
• Condition and switch: prefer flat names. Set flatten and write request_intent == "cancel_order".

With flatten: true, each top-level property is also written to <outputVar>_<name>; with flatten: "req_" it is req_<name>. For the object above and outputVar: "request", that is request_orderId, request_intent and request_sentiment. A property the model answered with null is written as null. The names must be valid variable names (letters, digits and underscore, starting with a letter, at most 64 characters), must not start with _, must not be a reserved context key and must not equal outputVar; a prefix that breaks this is refused at save.

How the model is asked: structuredMode

How the platform asks for JSON depends on the model that serves the call. You do not choose it per model:

Mode	Used for
native	Gemini models, and the Bedrock Mantle models the platform has verified: the provider’s own JSON Schema option.
tool	Claude models through the platform’s default provider: the model is made to call one tool whose input is your schema.
prompt	Every other model (including the gpt-oss and magistral models, which accept a schema option but do not follow it): an instruction with the schema is added to the system prompt.

When the platform falls back to another provider during a call, the new provider uses its own mode. structuredMode: "prompt" forces the prompt instruction on every model; leave it at "auto" unless a model’s native mode misbehaves for your schema.

Whatever the mode, the reply is always parsed and validated by the platform itself; no mode is trusted on its own. The parser strips one Markdown code fence and tolerates a stray word before or after the object. Anything that is not a JSON object matching the schema is rejected.

A structured call is limited to between 2,048 and 4,096 output tokens, more for a larger schema (never above the model’s own limit). Keep schemas small and prompts focused: a reply cut off at the limit is a failure.

One repair call, then the failure edge

When the first reply is rejected, the node makes one repair call: the same request plus the rejected reply and the validator’s message (for example result/orderId must match pattern "^ORD-[0-9]{6}$"), asking for the JSON object only. Then:

• The repair call is a second, separately billed model call, recorded as its own usage entry at the price of the model that served it. Your spend cap is checked again before it is made. The node’s token and cost figures in the trace are the sum of both calls.
• No repair after a cut-off reply. A reply that stopped at the output-token limit fails at once, because repeating it would only double a slow, costly call.
• A reply still rejected after the repair fails the node. With a failure edge the run follows it, _lastNodeErrorCode is INVALID_DATA and _lastNodeErrorPublic is The information could not be processed.; without one, the whole run fails and the user gets the standard error reply. _lastNodeError names the node and the validator’s message but never contains the model’s reply.

Give every structured node a failure edge. The LLM_STRUCTURED_WITHOUT_FAILURE_EDGE diagnostic (info) points out one that has none.

Valid JSON is not the same as correct values

The schema guarantees the shape: the right fields, the right types, an allowed enum value, a string that matches pattern. It does not guarantee the values are true. A model can still invent an order number that fits the pattern, or pick the wrong intent. In testing, one model filled in an id it had not been given on every try, even though the field allowed null.

• Make every value the model might not know nullable, and say in the prompt and the field’s description to answer null rather than guess.
• Constrain ids and codes with pattern; the platform enforces it on every reply.
• Check the meaning in a later node before anything irreversible: look the id up with a tool or action node and branch on the result, or put an approval step in front of the action.

Structured output in the trace

The node’s row in nodeTrace gains three fields:

Field	When	Meaning
structuredMode	structured nodes	native, tool or prompt: the mode of the call whose reply was accepted (or of the last call, on failure).
repaired	structured nodes	true when the repair call ran.
structuredRaw	when a reply was rejected	The rejected reply, with secret-looking values redacted and cut to 2,048 characters.

With flatten, the row’s assigned lists outputVar followed by the flattened names, and outputValue holds each of them.

Moving from the text helpers

Today	With outputSchema
matchAnyOf: ["billing", "technical", "general"]	A property "topic": { "type": "string", "enum": ["billing", "technical", "general"] }; with outputVar: "ticket", route on ticket.topic or on ticket_topic with flatten: true.
extractFirstInteger with 0 as “no match”	"slotId": { "type": ["integer", "null"] }; “no match” is null instead of 0.
extractAlphanumericId with NONE	"trackingNo": { "type": ["string", "null"], "pattern": "^[A-Z0-9]{8,20}$" }; “none” is null instead of "NONE".
Two llm nodes, one per value	One node, one property per value.

Remove the helper in the same edit (the two cannot be combined), set flatten if a condition or switch reads the result, add a failure edge, and update every node that reads the old variable: it now holds an object, so {{ticket}} becomes {{ticket.topic}} (or the flattened name). Run a draft test for each branch before publishing.

Example: order id, intent and sentiment

One node reads a customer message; a switch routes on the intent; the reply uses the order id when there is one. The full graph (a start node, these two nodes and four response nodes) passes the platform’s save-time and publish checks with no diagnostics.

{
  "id": "read_request",
  "type": "llm",
  "label": "Read order request",
  "config": {
    "promptTemplate": "Read the customer's message and fill in the fields. orderId is the order number exactly as written (ORD- followed by 6 digits), or null if the message has none. Never guess an order number. intent is what the customer wants. sentiment is the customer's tone.",
    "outputVar": "request",
    "outputSchema": {
      "type": "object",
      "properties": {
        "orderId": {
          "type": ["string", "null"],
          "pattern": "^ORD-[0-9]{6}$",
          "description": "The order number, or null when the message has none"
        },
        "intent": {
          "type": "string",
          "enum": ["track_order", "cancel_order", "refund", "other"]
        },
        "sentiment": {
          "type": "string",
          "enum": ["positive", "neutral", "negative"]
        }
      },
      "required": ["orderId", "intent", "sentiment"],
      "additionalProperties": false
    },
    "flatten": true,
    "excludeHistory": true
  },
  "position": { "x": 0, "y": 0 }
}

{
  "id": "route_intent",
  "type": "switch",
  "label": "Route by intent",
  "config": {
    "cases": [
      { "label": "track", "expression": "request_intent == \"track_order\"" },
      { "label": "cancel", "expression": "request_intent == \"cancel_order\"" }
    ]
  },
  "position": { "x": 250, "y": 0 }
}

The cancel branch replies with I can help you cancel{{#if request.orderId}} order {{request.orderId}}{{/if}}. A team member will confirm shortly., and the failure branch with Sorry, I could not read that request. {{_lastNodeErrorPublic}} Could you tell me your order number and what you need?. The edges:

[
  { "id": "e0", "source": "start", "target": "read_request" },
  {
    "id": "e1",
    "source": "read_request",
    "target": "route_intent",
    "label": "success"
  },
  {
    "id": "e2",
    "source": "read_request",
    "target": "sorry_reply",
    "label": "failure"
  },
  {
    "id": "e3",
    "source": "route_intent",
    "target": "track_reply",
    "label": "track"
  },
  {
    "id": "e4",
    "source": "route_intent",
    "target": "cancel_reply",
    "label": "cancel"
  },
  {
    "id": "e5",
    "source": "route_intent",
    "target": "other_reply",
    "label": "default"
  }
]

How the platform judges some replies to this schema:

Model reply	Result
{"orderId":"ORD-482913","intent":"cancel_order","sentiment":"negative"}	Accepted.
{"orderId":null,"intent":"track_order","sentiment":"neutral"} inside a Markdown code fence	Accepted (the fence is stripped).
{"orderId":"482913", ...}	Rejected: result/orderId must match pattern "^ORD-[0-9]{6}$".
{"orderId":null,"intent":"cancel", ...}	Rejected: result/intent must be equal to one of the allowed values.
The valid object plus "note": "x"	Rejected: result must NOT have additional properties.
Sure! Your order is ORD-482913.	Rejected: the reply is not valid JSON.

Each rejected reply leads to the one repair call; a second rejection takes the failure edge. After an accepted reply the run holds request (the object) and request_orderId, request_intent and request_sentiment. The id still has to be checked: look it up before cancelling anything.

Success/failure edges

Added 2026-10-15. An llm node supports the same opt-in routing as a tool node: draw one outgoing edge labeled success and one labeled failure. When the node fails, the run follows failure instead of failing, the trace row is FAILED_HANDLED, and the error message is in the _lastNodeError context variable. outputVar is not written.

What takes the failure edge: no model answered (only after the platform’s own provider fallback has given up), the model is not allowed for your workspace or for the API key that started the session, a modelOverride that is not in the catalog, an image the node refuses (an incapable provider, an unusable imageVars value, or imageUrlsFailPolicy: "fail"), a structured reply that is still invalid after its repair call or was cut off at the token limit, and any other error raised while the node runs. Routing works the same with streamOutput on or off.

A typical use is a friendly fallback reply when the model step fails:

[
  {
    "id": "e1",
    "source": "classify",
    "target": "route_topic",
    "label": "success"
  },
  {
    "id": "e2",
    "source": "classify",
    "target": "sorry_reply",
    "label": "failure"
  }
]

A failure edge without a success edge is refused at publish, and a node with neither label behaves exactly as before. See Failure routing for the full rules.

Worked example

A generic appointment-scheduling assistant uses one llm node to extract a requested time slot as free text, and a second, extractFirstInteger-enabled llm node to resolve a human-given slot code to an internal numeric ID before calling a booking tool.

{
  "id": "extract_slot_id",
  "type": "llm",
  "label": "Match Requested Slot to Internal ID",
  "config": {
    "outputVar": "slotId",
    "promptTemplate": "You are a scheduling assistant's internal matching step. You are not talking to the user — output only a number.\n\nThe user asked for this slot: \"{{userMessage}}\". Available slots are: {{availableSlots}}. Respond with ONLY the numeric id of the best-matching slot from the list, and nothing else. If none match, respond with 0.",
    "extractFirstInteger": true,
    "streamOutput": false
  },
  "position": { "x": 200, "y": 120 }
}

Because extractFirstInteger is set, even a stray reply like “I think slot 4 works best” still resolves slotId to the number 4 instead of breaking a downstream tool node’s argsTemplate. streamOutput: false is set because this node’s output never reaches the user directly.

A second node in the same workflow, this one voice-facing and feeding a response node:

{
  "id": "draft_confirmation",
  "type": "llm",
  "label": "Draft Booking Confirmation",
  "config": {
    "outputVar": "confirmationText",
    "promptTemplate": "Confirm the appointment was booked for slot {{slotId}}. Be warm and brief.",
    "applyVoiceFormatting": true
  },
  "position": { "x": 400, "y": 120 }
}

Here applyVoiceFormatting is appropriate — confirmationText flows straight into a response node meant to be read aloud, not into a router or condition.

In a mock-mode test

Live in production since 2026-10-24. A mock-mode draft test or regression replay still calls the model for this node, so the test checks your prompt; at most 20 real model calls are allowed per test turn. The node is mocked only when the test passes draftMockLlm: true, names the node in draftMocks, or the node has a mockResponse (any JSON value up to 16 KB). A mocked node makes no model call and is not charged; outputVar gets the mock as text (default [mock LLM output for node <id>]), and options that post-process the answer, such as extractFirstInteger or matchAnyOf, are not applied. See Testing a Workflow Safely.

Next steps

• Workflow Overview — node/edge envelope and publishing.
• Condition Node and Router Node — common consumers of an llm node’s classification output. Never set applyVoiceFormatting on a node feeding either of these.
• Response Node — the usual consumer of a voice-facing llm node’s output.
• Image Understanding for an Agent — the full end-to-end walkthrough for imageVars and the URL options: getting an attachment into the conversation, the graph shape, and the vision-capable-provider requirement.
• Best Practices — templating and LLM-reliability patterns.
• Examples — complete end-to-end workflows.
• Workflows API Reference — mutations for creating and publishing workflows.