Skip to content
Wetel Docs

Testing a Workflow Safely

This content is not available in your language yet.

A draft test runs your unpublished graph from a test session (sendMessage with testDraft: true). The graph is not live yet, but its steps are: a webhook, action or tool node calls the real system, and a pdf_export node writes a real file. Mock mode stops that. It is the default for every draft test and every regression replay.

What mock mode does

Section titled “What mock mode does”

In a mock-mode run, each node that would reach the outside world returns a mock result instead. Everything else runs as usual.

Node typeIn mock mode
tool, action, webhookThe request is built exactly as live (same templates, the same connector or custom action lookup), then the node stops before the network: no address check, no credentials read, no call, no metering. The rendered request is still in the trace row’s renderedRequest, so you can check what would have been sent, and a broken template still fails the node.
pdf_exportThe config, sections and title are checked as live, then nothing is rendered or stored: no PDF, no artifact, no file written.
llmCalls the model for real, so the test checks your prompts (and is charged as usual). Mocked only when you ask: draftMockLlm: true for every LLM node, a draftMocks entry for that node, or a mockResponse on it. A mocked LLM node returns the mock text as it is; options that post-process the model’s answer, such as extractFirstInteger, are not applied.
sub_agentWith a mock for the node, it dispatches nothing and its resultVar is the mock. Without one, it runs its child agents, and they run in mock mode too.
approvalSame as any draft test: the node fails unless the turn passes draftApprovalOutcome. A mock-mode run never opens a real request or notifies anyone.
everything elseRuns normally (condition, router, switch, set_variable have no outside effects; response, await_reply and end_session only touch the test session).

Where a mocked node’s result comes from, first match wins:

  1. The test’s own draftMocks entry for that node id (see draftMocks rules).
  2. The node’s mockResponse config field.
  3. A default marked as a mock: { "mocked": true, "nodeId": "<id>" } (a tool node adds toolName, an action node adds customActionId, a pdf_export node returns { "artifactId": null, "mocked": true }), or the text [mock LLM output for node <id>] for an llm node.

A mocked result lands where the real one would: in outputVar (an action node also sets <outputVar>Status to OK, as a live success does), in resultVar for sub_agent. A tool or llm node’s output is always text, so an object mock is stored as its JSON string. An async webhook (async: true) has no output, live or mocked.

Limits and marks

Section titled “Limits and marks”
  • At most 20 real model calls per test turn, across the run and any sub-agent children it starts. The 21st real call fails the run with a message telling you to mock some LLM nodes; it never takes a failure edge.
  • The run is marked. WorkflowRunDto.mocked is true, and every trace row whose node returned a mock carries "mocked": true. A mocked run never counts as real activity.
  • A paused test stays in mock mode. When a mock-mode run pauses at an await_reply node, the reply’s run is mock mode too, even if that turn does not repeat draftSideEffects. draftMocks apply only to the turn that sends them, so send them again on each turn that needs them.

The sendMessage options

Section titled “The sendMessage options”

All of these apply only with testDraft: true; on any other turn they are refused with BAD_REQUEST.

FieldTypeNotes
draftSideEffectsDraftSideEffectMode (MOCK or LIVE)Defaults to MOCK. LIVE runs every node for real and is allowed only for the workspace owner; anyone else gets FORBIDDEN (Only the owner of this workspace can do this).
draftMocksJSONPer-node results, keyed by node id. Requires mock mode. Pass it through variables, not as an inline literal.
draftMockLlmBooleantrue mocks every llm node too (no model call, no charge). Requires mock mode. Default false.

Worked example: an order lookup

Section titled “Worked example: an order lookup”

The graph below extracts an order number, checks it, looks the order up through a custom action, and answers. The lookup has success and failure edges, so a failed call gets a polite reply instead of failing the run.

flowchart LR
  start(["start"]) --> extract["extract_order_id (llm)"]
  extract --> check{"has_order_id: orderId > 0"}
  check -- true --> lookup["lookup_order (action)"]
  check -- false --> ask["ask_for_order_id (response)"]
  lookup -- success --> found["reply_status (response)"]
  lookup -- failure --> sorry["reply_sorry (response)"]

The flow reads: start, extract the order number with an LLM node, check it is above zero, and either look the order up (answering with the status, or with an apology on failure) or ask the user for the number.

{
  "nodes": [
    {
      "id": "start",
      "type": "start",
      "label": "Start",
      "config": {},
      "position": { "x": 0, "y": 0 }
    },
    {
      "id": "extract_order_id",
      "type": "llm",
      "label": "Extract order number",
      "config": {
        "outputVar": "orderId",
        "extractFirstInteger": true,
        "streamOutput": false,
        "excludeHistory": true,
        "promptTemplate": "You are a data extraction step, not the assistant. Output ONLY the order number in this message, or 0 if there is none: \"{{userMessage}}\""
      },
      "position": { "x": 0, "y": 100 }
    },
    {
      "id": "has_order_id",
      "type": "condition",
      "label": "Order number found?",
      "config": { "expression": "orderId > 0" },
      "position": { "x": 0, "y": 200 }
    },
    {
      "id": "lookup_order",
      "type": "action",
      "label": "Look up order",
      "config": {
        "customActionId": 7,
        "argsTemplate": "{\"orderId\": {{orderId}}}",
        "outputVar": "orderStatus",
        "timeoutMs": 8000,
        "mockResponse": "Shipped on 21 October, arriving 24 October"
      },
      "position": { "x": 250, "y": 200 }
    },
    {
      "id": "reply_status",
      "type": "response",
      "label": "Status reply",
      "config": { "messageTemplate": "Order {{orderId}}: {{orderStatus}}" },
      "position": { "x": 500, "y": 150 }
    },
    {
      "id": "reply_sorry",
      "type": "response",
      "label": "Failure reply",
      "config": {
        "messageTemplate": "{{_lastNodeErrorPublic}} Please try again in a few minutes."
      },
      "position": { "x": 500, "y": 250 }
    },
    {
      "id": "ask_for_order_id",
      "type": "response",
      "label": "Ask for the number",
      "config": {
        "messageTemplate": "Could you share your order number so I can look it up?"
      },
      "position": { "x": 250, "y": 320 }
    }
  ],
  "edges": [
    { "id": "e1", "source": "start", "target": "extract_order_id" },
    { "id": "e2", "source": "extract_order_id", "target": "has_order_id" },
    {
      "id": "e3",
      "source": "has_order_id",
      "target": "lookup_order",
      "label": "true"
    },
    {
      "id": "e4",
      "source": "has_order_id",
      "target": "ask_for_order_id",
      "label": "false"
    },
    {
      "id": "e5",
      "source": "lookup_order",
      "target": "reply_status",
      "label": "success"
    },
    {
      "id": "e6",
      "source": "lookup_order",
      "target": "reply_sorry",
      "label": "failure"
    }
  ]
}

Save it with updateWorkflow, attach the workflow to a test agent, and start a test session. Every turn below goes through the same mutation:

mutation TestTurn($input: SendMessageInput!) {
  sendMessage(input: $input) {
    id
  }
}

1. The happy path, with no setup

Section titled “1. The happy path, with no setup”
{
  "input": {
    "sessionId": 123,
    "text": "Where is order 4821?",
    "testDraft": true
  }
}

draftSideEffects is omitted, so the turn runs in mock mode. The LLM node calls the model and extracts 4821; the action node renders {"orderId": 4821}, sends nothing, and returns its mockResponse. The agent answers Order 4821: Shipped on 21 October, arriving 24 October. Without the mockResponse field the reply would show the default mock object instead, which is enough to check the route but not the wording.

2. A specific result for one test

Section titled “2. A specific result for one test”
{
  "input": {
    "sessionId": 123,
    "text": "Where is order 4821?",
    "testDraft": true,
    "draftMocks": { "lookup_order": { "output": "Held at customs" } }
  }
}

A draftMocks entry wins over the node’s mockResponse, for this turn only. The agent answers Order 4821: Held at customs.

3. The failure branch

Section titled “3. The failure branch”
{
  "input": {
    "sessionId": 123,
    "text": "Where is order 4821?",
    "testDraft": true,
    "draftMocks": {
      "lookup_order": { "fail": "Request failed with status code 503" }
    }
  }
}

A { "fail": "<message>" } entry makes the node fail with Mocked failure at node 'lookup_order': <message>, and the failure is handled exactly like a real one: the run follows the node’s failure edge, the trace row is FAILED_HANDLED with "mocked": true, and the failure variables are set. _lastNodeErrorCode is worked out from your message the same way as from a real error, so status code 503 gives UPSTREAM_ERROR and the agent answers The service had a problem handling the request. Please try again in a few minutes. A message the rules do not recognise gives UNKNOWN. A node with no failure edge fails the run, as it would live.

4. No model calls at all

Section titled “4. No model calls at all”
{
  "input": {
    "sessionId": 123,
    "text": "Where is order 4821?",
    "testDraft": true,
    "draftMockLlm": true,
    "draftMocks": { "extract_order_id": { "output": "4821" } }
  }
}

With draftMockLlm: true every LLM node is mocked and nothing is charged for them. The extraction node returns its draftMocks entry as the text 4821 (extractFirstInteger is not applied to a mock, so give the exact value you want); without an entry it would return [mock LLM output for node extract_order_id], which orderId > 0 treats as not found. Use this to check routing quickly; leave LLM nodes real when you want the test to judge your prompts.

5. Read what happened

Section titled “5. Read what happened”

workflowRuns(sessionId) shows each turn’s run with mocked: true, and each mocked node’s nodeTrace row with "mocked": true and the renderedRequest it would have sent. Check that request before you publish: it is exactly what the live call will send.

6. Run it for real, once

Section titled “6. Run it for real, once”

When the mocks agree with the real system, the workspace owner can send one turn with "draftSideEffects": "LIVE" against a test record, or publish and test from a normal session.

mockResponse on a node

Section titled “mockResponse on a node”

mockResponse is an optional config field on tool, action, webhook, llm, sub_agent and pdf_export nodes. It holds any JSON value up to 16 KB, and only a mock-mode run reads it; a live run ignores it. Saving it on any other node type is refused with BAD_USER_INPUT, as is a value over 16 KB. A node that has one gets the MOCK_RESPONSE_SET diagnostic (an INFO reminder; nothing to fix). Keep it close to what the real call returns, and update it when the real response changes.

draftMocks rules

Section titled “draftMocks rules”

draftMocks is an object keyed by node id. Each value is exactly one of:

  • { "output": <any JSON> }: the node returns this value.
  • { "fail": "<message>" }: the node fails with this message (1 to 500 characters) and takes its failure edge if it has one.

At most 50 nodes and 64 KB in total, node ids up to 255 characters. An entry for a node that is not mocked in this run (a condition, say) has no effect. An llm node named in draftMocks is mocked even when draftMockLlm is false. draftMocks are never stored on the workflow or the session.

Regression replays

Section titled “Regression replays”

runRegressionScenario replays a saved conversation against the current draft. It runs in mock mode by default too, with mocks built from the scenario’s own recorded run: each tool, action, webhook, pdf_export and sub_agent node returns what it returned in the recorded run, and a node whose failure was handled there fails the same way, so the replay takes the same edges. LLM nodes run for real, since checking the prompts is the point of a replay.

  • Pass sideEffects: LIVE (workspace owner only) to call the real systems instead, for example to find out whether another system’s API changed.
  • Each result records its mode in WorkflowRegressionResultDto.sideEffectMode; replays recorded before the field existed are LIVE.
  • One mock per node. A node visited more than once (a polling loop) gets the last value it returned in the recorded run on every visit, so the replay can diverge. Replay such a scenario LIVE.
  • Images and base64 values replay as placeholders, because the recorded run stores them that way.
  • An approval node fails a replay in either mode.
  • If the recorded mocks would exceed the draftMocks limits, the replay is recorded FAILED rather than run live; replay it LIVE.

Next steps

Section titled “Next steps”