Playground
In the app: AI → Assistant → Playground (in the "More" row at the bottom)
Needs the 'View AI Config' permission.
The Playground is a test bench for your AI agent. You can write to the agent as a new customer, or replay a customer message from a real ticket and compare the reply the agent writes now with the reply that was sent. Nothing in the Playground reaches a customer.
Opening the Playground
There are four ways in:
- the "Playground" link in the "More" row at the bottom of AI → Assistant;
- "Replay in Playground" on an AI response in a conversation's AI activity, which opens the Playground with that conversation and that reply already selected;
- "Test in Playground" on a Squid Agent proposal that changes a knowledge item and is still waiting for approval, which runs replays with the proposed change applied;
- "Test in Playground" on the "Tone of voice" tab of a configuration set, which runs replays with a tone you have not saved yet.
"Target" at the top left chooses what you test: "New customer" or "Existing conversation". "Existing conversation" is greyed out when you may not view conversations.
When a conversation is loaded, the "Back to #…" button at the top right returns to that ticket in the inbox.
New customer
"New customer" shows the "Try it" conversation at full size. You write a message as a customer and the agent answers the way it would for the selected store. It always runs the settings the store runs now; "Run with" is disabled for this target.
- Channel buttons ("Chat", "Email", "WhatsApp"): the channel the reply is written for. WhatsApp uses the same configuration as chat, with WhatsApp's formatting rules. Switching channel after the first message asks you to confirm, because it clears the conversation: "Switch and start over" or "Keep …". Your remaining test runs are not affected.
- Suggested questions: the panel offers a few example questions. Clicking one puts it in the message box; it is not sent until you press Enter or "Send".
- Message box and "Send": type a message and press Enter or "Send". Shift+Enter starts a new line.
- "Start over": clears the test conversation.
A test conversation holds at most 12 exchanges. After that, another message is refused until you choose "Start over". It is kept for six hours after its latest saved turn for you on that store, so you can leave the page and come back. Each teammate has their own test conversation.
Under each answer the panel lists the steps the agent took. A reply can also carry a badge: "Would escalate to a human" when the live agent would hand the ticket to your team, "Your store's handoff message: your agent wrote no reply" when the text shown is the store's fixed handoff message, and "Would close the conversation" or a similar line when it would close or snooze it. Nobody is assigned and nothing is closed in a test.
Testing works the same when the agent is switched off for the store, and still uses a test run; the Playground does not say whether the agent is on. A warning such as "2 actions not available on this store's default setup" lists actions the agent cannot use on this store, with the reason for each. It names at most four actions.
Existing conversation
A replay gives the agent a real conversation up to one customer message and lets it write the next reply. Replaying needs permission to view conversations as well as permission to view AI configuration, and you can only replay conversations of stores you have access to.
- "Conversation": paste a conversation link or ID and click "Open". Once loaded, the field shows the channel, the ticket number, the customer and the store. "Change" lets you pick another conversation.
- "Replay from": the customer message the agent answers. The list holds every customer message of the conversation, with the time and the first words. When you came from an AI response, the message that response answered is selected for you. If that response answered an internal note or a return request rather than a customer message, the latest customer message is selected and a note says so.
- "Run replay": starts the replay. After a run the button reads "Run again".
The agent is given the conversation as it stood before the chosen message, up to the 50 most recent messages, the same amount the live agent sees. A customer message that was only an attachment is passed as a short placeholder saying the customer sent an attachment without text.
Some conversations cannot be replayed:
- A conversation of a store that shares its mailbox with other stores shows "Can’t replay this conversation", because the test would look orders up in the wrong store.
- A conversation with no customer message shows "Nothing to replay".
- A conversation that is not in one of your stores is reported as not found.
Only one replay per person and store runs at a time. A second one is refused until the first finishes.
Run with
"Run with" chooses the settings a replay runs. It applies to "Existing conversation" only.
- "Live settings": what the conversation's store runs now. This is the default.
- "Draft: …": the editable configuration set, rather than one of its releases. This runs its active main prompt and enabled specialists; it does not run a newer main prompt that you have only saved as an inactive draft. If the set has no active main prompt, the store’s older store-specific active prompt is used when one exists; otherwise the replay is refused.
- A release, shown as the set's name and the release name: a published version of a set that uses version control.
Configuration sets that belong to one other store are left out of the list. The line under the setup row says which settings the next run uses.
When you arrived with a proposal or an unsaved tone, "Run with" is locked to it and a blue note explains what is being tested: "Proposal #… (not live yet)" changes one knowledge item for these runs only, and "Draft + unsaved tone" runs the configuration set with the tone you have not saved. "Run without it" drops the proposal or tone and unlocks the list. An unsaved tone is kept in the browser tab it was made in; if it is gone, go back to the "Tone of voice" tab and press "Test in Playground" again.
If a proposal changes a different configuration set than the one that answers this conversation, a warning says so; the replay then shows the change on this conversation's set.
In every case the replay uses today's data: orders, tracking and knowledge are read as they are now, not as they were when the ticket came in.
Reading the result
A finished replay shows the comparison and the steps behind it.
"What changed" is a one-line summary: whether the new reply is shorter or longer than the one that was sent, whether the sent reply was written by a teammate, whether the new run would hand the ticket to a person or wrote no reply at all, and how many lookups used live data and how many changes were made up.
"Conversation so far" shows the messages the agent was given, ending with the customer message it answered.
"Original" and "New" stand side by side. "Original" is the reply that was sent after that message, with the time it was sent, or "No reply was sent after this message." "New" is the reply the agent wrote now, labelled with the settings it ran with. "Would hand this over to a human." appears under it when the agent would have escalated.
"What the agent did" lists every step of the new run. Click a step to see what the agent asked for and what came back. Each step has a label:
- "Used your live data": a lookup that ran for real and returned usable data from your store.
- "Made up (nothing real)": an action that would change something, so the agent was given a stand-in result.
- "Ran (read nothing)": a step that only decides what happens to the conversation, such as closing or snoozing it.
- "Ran (nothing connected)": a lookup whose integration is not connected to this store.
- "Ran (failed)", "No data returned" and "Blocked": the step ran with an error, returned nothing usable, or was not allowed to run.
A "Consulted …" label names a specialist the main agent asked for help.
A slow replay keeps running when the page stops waiting; the page shows "Still finishing… fetching the result" and picks the result up. A finished result is kept for 15 minutes. A replay that takes longer than 15 minutes is stopped.
What the sandbox does and does not do
Every run in the Playground calls the AI in a sandbox. It tests reply writing and tool choices, rather than every step of live ticket processing:
- No reply is sent, on any channel.
- Nothing is written to the conversation: no message, note, tag, status change or assignment, and nothing appears in the ticket's AI activity.
- A fixed set of lookups runs for real: orders, tracking, refund status, return addresses, products, return portal details, order age, handoff reasons and Google Sheets lookups. Other tools receive stand-in results. A real lookup can return no data or fail; the step shows that outcome.
- Anything that would change an order, a refund, a return or a customer record is not carried out. The agent is told it succeeded, so it answers as it would in real life, and the step is labelled as made up.
- A question about the built-in sample order is answered from a stand-in order, not from your store.
New-customer tests and replays follow the store’s current automatic-send setting when deciding which action tools to offer. They do not reproduce the live decision to send a customer to the return portal first, or the recovery steps used when a live answer fails its checks.
The Playground does not show what an answer would have looked like in the past: it always uses the current data and the settings you chose.
Test runs
Every message you send as a new customer and every replay is one test run. Your organization gets 50 free test runs per billing period. The allowance is shared by the Playground, "Try it" on the Assistant page and the "Test" tab of the Builder, and by everyone on your team. The line under each result, and the line in the "Try it" panel, shows how many are used.
After the free runs are used, you can keep testing: each further run counts as one AI message on your plan. On a trial, further runs come out of the trial's AI message allowance and stop when that is spent. When your plan is not active, test runs cannot run at all until billing is sorted out under Settings → Billing.
An organization that is still setting up gets 25 extra test runs, once, not per billing period. It counts as still setting up until the first time one of its stores has completed every step of the setup checklist. During that time a question typed as a new customer to a store's running agent uses one of these 25 runs, wherever it is typed: here, in "Try it" on the Assistant page, in "Test an answer" in AI Advanced settings or during store setup. Such a run does not use the 50 free test runs, does not count as an AI message, and still works when the trial's AI message allowance is spent. It does not help when the plan is not active. Replays of real tickets and tests in the Builder never use these runs. After the 25 runs, or once setup is complete, the normal rules above apply.
A run that fails before the AI model was called is not counted. A run in which the agent ran but wrote no reply is counted, and the panel says so.