AI Tools report
In the app: Reports → AI Tools
Needs the 'View Reports' permission.
Shows how often the AI agent used each of its tools while answering customers, and how often a tool failed. Use it to find a tool that fails often, for example because a store's connection is broken.
When the report is available
The tab is only in the Reports strip for organizations that have Squid Agent switched on. For other organizations the page shows "Available with Squid Agent" with a link back to Reports, and no figures.
Period and data freshness
"Last 7 days", "Last 30 days" and "Last 90 days" choose the period. The default is the last 30 days. There are no date fields and no store filter on this page.
- The figures are built once a night, so today is never included. The period ends yesterday.
- Days are UTC days, not days in your organization's timezone.
- "Data through" shows the last day that has been built.
- When part of the period has not been built, an amber line reads "Counts cover … to …; the other days aren't built yet (not zero)". When none of it has, the page says "Not built yet" with the reason instead of showing zeros.
- The first build goes back 90 days, because the detailed record of tool calls is kept for 90 days. The longest period on this page is 90 days.
A teammate whose access is limited to certain stores only sees the tool calls of those stores.
Calls and failures
- "Calls": every time the AI ran a tool on a customer conversation in the period, all tools together. One reply can use several tools, and each recorded use counts. A recorded refusal also counts, such as a supplier tool being unavailable because it was not enabled. A call is placed on the day the tool ran.
- "Failed": the calls that did not succeed. A call fails when the tool stopped with an error, and also when it ran but reported that it could not do what was asked, such as an order it could not find.
What is not counted. Test chats, such as the ones in the AI Builder and the preview of an AI test, and Squid Agent simulations are not counted: their tool calls are never recorded. The same goes for background comparison runs, where a second model answers the same ticket without anything reaching the customer. Tickets that are part of a running AI test are real customer conversations, so their tool calls are counted like any other.
A failed call does not mean the customer got a wrong answer. The AI may have tried again, used another tool or handed the conversation to a person. Whether a successful call gave the right result is not measured here.
Tool table
One row per tool that was used in the period, the tool with the most failed calls first.
- "Tool": the tool's name.
- "Calls" and "Failed": as above, for this tool.
- "Failure rate": failed calls as a percentage of this tool's calls, to one decimal.
- "Most common error": the kind of error behind most of this tool's failures. The kinds are "Timed out", "Rate limited", "Connection or permission" (usually a store's own connection: a wrong key or a revoked access), "Not found", "Invalid input", "Service error" and "Other error". The kind is read from the error text, so it is an indication.
- "p95": how slow the tool was on its worst day in the period. The figure is worked out per store and per day: on the slowest store-day, 95 out of 100 calls finished within this number of seconds. It is not the figure for all calls in the period together. A dash means no duration was recorded.
"No tool calls in this window." means no tool calls were recorded for the built days in the period.
Which tools the AI may use is set per configuration set, on its "Tools" tab under AI → Advanced → Config Sets.