01 / Good customer service starts with resolving the issue
Reviewed October 12, 2026. An AI assistant can reply politely yet still disappoint if it keeps asking you to explain the same issue or promises an action it never completes. We care more about whether it checks the relevant rules and follows through.
This edition uses the public τ³-Banking customer service task evaluation as its reference. Qwen 3.8 Max ranks first, followed by Claude Opus 5 and Grok 4.5. The results reflect completion of this type of service task. They are neither a customer satisfaction survey across industries nor a hands-on review of a particular support platform.
Sierra: τ³-Banking leaderboard ↗Sierra: knowledge and customer service task design in τ³ ↗
02 / Customer service task results for 10 models
Single-attempt task success compares the ability to complete a task in one attempt. These scores do not mean that the same percentage of 100 real customers would be satisfied: simulated tasks and actual service experiences differ. Positions 8 and 9 have the same score when displayed to one decimal place, so we retain the source order.
Sierra: τ³-Banking leaderboard ↗
| Rank | Model | Single-attempt task success | What to consider |
|---|---|---|---|
| 1 | Qwen 3.8 Max | 55.2% | Leads this edition and deserves a first look |
| 2 | Claude Opus 5 | 48.7% | A leading candidate, with tasks still left unresolved |
| 3 | Grok 4.5 | 47.9% | Close to second place; avoid overstating the gap |
| 4 | GPT-5.6-sol | 46.9% | Strong single-attempt results; also consider consistency |
| 5 | GPT-5.5 | 44.6% | Higher repeated-success score than the model above it |
| 6 | Muse Spark 1.1 | 40.5% | Keep the exact version; these results do not apply to 1.3 |
| 7 | Claude Opus 4.7 | 40.2% | An older version with results still worth comparing |
| 8 | Claude Opus 4.8 | 39.7% | A newer version does not guarantee higher scores on every measure |
| 9 | Claude Fable 5 | 39.7% | Same single-attempt score, different repeated-success result |
| 10 | GPT-5.4 | 39.4% | An earlier evaluation record that provides a reference point |
03 / 1. Qwen 3.8 Max: the leading candidate in this edition
Qwen 3.8 Max achieved the highest single-attempt success rate in the service tasks used for this edition. If you want AI to understand service policies and move an issue toward resolution, this is more informative than conversational tone alone.
The limitation is equally clear: even the leader falls short of the near-perfect resolution many people would expect. On the measure requiring success on all four attempts at the same task, it scored 35.1%. We recommend examining its strengths while keeping realistic expectations for an actual service workflow.
Sierra: original model submission, date, and evaluation settings ↗
04 / 2. Claude Opus 5: another leading candidate
Opus 5 scored 48.7% on single-attempt success and 32.0% on success across all four attempts. It ranks near the top on both measures in this edition and is worth considering alongside the leader.
These results do not determine which model is most patient, best understands conversational Chinese, or makes users wait the least. Its place on this list reflects task completion, which cannot be extended into a claim of a better support experience in every respect.
Sierra: original model submission, date, and evaluation settings ↗
05 / 3. Grok 4.5: do not overstate the gap from second place
Grok 4.5 is less than one percentage point behind Opus 5 on single-attempt success, and its score for success on all four attempts is also displayed as 32.0%. Treating them as close candidates is more useful than focusing only on second and third place.
This is performance on a fixed set of service tasks. It does not establish that every product using Grok provides better customer service. What an assistant can look up or resolve still depends on the capabilities that its product makes available.
Sierra: original model submission, date, and evaluation settings ↗
06 / 4. GPT-5.6-sol: strong single attempts, with a consistency tradeoff
GPT-5.6-sol achieved a single-attempt success rate of 46.9%, placing it near the top of this edition. It belongs on the shortlist of models worth examining for complex service tasks.
However, its score for success on all four attempts was 27.8%, below GPT-5.5 in the next position. This difference illustrates why performing well once and performing consistently across repeated attempts deserve separate attention.
Sierra: original model submission, date, and evaluation settings ↗
07 / 5. GPT-5.5: a lower rank with a strength worth noting
GPT-5.5 scored below GPT-5.6-sol on single-attempt success, but its score for success on all four attempts was higher at 29.9%. Consistency has value for users who do not want to keep trying their luck.
This does not make GPT-5.5 better across the board. We retain the distinction to show that neither a newer model nor a higher single-attempt score captures the whole experience.
Sierra: original model submission, date, and evaluation settings ↗
08 / 6. Muse Spark 1.1: worth considering, with the version kept explicit
Muse Spark 1.1 made this edition's top ten on single-attempt success. However, its score for success on all four attempts was 20.6%, leaving a substantial gap from the leader. It is a candidate with public results to examine, but the brand name alone does not put it in the leading group.
The version compared here is 1.1. Muse Spark 1.3 in the speed ranking is a different version. One version's service results and another's speed cannot be combined into a promise of what users will experience.
Sierra: original model submission, date, and evaluation settings ↗
09 / 7. Claude Opus 4.7: older models can still have strengths
Its single-attempt score was 40.2%, slightly above Opus 4.8 on this task ranking. This is a clear example of why a higher version number does not guarantee a higher rank on every type of task.
Results for older models can help explain product differences, but they do not establish the best purchase today. Account availability and subscription options are questions this task evaluation does not answer.
Sierra: original model submission, date, and evaluation settings ↗
10 / 8. Claude Opus 4.8: a small score gap may not mean a noticeable difference
Opus 4.8 trails the previous model by only half a percentage point on single-attempt success. That difference alone does not establish that ordinary users would reliably notice which one performs better.
It is worth comparing with other models in the same family. In actual use, understanding your issue, avoiding repeated requests for information you already supplied, and clearly explaining the outcome are more concrete concerns than a rank alone. This article does not score those experiences separately.
Sierra: original model submission, date, and evaluation settings ↗
11 / 9. Claude Fable 5: the same single-attempt score, different consistency
Fable 5 and Opus 4.8 both display a single-attempt score of 39.7%. However, Fable 5 scored 28.9% for success on all four attempts, above Opus 4.8's 22.7%. Looking only at the main ranking column can hide this distinction.
If you care about reliably resolving similar support issues, consider this measure as well. We do not convert it into a real-world complaint rate or satisfaction score, or add subjective conversational ratings for either model.
Sierra: original model submission, date, and evaluation settings ↗
12 / 10. GPT-5.4: a dated reference point for comparison
GPT-5.4 scored 39.4% on single-attempt success in the selected tasks. The original evaluation date for this record is May 6, 2026. Its inclusion in this edition's top ten rests on existing public results, not a recent product launch.
For readers, it provides a way to compare generations. Updating an article in October does not turn a May test into a new October run, nor establish that this is necessarily the best version to choose today.
Sierra: original model submission, date, and evaluation settings ↗
13 / How to judge the AI customer service you actually encounter
Pay attention to three things: whether it understands your request, whether the rules and outcome are clear, and whether it explains the situation and hands you over to a person when it cannot resolve an issue. The model name provides context, but cannot replace those observations.
If you simply want to know which models stand out in this edition, start with Qwen 3.8 Max, Claude Opus 5, and Grok 4.5. If consistency matters more, also consider the repeated-success results in each review. Model capability and the service a finished support product delivers remain separate questions.
14 / Evaluation sources, dates, and scope
Sources checked: October 12, 2026 (Asia/Shanghai); article version: v1.1. The ranking comes from Sierra's standard text-agent evaluation for τ³-Banking. Tasks require consulting nearly 700 knowledge documents and completing customer service actions. We have not added new hands-on testing with real customer support services.
Single-attempt success in the main table corresponds to pass¹. Success on all four attempts in the individual reviews corresponds to pass⁴: succeeding at the same task four times, not succeeding just once within four tries. All ten submissions record tau2-bench software version 1.0.1, the banking_knowledge domain, alltools, and the gpt-5.2 user simulator. Reasoning settings are listed below; different settings do not imply equal compute budgets.
These results do not directly represent Chinese-language presales support, retail returns, emotionally sensitive conversations, or phone support. Original evaluation dates are kept separate from the article's source-check date. We do not invent missing costs, latency figures, or exact API snapshots, and small score differences are not treated as significant leads.
Sierra: knowledge and customer service task design in τ³ ↗τ-bench: v1.0.1 scoring correction notes ↗τ-bench: metrics and submission format ↗
| Model | Original evaluation date | Original reasoning setting |
|---|---|---|
| Qwen 3.8 Max | 2026-08-03 | xhigh |
| Claude Opus 5 | 2026-08-03 | max |
| Grok 4.5 | 2026-07-24 | high |
| GPT-5.6-sol | 2026-07-22 | xhigh |
| GPT-5.5 | 2026-07-22 | xhigh |
| Muse Spark 1.1 | 2026-07-23 | xhigh |
| Claude Opus 4.7 | 2026-07-23 | max |
| Claude Opus 4.8 | 2026-07-23 | max |
| Claude Fable 5 | 2026-07-23 | max |
| GPT-5.4 | 2026-05-06 | xhigh |