Frontier models: pick for the job, not the score

Gemini 4 Argon opened to broad access on October 13. It joins Claude Opus 5.5, GPT-6.1 Sol and Claude Sonnet 5.5, all September releases. And that is a small slice. Artificial Analysis lists at least 49 named models for August and September alone, and 123 for 2026 so far. That counts models, not reasoning-setting variants, and not all are frontier. Benchmark and pricing snapshot: October 15, 2026.

Picking the leader is harder than it looks. On the same source’s intelligence index, GPT-6.1 Sol scores 52 at $0.72 per task, while GPT-6 Astra scores 53 at $3.26. Claude Opus 5.5 costs $5.98 per task and Claude Fable 5.1 costs $7.63. Fable 5.1 also scores 53. So a one-point score gap can mean roughly a 5x to 11x cost gap. And GPT-6.1 Sol replaced GPT-6 Sol after just seven days.

If your product’s edge depends on how well a model solves hard problems, this choice is a business decision. A wrong pick can mean slow responses, a bill that breaks your margins, or errors your team has to fix. But switching at every launch costs something too. Each release needs a test, not an automatic migration: rerun benchmarks on your own tasks, recompute cost per completed job, tune reasoning effort, trim context, and update prompts, limits and fallbacks. This work is called inference optimization, and a launch chart cannot tell you whether it pays off.

This post gives product and program managers the criteria to choose the right frontier model for each use case, and a way to know when a new release has earned a switch.

What frontier models are

Of those 123 models, only a handful are frontier. The term means the models leading the capability race right now, measured on benchmarks for reasoning, coding, knowledge and other tasks. It describes a position, not a permanent category or a badge a model keeps.

The Frontier Model Forum puts it the same way. It describes frontier AI as general-purpose models at the state of the art:

"a collection which will shift over time as the field progresses."

For its own membership, the Forum uses a looser test: a general-purpose model that beats every model widely deployed for at least 12 months. So institutions draw the line in different places.

Because frontier is a moving position, it has three practical effects for anyone choosing a model: the leaders sit close together, one company can have several, and settings change the answer.

The leaders sit close together - On the Artificial Analysis Intelligence Index, Claude Opus 5.5 scores 58. Muse Wonder Full and Claude Sonnet 5.5 score 56. GPT-6 Astra, Gemini 4 Argon and Claude Fable 5.1 score 53. GPT-6.1 Sol and Muse Wonder score 52. That is eight models inside a six-point band.

One company can have several - OpenAI has Astra and the lower-cost Sol. Anthropic has Fable 5.1, Opus 5.5 and Sonnet 5.5. Meta released Muse Wonder and Muse Wonder Full on October 12: Muse Wonder is available through its API, while Wonder Full is limited to Muse Code subscribers, with no API. If you build on an API, that access rule matters as much as the score.

Settings change the answer - Argon’s 53 comes at high reasoning, while Astra and Fable 5.1 score 53 at max reasoning. Comparing maximum effort against a faster setting answers a different question from comparing both at the same effort.

So “frontier” gives you a shortlist, not a winner. The job decides which comparison matters. That leads to the next question: what should you measure?

Why the choice matters more in agentic apps

The eight frontier models listed above score within six points of each other on the Artificial Analysis index, from 52 to 58. That score is a good start, but it hides three differences: what each task costs, how often the model is wrong, and whether it gets the same task right every time. In a chat, these are small annoyances. In an agentic app, where the model calls tools and changes things outside the chat, each one becomes a real cost.

Cost. GPT-6.1 Sol scores 52 at $0.72 per task. GPT-6 Astra scores 53 at $3.26. Claude Fable 5.1 scores 53 at $7.63. As an illustration, if your task cost the same as the index task, 10,000 tasks a month would run about $7,200, $32,600 and $76,300.

Errors. Gemini 4 Argon and Astra both score 53. But on a factual test that penalizes wrong answers, Argon’s hallucination rate is 15 percent and Astra’s is 51 percent. In a chat, a user may catch a wrong fact. In an agent, it becomes a wrong action.

Consistency. The tau-bench research paper scores agents by comparing:

“the database state at the end of a conversation with the annotated goal state.”

In other words: did the correct work actually happen? It also repeats each task. In the original 2024 results, the best agent, GPT-4o, completed fewer than 50 percent of tasks. In retail it got the same task right on all eight tries less than 25 percent of the time. Those are older models, so they say nothing about today’s ranking. The lesson is the test: measure the outcome and its consistency, not the agent’s claim that it finished.

Take a support assistant. Summarizing a returns policy is one task. Finding the right order, checking eligibility, asking for approval and starting the return is another. A cheap model that picks the wrong order creates a refund dispute. A costly one makes routine returns too expensive. A cautious one sends more cases to your human queue.

A score tie is not a tie in practice, and more capability does not remove the need for approval rules, good tools and recovery paths. So the real question is whether a model’s gain pays for its bill, wait time, correction work and migration cost in your workflow. The next section covers what to measure.

What to look for

The scores above hide differences that only show up on your own work. Six things are worth checking before you commit: cost per task, accuracy, fit for your job, speed, access and stability.

Cost per task, not cost per token

A token is a small chunk of text, and providers charge per million of them. That price is a rate, not a bill. The bill depends on how many tokens a job burns, including hidden reasoning, tool calls and retries. Models differ a lot here. One index measured Gemini 4 Argon at about 62,000 output tokens per task, against 27,000 for GPT-6 Astra. Those counts include reasoning, not just the answer a user sees.

Here is a made-up example - Model A costs $5 per million tokens and uses 20,000 tokens per task. That is $0.10 a task. Model B costs $3 per million tokens and uses 60,000 tokens per task. That is $0.18 a task. B has the cheaper price tag and the higher bill. Now add quality. If A passes your bar 90 percent of the time and B only 70 percent, the cost per passed task is about $0.11 for A and $0.26 for B. Every failed task also lands on a person who has to fix it, and that time is often the biggest cost of all.

So the number to track is this: (model cost + human fix time) divided by the number of tasks that pass your quality bar. Before you pick a model, measure three things on your own tasks: total cost per attempt, pass rate, and minutes of human correction. A cheap model that sends work back to your team is not cheap. The numbers in the A and B example are invented for illustration. The mechanics are what matter.

All figures in this table are invented, and human correction is not yet included. In the formula above, convert human fix time to dollars before adding it to model cost. The 62,000 and 27,000 figures are measured benchmark totals that include reasoning, not just the answer a user sees.

Accuracy and hallucination

An answer can be wrong, or the model can decline. Those are different outcomes. Hallucination means it gives an unsupported answer rather than admitting the gap.

Imagine 100 support questions. In this invented example, Model A answers 80 correctly, guesses wrongly on 10 and declines 10. Model B answers 70 correctly, guesses wrongly on two and declines 28. B makes fewer confident mistakes but sends more work to a person. Which is better depends on the damage a wrong answer does and how well your handoff works.

This matters when you read benchmarks. The test behind Argon’s 15 percent hallucination rate penalizes wrong answers but not refusals, so declining is never punished. Track correct answers, unsupported guesses and declines separately, and include questions where the information is missing or conflicts. Decide what the model should do then: look up evidence, ask a question or hand off. Do not reward a confident fabrication because it looks complete.

Fit for your job

A coding score is not a customer-support score, and the leaders change by test. On FrontierSWE v2, Argon trails Astra (55.0 against 65.5 percent). Anthropic's launch comparison reports Opus 5.5 at 66.4 percent on Terminal-Bench 4.0 under its own configuration; Artificial Analysis uses a separate setup, discussed below.

For a document-review feature, try a real contract with an exception buried in an appendix. Can the model find it, explain it and point to the evidence? If users upload scans or charts, include them. A large context window means room for more information, not proof that the model will use the important detail.

Run each test more than once. The tau-bench research found that agents which pass a task once often fail it on repeat runs.

Speed and verbosity

The useful measure is time until the job is done, not how fast the first words appear. Reasoning, tool calls, retries and checks all add time. A newer version is not always faster: Artificial Analysis's current Max-setting comparison clocks GPT-6.1 Sol at about 54 tokens per second, against about 102 for GPT-6 Sol.

In an invented example, a faster model answers in two seconds but usually needs two retries, about six seconds in all. A slower one takes five seconds and passes the first time. Timing only the first response picks the wrong winner. For live support, measure the slow cases, not just the average. For overnight analysis, a longer wait may be fine if quality improves.

Access

A model you can try in a chat app is not necessarily one your engineers can deploy. Check API access, rate limits, supported inputs and data-handling terms before it goes on the roadmap. Muse Wonder Full scores 56 on the index, but Meta does not serve it through an API.

Your test may pass with one request at a time, while a launch brings hundreds at once. Test expected traffic and decide what happens when requests are limited: wait, use another tested model or hand the work to a person. Restricted access is a product constraint, not an engineering detail to discover after choosing the winner.

Stability

A model name can stay the same while prices, defaults, safety behavior or versions change. GPT-6.1 Sol replaced GPT-6 Sol after seven days. Record the exact version, prompt and settings that passed your tests.

A new release may change how it returns tool arguments, so a workflow that worked yesterday may reject the output even if the benchmark score improves. Test that boundary before switching, and keep a rollback route.

Price changes count too. Argon’s index cost is $1.99 per task at its discounted price and $3.98 at the standard price, which is above Astra’s $3.26. Check that your feature still works at the standard rate, not just during the discount. A model choice needs an owner for re-testing and migration, not a one-time thumbs-up.

How the frontier models compare: no model wins everywhere

Among the five models below, Claude Opus 5.5 posts the highest score on the Artificial Analysis Intelligence Index (58), and GPT-6.1 Sol is the cheapest per task ($0.72). Gemini 4 Argon has the lowest hallucination rate of any model scoring 45 or higher (15 percent), and GPT-6 Astra tops FrontierSWE v2. So the useful question is not which model is best, but best at what, at what price and at which setting.

This section compares them on cost, accuracy, fit and access. Speed and stability depend mostly on your own setup. Two cautions apply throughout. The index’s cost per task measures index tasks, not your support agent or document review feature. And the settings differ: Argon runs at High reasoning, the others at Max, and the Claude models include their default fallbacks. These are operating points, not permanent facts about each model.

Cost and score on one independent index

Sources: Argon analysis, Sol versus Astra, and Opus versus Fable.

Opus 5.5 scores six points above Sol at about eight times the cost per task. Fable 5.1 scores 53 and is the most expensive of the five. Whether extra points are worth the price depends on your hardest cases, and one index cannot tell you that.

Don’t budget a long-running feature around Argon’s discount. Google specifies $2 input and $10 output per million tokens through November 30, then $4 and $20 from December 1.

Token prices can hide the work

Opus 5.5 lists at $4 input and $20 output per million tokens, Astra at $10 and $50. But Argon used about 62,000 output tokens per index task, against 27,000 for Astra, and those totals include reasoning. Generating more can eat into a low token price. Caching, long-input surcharges and tool fees change the math again. Sol’s documentation raises rates above 272,000 input tokens, so a document-heavy feature can land in a different price bracket than a short benchmark prompt.

Low hallucination is not high coverage

On AA-Omniscience, Argon’s hallucination rate is 15 percent against Astra’s 51 percent. The rate is defined as:

“incorrect / (incorrect + partial answers + not attempted)”

So it asks how often a model guesses wrong among answers that were not correct, not how many questions it got wrong overall. Argon more often admits uncertainty. It also answers fewer questions correctly: 50 percent accuracy against Astra’s 63 percent.

For product design, an assistant that declines uncertain questions needs a good retrieval or escalation path. One that answers more needs stronger checks against confident mistakes. This test also runs without tools or supplied context, so it does not measure a retrieval-based assistant.

Different jobs produce different leaders

• Long software tasks. On FrontierSWE v2, Astra scores 65.5 percent, Opus 5.5 62.3 and Argon 55.0. The ranges are wide and overlap, so treat the order as a result in that setup. Google reports Argon at 77.9 percent on DeepSWE v1.1, a different task set.
• Terminal work. On Terminal-Bench 4.0, Sonnet 5.5 leads at 63.6 percent and Opus 5.5 is at 59.6. Anthropic’s own comparison reports a different result under its configuration.
• Business workflows. OpenAI reports Sol 2.2 points ahead of Opus 5.5 on AutomationBench 1.0.6 at medium effort. Don’t mix that with Artificial Analysis’s separate scores.
• Visual reasoning. On Scale’s Humanity’s Sixth Sense test, humans score 93.1 percent. Argon scores 55.4 and Astra 53.6, with overlapping uncertainty. These are overall results, not evidence that either model leads on every visual task.

Benchmarks help you pick candidates. Your own tasks decide whether one belongs in the product.

Availability is a moving target

Argon went first to trusted cyber defenders in September and, as of October 13, is broadly available through the paid API. Sol is live in the API. Claude Fable 5.1 is generally available. Muse Wonder has an API, but Wonder Full is Muse Code subscription-only.

New challengers keep arriving. Paper Instruments’ Ultramarine 1 Flash, added October 14, claims 59.3 at $0.13 per task against Fable 5’s 57.8 at $8.19. That is vendor-reported on Paper Instruments' internal benchmark, compared with Fable 5 rather than 5.1, and its weights are listed as “coming soon.” It is a reason to test beyond the familiar names, not to crown a new winner.

Blind spots that undo a good model choice

Trusting launch charts

A vendor chooses what to show. An independent benchmark also makes choices about tasks, settings and scoring. GPT-6.1 Sol shows the gap: it lands within a point of Astra on the headline index, but trails it on ExploitBench Internal Port (21.5 against 31.5 percent) and TroubleshootingBench (47.96 against 63.46). Read how a result was produced before treating it as your acceptance test.

Chasing the newest model

GPT-6.1 Sol arrived seven days after GPT-6 Sol. It gained 4 points on the index and cut cost per task from $1.05 to $0.72. It also uses about 10 to 30 percent more output tokens at the same list price, so even a clean upgrade moves your bill per job. That does not mean the older model was retired, or that you should switch every week. Migration work has a cost too.

Ignoring the long tail

Average performance can hide rare, expensive failures: the wrong account, missing evidence, an ambiguous instruction or a tool that failed after the model assumed success. Test the ugly cases, and repeat important tests to check consistency.

Forgetting that agents compound

If ten independent steps each succeed 95 percent of the time, all ten succeed only about 60 percent of the time. At 98 percent per step, it is still only about 82 percent. That is an illustration, not a prediction, because real steps can have correlated failures and recovery changes the result. The design question is where you will detect, stop and repair a failure before it travels further.

Confusing capacity with a better experience

Argon’s guide describes a shared one-million-token window for input, thinking and output. Beyond 65,536 output tokens on its Interactions API, it requires background execution. Long jobs need progress states, interruption and spending limits. The model’s ability to keep working does not decide when the product should stop.

Treating confidence as permission

A better model does not earn the right to send, delete or buy. Put approval and verification around consequential actions. Safe refusal and useful escalation are product behaviors, not inconveniences to prompt away.

The playbook

1. Define the job. Name the task, the quality bar and a cost ceiling per run. Specify what "done" means and what the system must never do. For an order-change agent, that means the correct order actually changed with the required approval, not that the answer sounded helpful.
2. Build your own test set. Start with 50 real examples, including missing data, conflicting instructions and broken tools. Fifty is a starting point, not evidence that rare risks are covered. Expand as failures emerge and keep some examples aside to check that improvements generalize.
3. Run three models. Include a capable baseline and a lower-cost alternative. Keep information, tools and criteria consistent; record model versions and reasoning settings. Measure completed work, serious errors, elapsed time, total spend and human correction. Repeat high-risk scenarios.
4. Route by task. Try a cheaper model for straightforward steps and deeper reasoning for hard ones. Test the handoff and routing decision too. Saving on one call isn't helpful if a weak routing decision sends the whole job down the wrong path.
5. Add a swap layer. Keep provider-specific calls and model names out of core business logic. Maintain versioned prompts and tests. A shared interface makes switching easier; it doesn't erase differences in tool behavior, safeguards or output formats.
6. Set a fallback. Know your plan for outages, access limits and repricing. That might be another tested model, a person, or an honest pause. Preserve approval rules during the fallback; don't lower them just to keep the workflow moving.
7. Re-test quarterly, and after major model or workflow changes. Use monitoring to catch changes sooner when the risk calls for it. A new price, tool, prompt or information source can invalidate an old result even when the model name stays the same.

For product managers, the work is deciding what quality and risk mean for the user. For program managers, it's making sure testing, monitoring, migration and recovery have owners. Both need the same thing: evidence tied to a real task.

Don't guess the winner. Make switching cheap, and make success measurable.

-amrita

Subscribe | LinkedIn | X