Claude alternatives for coding: GPT-6 Sol, Gemini 3.8 Flash, Kimi K3, DeepSeek and Grok ranked by cost per passing task

Price per million tokens is the wrong number for choosing a coding model. A model at half Claude's token price can still cost more per finished task if it burns ten times the tokens or fails often enough that you pay twice.

What matters is the cost of a task that passes its tests. This post works it out for GPT-6 Sol, Gemini 3.8 Flash, Kimi K3, DeepSeek V4 Pro, DeepSeek V4.1 Flash and Grok 4.7 against Claude Sonnet 5.5 and Opus 5.5, from public benchmark runs that publish cost per task, then turns it into a routing pattern: try a low-cost model, run the tests, send failures to Claude.

The short answer

• DeepSeek V4.1 Flash is the clearest saving. On Vals AI's Terminal-Bench 2.1 run it passed 74.5% of tasks at $0.13 per passing task. Claude Opus 5.5 came to $0.57 and Sonnet 5.5 to $0.75.
• GPT-6 Sol has Sonnet 5.5's input and output prices and matched its 83.1% pass rate for $0.45 per passing task. GPT-6.1 Sol, same price, did better on the harder benchmark.
• Gemini 3.8 Flash has lower token prices than Sonnet 5.5 and Grok 4.7 a lower output price, yet both cost more per passing task.
• Kimi K3 costs more per token than Sonnet 5.5, held up on everyday tasks and fell to 17.2% on hard ones.
• On long, hard tasks Claude leads: Opus 5.5 passed 65.2% and Sonnet 5.5 64.1% of Terminal-Bench 4.0. No alternative in this comparison passed more than 55.1%.
• Opus 5.5 cost less per attempt than Sonnet 5.5 on both Terminal-Bench runs, so test it before assuming Sonnet is the economical Claude.

List prices, checked 31 October 2026

Per million tokens, input / output. Cached input is the rate for a reused prompt prefix, most of the input in an agent loop.

Long sessions can trigger higher rates. OpenAI bills a whole request at 2x input and 1.5x output once the prompt passes 272,000 tokens on both Sol models. Grok 4.7 goes to $4.00 / $12.00 above 200,000 tokens at xAI and above 128,000 on Aight. Claude Opus 5.5 and Sonnet 5.5 bill their full 1M window at the standard rate. DeepSeek's peak hours are in the V4 Pro vs V4.1 Flash post.

Note the cached column: Grok 4.7's cached input costs five times Sonnet 5.5's, and Kimi K3's three times.

How cost per passing task works

cost per passing task = cost per attempt ÷ pass rate

GPT-6 Sol on Terminal-Bench 2.1 cost $0.37 per attempt and passed 83.1% of tasks: $0.37 ÷ 0.831 = $0.45 per passing task.

Cost per attempt depends as much on how many tokens a model burns as on its price. On the same 89 tasks, Vals recorded about 0.57 million tokens per task for Sonnet 5.5, 1.14 million for Grok 4.7 and 3.71 million for Gemini 3.8 Flash. Almost all of it is cached input: in Vals' Terminal-Bench 4.0 run, cache reads were 95% to 99.7% of input tokens for every model here. For agent work, the cached rate is often the price that counts.

Two limits on the formula:

• It assumes retries are independent. They aren't. On the official Terminal-Bench 4.0 leaderboard, Opus 5.5 in Claude Code passed 64.9% of tasks on one try and 78.8% within five; independent retries would have reached 99.5%. A model that fails a task usually fails it again, which is the case for escalating to a different model rather than retrying.
• Each benchmark uses one agent harness. Yours differs, so trust the ratios more than the dollar amounts.

Everyday tasks: Terminal-Bench 2.1

Vals AI ran every model through the Terminus 2 harness on Terminal-Bench 2.1: 89 terminal tasks, mostly software engineering and system administration, scored pass@1, and a task counts only if all its tests pass. Last updated 1 October 2026; GPT-6.1 Sol and DeepSeek's 17 October Flash build aren't in it. Cost per attempt is Vals' published cost per test; its model pages list the makers' token prices, with DeepSeek at peak.

GPT-6 Sol, Kimi K3 and V4.1 Flash each cost less per passing task than either Claude model. So does V4 Pro, but it fails almost half the tasks, and each failure still costs a review or a retry.

One oddity: Vals lists Gemini 3.8 Flash at $0.75 / $3.75, but its published Gemini costs work out to twice that price from its own token counts, which matches Google's 1 January price. If that's the cause, the cost through 31 December is half: $0.95 per passing task here, still above Opus 5.5.

Long, hard tasks: Terminal-Bench 4.0

Vals AI, updated 29 October 2026: 66 new tasks with a median expert time estimate of 4 hours, run with mini-swe-agent, an 8-hour limit per task, and scores averaged over three full runs. About three quarters of the tasks fall outside traditional software engineering (science, ML systems, operations, hardware), so read it as a stress test of long agent work.

Here low token prices stop helping: Kimi K3, Grok 4.7 and Gemini 3.8 Flash each cost more per passing task than Opus 5.5, because they fail most tasks and still spend heavily on every attempt. GPT-6.1 Sol stands out: 55.1% at $1.72 per attempt, about an eighth of Opus 5.5's $13.20.

Two newer models did well but aren't on Aight yet: Moonshot's Kimi K3.1 Pro ($2.00 / $10.00, released 23 October) passed 63.6% at $4.57 per attempt, and Claude Haiku 5.5 ($0.10 / $0.50 for prompts up to 100,000 tokens) passed 36.4% at $3.12.

The official Terminal-Bench 4.0 leaderboard, which runs each maker's own agent, ranks the shared models in the same order: Opus 5.5 in Claude Code 64.9% at $14.30 per attempt, GPT-6.1 Sol in Codex 58.2% at $1.92, GPT-6 Sol 49.4% at $4.40, Grok 4.7 in Grok Build 37.6% at $11.16.

Where the benchmarks disagree

DeepSWE from Datacurve (113 tasks written from scratch in five languages, mini-swe-agent, four runs, updated 22 September 2026) tells a different story for Gemini. Gemini 3.8 Flash at high effort passed 73.8% at $2.36 per attempt, matching Claude Opus 5 at $11.84. Kimi K3 passed 68.5% at $4.65 and DeepSeek V4 Pro 62.8% at $1.67. DeepSWE hasn't run Sonnet 5.5, Opus 5.5, GPT-6 Sol, Grok 4.7 or V4.1 Flash.

Vals' Vibe Code Bench v1.1 (29 October, OpenHands harness) has each model build 50 web apps from a spec and scores the share of each app's tests that pass, so the division doesn't apply directly. Sonnet 5.5 scored 92.4% at $31.25 per app, Opus 5.5 90.3% at $57.92, GPT-6.1 Sol 88.9% at $6.23, and V4.1 Flash (17 October build) 87.7% at $0.58. On greenfield app work the gap is small, and Sonnet beat Opus at about half the cost.

The usual coding leaderboards have no current data for most of these models. The swebench.com leaderboard's newest entry is from 26 February 2026. Vals retired its SWE-bench Verified run on 1 September 2026 as saturated, before Claude 5.5, GPT-6 Sol and Grok 4.7. Scale's public SWE-Bench Pro board lists none of these models, Aider's polyglot leaderboard was last updated on 20 November 2025, and Vals' LiveCodeBench run (1 September) lists no cost per problem.

So treat these numbers as a shortlist. Take 50 recent tasks from your own repository that have tests, run two or three candidates, and do the same division.

What changes on Aight's prices

Claude Opus 5.5, Sonnet 5.5, GPT-6 Sol, GPT-6.1 Sol and Grok 4.7 are at the makers' list prices on Aight, so the tables apply as they are. DeepSeek V4.1 Flash is a flat $0.30 / $1.20 with cached input at $0.006, the same as DeepSeek's peak rate that Vals used. Off-peak, DeepSeek's own API costs half. Vals tested two V4.1 Flash builds, and the 17 October build that DeepSeek's API now serves did far better on the hard set, so ask any host which build it runs.

Two models cost more on Aight than direct:

• Kimi K3 is $3.30 / $16.50, 10% above Moonshot. Priced from Vals' token counts at Aight's rates, a Terminal-Bench 4.0 attempt costs $14.12 instead of $12.48.
• DeepSeek V4 Pro's cached input is $0.145 against DeepSeek's peak $0.044, and cached input is nearly all of an agent's tokens. A Terminal-Bench 4.0 attempt comes to $8.79 instead of $3.31, a DeepSWE attempt $4.10 instead of $1.67. On Aight, use V4.1 Flash: it beat V4 Pro on both Terminal-Bench runs and on Vibe Code Bench.

For Gemini 3.8 Flash, compare the price on Aight's models page with Google's $0.75 / $3.75 before routing work there.

Two limits to check:

• Check each model's context window and maximum output on Aight's models page before you send long diffs, since they can differ from the maker's own figures. The router below treats a cut-off reply as a failure, so a limit costs one rung, not a broken patch.
• Aight documents only Chat Completions. OpenAI says GPT-6.1 Sol needs the Responses API for tool calling, and GPT-6 Sol allows function calling in Chat Completions only with reasoning_effort set to none. Through Aight, use the Sol models where edits come back as plain text, as in the router below.

A routing pattern: low-cost model first, Claude on failure

You need tests that fail before the fix and pass after it. Then:

1. Send the task to the lowest-cost model and ask for a diff.
2. Apply it and run the tests.
3. If they pass, stop. If they fail or the reply was cut off, undo the diff and try the next model up.
4. If Claude fails too, hand the task to a person with every attempt's output.

With two rungs, expected cost per task is the first model's cost plus its failure rate times Claude's cost. On Terminal-Bench 2.1, V4.1 Flash then Opus 5.5 comes to $0.099 + 25.5% × $0.496 = about $0.23, against $0.50 for Opus alone. Treat that as a lower bound: the tasks Flash fails are the harder ones, and Claude spends more on those.

The same arithmetic shows when a cascade doesn't pay. GPT-6 Sol first, then Opus: $0.373 + 16.9% × $0.496 = about $0.46, barely under Opus alone. A first rung pays only when it costs a small fraction of a Claude attempt, which is why the ladder starts with V4.1 Flash and the middle rung is GPT-6.1 Sol ($1.72 per hard-task attempt against Opus's $13.20).

The code

This uses the OpenAI Python SDK (3.42.0, released 31 October) pointed at Aight and sends only model and messages. Newer Claude models reject temperature, as the temperature post explains. Run it on a clean branch: it applies patches to your working tree.

import os
import re
import subprocess
from openai import OpenAI, APIStatusError

client = OpenAI(
    base_url="https://api.getaight.ai/v1",
    api_key=os.environ["AIGHT_API_KEY"],
)

# Lowest cost first, Claude last. Aight prices, US$ per 1M tokens (input, output), 31 Oct 2026.
LADDER = [
    ("DeepSeek-V4.1-Flash", 0.30, 1.20),
    ("gpt-6.1-sol", 2.00, 10.00),
    ("claude-opus-5-5", 4.00, 20.00),
]

FENCE = "`" * 3
PROMPT = """Make the failing tests pass. Change as little as possible.
Reply with one unified diff against the repository root, in a {fence}diff block, and nothing else.

Test output:
{tests}

{files}"""


def run_tests(cmd):
    result = subprocess.run(cmd, capture_output=True, text=True)
    return result.returncode == 0, (result.stdout + result.stderr)[-4000:]


def git_apply(patch, *flags):
    cmd = ["git", "apply", "--recount", *flags, "-"]
    return subprocess.run(cmd, input=patch, text=True).returncode == 0


def fix(paths, test_cmd):
    passed, output = run_tests(test_cmd)
    if passed:
        return "already passing", 0.0
    files = "\n\n".join(f"File: {p}\n{open(p).read()}" for p in paths)
    prompt = PROMPT.format(fence=FENCE, tests=output, files=files)
    spent = 0.0
    for model, price_in, price_out in LADDER:
        try:
            resp = client.chat.completions.create(
                model=model,
                messages=[{"role": "user", "content": prompt}],
            )
        except APIStatusError as e:
            if e.status_code == 402:
                raise  # out of balance: a bigger model won't help
            continue
        usage = resp.usage  # cached input is billed for less, so this slightly overstates
        spent += (usage.prompt_tokens * price_in + usage.completion_tokens * price_out) / 1e6
        choice = resp.choices[0]
        found = re.search(FENCE + r"diff\n(.*?)" + FENCE, choice.message.content or "", re.S)
        if choice.finish_reason != "stop" or not found:
            continue  # cut off, or no diff: try the next model
        patch = found.group(1)
        if not git_apply(patch):
            continue
        if run_tests(test_cmd)[0]:
            return model, spent
        git_apply(patch, "-R")  # undo before the next model tries
    return None, spent


if __name__ == "__main__":
    model, spent = fix(["app/pricing.py"], ["python", "-m", "pytest", "-q", "tests/test_pricing.py"])
    print(model or "no model fixed it", f"${spent:.4f}")

• --recount lets git apply work out hunk line counts itself. Models often get those counts slightly wrong, and without the flag a correct change is rejected.
• Every model sees the original failing output, not the previous attempt, so a wrong diff doesn't anchor Claude on the wrong fix.
• Log model and spent per task to get your own pass rate per rung. For greenfield app work, try claude-sonnet-5-5 as the last rung, given the Vibe Code Bench result.

Using the same models in coding tools

The same key and model IDs work in tools that accept an OpenAI-compatible base URL:

• Aider reads OPENAI_API_BASE and OPENAI_API_KEY and takes openai/<model-id>. It gets edits back as plain text, so it doesn't need tool calling.
• Cline has an "OpenAI Compatible" provider with fields for base URL, key and model ID.
• Continue takes provider: openai with an apiBase; add useResponsesApi: false so it calls Chat Completions.

export OPENAI_API_BASE=https://api.getaight.ai/v1
export OPENAI_API_KEY=$AIGHT_API_KEY
aider --model openai/DeepSeek-V4.1-Flash

The one-key setup post covers more tools and frameworks.

Where Aight fits

Aight puts every model in this post behind one OpenAI-compatible key and one prepaid US dollar balance, so the ladder is three strings in a list, not four accounts.

It isn't the lowest-cost way to buy every rung: DeepSeek's own API costs half off-peak, and Kimi K3 and V4 Pro cost more on Aight than direct. What one key adds for a router like this:

• A key can be limited to the models on the ladder. Anything else returns a 403.
• A monthly cap per key stops a router that loops on a hard task with a 403 key_cap_reached.
• Kimi K3 runs on AWS Bedrock in the United States, and DeepSeek on Microsoft Azure, not DeepSeek's own servers. The models page shows where each model runs.
• Every recharge adds 10% extra balance: after the 3% processing fee outside India, about $0.94 per $1 of list-price usage. Unspent paid balance is refundable.

For how Aight compares with other multi-model services, see the aggregator pricing post. To try the ladder on your own tests, create a key limited to those three models.

Sources

• Vals AI, Terminal-Bench 2.1, Terminal-Bench 4.0, Vibe Code Bench v1.1, SWE-bench Verified, LiveCodeBench and methodology; model pages for Gemini 3.8 Flash, Kimi K3.1 Pro and Claude Haiku 5.5
• Terminal-Bench, official Terminal-Bench 4.0 leaderboard
• Datacurve, DeepSWE leaderboard v1.1
• SWE-bench leaderboards, Scale AI SWE-Bench Pro public leaderboard, Aider LLM leaderboards
• Anthropic, Pricing
• OpenAI, Pricing, GPT-6 Sol, GPT-6.1 Sol
• Google, Gemini API pricing
• Moonshot AI, Kimi model inference pricing
• DeepSeek, Models and pricing
• xAI, Models and pricing
• Tool docs: Aider with OpenAI-compatible APIs, Aider edit formats, Cline OpenAI Compatible, Continue OpenAI provider, openai on PyPI
• Aight, Models and pricing, API reference, Refund policy

Checked on 31 October 2026.