Reasoning Effort as a Parameter

The reasoning_effort Parameter

Every request to Astra through the Responses API carries a reasoning configuration. When you omit it, the model defaults to medium effort. When you include it, you control how much compute the model spends on internal reasoning before producing output. The parameter sits in the reasoning object, under the key effort:

from openai import OpenAI

client = OpenAI()

response = client.responses.create(
    model="gpt-6-astra",
    reasoning={"effort": "medium"},
    input="Refactor this function and explain each change.",
    max_output_tokens=4000
)
print(response.output_text)

Astra supports five effort values. none and minimal are rejected — the lowest available setting is low.

low — Fast, inexpensive responses. Minimal internal iteration before output. Use for classification, extraction, or tasks where the correct answer is unambiguous.

medium — The default. Balanced cost and quality. Suitable for most production tasks: summarization, code review, question answering, multi-step instructions.

high — Noticeably more thinking. Suitable for tasks where errors are expensive — contract analysis, security review, complex code generation.

xhigh — Extended reasoning. Use for problems that require multi-step planning or tasks that previously needed many retries on lower settings.

max — Several minutes of dedicated compute for genuinely novel or hard problems. Benchmarks use this tier. Production applications rarely need it unless accuracy on a single expensive task is the primary concern.

An important calibration point from OpenAI's own guidance: Astra on low frequently outperforms GPT-5.6 Sol on high. If you migrated from Sol and defaulted to high to get acceptable results, test medium first — you may get equivalent quality at significantly lower cost.

Cost and Token Tradeoffs Per Tier

Astra's base API pricing is $10 per million input tokens and $50 per million output tokens on the standard tier. The critical detail for reasoning: reasoning tokens bill as output tokens at $50 per million. Every internal thinking step the model takes is added to your output token bill.

This makes reasoning_effort the single largest cost lever in your request. Independent benchmarking found a roughly fourfold difference in total token consumption between low and max on the same input. A task that costs $0.02 at low will cost approximately $0.08 at max.

The non-obvious insight: max effort on a hard task can be cheaper than low effort when low produces unusable output. A response the application discards and retries costs twice — the failed request and the retry. If 30% of your low-effort responses are discarded, the effective price is 1.4× the stated low rate, approaching what a single medium request would cost. Measure before assuming the cheapest tier is cheapest.

Other cost levers: the Batch and Flex service tiers run Astra at half price ($5/$25 per million input/output). For latency-tolerant workloads — bulk document processing, overnight analysis runs — batch mode reduces cost independently of effort level.

Persisted Reasoning Across Turns

In a standard multi-turn Responses API session, each request processes the full conversation history from scratch. The model reads every prior message and rebuilds its context fresh. This is correct, but expensive when turn 2 reasoning largely builds on reasoning already done in turn 1.

Persisted reasoning addresses this. When enabled, the model renders compatible reasoning items from earlier turns into its current context rather than discarding them. The result is continuity of thinking state across turns — not just memory of prior messages, but continuity of reasoning without re-processing the full context.

The parameter is reasoning.context. Two values are available:

• all_turns — Reasoning items from all available prior turns are rendered into context. GPT-6 Astra supports this. Token cost increases because more reasoning items are included, but each subsequent turn benefits from prior reasoning work.
• current_turn — Default for older reasoning models. Reasoning is scoped to the current turn only and not carried forward.

from openai import OpenAI

client = OpenAI()

# First turn: initial diagnosis
response1 = client.responses.create(
    model="gpt-6-astra",
    reasoning={"effort": "high", "context": "all_turns"},
    input="Here is a failing test suite. Diagnose the root cause.",
    max_output_tokens=4000
)

# Second turn: build on prior reasoning state via previous_response_id
response2 = client.responses.create(
    model="gpt-6-astra",
    reasoning={"effort": "high", "context": "all_turns"},
    previous_response_id=response1.id,
    input="Now generate the fix for the root cause you identified.",
    max_output_tokens=4000
)

Two things to keep in mind. First, the reasoning items stay opaque — the API never returns the internal reasoning text. Persisted reasoning improves quality without exposing the model's thinking. Second, because all_turns renders more items into context, it increases tokens billed per request. For short sessions of two or three turns the improvement is usually worth it. For sessions with many turns, monitor token consumption per turn and evaluate whether the quality gain justifies the cost at each step.

The most practical use case: multi-turn debugging workflows where the model needs to hold a consistent mental model of a codebase, error state, or diagnostic chain across several exchanges without losing the thread of prior analysis.

Measuring the Right Tier for Your Use Case

The right effort tier for your application is not determined by intuition — it is determined by measurement. The metric that matters most is accepted-task rate: the percentage of model responses your application actually uses, versus the percentage discarded and retried because the output was wrong, incomplete, or unusable.

Cost per accepted task = total cost / accepted tasks. This is the real price of a tier, and it often changes the comparison significantly. A tier with twice the stated token cost but a 50% higher accepted-task rate has the same or lower real cost per useful response.

Here is a minimal instrumentation pattern:

import time
from openai import OpenAI

client = OpenAI()

def run_with_measurement(prompt, effort="medium"):
    start = time.time()
    response = client.responses.create(
        model="gpt-6-astra",
        reasoning={"effort": effort},
        input=prompt,
        max_output_tokens=4000
    )
    elapsed = time.time() - start
    input_tokens = response.usage.input_tokens
    output_tokens = response.usage.output_tokens
    cost_usd = (input_tokens / 1_000_000) * 10 + (output_tokens / 1_000_000) * 50
    return {
        "output": response.output_text,
        "cost_usd": round(cost_usd, 6),
        "latency_s": round(elapsed, 2),
        "input_tokens": input_tokens,
        "output_tokens": output_tokens
    }

Run this across 100 representative tasks at medium. Record total cost, latency, and which responses your application accepted. Compute accepted-task rate and cost per accepted task. Then decide whether escalation is warranted.

Escalate to high only when: accepted-task rate at medium falls below your target; and the cost per accepted task after escalation stays within your budget. Do not escalate preemptively — the assumption that harder reasoning always produces better output is not always borne out, and the token cost is real and immediate.

If medium has an acceptable task rate but latency is the problem, explore Batch mode before reaching for a higher tier. Cost and latency are different constraints — escalating effort addresses quality, not latency, and adds cost on top of an already acceptable task rate.