Astra for Coding and Software Engineering

What the Benchmarks Actually Tell You

Three benchmarks define Astra's coding story. Understanding what each one measures — and what it doesn't — prevents both over-reliance and dismissal.

DeepSWE v1.1 (74.1%). This is a 113-task agentic coding benchmark built on real open-source repositories. A task is not a code completion prompt. The agent is given a bug report or feature request, navigates the repository, writes a fix across multiple files, runs the existing test suite, and verifies the result. Astra scores 74.1%, against 72.7% for GPT-5.6 Sol, 73.8% for Gemini 3.8 Flash, and 69.9% for Fable 5. The models are clustered — the gap between first and last is four points. What matters is that 74.1% means working end-to-end on real engineering problems, not generating plausible-looking code you still have to verify by hand.

SRE-Bench (88% in one attempt, 99.2% in four). This benchmark measures whether a model can reverse-engineer compiled software binaries to understand their logic — without access to source code. Astra solved 88% on the first attempt, versus 55.9% for Sol. The 32-point gap is the largest across any coding benchmark and changes what's practical: tasks that required multiple retry attempts or human fallback under Sol become single-pass operations under Astra.

AutomationBench (41.4%). This measures chaining multiple coding tasks in sequence — requirements, repository analysis, plan, code changes, tests, and verification in a single agent run. Astra's 41.4% represents a doubling of the prior baseline of 18.1%. Full-pipeline automation wasn't viable at that baseline; it's worth building now.

One calibration note: on Artificial Analysis's broader Intelligence Index, Fable 5.1 scores 66 against Astra's 61. On the Coding Agent Index, Fable 5.1 leads 70 to 67. OpenAI's launch materials call Astra the best coding model to date; independent aggregators tell a more mixed story. The SRE-Bench result is the unambiguous gap. DeepSWE and AutomationBench reflect real improvement without large separation from the field.

Full-Codebase Review via 1M Context

The 1M-token context window is large enough to hold a complete small-to-medium codebase in a single session. You don't have to slice code into pieces, curate what's relevant, or paste individual files. You can upload the entire repository and ask Astra to reason across it without deciding in advance what matters.

Four review categories benefit most from this approach:

• Architecture review. Ask for a summary of the entry points, how the layers are organized, and where the major interfaces between components are. A useful prompt: "Walk me through how a request enters this system, what it touches, and where it exits. Identify any architectural inconsistencies or missing separation of concerns."
• Auth flow audit. Ask Astra to trace all authentication and authorization paths. Prompt: "Identify every place where access control decisions are made and describe what those checks are doing. List any paths where the check is missing or incomplete."
• Data validation coverage. Ask for a survey of where external input enters the codebase and where it's validated before use. The goal is a list of unvalidated or under-validated inputs, not a description of validation that exists.
• Dependency risk review. Paste your dependency manifest alongside the codebase and ask Astra to identify packages with high usage and low maintenance signal, or packages with known CVEs within the model's knowledge cutoff.

Structuring the prompt for actionable findings matters as much as what you ask. Specify the output format upfront: "Give me a numbered list of findings, each with the file path and line range, what the problem is, and a recommended change." Narrative summaries are harder to act on than structured findings. For security audits, ask Astra to distinguish confirmed issues from patterns worth investigating — mixing certainty levels in a single list slows triage.

One constraint to keep in mind: Astra's knowledge cutoff is April 30, 2026. For dependency CVE checks, supplement with a current database. For anything that touches recent framework releases or security advisories published after that date, pass the relevant docs as additional context rather than relying on training data alone.

Bug Triage from Logs and Code

The most effective bug-triage session combines three context sources: the stack trace or error output, the relevant source files, and the recent commit history. All three together give Astra what it needs to identify root causes rather than surface symptoms.

The prompt pattern that works reliably:

1. State the symptom precisely: "This function returns null when the input list contains more than 50 items."
2. Paste the full stack trace or error output in a code block.
3. Paste the relevant source files — not the entire codebase for a targeted bug, but the files the trace points to plus the functions those call.
4. Include the last 5–10 commit messages and diffs that touched these files, if the bug appeared recently. Ask: "Which of these changes is most likely to have introduced this behavior?"
5. Close with: "Identify the root cause. Propose a minimal fix. Explain the risk of applying that fix — what else might break."

The "minimal fix" framing is important. If you ask Astra to fix a bug without scoping it, you'll sometimes receive a refactored function or structural change larger than what you need. "Minimal fix" keeps the answer focused on the specific failure. You can always ask for a larger refactor as a follow-up once the root cause is confirmed.

The risk-explanation step prevents surprises. Astra will often catch that a minimal fix trades one failure mode for another — for example, that adding a length check prevents the null return but exposes an empty-list path that downstream code doesn't handle. Asking for that analysis upfront means you see the tradeoff before you commit the change.

Defensive Security Use Cases

Astra is the first OpenAI model rated "Critical" under the company's Preparedness Framework — an internal safety classification that defines Critical in cybersecurity as the ability to find and exploit novel vulnerabilities in hardened systems without step-by-step human guidance. OpenAI delayed parts of the release by several weeks to strengthen protections against cyber misuse before shipping.

The context for why: Astra scored 100% on ExploitBench (OpenAI's benchmark for developing working exploits from known software vulnerabilities) and demonstrated the ability to discover two previously unknown zero-day vulnerabilities. The model is restricted from using those capabilities offensively. What it will do for you is the defensive half of the same skillset.

What Astra will do:

• Identify OWASP Top 10 vulnerabilities in submitted code: injection flaws, broken authentication, insecure deserialization, misconfigured security headers, and similar
• Review a pull request for security regressions — new code that weakens or bypasses existing controls
• Validate a patch: confirm it actually closes the vulnerability it claims to fix and doesn't introduce a new one
• Generate Semgrep rules for a class of vulnerability found in your codebase
• Produce a structured vulnerability report with severity, affected code paths, and remediation steps

What Astra will refuse:

• Writing proof-of-concept exploit code for a described vulnerability
• Weaponizing a zero-day — turning a discovered vulnerability into working attack code against a live system
• Generating shellcode, payloads, or anything functionally equivalent to an attack tool

These refusals are consistent and not easily circumvented. Phrasing a request as "educational" or "for a CTF" doesn't change the outcome for serious exploit creation requests. The Preparedness Framework's Critical classification means OpenAI's monitoring is tighter on these requests than for any prior model.

For cybersecurity defenders who need less-restricted access — malware analysts, penetration testers with documented scope, incident responders — OpenAI runs a program called Daybreak that provides early, less-restricted access to frontier models for defensive use cases. It requires organizational verification and is separate from standard API access.

Setting Up Your Coding Workflow

Three decisions shape how useful Astra is for ongoing coding work: how you handle large files, whether you use Projects, and which reasoning tier you request by default.

Handling large files. In ChatGPT, you can paste code directly into the message box or use the attachment button to upload files. For files over a few hundred lines, attachment is cleaner — it preserves the file name as context and doesn't fragment across the message scroll. For an entire codebase, zip the repository and attach the zip file. Astra can read the directory structure and individual files from a zip. For a targeted session focused on a specific bug or module, pasting the relevant files inline is faster.

Using Projects. ChatGPT Projects persist context across sessions. Set up a Project for a codebase you return to regularly and include the files that don't change often — schema files, core library modules, configuration — in the Project instructions. Astra will carry that context into every conversation in the Project without you re-pasting it. For context that changes with each session (new stack traces, recent commits), paste it inline in the individual conversation.

One habit that pays off: keep a short context.md file in the Project instructions with a paragraph describing the codebase — its purpose, main technologies, and any conventions the team uses for naming, error handling, and test patterns. Astra's suggestions will match your codebase's style more consistently than if it's inferring conventions from code alone.

Reasoning tier for coding tasks. OpenAI's own guidance puts agentic coding at medium effort. That's the right default for most coding tasks — debugging, code review, feature implementation with clear requirements. Move to high for multi-file refactors, architecture analysis, or any run where the agent works autonomously for an extended period. Use xhigh for tasks where a wrong answer cascades: a migration that modifies production data, or a security audit where a missed vulnerability has real consequences.