GPT-6 Astra의 컴퓨터 사용, 모델 라우팅, 에이전트 실행 계층의 재편

This is GPT-6 Astra. Anything you can do on a computer, Astra can do for you. Fast.

What a week for Terminal-Bench-Science & AI for Science in general! 🥂 Just 1 week after launch, Terminal-Bench-Science 0.1 is now also the #1 featured benchmark on today's GPT-6 Astra release by @OpenAI . GPT-5.6 Sol → 22.4% GPT-6 Astra → 64.6% The +42.2pp jump is out of this world, and that's on top of the Claude Fable 5.1 numbers from two days ago. To all scientists out there: We need to buckle up and bring the world's toughest scientific challenges together for Terminal-Bench-Science 0.2. Deadline is October 5. Let the games begin!

Researchers found a surprisingly simple path to 82% fewer LLM errors. Instead of searching for one universally best model, they combined the strengths of many. 21 LLMs were evaluated across 16 benchmarks, running every model 10 times on each prompt. Different models failed on different problems. Even the same model could fail once and succeed on another attempt. So they calculated what happens when the optimal model is selected for every prompt. At matched cost, this reduced the average error rate by 54% compared with the best individual model. It also matched SOTA quality at 85% lower cost. With up to 10 answers and a perfect judge selecting the best one, the error reduction reached 82%. Model routing itself is not new. What is new is measuring its true ceiling while correcting the statistical bias created by selecting lucky results from noisy runs. Production routers do not select perfectly today and this research reveals how much performance existing models may already contain when used together. Martian also turned the research into AI Frontier, an interactive dashboard for exploring the results, and I worked with their team today. More details below.

This chart will worry open-source labs! Researchers just measured the actual dollars models consume per solved task across dozens of LLMs and 16 benchmarks, then compared that against quoted per-token pricing. Sorted by how far real cost exceeds quoted pricing, all 8 models at the worst end are open-weight, coming from Kimi, DeepSeek, Qwen, and GLM. The first closed model appears tenth, and on the favorable end, the first open-weight model shows up ninth. A pricing page quotes dollars per token, but it cannot quote how many tokens a model needs, and that second number is where open-weight reasoning models spend heavily. R1-style models are trained with rewards on the final answer only. A longer chain of thought raises the odds of hitting a correct step, so training reinforces long chains, and every one of those thinking tokens is billed as output. Closed labs appear to tune this harder, with length-aware rewards and effort controls that spend tokens only where the task needs them. A recent benchmark, OckBench, measured the same efficiency gap between open-weight and proprietary models and called it the overthinking tax. Tokenizers widen the gap further, since every lab counts tokens differently. Tibo from OpenAI made that half of the argument recently, that a lower price per token doesn't guarantee a lower bill. Open-weight models still make sense for self-hosting, fine-tuning, and data control. But pricing is not the right way to compare them. All of this is now documented in AI Frontier. It's an interactive dashboard by Martian, and the method behind it is a peer-reviewed ICLR 2026 paper. The screenshot below depicts the published dashboard, and I worked with the Martian team today to share it. More details in the quote below.

ClaudeDevs

@ClaudeDevs

7시간 전

We're exploring a new way to let you extend and customize Claude Code: Function Hooks. Here's a couple videos showing what you'd be able to do. It hasn't shipped yet, we'd love feedback on this on our GitHub issue.

Thank you to the @SemiAnalysis_ team for the shoutout and for the collaboration on AgentX 🙏 Benchmarks are only useful when they measure the workloads people actually run, and AgentX measures the real thing: multi-turn, long-context agent traffic. vLLM is the engine for production agentic workloads. For teams serving tokens at scale, revenue depends on optimized inference over long multi-turn contexts. Our AgentX deep dive blog is coming soon! Stay tuned 📖

We’re bringing new voice capabilities to @GoogleWorkspace to help you tackle daily tasks. These are now rolling out across @Gmail , @GoogleDocs , and Keep to help you search your inbox, organize your thoughts, or brainstorm new ideas conversationally. Here's how you can use them to get things done: 📤 Gmail Live: Skip the manual search and use your voice to quickly find specific details buried in your inbox. 🗣️ Docs Live: Talk through your ideas and build structured, context-aware documents on the fly. 🧠 Keep Live: Turn your stream-of-consciousness brain dumps into organized lists and notes without typing a word.

Google Photos in Gemini Spark is here. Put your photo library into action to handle end-to-end tasks with a single prompt. 📸 100+ vacation shots ➡️ Cleaned-up photos in a shared album & emailed recap 📝 Whiteboard snapshots ➡️ Drafted team emails 🗓️ Event flyers ➡️ Checked for

Can Enterprise AI Assistants Be Trusted Under Pressure? Even though AI assistants can follow rules when asked directly, they may still break them when speed, cost, or convenience are at stake. This paper introduces PACT, a benchmark that tests whether models keep following

Google Research

@GoogleResearch

6시간 전

Today on the blog, we evaluate ways to improve cross-population genetic risk prediction. While transfer learning from European cohorts improves prediction in small populations, it degrades accuracy as target cohort sample sizes grow. Learn more: https:// goo.gle/4zR9XFf