프런티어 AI 속도 조절, 독립 평가자와 오픈 웨이트의 검증 구조

Day 1: Anthropic employee quits to work at METR. Day 2: Anthropic nominates METR as "third-party evaluator".

Yuchen Jin

@Yuchenj_UW

1일 전

It’s shocking that Dario, Elon, and Sam all agree today that we should slow the pace of AI. I’m generally supportive of embedded third-party evaluators, but I have a few concerns: 1. who evaluates the evaluators? How do we make sure groups like METR remain fair, competent, and unbiased? 2. slowing AI progress can conflict with the incentives of frontier labs, especially when they’re preparing for IPOs. How do we make those incentives compatible? 3. you can only pace what you can measure and verify. What exactly are we measuring? How do you define and measure something like RSI? The idea sounds good. The implementation seems extremely hard.

Sholto Douglas

@_sholtodouglas

1일 전

Dario is making the case for the opposite. This actually makes our life harder and makes it easier for others to catch up with us, but we still think it is the right thing to do. Happy to come on the pod next week and talk about it!

Cline

@cline

1일 전

It's incredible seeing Anthropic commit to third-party evaluators. However, we believe open weights takes this same idea further. Anyone can inspect, evaluate, and red-team the model. We hope to see more open collaboration in building this powerful technology.

Self-improving agents are a top research topic right now. This new survey is a good map of the area. (bookmark it) It splits recursive self-improvement into stages of autonomy. An agent first executes improvements someone else designed. Then it chooses its own improvement strategy, collects its own experience, adapts to new environments, and finally improves the process of improvement itself. That staging makes claims easier to check. When a paper says its agent is self-improving, you can ask which of these stages it actually automates. The survey also uses a Headroom-Closed Index to show where current LLMs fall short, and compares requirements across scientific discovery, embodied agents and software engineering. Paper: https:// academy.dair.ai/papers/the-las t-ai-built-by-humans-toward-genuine-recursive-self-improvement-2609.11873 …

Why you can’t do RSI as fast as you like -

Insane progress for small language models! MiniCPM5-2B is a dense 2B-parameter model by OpenBMB from China that's built for reasoning, coding, and tool use on resource-constrained hardware. The model specifically excels at coding and tool calling, two capabilities central to the shift from on-device LLMs to on-device agents. Instead of only answering prompts, it can use tools, generate code, carry information between steps, and complete multi-step tasks. I ran it 100% locally and connected it to a small investigation agent with one request: > Revenue dropped last week. Investigate what happened, quantify the impact, identify the likely cause, and produce an incident report with supporting evidence. The evidence was spread across orders, traffic, payments, refunds, and deployment logs. The model inspected the files, wrote its own queries, analyzed the intermediate results, and decided what to investigate next. Each tool result informed the next action, so the final report depended on the model maintaining a coherent investigation across the complete trajectory. The recording shows the actual task from beginning to end. It starts with the revenue question, follows the tool calls and supporting evidence, and ends with a quantified diagnosis and incident report. The data, tool execution, and model inference all remained on my machine. These capabilities were optimized through Agentic Pre-training, SFT, and large-scale RL. They do not come entirely from an application-level agent framework. MiniCPM5-2B supports SGLang, vLLM, llama(.)cpp, Ollama, iOS, Android, and HarmonyOS. OpenBMB has also released the model weights and parts of the training recipes and data resources behind it. Download MiniCPM5-2B: https:// huggingface.openbmb.cn/model/openbmb/ MiniCPM5-2B?from=_avichawla … A 2B model can now maintain enough state to coordinate tools and complete a useful investigation on local hardware.

Very interesting paper if you are building with agents. How should a website tell an AI agent what it may access, for what purpose and at what price? robots.txt can only allow or disallow paths. It cannot say who is crawling, why, or on what terms, and automated clients now make up most web requests. This paper specifies terms.txt, a file that sets machine-access terms per path and per purpose. It pairs the file with a signed exchange built on Web Bot Auth signatures, signed intent, delegation tokens, HTTP 402 negotiation and signed receipts, all enforced by the origin server. The author is explicit about scope, separating what the exchange can enforce, what it can only audit and what is left to contracts. Paper: https:// arxiv.org/abs/2609.11152 Chat with Paper: https:// academy.dair.ai/papers/terms-t xt-a-consent-and-compensation-protocol-for-agentic-web-access-2609.11152 …