에이전트 감독 난제, 실제 협업 데이터 개방, 컴퓨터 사용 모델과 오픈 웨이트 확장

METR & Redwood Research investigated agent behavior in the Hugging Face incident. We found agents developed a universal cheat for ExploitGym within 4 hours, then coordinated multi-day R&D efforts to trick the scorer into accepting cheats, including trying to tamper with logs.

I was the main person doing transcript analysis for this investigation of the Hugging Face incident. My main takeaway: We don't have good approaches for understanding/overseeing the activity and aims of AI 'swarms'. I semi-jokingly called our efforts a "slop-vestigation" because we were so reliant on AIs to analyze what happened and there were a huge number of different important things to analyze. The total quantity of data—over a thousand extremely long transcripts from agents that ran for multiple days—made it impossible to understand what was happening, especially in aggregate, without heavy reliance on AI tools. The agents we used for classification and analysis were similarly capable to the agents involved in the incident, but this didn't mean these agents could be easily used to oversee and understand the incident. Outputs from analysis agents were often missing key details, wrong, overconfident, or really hard to understand. We discuss various examples in our report, mostly in the limitations and methodology sections. Additionally, AI agents themselves seemed to have a hard time understanding what happened and their explanations of what happened were often overconfident. Keep in mind that a single analysis agent would itself only be able to read a tiny fraction of all of the transcript data into context, and AIs may themselves have trouble getting subagents to do informative analysis for them. We did our best to manually check the most important claims and we tried to get the AIs doing this analysis to write up their argument (with evidence) clearly enough that we could check whether it made sense. But overall, it was difficult to get a precise understanding of events and we were missing aspects of the story that we now think of as key until almost the end of our investigation. Our understanding of what overall happened and the aims of the 'collective' shifted greatly during our last time on premises (when we obtained a much more complete dataset). For instance, we didn't learn about the tool call spoofing, the story of PHASEONE[big], and the broad structure of different workstreams until this third time on premises (and for many of these, we only reached a decent understanding on the final day of that visit). It was legitimately hard to figure out what was going on and why! The difficulty of understanding incidents and overseeing AI agents appears to be growing faster than the rate at which more capable AIs help us with oversight and understanding. As in, AI capabilities (and propensities?) for achieving large, ambitious, and misaligned objectives are growing faster than our ability to understand what these agents are doing. While understanding this incident was difficult, there were some mitigating factors in this case that probably won't apply in the future: - Models communicated and reasoned in natural language. In the future, this reasoning may occur (entirely or almost entirely) in activations. - While the scope of this incident was massive, the scale of agentic activity was still less than we'll see in the future (e.g., misalignment incidents that involve agent teams running entire companies). - The AIs involved in this incident weren't generally much more capable than humans (though they may have been somewhat superhuman at some limited and very narrow abilities). - We didn't have strong reason to believe that the AIs we used to help us investigate this incident would try to intentionally sabotage or otherwise undermine our investigation. In the end, I think we were able to get some understanding of the events, map out the overall story, and get a pretty good aggregate understanding of the chain-of-thought reasoning on some important topics (e.g., how did the AIs reason about helping other AIs, did the AIs know what they were doing was undesired, what deception did the AIs engage in, and how did they think about it). But overseeing AIs and understanding misalignment incidents is difficult and it looks like it is going to get harder.

It’s such a rare opportunity to study human–AI collaboration at scale, given the constraints around data access and privacy. Huge thanks to @AnthropicAI for making this possible through the Anthropic Insights system, and for their support throughout the project. We started this work with several hypotheses we’d been thinking about for a long time (around task criticality and delegation, human agency, and friction). But what surprised us is that the findings consistently point back to the **human side** of human–AI collaboration. More details in

“Human–AI Collaboration at Scale” This new Anthropic study shows how people actually work with AI using 249,834 real Claude conversations, focusing on task criticality, human agency, learning, and friction. They found that AI is already being used for consequential work, but mostly as augmentation rather than full automation, with 72% of conversations still human-led and users engaging more deeply as stakes rise. AI also teaches in 67% of relevant conversations, while friction appears in roughly half of interactions and is often productive when users actively repair, challenge, or clarify the model’s output. So the benefits of AI depend heavily on how much agency and critical engagement the human keeps during the interaction. https:// alphaxiv.org/pdf/2608.human -ai-collaboration-at-scalev1 …

People were always surprised when I told them small models can beat big models in a specific domain and a specific domain doesn't mean niche. I am glad that Yutori annouces the param size and now I have a proof how efficient the model is. When a field is growing so fast, no one cares about cost, and tokenmaxing and scale maxing were the brute force way to deliver results. However, as it becomes more mature, efficiency, reliability and cost-effectiveness become the blocker to mass-adoption. I do think AI is entering adolescence. Don't get me wrong. Bigger model is smarter, scaling law will continue, and I am sure the unrelased models in Anthropic and OpenAI are scary. Model scaling is critical in pushing the boundary of humanity, e.g. AI4S, or "immortality"? That said, 99.9% of humans on Earth care more about living a better life today than achieving the big dreams. For every day task, do we really need Einstein-level-or-higher IQ? Or would we prefer an assistant who is efficient, reliable, and understands our preferences?

The computer use frontier is being pushed by @yutori_ai - and I think their strategy is spot on. As @DhruvBatra_ told me a few months ago, most of the web will not get APIs for agents anytime soon: https:// chainofthought.show/podcast/63-mos t-of-the-web-will-never-get-apis-for-ai-agents-dhruv-batra/ …

DogeDesigner

@cb_doge

1일 전

Grok Build made this fun lil Mars simulator game for my nephew. I gave it simple prompts. It did the rest: wrote systems, built and tested it, studied gameplay, found and fixed bugs, then kept iterating. Grok Build is very powerful. It feels like having a game studio on demand.

Aravind Srinivas

@AravSrinivas

1일 전

We are doubling down on making Computer more useful for financial researchers and analysts. Computer now connects to new licensed data sources, including Dun & Bradstreet, Guidepoint, IBISWorld, and 20+ others. An analyst can ask Computer a question and get an answer built from the firm's own licensed sources without APIs or separate logins to work through. Every figure traces to the source record it came from. Teams doing high-stakes work would know exactly which data set produced which number. To connect, select a provider in Settings → Connectors and sign in with an existing license. Available now to all Perplexity users. https:// perplexity.ai/hub/blog/compu ter-connects-to-20-new-licensed-finance-data-sources …

The International Math Olympiad is the hardest math competition in the world, where students compete on proof problems most math PhD’s even struggle with. DeepSeek V4 Flash won a gold medal for only 12 cents.

Notably, DeepSeek V4 Flash isn’t post-trained for Olympiad problems. It has 284B total MoE parameters, with about 13B activated per token. This is the only model (so far) that can be run on small local GPU setups and still win an IMO gold.

NVIDIA Newsroom

@nvidianewsroom

22시간 전

NVIDIA and @awscloud are expanding the partnership across the full stack — GPUs, CPUs, networking, open models and software — to make agentic and physical AI real at a pace and scale that only we can deliver together. What that looks like: 2 million additional NVIDIA GPUs across AWS’s global infrastructure NVIDIA Vera CPU coming to AWS for agentic AI Faster, more power-efficient memory with NVLink Fusion with NVHBM AI factories for the U.S. government, including 100,000 GPUs on secure AWS infrastructure Demand is running ahead of every forecast and we’re building to match it. Read the release: