에이전트 안전 공개 기준, 금융 평가셋 확장, Astra·Percept-Lens의 도구화

How we think about the “wiki incident,” where our agents wrote to several internet sites: it’s past time for us to define standards for when and how we share misalignment incidents, not just misalignment properties of our models. Historically, we have treated misalignment largely as a research question, which gets communicated in research publications such as systems cards. This year, we’ve started to see misalignment cause new types of real-world impact. For the Hugging Face incident, where misalignment led to security impact to us and third parties, we followed a traditional security incident response playbook. We immediately started working with Hugging Face to understand what had happened and also disclosed publicly the very next day. Our investigation continues, and we are continuing to notify parties whom our models impacted in less significant ways. Prior to the Hugging Face incident, we saw early signs of agents using the internet in unintended ways, as reported in https:// openai.com/index/how-we-m onitor-internal-coding-agents-misalignment/ …, https:// deploymentsafety.openai.com/gpt-5-6, and https:// openai.com/index/safety-a lignment-long-horizon-models/ …. We considered the wiki incident to be an instance of misalignment similar to the ones we’d shared. Our misalignment disclosure practices need to expand for this new phase of model capabilities. We and the larger AI community do not yet have a clear standard for how to report misalignment that shows up during training, evaluation, and deployment, including examples that don’t look like traditional security incidents but could provide insight into AI behavior and future risks. We’re working on a framework and will share it in upcoming weeks, and in parallel we're working with dozens of government regulatory agencies worldwide on these issues.

Since its release, FinFIRST has drawn a lot of interest. Built with finance experts, it uses atomic rubrics to assess both answers and evidence, including how agents search, handle timely real world tasks, combine sources, choose reliable evidence, calculate and make results easy to verify. 🧵

FinFIRST V2 is on the way, with broader coverage of foundational financial search and advanced research tasks. Hope FinFIRST helps you uncover useful answers—and sparks a few new questions to explore. 🤝 Hugging Face Dataset: https:// huggingface.co/datasets/inclu sionAI/FinFIRST … ModelScope Dataset: https:// modelscope.cn/datasets/inclu sionAI/FinFIRST … Technical Report: https:// huggingface.co/datasets/inclu sionAI/FinFIRST/blob/main/FinFIRST_paper.pdf …

Greg Brockman

@gdb

1일 전

Astra for helping in your personal and work life

Teknium 🪽

@Teknium

1일 전

You can now choose @perplexity_ai as your web search and web scrape tool backend in Hermes Agent. Enjoy!

Interesting project: Reef tries to make the agent improve itself, and that improvement doesn’t have to mean retraining the model. It can also evolve prompts, rules, and skills.

Percept-Lens: A Deep Dive into AI-Generated Image Detection