OpenAI 평가 사고와 에이전트 안전·장기 문맥 논쟁

Elon Musk

@elonmusk

1달 전

Troubling …

GPT 6 escaped its sandboxes through zero day exploits to try to figure out how to benchmax For the good of all please nobody release a paper clip benchmark for future models to max

Hidden Hermes gem: /rollback I just watched an AI agent destroy a codebase in 3 seconds. rm -rf src/ rm -rf tests/ git reset --hard It ran them all in a single turn. The agent made a logic error early on and cascaded into catastrophic failure. So I typed: /rollback 1 Everything came back. Files restored. Agent context rewound. Like it never happened. This is the checkpoint system. Here's what it does: Hermes auto-snapshots your project before EVERY destructive operation: - File writes (write_file) - File patches (patch) - Dangerous terminal commands (rm, git reset --hard, sed -i, etc.) One snapshot per directory per turn. No spam in long sessions. Then when things go wrong (or right, but you want a different approach): /rollback lists all checkpoints with change stats /rollback <N> restores to checkpoint N AND rewinds agent context (filesystem state matches agent context) /rollback diff <N> previews what changed since checkpoint N /rollback <N> filename restores a single file from checkpoint N How to enable: Per-session: hermes chat --checkpoints Or globally in ~/.hermes/config.yaml: checkpoints: enabled: true How it actually works under the hood: Single shared git store (~/.hermes/checkpoints/store/) Git's content-addressable object DB deduplicates across projects Per-project refs track checkpoint history independently Result: after first snapshot, each new snapshot costs near-zero storage Safety guardrails: Skips files larger than max_file_size_mb (default 10 MB) Skips overly broad directories (root /, home $HOME) Skips repos with 50k+ files Auto-prunes orphan projects (whose working directory no longer exists) Respects max_snapshots (default 20) and max_total_size_mb (default 500) If git is not on PATH, checkpoints transparently disable Non-fatal errors log at debug level; your tools continue running What it's NOT: Not a git replacement Not full version control Not recursive backup magic Just a safety net for AI operations. Opt-in. Silent. Near-zero cost. Best for: Unattended automation (cron jobs, deployments, CI/CD) Multi-step refactoring where mistakes cascade Bulk file operations you want to abort instantly Experiments you're not 100% confident in Real example: Agent refactoring your package structure: Snapshot 1: write_file for new modules (+847 lines) Snapshot 2: patch imports across 15 files Snapshot 3: terminal rm -rf old_src/ Somewhere in step 3, it deletes more than intended. /rollback 1 Back to before the refactoring started. Agent context rewinds. You're in control again. This is what separates: - Hands-on automation (you watch every step) - Trusted automation (you set it loose, but you have an undo button) CLI commands (outside a session): hermes checkpoints hermes checkpoints status hermes checkpoints prune hermes checkpoints clear Config options: checkpoints: enabled: false max_snapshots: 20 max_total_size_mb: 500 max_file_size_mb: 10 auto_prune: true retention_days: 7 Why this matters now: Most AI agent frameworks have no undo. You're supposed to use git manually. Hermes built it in, transparent, automatic, intelligent.

Elon Musk

@elonmusk

1달 전

You can talk to Grok like a person to accomplish tasks via Grok Build http:// X.ai/cli

Elon Musk

@elonmusk

1달 전

Try it out!

Open weight models galore! Laguna S 2.1 went from the start of training to launch in under 9 weeks! Custom & open models are putting a lot of pressure on the frontier closed and general models. No wonder they want to ban them. Diverse intelligence is good for the market.

Soumith Chintala

@soumithchintala

1달 전

this looks pretty good for agentic. that it fits on a dgx spark is **chef's kiss**

Progressive disclosure in agents doesn't scale. And its benefits seems agent harness dependent. (bookmark this one) Finally there is a proper study on using agent skills and the effect of progressive disclosure. Progressive disclosure is the agent skills pattern where you hand an agent a document path and let it decide what to read, from a short description down to specific passages. Practitioners adopted it fast for book-length tasks, purely on vibes. Researchers ran it across three agent harnesses and three model families on InfiniteBench. On a single book: > the gain is harness-dependent, > large when the agent navigates raw documents poorly, > near zero when a strong harness already retrieves on its own. Scale to many books and raw navigation falls apart while one level of disclosure pulls ahead. A second routing level never helps and sometimes breaks accuracy. It feels like progressive disclosure buys context, but not intelligence. It is redundant while a strong agent can find the passage itself, and decisive once the corpus is too large to read. Paper: https:// arxiv.org/abs/2607.17598 Learn to build effective AI agents in our academy: https:// academy.dair.ai

Very cool idea to convert memory to skills. (bookmark it) Most agent memory systems retrieve past traces as passive context. MSCE turns them into executable skills instead. The training-free framework organizes agent experience into grounded step traces, reusable procedural policies, and declarative environmental cognition. Evidence-backed policies with positive estimated gain get converted into callable skills that carry their applicability boundaries, verification rules, and reliability estimates. So a skill knows when it applies and how to check itself. The other piece is reflection-weighted value backfilling, which propagates sparse terminal feedback through dense local self-reflections to produce evidence-calibrated trace values. Those values govern which memory and skills survive. On EvoAgentBench and LoCoMo, MSCE outperforms strong skill-augmented and memory-driven baselines with cross-domain transfer. The gap between memory-as-context and memory-as-capability seems to be where long-horizon agents actually compound. Paper: https:// arxiv.org/abs/2607.16621 Learn to build effective AI agents in our academy: https:// academy.dair.ai