Open Secure AI Alliance 출범과 추론·에이전트 인프라 혁신

Jensen Huang

@JensenHuang

Attackers have frontier AI. Defenders need a frontier AI ecosystem—the best open and closed models, force-multiplied by a global community. During the Hugging Face incident, closed AI blocked essential forensics. An open-weight frontier model helped contain the intrusion. That’s why we created the Open Secure AI Alliance.

Proud to be an inaugural partner alongside @NVIDIA and other industry leaders in the Open Secure AI Alliance.  We look forward to accelerating the adoption and trust of open models and harnesses through this initiative.

every inference engine makes the same mistake. an inference engine like vLLM or SGLang is the software sitting between your request and the model weights. its job is to load the model onto the GPU, run your prompt through it, and stream tokens back as fast as the hardware allows. everything it does well comes down to keeping the GPU busy doing math. as context windows grew, engines took on cache management too, and it went into that same process. the KV cache is the model's saved understanding of tokens it has already read, and storing it, loading it back, and shuttling it between GPU and CPU all compete with attention for the same resources. so the two take turns. Google's TurboQuant shows how tight that constraint is. it shrinks the KV cache to 3 bits per value with no loss in accuracy, so cached blocks take up less GPU memory and travel faster out to CPU RAM or disk. the shrinking is not free, though. something has to compress each block on the way out and decompress it on the way back, and that work runs on the same GPU that is generating tokens. switch TurboQuant on and inference throughput drops 20%+ compared to leaving it off. the compression is not the problem, and the process it runs in is. LMCache is an open-source solution that moves cache management into a separate process running alongside the engine. three things follow from that split. → the handoff is tiny. the engine sends the block IDs it needs, which are a few identifiers rather than gigabytes of data, and all the actual movement of cached vectors happens on the other side while the engine keeps computing attention. → both processes share GPU memory directly. passing cached state between two GPUs normally costs several copies of the data, and here they read and write the same region, so a cached document stops being duplicated per worker. → lookups run in parallel. cached blocks live in four places, GPU memory first, then CPU RAM, then local SSD, then cloud storage. checking them one at a time means the slowest one sets your floor, so LMCache queries all four at once and streams from whichever answers first. that's the real shift. cache work is I/O-heavy and inference is compute-heavy, and keeping them in one process means every improvement to your caching costs you inference time. on H200s with Qwen3-235B and 50 concurrent users, separating them gives 14x faster time-to-first-token and 4x faster decoding. startup drops from over three minutes to about 30 seconds. @lmcache directly plugs into vLLM, SGLang, and TensorRT-LLM, runs on NVIDIA and AMD, and survives a crash on either side, with the engine falling back to uncached inference until the cache process reconnects. check it out on GitHub: https:// github.com/LMCache/LMCache (don't forget to star ) the diagram below traces one prompt through the whole path. i wrote the full breakdown of how LMCache works, and the article is quoted below.

Sydney Runkle

@sydneyrunkle

new deepagents version is about to drop! we just released a v0.7 beta: 1. much more minimal prompting -- we cut base input tokens by more than 1/2! 2. more configurability -- override any default middleware w/ your custom settings test it out and let us know what you think!

LangChain

@LangChain

We just released Deep Agents v0.7.0b2 This release offers a more performant, more configurable base harness. Test it out and let us know what you think! https:// docs.langchain.com/oss/python/rel eases/changelog#deepagents-v0-7-0b2 …

Your Claude chats might already be public on i using Google dork. Including your mental health data, Api key, DB_PASSWORD, documentation, business idea, Full resumes with real names & contacts , Internal company projects , Legal discussions, Deeply personal conversations, Todolist, mail addresses, When you use Claude’s Share feature, it creates a public web page. Turns out those pages are getting indexed by search engines. This is the same mistake ChatGPT made last year.

A free 1,900+ AI models in playground live list of (and 19k+ nodes) that are publicly reachable no login in needed Thousands of large models (Kimi 1T, DeepSeek 700B+) GLM, Mistral variants… all usable right now sitting openly on the internet. Direct use No login. No credit card. Just pure compute. - http:// stolencompute.com