Gemini Omni 영상 생성, llm-d 추론 인프라, 에이전트 안전성 연구, 구조 보존형 문서 추출

Meet Gemini Omni 1.1 Flash Our newest multimodal model for video generation and editing. It now features your favorite creative controls from Veo, plus brand new capabilities. Enjoy features like 4K upscaling, first / last frame control, and fast 360p drafting. But, the biggest upgrade? Next-level scene extension. With Omni 1.1 you can extend scenes based on 10 seconds of context from your original video, a big jump from Veo's 1 second! That means tighter consistency, deeper control, and longer, more cohesive storytelling. See it in action ↓

Big news: Gemini Omni 1.1 Flash has landed #1 in the Text-to-Video Arena and #2 in the Image-to-Video Arena! For Text-to-Video the latest 1.1 model is +20pts above FLUX 3 Video at #3 (1495 pts). For Image-to-Video the release is a strong +25pt improvement from Gemini Omni Flash at #5, and is only behind MiniMax-H3 in the #1 spot (1494 pts). See more details in the thread. Congrats to the @Google team on this release!

Kubernetes meets LLM inference. Google, NVIDIA, IBM, and Red Hat are all backing the same open-source project to make it work. the problem is that LLM inference does not scale the way normal web services do, and the usual Kubernetes answer makes it worse. let me explain: run one vLLM or SGLang server and the KV cache is a clean win. the server keeps the attention keys and values for tokens it has already processed, so a prompt that shares a prefix with an earlier one skips past that computation and starts generating. put a standard Kubernetes Service in front of several replicas and that saving mostly evaporates. the Service hands each request to whichever pod is next in rotation, and that pod usually never saw the prefix, so it recomputes the entire context from scratch. round-robin assumes every replica serves every request equally well. true for stateless web traffic. false the moment prefill caching exists, because replicas now differ by what they remember. teaching Kubernetes that difference turns into four problems. → knowing which replica holds the prefix. each server streams an event every time it creates or evicts a cache block, and the router keeps a live index of who holds what. → knowing when to ignore that index. cache affinity pulls traffic onto warm replicas, so past a load threshold the router drops affinity and picks on load alone. otherwise the warm replica turns into the bottleneck. → extending where the cache lives. accelerator memory fills fast, so blocks spill to CPU memory and then disk. on four H100s at 250 concurrent users, that hierarchy delivered 13.9x the throughput of keeping everything on the GPU. → separating prefill from decode. prefill is compute-bound, decode is memory-bandwidth-bound, and running both on one replica underuses each. AWS measured up to 70% higher tokens per second after splitting them onto dedicated pools, though the KV cache now has to cross the network before the first token appears. the KV cache stops being something a single server manages. it becomes cluster state, and the routing layer has to track it. solve that and you get roughly 3x the output throughput and half the time to first token, on the same hardware, running the same model. llm-d is the project handling all four on Kubernetes. it sits above vLLM and SGLang rather than replacing them, so you keep whichever engine you already run and it takes over the routing, the cache index, the offloading, and the prefill/decode split. Apache 2.0, CNCF sandbox, with Tesla, Snowflake, Cohere and DigitalOcean running it. check it out on GitHub: https:// github.com/llm-d/llm-d i wrote the full breakdown of how inference works underneath all of this. the article is quoted below.

ClaudeDevs

@ClaudeDevs

3일 전

We've added a new cookbook: connect a Claude Managed Agent to @vercel 's Chat SDK. This gives an agent access to a universal chat layer. https:// github.com/anthropics/cla ude-quickstarts/tree/main/managed-agents/chat-sdk …

ClaudeDevs

@ClaudeDevs

3일 전

Chat SDK gives you the chat interface: one type-safe `onDirectMessage` handler, 15+ adapters across Slack, Teams, Discord, web and more. Managed Agents runs the agent server-side: harness, session management, and memory. Each conversation is one persistent session.

Important read if you build with agent skills. Shared skill libraries are treated as a safe way for coding agents to reuse each other's work. New research shows they propagate malware. EvoMal plants a malicious skill in the library and never invokes it. The agent retrieves it as an authoring template, writes a new skill that preserves the payload, stores it, and runs it. Each authored copy re-enters the library and gets imitated again. Across six models on 153 tool-relevant SWE-bench Verified tasks, the agent self-poisoning rate runs 20.3% to 41.8%. Poisoned libraries end up holding 4.9 to 9.0 times as many malicious skills as were planted. Deleting every planted skill does not clean it up. Qwen3 still shows 68% at round five because the agent-authored copies remain. A counter-prompt that discourages banner-style copying drops it to 6.7% with no significant task-completion loss. Paper: https:// arxiv.org/abs/2608.25776 Chat with Paper: https:// academy.dair.ai/papers/evomal- self-poisoning-in-self-evolving-coding-agents-2608.25776 …

// Automata from agent traces // How much of your agent's behavior comes from the model, and how much from the harness you wrapped around it? New work collapses an entire corpus of agent traces into a single compact finite-state machine. Across twelve public datasets the induced machines run 7 to 43 states, replay held-out data at 0.997 fitness with near-identical topology across splits, and build in milliseconds. FSM-state context beats Agent Workflow Memory on every ground-truth-matched dataset for next-step prediction. Per-state behavioral features reach held-out AUROC up to 0.94 for failure prediction, and an online monitor ranks failing runs above passing ones from a partial trace, triggering early stopping well before completion. The authors suggest that behavioral topology gets shaped more by the deployment harness than by the LLM underneath it. Paper: https:// arxiv.org/abs/2608.23670 Chat with Paper: https:// academy.dair.ai/papers/automat a-from-agent-traces-2608.23670 …

Most extraction tools treat spreadsheets like PDFs. They flatten the file into text or markdown, then ask a model to infer the original structure. But spreadsheets depend on structure. Headers, formulas, merged cells, and hidden rows give every value its context. Strip that away, and you map the right number to the wrong metric or period. That's why we built native spreadsheet extraction into the LlamaParse platform. Instead of flattening your workbook to text, it reads the raw cells directly and maps the data to your schema. Available today in beta on the agentic_plus tier. Give it a spin on your messiest .xlsx, .xls, or .csv files. Docs: https:// developers.llamaindex.ai/llamaparse/ext ract/guides/configuring-extract/#spreadsheet-mode …

Jerry Liu

@jerryjliu0

3일 전

We've introduced native, agentic spreadsheet extraction into LlamaParse. Spreadsheets are a wildly different format from PDFs (or any other document format). They can span arbitrarily many rows/columns, lack any guarantees on tabular structure, and can have information linked across multiple sheets. The native ways to deal with spreadsheets is usually through a code interpeter as opposed to OCR. We've introduced a tuned agentic engine (model+harness) that is equipped for large-scale schema guided extraction from spreadsheets. You can extract out dense sheets like balance sheets into clean structured fields. Check out the docs for how to integrate into the API: https:// developers.llamaindex.ai/llamaparse/ext ract/guides/configuring-extract/#spreadsheet-mode … On the UI, select "agentic plus", and see "spreadsheet options" in the advanced options section. Signup for LlamaParse here! http:// cloud.llamaindex.ai

Vector Database by hand ~ 10 steps walkthrough below Vector databases are the backbone of Retrieval Augmented Generation (RAG). How do they actually work? Goal: index three sentences, then answer a query by finding the nearest one, filling in every cell yourself. = 1. Given = A dataset of three sentences, three words each. In practice it is millions of them. = 2. Word embeddings = Let us look up each word in an embedding table. Here the vocabulary is 22 words; in practice it is tens of thousands, and the vectors have thousands of dimensions rather than four. = 3. Encoding = We feed the sequence to an encoder, one linear layer and a ReLU, and get one feature vector per word. In practice the encoder is a transformer. = 4. Mean pooling = Let us average across the columns. Three word vectors collapse into one, which is what people mean by a text embedding or a sentence embedding. = 5. Indexing = We multiply by a projection matrix and the four dimensions become two. It is doing the job of a hash: a short representation that is faster to compare, and it is what gets saved in the vector storage. = 6. Process "who are you" = Let us repeat steps 2 to 5 on the second sentence. = 7. Process "who am I" = We do it a third time. The database is now indexed. = 8. Query "am I you" = Let us push the query through the very same pipeline: lookup, encoder, mean pooling, projection, and it lands as a 2D vector in the same space. = 9. Dot products = We transpose the query and multiply, which takes the dot product against every stored vector at once. The dot product is the estimate of similarity. = 10. Nearest neighbour = Let us scan for the largest: 60/9 beats 44/9 and 40/9, so the answer is "who am I". Scanning billions of vectors one at a time is what makes this the slow step in practice, which is why real databases use an approximate nearest neighbour index like HNSW. The outputs: Stored index vectors = [5/3, 2/3], [5/3, 0], [7/3, 2/3] Query vector = [8/3, 2/3] Dot products = 44/9, 40/9, 60/9 Nearest neighbour = "who am I" The takeaway: a vector database is an embedding pipeline, a projection, and a dot product. Every step here is arithmetic you can do in pen, which is worth remembering when the word "database" makes it sound like something else. Save this post!

Google Gemma

@googlegemma

3일 전

Addressing the world's most pressing challenges requires robust solutions. These Gemma 4 Good projects fine-tuned models to address key challenges in anticipating and mitigating problems that affect our environment and society. This Global Resilience Prize winner, Acuífero·Vigía, provides an offline disaster response system for early flood warnings in Argentina. Powered by Gemma 4 E2B running entirely locally on Raspberry Pi and Android phones, it fuses real-time visual camera data with local citizen voice reports to track water rise and issue critical alerts without relying on cloud connectivity. Watch the project video: