오픈소스 토큰 점유율 급등, 재순환 추론과 로컬 MoE로 이어진 효율성 경쟁

More data than open-source AI is taking share from OpenAI and Anthropic. Open source has gone from 28% token share to 62% token share @vercel over the last 2 months. Chart from @rauchg Super impressive given that the sum of OpenAI and Anthropic accelerated in July. So net token/AI infra demand accelerated even more than the acceleration we saw at the frontier. And suspect Grok growing even faster than open-source and we saw some of this in the @tryramp data. Open-source AI taking share is positive for AI infrastructure demand as it lowers margins at the model layer and an open-source token costs just as much compute to produce as a frontier token. Nothing about open-source AI inference is “free.” Most likely end state IMO is that closed, frontier tokens are 60-90% of economic value but only 15 to 25% of tokens.

2026 is the year companies start seriously caring about model efficiency and reliability as it becomes critical infrastructure

You don't often see one-word titles in AI papers. That aside, strong recommend this paper from Google DeepMind. I think this is an interesting training-free approach to evolve model architectures by leveraging the model itself to inform architectural modifications. Something like this could also inspire even more robust recursive self-improvement approaches. Approach details below: A feedforward transformer can only update its internal state as many times as it has layers. Long generations need more updates than that, so chain-of-thought ends up doing basic state tracking in text. Recirculation adds recurrence at inference time. The model feeds activations back through itself during prefill, which lets it act like a dynamical system and track belief states without any retraining. Generation cost stays flat. All the serial work happens in prefill. On the Gemma3 family, the adaptive variant cuts perplexity 23% and lifts GSM8k accuracy 21%, with the original weights frozen and only light hyperparameter tuning. Paper: https:// arxiv.org/abs/2608.17981 Track more trending AI papers in our academy: https:// academy.dair.ai

Scientific terms should have precision. If we use the terms VLM, VLA, WAM in an indiscriminate fashion, as is becoming common in robotics, we are not helping clarity in communication. Let's keep the historical origins of these terms in mind. VLMs arose as multimodal extensions of LLMs-the training was for tasks like VQA (VIsual Question Answering). These capture the static semantics of the scene behind an image. No dynamics. World Models (e.g. @ylecun , Ha & Schmidhuber 2018) on the other hand are primarily dynamics models, which go back to control theory -1960 (Bellman, Kalman etc.) This makes them natural for robotics planning / policies- I am in a state s, what action a should I perform to get to state s'. In classical control, these models were written down a priori by modeling the physics of the system; today we think of them as learned neural networks trained from temporal data e.g. video, robot trajectories. But the concept is the same. We shouldn't mix this concept with VLMs.

Your 8GB laptop can now serve a 35B frontier MoE at 39 tokens per second. Your 5090 can run a 284B model interactively. UC Berkeley just made frontier MoE models actually usable on normal hardware. - Qwen3.6 35B flies at 39 tok/s on an 8GB RTX 4060 laptop - DeepSeek-V4-Flash 284B hits 22-25 tok/s on a single 5090 - GLM-5.2 753B runs at 15 tok/s on one RTX PRO 6000 And it’s significantly faster than Ollama. FreeToken lets you run frontier model locally at interactive speeds on hardware you already own. No extreme quant, No datacenter required. - http:// arxiv.org/pdf/2608.16157

Jerry Liu

@jerryjliu0

6일 전

The latest RAG trend for the current agent harnesses (Codex, Cowork) is to do two passes of document processing to solve a knowledge work task over a data room of documents: A fast and light pass, oftentimes using a free/OSS doc parsing tool. This can be cheaply run across 10-100-1k’s of files, and enables the agent to then do retrieval (e.g. grep, semantic) to find relevant subsets of context. A “just-in-time” VLM-based pass. Once the agent finds the relevant pages of context, it will screenshot the documents can call its own VLM (or write code) to dissect the pages. The issue with only using VLM-based OCR tools over massive ad-hoc customer file dumps is that it’s slow and expensive. Doing JIT VLM OCR allows the agent to filter through the data cheaply, but still preserve accuracy for the context that’s needed for the task. The agent harnesses do two-pass document processing by default using off the shelf-tools: pdf2text as the first pass, and using itself (Opus 5) as the second pass. See the below video where Cowork runs over a bunch of PDFs to answer a question about a benchmark graph in the Kimi k3 paper. The main issues here with the “out of the box” doc processing these agents offer are: * Opus 5 is not the best VLM for OCR. It is also way too expensive at scale and lacks grounding * The OSS tools like pypdf, pdf2text, may not be versatile enough as the first pass. * The agent will write a lot of throwaway code to rewrite things an OCR tool would’ve provided out of the box, like chart processing, bounding boxes, confidence scores, leading to increased cost and speed. We have all the tools within @llama_index to help any agent do two-pass document processing with higher accuracy and lower cost. We have liteparse for the first pass - a free/OSS parser written in Rust that’s faster/more accurate than other OSS parsers, and supports 50+ document types We have LlamaParse for the second pass - an agentic document engine that uses VLMs+harnesses to achieve SOTA in accuracy and cost across various doc parsing and extraction tasks. It can be called from any agent harness as an MCP or skill. It takes in page numbers as input, so that the agent can choose to run LlamaParse over a subset of the doc instead of the full doc as a “zoom-in” pass. Come check it out! LiteParse: https:// github.com/run-llama/lite parse … LlamaParse: https:// cloud.llamaindex.ai All the relevant docs, including MCP, are here: https:// developers.llamaindex.ai/llamaparse/for -agents/mcp/ …

Graph Convolutional Network by hand ~ 12 steps walkthrough below Graph Convolutional Networks (GCNs), introduced by Thomas Kipf and Max Welling in 2017, are the tool for data shaped like a graph: social networks, recommendations, biological networks, drug discovery, molecular chemistry. I drew and calculated a simple GCN entirely by hand. Goal: run a two-layer GCN, then a small classifier, on a five-node graph, filling in every cell yourself. 1. Given A graph of five nodes, A to E, with edges between some of them. 2. Adjacency matrix (neighbors) Put a 1 wherever two nodes share an edge, in both directions. 3. Adjacency matrix (self) Add 1s down the diagonal, one self-loop per node. That is just adding the identity matrix. 4. Messages Multiply each node's embedding by the weights and biases, then ReLU. Negatives become 0. 5. Pooling Multiply the messages by the adjacency matrix. Each node gathers the messages of its neighbours and itself. 6. Visualize Node A pools [3,0,1] + [1,0,0] = [4,0,1]. 7. Second GCN layer Messages again: weights, biases, ReLU. 8. Pooling again Pool over each node and its neighbours, once more. 9. Visualize Node C pools [1,2,4] + [1,3,5] + [0,0,1] = [2,5,10]. 10. Fully connected layer Weights, biases, ReLU. This time there are no neighbours to pool, just the node itself. 11. Linear layer One more: weights and biases. 12. Sigmoid Squash each score to a probability (≥ 3 → 1, 0 → 0.5, ≤ -3 → 0). That is the classification for each node. You have just classified every node in the graph by hand. The outputs: A: 0 (very unlikely) B: 1 (very likely) C: 1 (very likely) D: 1 (very likely) E: 0.5 (neutral) The takeaway: a GCN layer is two parts. The top part pools each node with its neighbours through the adjacency matrix. The bottom part is an MLP that transforms each node on its own. A transformer layer has the same two parts, with an attention matrix where the adjacency matrix was. Both matrices do one job, mixing across positions: attention over tokens, adjacency over nodes. In my class I call the GCN the transformer's little cousin: a bit more stubborn, because its attention is fixed by the graph rather than computed from Q, K, and V. Draw the two side by side and the resemblance is hard to miss. Save this post! #AIbyHand #GraphNeuralNetworks #DeepLearning

[DEMO] More models, coding agents, and MCP tools usually means more places to keep track of. Unity AI Gateway gives admins one place to see usage and costs, set budgets, control which MCP tools agents can access, and block sensitive PII before it reaches an external model. Amber Roberts walks through the setup, including dashboards, shared and per-user budgets, MCP controls, and input guardrails. https:// youtube.com/watch?v=ujYLgY lo6XM …

Stripe claims "the singularity" is here, and justifies its $7.5B acquisition of OpenRouter. The truth, as @EquityPod highlights, is much more practical, as unglamorous as token expense management may seem: http:// spr.ly/6018B1m8AA