Grok 4.5 HighWalk 1위, Qwen Audio 3.0 음성 경쟁, 모델 파일 포맷 트레이드오프, SIE GPU 공유, vLLM ROCm 지원

Elon Musk

@elonmusk

Grok now in Copilot

Tesla Owners Silicon Valley

@teslaownersSV

BREAKING: Grok 4.5 just claimed the top spot on the new HighWalk Benchmark. The independent test measures how well AI models update real technical specifications from 46 Laravel commits — heavy on code analysis, abstraction, and precise writing. Results: • Grok 4.5 (high) → overall #1 (best quality + efficiency combo) • Claude Opus 5 (high) → highest raw quality, zero hard failures • GLM 5.2 → strongest open-weight model Higher reasoning effort didn’t always help. Full results just dropped.

Google and Anthropic agree on one thing about LLM inference: The faster a model runs, the fewer machines can run it. This sounds counterintuitive, but a model file gets faster by locking in hardware-specific choices into the file. And those choices only fit the hardware they were made for. That is why they build the same model more than once, one version per chip or runtime it has to run on. The visual below covers the six formats used to run LLMs in production: 1-2) Pickle and safetensors hold raw weights. Both run anywhere Python runs, so neither is faster than the framework around it. A .pt file is a list of instructions that torch.load executes and loading a checkpoint can also run a payload (possibly malicious). Safetensors holds plain numbers instead, laid out the way memory needs them, so there is nothing to rebuild when the file opens. Both still need Python, the model code, and a tokenizer alongside. 3) GGUF is llama.cpp's format, built so a model can run on a machine with no ML framework installed. To achieve this, weights, tokenizer, and chat template ship in one file, and llama.cpp or Ollama runs it directly. 4) ONNX is an interchange format, built so a model trained in one framework can run somewhere that framework was never installed. It decides the operations and the order they run in, but leaves the hardware open, so the runtime picks CPU, CUDA, or an NPU when the file loads. 5) MLX is Apple's framework for its own silicon, where the CPU and GPU share a single pool of memory. It decides where the weights live, so they are never copied between the two, and that advantage does not help on any other machine. 6) TensorRT is NVIDIA's compiler, which turns a model into machine code for one specific GPU. Everything is settled at build time, down to instructions tested on the exact card it compiles for, which is why the file will not load on a different GPU architecture. The safe default is to stop as high on the chart as the latency budget allows, because every row down trades speed with the types of hardware it can run on. That said, picking the right format only solves how ONE model behaves. But production systems run a fleet of models, usually an embedding model, a reranker, an extractor, and a generator, so the bottleneck moves from the file to the GPU that they all have to share. Except they mostly cannot share it. For instance, each vLLM process claims 90% of GPU memory at startup and holds it, so teams end up provisioning a GPU per model, no matter how carefully each file was built. SIE is an open-source inference server to solve this problem. Models load when a request needs them and get evicted when they go cold, so one GPU serves embeddings, reranking, extraction, and generation behind a single API instead of running one process per model. GitHub repo: https:// github.com/superlinked/sie (don't forget to star it ) I wrote a full breakdown of this small-model serving problem and how SIE solves it. It covers why switching to small specialized models doesn't reduce inference costs unless those models share GPUs, and why vLLM's design makes that impossible. Read it below.

Google and Anthropic agree on one thing about LLM inference: The faster a model runs, the fewer machines can run it. This sounds counterintuitive, but a model file gets faster by locking in hardware-specific choices into the file. And those choices only fit the hardware they were made for. That is why they build the same model more than once, one version per chip or runtime it has to run on. The visual below covers the six formats used to run LLMs in production: 1-2) Pickle and safetensors hold raw weights. Both run anywhere Python runs, so neither is faster than the framework around it. A .pt file is a list of instructions that torch.load executes and loading a checkpoint can also run a payload (possibly malicious). Safetensors holds plain numbers instead, laid out the way memory needs them, so there is nothing to rebuild when the file opens. Both still need Python, the model code, and a tokenizer alongside. 3) GGUF is llama.cpp's format, built so a model can run on a machine with no ML framework installed. To achieve this, weights, tokenizer, and chat template ship in one file, and llama.cpp or Ollama runs it directly. 4) ONNX is an interchange format, built so a model trained in one framework can run somewhere that framework was never installed. It decides the operations and the order they run in, but leaves the hardware open, so the runtime picks CPU, CUDA, or an NPU when the file loads. 5) MLX is Apple's framework for its own silicon, where the CPU and GPU share a single pool of memory. It decides where the weights live, so they are never copied between the two, and that advantage does not help on any other machine. 6) TensorRT is NVIDIA's compiler, which turns a model into machine code for one specific GPU. Everything is settled at build time, down to instructions tested on the exact card it compiles for, which is why the file will not load on a different GPU architecture. The safe default is to stop as high on the chart as the latency budget allows, because every row down trades speed with the types of hardware it can run on. That said, picking the right format only solves how ONE model behaves. But production systems run a fleet of models, usually an embedding model, a reranker, an extractor, and a generator, so the bottleneck moves from the file to the GPU that they all have to share. Except they mostly cannot share it. For instance, each vLLM process claims 90% of GPU memory at startup and holds it, so teams end up provisioning a GPU per model, no matter how carefully each file was built. SIE is an open-source inference server to solve this problem. Models load when a request needs them and get evicted when they go cold, so one GPU serves embeddings, reranking, extraction, and generation behind a single API instead of running one process per model. GitHub repo: https:// github.com/superlinked/sie (don't forget to star it ) I wrote a full breakdown of this small-model serving problem and how SIE solves it. It covers why switching to small specialized models doesn't reduce inference costs unless those models share GPUs, and why vLLM's design makes that impossible. Read it below.

Great to see @RedHat_AI close the format gap this fast: FP8 is what Hopper's tensor cores run natively, so their FP8-Block K3 serves faster on H100s and H200s. Spin it up now!

Congrats to our own maintainer Tun Jian Tan @Rxday000 , who received the AMD AI Core Contributor award from CEO @LisaSu at #AdvancingAI. Tun Jian landed the first AMD GPU PR and has led vLLM's ROCm stack for 2.5+ years. Huge thanks to him, the @EmbeddedLLM team, and the @AIatAMD engineers working alongside them for making high-performance @AMD support a first-class part of vLLM.

Qwen Audio 3.0 Realtime Plus is now the world's #1 Speech-to-Speech model on the Artificial Analysis Index (84.1%, ahead of GPT-Realtime-2.1 High at 79.1%), leading all three component benchmarks — Speech Reasoning, Conversational Dynamics, and Agentic Performance. Model Studio: https:// click.alibabacloud.com/m/20000000926/ Qwen Cloud: https:// click.qwencloud.com/m/20000000934/

Most people treat video like a bag of frames or a transcript with timestamps. But you end up missing the most important parts of it: - Motion - Causality - Temporal progression - The relationship between what happens and why it matters At Vector Space Day SF, James Le from @twelve_labs walked through 3 failure modes that come from this: - Wrong Context - Wrong Memory - Wrong Reasoning And he introduced the stack built to fix it: → Marengo: a retrieval model that makes video searchable → Pegasus: a video language model that turns retrieved moments into structured answers → Jockey: an agentic framework and memory layer for video corpus workflows With Qdrant handling the storage and retrieval underneath it all. Full talk is live on our YouTube channel: https:// youtube.com/watch?v=i8xZeK P6t6k …