Qwen3.8-Flash API 출시, LLM 양자화·Prefix Sliding, Nvidia의 Hugging Face 인수 보도, WorkBuddy 공동 집필

Qwen

@Alibaba_Qwen

2일 전

Qwen3.8-Flash on @qwen_cloud : $0.15/1M input tokens, $0.47/1M output tokens, and just $0.016/1M on cache hits. Come give it a try!

Qwen3.8-Flash is now live on @OpenRouter ! Coding assistants, agentic workflows, long-video understanding, all one API call away.

5 LLM quantization techniques, clearly explained: (bookmark this) A 70B model in FP16 needs 140GB for weights alone. At 4-bit, that drops to 35GB, which fits on one card. But naive rounding fails on large models. Roughly 0.1% of hidden dimensions carry values up to 20x larger than anything else in the tensor, and they wreck the quantization grid for everything else. Each of these 5 methods handles those outliers at a different point: 1. RTN: ignores them. Rounds every weight to the nearest grid level with no calibration data. Cheapest option, weakest at low bit widths. 2. GPTQ: repairs after rounding. Quantizes a layer column by column and adjusts the remaining weights to absorb the error before moving on. 3. AWQ: protects before rounding. Finds the ~1% of weight channels that matter most and scales them up so they survive quantization. Everything still ends up in plain INT4. 4. LLM. int8(): isolates at inference. Outlier dimensions run in FP16, the other 99.9% run in INT8, and the results are merged. 5. QAT: solves it during training. The model is fine-tuned with rounding baked into every forward pass, so it adapts to the damage before quantization is actually applied. All five produce the same artifact, a model at a fraction of its trained precision. They differ only in where the outlier problem gets addressed. The visual below nicely summarises these techniques. As further reading, the article below is a first-principles guide to LLM inference that walks through everything between your prompt and the streamed response, covering tokenization, embeddings, attention, the prefill and decode split, KV caching, and quantization. It will give you a complete mental model of how inference actually works under the hood. Read it below.

Prefix Sliding for efficient test-time scaling "we find most intermediate reasoning tokens lose importance as the model continues reasoning. This calls into question whether retaining them is worth the cost. Based on this insight, we propose Prefix Sliding, which discards tokens during reasoning that are not part of the prefix or the window of the last few thousand tokens." "Without training, Prefix Sliding can make existing models 3x faster while maintaining performance. Training with Prefix Sliding using reinforcement learning can achieve better performance by enabling scaling to reasoning traces beyond a hundred thousand tokens." code: https:// github.com/Muennighoff/pr efix-sliding … link: https:// arxiv.org/abs/2608.26070

Demystifying Reinforcement Learning Post-Training of Language Models A very useful exploration of RLVR in a simplified setup, lots of good insights... "Result 1: sparse RL cannot find what the model never samples" "Result 2: dense rewards break that ceiling" "Result 3: "spurious rewards" are a story about the prompt set" "Post-training is best understood as redistributing probability mass inside the pretrained distribution. That framing suggests the measurement: track the probability the model assigns to the behavior you want, and track the entropy of its output distribution, throughout training." project page: https:// minjang10.github.io/demystifying-r l-finetuning-web/ … code: https:// github.com/sankarh-1/demy stifying-rl-finetuning … link: https:// arxiv.org/abs/2608.24949

Nvidia has reportedly agreed to buy Hugging Face, the popular open-source AI hub, for $12.9 billion in a move that would let Nvidia both protect its chip empire and jump back into the cloud business.

Human-AI Co-writing is now live globally in @WorkBuddy_AI Most AI writing tools take over. WorkBuddy sits beside you. Highlight： · Same document, same time: you and AI edit together, both changes visible instantly · Select what you mean: a sentence, a cell range, a whole slide. Only that part gets touched · Your actual files: Word, Excel, PPT, Markdown. Just open and go Ctrl+Z works on the AI too. Control stays with you, AI just keeps up. Free Hy3 model. Free credits for new users.