Wan3.0 플랫폼 확장, 작업 단위 모델 고정, 오프라인 로봇 검색

Avi Chawla

@_avichawla

11일 전

How to think about model routing in production: Traditional routers classify by intent, and inside one task the intent keeps changing. Each call gets scored on its own, usually with no context that it's the middle of something. So if the routing layer switches the model midway, the new model has to build the whole cache again, at cold rates. Routing per call is the problem, not routing itself, so one common solution is to route once per task, not once per call. First LLM call routes normally, and that model gets pinned to a session ID. Every subsequent call goes to the same model. When the task changes, the pin resets, and the router picks again. This still gives you routing, but it's just that it happens at the point where switching is actually (or mostly) free. Easy tasks get routed to the cheap model and hard ones to the expensive model, and the model stays pinned during the same task. The idea is called model affinity, and the visual below explains this. To use this in practice, Plano (open-source) already implements it behind an OpenAI-compatible endpoint. You can integrate it by just changing the base URL in the actual code while the whole model routing config is defined in a YAML file. Basically, you generate one ID per task and pass it as a header on every call in the loop: ``` affinity_id = str(uuid.uuid4()) response = client(.)chat(.)completions(.)create( ..., extra_headers={"X-Model-Affinity": affinity_id}, ) ``` The pin lasts 10 minutes by default, and you can back it with Redis if you run more than one replica. When the task changes, you can swap in a new ID, and it routes again. Here's the repo: http:// github.com/katanemo/plano Btw, there's also early research trying to remove the KV transfer constraint. NVIDIA published a method recently that converts one model's KV cache into the format another model expects, so the target skips prefill entirely, and the conversion runs about 2.7x-25x faster than reprocessing the context. It's closed-form and training-free, which is new. But every pair they tested is within one family, Qwen to Qwen and Llama to Llama, and cross-family is listed as future work. So it's not production-ready yet. But if cross-family transfer gets solved, you could switch models mid-task without repaying for the cache. NVIDIA paper: https:// arxiv.org/abs/2608.03893

Teknium

@Teknium

11일 전

Just FYI. If you are switching models in a session all the time - you are doing it wrong. Every time you switch, your entire prompt cache is invalidated on the new model you switch to, and you have to repay the full input tokens price for all of it. Stop doing this unless those models are free. This is not a hermes thing - this is a fundamentals of inference thing.

What is a robot doesn't need the network to know what it’s looking at Rahul Sharma shows how Qdrant Edge + MobileCLIP2 can power offline visual search for warehouse robots, with the vector index and vision model running directly on the robot. He also uses multivectors + MAX_SIM to match products across different views, even when they’re crumpled or partially visible. Read more: https:// levelup.gitconnected.com/solving-the-mi xed-bin-offline-visual-search-for-apparel-robotics-1a0c7c4b4f3 …

What if warehouse robots could search product catalogs without the network? Rahul Sharma shows how Qdrant Edge + MobileCLIP2 enables offline visual search, with multivectors helping recognize products across different views. Read more: https:// levelup.gitconnected.com/solving-the-mi xed-bin-offline-visual-search-for-apparel-robotics-998a0a3ce8e4 …

Wan 3.0 is now live on @OpenRouter . Generate 2–30s videos from text, images, or references, with support for up to 1080p. A limited-time launch offer is also available across resolutions. More API access： • Model Studio: https:// click.alibabacloud.com/m/20000002017/ • Qwen Cloud: https:// click.qwencloud.com/m/20000002025/

Wan 3.0 is now available on @ComfyUI ! 30 seconds. One generation. More room for your story. Explore what’s possible with Wan3.0 on Comfy