서빙·RAG·멀티모델 전략: SIE의 메모리 공유 제안과 AWS·Alibaba의 실무 패턴

Serverless vs on-prem vs edge deployment. (read before your next deploy) These are three different answers to the same question of where your model actually runs, and each one costs you something different. 1) Serverless means you hand your model to a provider that only turns on a machine when a request arrives. Between requests the container shuts down, so you pay nothing while idle. The weights get unloaded along with it, so the next call has to pull gigabytes back into memory before it can answer, which adds up to 90 seconds. A reranker sitting in the middle of a search cannot spend that long waking up, so you keep instances warm, and you are back to paying for idle GPUs. 2) On-premise means you rent or own the GPU yourself and keep the server running inside your own network. Nothing leaves your walls, there is no cold start because the engine already holds the weights, and you pay a flat hourly rate instead of a per-token meter. The card runs whether traffic arrives or not. 3) Edge means the model runs on the user's own device, on its NPU or CPU, through a small local runtime. The model has to be shrunk down to fit, so it works offline with nothing ever leaving the hardware, but it handles one narrow task rather than a full pipeline. On-premise is where serious teams land, and it only pays off if several models share one card. That part rarely happens, because vLLM pre-allocates 90% of the GPU at startup and ignores every other vLLM instance on it, while HuggingFace TEI (Text Embeddings Inference) takes one model-id per process. Neither one knows the other is there. So four small models end up on four separate cards. You switched to small models to spend less, and you are now renting four GPUs to run what one could hold. The serving engine box in that middle row is what decides whether on-prem is cheap or just differently expensive. The card was never the constraint. The missing piece is one server that holds all the models and shifts memory between them as traffic moves, instead of four servers each convinced they own the whole GPU. The Superlinked Inference Engine (SIE) is the open-source project built for that slot. One server handles embeddings, reranking, extraction, and generation, loads a model the first time a request needs it, and drops the least recently used one when memory runs short. Check it out on GitHub: http:// github.com/superlinked/sie (don't forget to star ) My co-founder wrote a detailed breakdown on what SIE is and how to get started with it. The article is quoted below.

How to build a RAG app on AWS! The visual below shows the exact flow of how a simple RAG system works inside AWS, using services you already know. At its core, RAG is a two-stage pattern: - Ingestion (prepare knowledge) - Querying (use knowledge) Below is how each stage works in practice. > Ingestion: Turning raw data into searchable knowledge - Your documents live in S3 or any internal data source. - Whenever something new is added, a Lambda ingestion function kicks in. - It cleans, processes, and chunks the file into smaller units that the model can understand. - Each chunk is embedded using Bedrock Titan Embeddings. - The embeddings are stored in a vector database like OpenSearch Serverless, DynamoDB, or Aurora. - This becomes your searchable knowledge store. This stage can run continuously in the background, keeping your knowledge fresh. That said, you need to be strategic about how you reindex your data. For instance, if you are fetching data from a document and only one character was updated in it, you don’t want to reprocess the entire document every time. So smart diffing, incremental updates, and metadata checks become important to avoid unnecessary cost and keep ingestion efficient. > Querying: Retrieving and generating answers: - A user asks a question through the app. - The request flows through API Gateway into a Lambda query function. - The question is embedded with Bedrock Titan Embeddings. - That embedding is matched against your vector database to fetch the most relevant chunks. - Those chunks are passed to a Bedrock LLM (Claude or OpenAI) to craft the final answer. - The response travels back to the user through the same API path. This pattern ensures the LLM is grounded in your actual data, not model guesses. This setup is the most basic form of RAG on AWS, but the pattern stays the same even as your system grows. You can add better chunking, smarter retrieval, caching layers, orchestration, streaming responses, eval pipelines, or multi-source ingestion, and the architecture still holds. That said, most RAG architectures rely heavily on vector search to gather relevant context from a vector DB. That layer can be made 32x more memory efficient using binary quantization. I covered the full implementation of that in the article below. P.S. The visual in this tweet on building a RAG app on AWS is inspired by ByteByteGo's post on a similar topic. I added more context to it for explainability.

Thrilled to welcome aspiring entrepreneurs from @NUS_Enterprise Summer Programme to Alibaba Cloud! It was incredible seeing them test-drive our flagship @Alibaba_Qwen models and explore how AI can accelerate their startups.The next unicorns are already building #AlibabaCloud

Usually a dev's nightmare. But with Qoder, our end-to-end AI coding agent, it’s actually possible. Skip the boilerplate. Focus on innovation. Explore: https:// qoder.com Free credits: https:// click.alibabacloud.com/m/20000000884/ #Qoder #AICoding #AlibabaCloudPH

Lyft and Baidu enter London’s robotaxi battleground as testing begins

A clarification on the timeframe for rebuttals and author discussion, for all three tracks at NeurIPS 2026. Rebuttals can be posted until August 3rd AOE. This includes any back and forth with the reviewers. The "rebuttal" button and "official comment" button will both be treated equivalently - the difference in terminology between the two is to facilitate discussion during the discussion period on OpenReview. This also means you are allowed to post several comments, with a total of over 10000 words. Please post your answers to the Area Chair's initial-meta reviews via a comment from July 28th and by August 3rd, so that reviewers can see it too. We emphasize that the entire period from July 27 through August 3 AOE is time for authors to respond to reviews, including with additional results or other responses to reviewers as applicable. It is your choice how you wish to divide this time into initial rebuttals and subsequent discussion with reviewers and ACs.

AI21 Labs

@AI21Labs

AI21 joined @nvidia , @Microsoft , @a16z , and dozens of others in signing the Open Weights and American AI Leadership letter. We build agentic systems and agent optimization products on top of models, open and closed alike. The competition between them is what keeps the field honest - on quality, on cost, on who gets access. Both kinds of models will exist, and both should. Closing off the open path would be a mistake the whole ecosystem pays for.

I'm pretty excited about this... Tim is, in my opinion, a legend in the field of modern self-supervised learning: he developed DINOv2, was involved in DINOv3, and invented ViT registers. He has a recent paper formulating a new SSL approach called CAPI, and he'll be presenting it in our journal club on Thursday! At @MedARC_AI we're very interested in exploring alternative SSL approaches apart from the standard DINO and MAE paradigms. We believe innovations in SSL training will lead to better medical imaging foundation models! So we're looking forward to learning more about CAPI, come join us!!

App vendors charged public schools millions of dollars for apps that take less than 10 minutes from prompt to finish.