Jeevansathi
Semantic cache for LLM calls
Implemented Redis semantic caching to avoid duplicate LLM calls.
- 85% cache hit ratio
- 40% lower message latency
What I do, 2 of 5
Make AI cheaper and faster
Most LLM bills pay for the same answer many times over. I cut repeat calls with semantic caching and keep answers reliable with fallbacks.
The problem
- Similar questions each trigger a full, paid model call.
- Slow answers in real-time flows like chat.
- One model provider is a single point of failure.
Try it
A small working tool. Change the numbers.
What would a semantic cache save you?
Set your traffic and your model's price. Then drag the cache hit rate. In production, mine runs at 85%.
–saved per month
–saved per year
–model calls you don't make, per day
Counts model tokens only. A cache adds a small embedding and Redis cost, and the right hit rate depends on how repetitive your traffic is. Mine also cut message latency by 40%.
Where I've done it
Jeevansathi
Real-time GenAI matchmaking
Delivered a real-time GenAI matchmaking engine using Spring Boot, Openfire (XMPP) and Aerospike, orchestrating GPT, Gemini and Qwen through a LangChain/LlamaIndex RAG pipeline on RedisVL vector search.
- 99.9% uptime
milanSathi
milanSathi: AI layer
Shipped four grounded LLM features on the engine: a personality reading, a couple's compatibility story, chart-based icebreakers and an AI astrologer chat, all with graceful fallback.
How I'd do it for you
Measure the repeats
Find how many requests are near-duplicates of earlier ones.
Cache by meaning
Match new requests to past answers by embedding similarity, not exact text.
Route across models
Use the cheapest model that is good enough, with a fallback when one fails.
Need this?
Tell me what you're working with. I'll tell you honestly whether it's worth doing and roughly what it takes.
Email me about this