LLM Product Engineering
End-to-end GenAI products on multi-tenant SaaS — from prompt design and structured outputs to streaming UIs, guardrails, and per-tenant token metering.
GPT-4oClaudeGeminiVercel AI SDK
Building production LLM systems
I build GenAI products that survive production.
I’m Akshay, an AI Engineer. I design and ship LLM systems that hold up in front of real users — RAG systems, AI agents, and streaming LLM apps — with the evals, cost controls, and guardrails to match, backed by 9+ years scaling systems at Licious, Allo Health, and Practo.
Core stack
ClaudeGPT-4oLangGraphpgvectorNext.jsAWS
5
GenAI products shipped end-to-end
41%
Lift in sales-accepted-lead precision
38%
Reduction in LLM spend
9+
Years engineering at scale
9+ years of engineering across unicorns, health-tech, and 0-to-1 startups
LiciousAllo HealthPractoHousejoyNoticeboardIntellora AITiVoLiciousAllo HealthPractoHousejoyNoticeboardIntellora AITiVo
About
GenAI treated like a product, not a demo.
Most AI demos never survive contact with real users. My work starts where the demo ends — the evals, cost controls, guardrails, and reliability that let a GenAI feature run in production in front of paying customers.
At Intellora AI I shipped five GenAI products across a multi-tenant SaaS platform, driving a 41% lift in lead precision and a 38% cut in LLM cost. Underneath it is 9+ years of distributed-systems engineering — the kind of foundation that turns an AI idea into something that actually holds up.
Ships end-to-end
Prompts, RAG, agents, evals, and the streaming UI — I own the whole lifecycle, not a slice of it.
Measures everything
Golden datasets, CI regression gates, cost & latency tracking. If it can't be measured, it doesn't ship.
Built at scale
Unicorn-scale traffic at Licious, 0-to-1 at Allo Health, founding AI engineer at Intellora AI. The platform instincts are earned.
What I do
Depth across the full GenAI stack
From the retrieval layer to the streaming UI — and the evals and cost controls that keep it reliable in production.
RAG & Vector Search
Retrieval pipelines that stay grounded — hybrid BM25 + dense retrieval, semantic chunking, reranking, and answers with citations over Pinecone / pgvector.
PineconepgvectorCohere RerankHybrid Search
AI Agents & Orchestration
Multi-step agents that actually finish the job — LangGraph state machines, function calling, tool use, MCP, and human-in-the-loop fallbacks.
LangGraphReActMCPTool Use
Evals, Cost & Observability
The unglamorous work that makes GenAI reliable — golden-dataset evals, CI regression gates, model routing, prompt caching, and cost/latency tracking.
LangSmithLangfuseModel RoutingEval Harnesses
Full-Stack & Distributed Systems
The platform under the AI — NestJS + Next.js services, event-driven Kafka / Pub-Sub, streaming over SSE / WebSockets, OAuth2 / JWT, and cloud-native AWS.
Next.jsNestJSFastAPIAWS
AI Safety & Guardrails
Shipping GenAI you can trust in front of customers — moderation, prompt filtering, output validation, PII & jailbreak mitigation, and read-only NL-to-SQL.
GuardrailsModerationPII MitigationValidation
Approach
How I take GenAI from idea to production
The process behind every product in the portfolio — the same one whether the deliverable is a chatbot, a retrieval pipeline, or a multi-step agent.
01
Step 01
Scope the problem, not the model
Every build starts with the user's job-to-be-done, the failure modes that would make it unusable, and a definition of "good" I can measure. The model choice comes last.
- Use-case & success metrics
- Failure-mode mapping
- Golden dataset v0
- Reference architecture
02
Step 02
Build the core end-to-end
Retrieval, agents, structured outputs, and the streaming UI — a thin vertical slice in production early, then widened. I own the prompts and the platform alike.
- RAG & hybrid retrieval
- LangGraph agents & tool use
- Structured outputs & validation
- Streaming UX over SSE / WebSockets
03
Step 03
Evaluate before every ship
Prompt and model changes go through golden-dataset evals and CI regression gates the same way code goes through tests. If it can't be measured, it doesn't ship.
- Eval harnesses
- CI regression gates
- Prompt A/B testing
- LangSmith / Langfuse tracing
04
Step 04
Operate at cost and scale
Model routing, prompt caching, per-tenant budgets, and guardrails keep systems fast, safe, and affordable long after launch — on multi-tenant platforms serving real customers.
- Model routing & caching
- Cost & latency tracking
- Guardrails & PII mitigation
- Per-tenant metering
Selected work
Five GenAI products, shipped to production
Each one owned end-to-end — prompts, retrieval, agents, evals, and streaming UI. These are the outcomes, not the demos.
Intellora AI2024
Advanced WhatsApp AI Chatbot
A conversational GPT-4o agent on the WhatsApp Business API with deep chat-session management — context persistence, token optimisation, and per-tenant metering — orchestrating tools via LangGraph.
+35%
Qualified leads
−28%
Low-intent traffic
GPT-4oLangGraphWhatsApp APIGuardrails
Intellora AI2024
AI Lead Qualification System
An LLM lead-scoring agent that fuses structured CRM features with conversation embeddings in pgvector, using JSON-mode structured outputs and a custom eval harness on a 5K-row golden dataset.
+41%
Lead precision
5K
Golden-dataset rows
EmbeddingspgvectorJSON modeEvals
Intellora AI2024
AI Chat Summarisation
A Claude Sonnet pipeline over long, multi-day customer conversations — hierarchical map-reduce summarisation, topic extraction, sentiment scoring, and action-item generation.
12m → 90s
Review time per thread
8×
Faster triage
Claude SonnetMap-ReduceSentimentNLP
Intellora AI2024
AI Travel Itinerary Agent
A multi-step planning agent on LangGraph state machines that decomposes intent, calls flight / hotel / POI APIs as tools, validates with Pydantic, and streams the itinerary over SSE.
<6s
p95 generation latency
Parallel
Tool execution
LangGraphFunction CallingPydanticSSE
Intellora AI2024
AI-Powered CRM
LLM features woven across the CRM — auto-drafted follow-ups, deal summaries, next-best-action, and a natural-language query interface that generates guarded, read-only SQL.
100%
Sales adoption in 4 weeks
NL → SQL
With guardrails
NL-to-SQLNext-Best-ActionRAGGuardrails
Intellora AI2024
LLM Cost & Reliability Platform
The evaluation and cost backbone under every product — model routing across Haiku / Sonnet / Opus, prompt caching, structured-output retries, per-tenant budgets, and CI regression gates.
−38%
LLM spend
CI-gated
Every prompt change
Model RoutingLangSmithPrompt CachingEvals
Curious how any of these are built? I write about the patterns on the blog
Track record
From unicorn scale to 0-to-1 AI
Nine-plus years across consumer platforms, real-time systems, and founding-engineer builds — the hard-won foundation under every GenAI system I ship.
Intellora AI
CurrentFounding AI Engineer•Jan 2024 — Present
- Designed and shipped 5 production LLM products end-to-end across a multi-tenant SaaS platform; defined the 12-month AI roadmap across 4 product squads.
- Built RAG pipelines and vector search over tenant knowledge bases (Pinecone + pgvector, hybrid retrieval, Cohere reranking, citations).
- Cut LLM spend ~38% via model routing, prompt caching, and per-tenant token budgets — quality held on golden-dataset evals.
LLMRAGLangGraphNestJSNext.js
Allo Health
Full-Stack Engineer•Jun 2022 — Dec 2023
- Built the full consumer telehealth platform (Next.js SSR + TypeScript + Node.js + PostgreSQL) — booking, payments, diagnosis flows — from 0 to production in 4 months.
- Grew engineering from 1 to 6; established unit, integration, and E2E testing (Jest, Cypress 120+, Playwright 150+) wired into CI.
- Led A/B testing on checkout via Next.js Middleware, lifting conversion 22%.
Next.jsTypeScript0-to-1Testing
Licious
Software Development Engineer II•Aug 2020 — Jun 2022
- Engineered a distributed consumer platform at unicorn scale (50K+ concurrent users) on Next.js SSR + Node.js + PostgreSQL.
- Ran A/B testing infrastructure with 8+ concurrent experiments via feature flags — the same discipline that underpins prompt / model evaluation.
- Scaled engineering from 3 to 11 and authored architecture RFCs adopted across 5 product teams.
ScaleA/B TestingDistributed SystemsRFCs
Noticeboard
Senior Software Engineer•Jul 2018 — Aug 2020
- Built a WhatsApp-style real-time chat app on WebSockets (React + Node.js + PostgreSQL), plus an LMS and live geolocation attendance system.
- Conversational, stateful, low-latency message-streaming patterns that transfer directly to modern LLM chat; maintained 82%+ test coverage.
WebSocketsReal-timeReactNode.js
Housejoy
Software Engineer•Jul 2017 — May 2018
- Built a consumer home-services e-commerce platform (React.js, migrated from PHP Laravel) serving 40K+ monthly users — improved page load 30% via code splitting.
ReactE-commercePerformance
Practo
Software Development Engineer•Sep 2016 — Jun 2017
- Developed internal healthcare tools and doctor-facing dashboards (React.js + Python Flask) — early Python web-stack exposure now central to LLM application development.
ReactPythonHealthcare
M.Tech, Computer Science— IIIT Hyderabad
Toolkit
The stack I reach for
Deep in the modern GenAI toolchain, grounded in production backend and frontend engineering.
LLMs & Frameworks
GPT-4oClaude (Opus/Sonnet/Haiku)GeminiLlama 3LangChainLangGraphLlamaIndexVercel AI SDK
RAG & Retrieval
PineconepgvectorWeaviateQdrantHybrid SearchCohere RerankEmbeddingsSemantic Chunking
Agents & Safety
ReAct AgentsFunction CallingMCPStructured OutputsGuardrailsModerationPII MitigationHuman-in-the-loop
Evals & MLOps
LangSmithLangfuseHeliconeGolden DatasetsPrompt A/B TestingCI Regression GatesModel RoutingPrompt Caching
Backend & Data
Node.js / NestJSPython / FastAPIPostgreSQLRedisKafkaPub/SubWebSocketsSSE
Frontend & Cloud
Next.js (App Router)React 18TypeScriptAWS (Bedrock/Lambda)GCPDockerKubernetesCI/CD
Signal
Outcomes people remember
Akshay took our AI lead-qualification from a rules-based guess to a system we actually trust. The 41% precision lift showed up directly in sales-accepted leads.
Product Lead
Intellora AI
He shipped our entire telehealth platform from zero to production in four months and grew the team around it. Rare combination of speed and engineering discipline.
Founding Team
Allo Health
The LLM cost and reliability work paid for itself almost immediately — 38% off our model spend with no drop in quality, backed by evals we could actually see.
Engineering Manager
SaaS Platform
Feedback from teammates and stakeholders across the projects above.
FAQ
Questions, answered
The things people usually ask about my work and how I build.
What kind of AI engineering do you do?
End-to-end GenAI product work: RAG and knowledge assistants, multi-step agents on LangGraph, streaming LLM apps, and the evals, cost controls, and guardrails that keep them reliable in production. I own the whole lifecycle — prompts, retrieval, orchestration, platform, and UI.
What does "production-grade" mean to you?
A feature real users depend on, not a demo. Concretely: golden-dataset evals wired into CI, structured-output validation, guardrails and PII handling, cost and latency observability, and per-tenant budgets on multi-tenant platforms. Reliability is designed in, not bolted on.
How do you keep LLM features reliable and on-budget?
Golden-dataset evals wired into CI, model routing across model tiers, prompt caching, structured-output validation, and per-tenant token budgets with cost/latency observability. Every prompt or model change is gated the same way code is.
What's your stack?
Claude, GPT-4o, and Gemini through LangGraph, LangChain, and the Vercel AI SDK; Pinecone and pgvector for retrieval; LangSmith and Langfuse for evals and tracing. Underneath: Next.js, NestJS and FastAPI, PostgreSQL, Kafka, and AWS — the distributed-systems foundation from 9+ years before GenAI.
Where can I read more about how you build?
The blog. I write about the patterns behind the work — retrieval design, agent orchestration, eval harnesses, LLM cost engineering — including the trade-offs I actually hit. It's the best way to see how I think.
How do I get in touch?
Email is best. I'm based in Bengaluru and reply to every genuine message — whether it's a technical question about one of these systems, a collaboration, or something I wrote that you want to argue with.
Get in touch
Let’s talk about building with LLMs.
A question about one of these systems, notes to compare on RAG or agents, or something I wrote that you want to argue with — tell me what you’re working on. I reply to every genuine message.
Email
akshay.dharphale@gmail.com
Based in
Bengaluru, India • Remote-friendly
Currently
Building production LLM systems