Blog

Notes from production

Practical field notes on shipping LLM systems that survive real users — cost, reliability, retrieval, and agents. No hype, just what works. Filter by who each piece is written for.

Filter by audience
Reading status
Featured Sep 27, 2025 10 min readLive product

Case study: Itinerary Beautifier — structured extraction under a no-hallucination constraint

A production system that turns a travel agent's messy PDF quote into a branded, print-ready itinerary — without inventing a single fact. The AI decisions, the ~90% cost win, and the trade-offs.

EngineeringProductFounders & Business
Read
Featured Aug 9, 2025 7 min read

Case study: shipping an agentic WhatsApp assistant for a real business

The end-to-end architecture of a customer-facing agent — FSM orchestration, LLM intent handling, Hinglish support, RAG, and the WhatsApp Business API — and the lessons from putting it in front of real users.

EngineeringProductFounders & Business
Read
Featured Mar 15, 2026 8 min read

System design: an autonomous document-evaluation agent

A walkthrough of designing an agent that ingests messy supplier submissions, evaluates them against a rubric, and recommends the top vendors — multi-format ingestion, a LangGraph state machine, rubric scoring, and a deterministic decision engine.

EngineeringProduct
Read
Featured Nov 16, 2025 7 min read

Advanced RAG retrieval: parent-child, hybrid fusion, and two-stage reranking

Basic RAG retrieves a chunk and hopes. Production RAG over long, structured documents needs real retrieval architecture — small-to-big chunks, BM25+dense fusion, and a two-stage rerank. Here's the toolkit.

Engineering
Read
Featured Jan 5, 2025 7 min read

How we cut LLM spend 38% without losing quality

Model routing, prompt caching, and per-tenant budgets sound obvious. The hard part is proving quality held. Here's the playbook — and the evals that made it safe to ship.

EngineeringFounders & Business
Read
Featured Aug 2, 2025 6 min read

Agents vs workflows: choose the least agency that solves the problem

Fully autonomous agents are seductive in demos and brittle in production. Here's a decision framework — chain, state machine, or true agent — and how to pick the least autonomy that gets the job done.

EngineeringProduct
Read
Mar 29, 2026 9 min read

The golden dataset that gates your CI: building it, scoring it, setting the bar

The eval-discipline post argues you should gate every change on a golden dataset. This is the practical how — worked through one real product: the support assistant for an online travel agency. What a row holds, how you build thousands of them, which metrics score them, and how a fuzzy 'is it good?' becomes a number that blocks a bad merge.

EngineeringProduct
Read
Mar 28, 2026 7 min read

When a tool call fails: validation, error-as-feedback, and escalation

A demo assumes every tool returns clean data. Production tools return malformed, empty, and quietly wrong results — and an agent is only as reliable as its worst tool call. Here's how to handle failure at the tool boundary, and how to know it happened at all.

EngineeringProduct
Read
Mar 22, 2026 6 min read

Why BLEU and ROUGE fail for LLMs: from word-overlap to LLM-as-judge

The instinct to score generated text by how much it overlaps a reference answer comes from machine translation — and it breaks the moment an output can be right in many wordings. Here's the ladder from n-gram overlap to embedding metrics to LLM judges, and how to pick a rung.

EngineeringProduct
Read
Mar 21, 2026 6 min read

Where RAG came from: REALM, RAG, RETRO, and FLARE

RAG feels like a 2023 invention, but the idea — pair a model with a memory it can look things up in — has a clear research lineage from 2020. Knowing it makes today's design choices obvious rather than arbitrary.

EngineeringProduct
Read
Mar 14, 2026 6 min read

Keep the decision engine deterministic: separating scoring from LLM orchestration

When an AI system has to rank, score, or decide, don't let the LLM do the math. Separate a deterministic decision engine (weighted scoring, MCDA) from the probabilistic orchestration that handles messy input.

EngineeringProduct
Read
Mar 8, 2026 6 min read

Does an LLM know when it's unsure? Confidence, self-reflection, and logprobs

An LLM states a wrong answer with the same fluency as a right one — it has no built-in confidence. So how do you get a usable confidence signal? Self-reflection, token logprobs, and their limits.

Engineering
Read
Mar 7, 2026 6 min read

RAG as an agent: retrieval as a worker node in a multi-agent system

Where does RAG live in a multi-agent system? Not in the orchestrator — as a dedicated knowledge-agent worker, decoupled from routing, running in parallel with the API workers. Here's why that's the right boundary.

EngineeringProduct
Read
Mar 1, 2026 6 min read

Post-retrieval processing: contextual compression, noise, and lost-in-the-middle

Retrieval and reranking get you the right chunks — but they arrive full of boilerplate, redundancy, and irrelevant surrounding sentences. What you do between reranking and the model is a stage of its own.

Engineering
Read
Feb 28, 2026 6 min read

Semantic chunking and ingestion: splitting documents so retrieval works

Fixed-size chunking slices ideas in half. Semantic chunking splits where the meaning shifts — and injecting document structure into each chunk keeps it from being stranded. Here's the algorithm and the ingestion around it.

Engineering
Read
Feb 22, 2026 6 min read

RAG variants: Naive, Hybrid, Adaptive, and Corrective (CRAG)

'Which RAG have you worked with?' has more than one answer. Beyond naive and hybrid retrieval, Adaptive RAG routes by query complexity and Corrective RAG grades its own retrieval and fixes it. Here's the ladder.

Engineering
Read
Feb 21, 2026 6 min read

API design for AI services: idempotency, retries, timeouts, and error contracts

'It works' isn't an answer; 'two retries with backoff, then a 502 with a typed error' is. How to design an API that behaves predictably when the model — or anything upstream — fails.

Engineering
Read
Feb 15, 2026 6 min read

FastAPI request lifecycle: routing, dependencies, validation, and response models

Every request to an AI service travels the same path — routing, dependency injection, validation, handler, response model. Understanding that spine is what lets you put auth, tenancy, and validation where they belong.

Engineering
Read
Feb 14, 2026 6 min read

Observability beyond LangSmith: Langfuse and structured logging

When a client needs self-hosted observability for data-residency reasons, LangSmith isn't the only answer — Langfuse is the open-source one. And LLM traces alone aren't enough; you also need structured app logs that correlate a request end to end.

EngineeringFounders & Business
Read
Feb 8, 2026 6 min read

How to choose an embedding model (criteria, not names)

The right embedding model is the one that works best on your data, not the one at the top of a leaderboard. Here's the decision by criteria — evaluate on your own golden set, weigh dimensionality against cost, and mind the rest.

Engineering
Read
Feb 7, 2026 5 min read

Bi-encoders vs cross-encoders: the retrieval trade-off

The difference between the model that finds candidates and the model that reranks them comes down to when the query and document meet. One scales to millions; the other is precise. Production retrieval uses both.

Engineering
Read
Feb 1, 2026 5 min read

Pausing an agent mid-run: interrupt() and human approval in LangGraph

Some actions are too consequential to let an agent take unsupervised. LangGraph can pause a run at a sensitive step, wait for a human to approve or edit, and resume exactly where it left off. Here's how — and why the checkpointer makes it possible.

EngineeringProduct
Read
Jan 31, 2026 6 min read

The supervisor pattern: an LLM whose tools are other agents

The cleanest way to coordinate several specialised agents is a supervisor — an LLM that routes each turn to a worker agent and decides when the job is done. Here's the pattern, the routing, and the guards against infinite loops.

Engineering
Read
Jan 25, 2026 6 min read

When summarization fails: structured facts, hierarchical summaries, and token budgets

The go-to fix for a growing conversation is to summarise it. But a single summary string quietly drops details or bloats over time. Here's the upgrade path when 'just summarise it' stops working.

Engineering
Read
Jan 24, 2026 6 min read

Long-term memory across sessions: persistent user profiles

Ask an agent your favourite colour today and it knows; ask it next week and it's forgotten — because chat history isn't memory. Here's the pattern that makes an agent remember: extract facts, store a profile, inject it next session.

EngineeringProduct
Read
Jan 18, 2026 6 min read

The LangGraph agent loop: tools_condition, ToolNode, and handling tool errors

How does LangGraph actually run the reason–act–observe loop? A conditional edge checks for tool calls, a ToolNode executes them, and a ToolMessage feeds the result back. Here's the wiring — and how errors self-correct.

Engineering
Read
Jan 17, 2026 5 min read

ReAct vs native tool calling: why we stopped regex-parsing the model

Before models had a structured tool-calling API, agents worked by prompting the model to write 'Action: ...' and parsing that text with regex. It was fragile. Native tool calling fixed it — here's the before and after.

Engineering
Read
Jan 11, 2026 6 min read

What an LLM actually outputs during a tool call — and how bind_tools gets it there

A tool call isn't the model 'running a function' — it's the model emitting a structured request that your code executes. Here's the exact anatomy of that payload, and how bind_tools puts the tools in the model's head.

Engineering
Read
Jan 10, 2026 6 min read

Grounding, citations, and abstention: making an LLM say 'I don't know'

The most dangerous LLM output is a confident, fabricated fact with a citation to match. Grounding, span-level attribution, and the discipline of abstention are how you make a model verifiable — and safe for high-stakes work.

EngineeringProductFounders & Business
Read
Jan 4, 2026 6 min read

Security and compliance for AI products: why clients buy on trust first

For clients handling confidential data, security and compliance aren't features you add later — they're the gate you clear before the sale. Encryption, least privilege, audit, and certifications are the price of entry.

EngineeringFounders & Business
Read
Jan 3, 2026 7 min read

Platform engineering essentials: CI/CD, IaC, containers, and safe deploys

Shipping AI features reliably needs the boring platform underneath: automated pipelines, reproducible infrastructure, containers, safe rollout strategies, and the monitoring to know when something breaks.

Engineering
Read
Dec 27, 2025 7 min read

Designing for scale: caching, async work, and latency budgets

Scaling a multi-hop AI pipeline is less about raw horsepower and more about caching what's repeated, doing slow work asynchronously, budgeting tail latency, and degrading gracefully when a dependency fails.

Engineering
Read
Dec 21, 2025 7 min read

Multi-tenant SaaS architecture: isolation, cost, and blast radius

Serving many customers from one system is a trade between isolation and cost, and for regulated or confidential data it's a hard requirement, not a preference. Here are the isolation models and how to choose.

EngineeringFounders & Business
Read
Dec 20, 2025 7 min read

Data pipelines that don't lose data: orchestration, idempotency, and backfill

Moving millions of documents through ingestion and indexing reliably is its own discipline. Orchestrators, queues, idempotent steps, and careful backfill are what separate a pipeline that heals from one that silently drops data.

Engineering
Read
Dec 14, 2025 7 min read

Polyglot persistence: the right datastore for each job (and keeping them in sync)

No single database is best at everything, so serious systems use several — search, vector, graph, relational, object storage — each for what it's good at. The hard part isn't picking them; it's keeping them consistent.

Engineering
Read
Dec 13, 2025 6 min read

Graph databases: modeling and querying connected data

When your questions are about relationships — who cites whom, what connects to what — a graph database answers them directly. Here's how Neo4j/Cypher and Neptune/Gremlin model and traverse connected data, and the traps.

Engineering
Read
Dec 7, 2025 8 min read

How Elasticsearch and OpenSearch actually work: Lucene, shards, and the write path

A search cluster is a distributed layer over Lucene, and most performance problems trace back to how documents are written, refreshed, merged, and sharded. Here's the mental model that makes it tunable.

Engineering
Read
Dec 6, 2025 6 min read

Classical NLP foundations still worth knowing: TF-IDF, BM25, clustering, and metrics

LLMs didn't make classical NLP obsolete — hybrid search, deduplication, and evaluation all rest on it. A refresher on the pre-transformer toolkit that still quietly runs production systems.

Engineering
Read
Nov 30, 2025 6 min read

Document AI: layout-aware extraction, OCR, and tables

Real documents are scans, PDFs, and forms — not clean text. Getting structured data out of them needs OCR, layout-aware models, and table extraction. Here's the stack before an LLM ever sees the content.

EngineeringProduct
Read
Nov 29, 2025 6 min read

Fine-tune, RAG, or prompt? A decision framework

The three ways to adapt an LLM solve different problems, and teams reach for the expensive one first. Here's when each is right — and why 'the model doesn't know our stuff' almost never means fine-tuning.

EngineeringProductFounders & Business
Read
Nov 23, 2025 6 min read

GraphRAG: retrieval over a knowledge graph, not just a vector index

Vector RAG finds similar chunks. GraphRAG builds a knowledge graph of entities and relations from your documents and traverses it — which is what you need for multi-hop questions and provenance.

EngineeringProduct
Read
Nov 22, 2025 7 min read

Vector search internals: HNSW, quantization, and filtered search

A vector database is an ANN index with knobs. Understanding HNSW's parameters, quantization, distance metrics, and filtered search is what lets you trade recall, latency, and memory deliberately at scale.

Engineering
Read
Nov 15, 2025 6 min read

Sub-agents in Deep Agents: handing off focused work

The highest-leverage move a deep agent makes is delegation — spinning up a focused sub-agent for a slice of work so the main agent's context stays clean. Here's how sub-agents work and when to use them.

Engineering
Read
Nov 9, 2025 6 min read

Agent Skills: giving an agent new capabilities with SKILL.md

A skill is a folder with a SKILL.md file that teaches an agent a capability — a workflow, expertise, scripts — loaded through progressive disclosure so it costs almost no context until it's needed.

EngineeringProduct
Read
Nov 8, 2025 6 min read

The Playwright MCP server: giving an agent a real browser

Microsoft's Playwright MCP server lets an AI agent drive a real browser through MCP — navigate, click, type, read the page — using the accessibility tree, not screenshots. Here's how it works and where it's used.

EngineeringProduct
Read
Nov 2, 2025 6 min read

Deep Agents: planning, sub-agents, and a filesystem for long tasks

Deep agents are the harness layer — you supply intent, and the framework brings planning, sub-agents, and a virtual filesystem for long-horizon work. Here's what create_deep_agent gives you, including the backend that stores its files.

Engineering
Read
Nov 1, 2025 5 min read

create_agent: LangChain's prebuilt agent in one call

create_agent hands you the agent loop prebuilt — a model, tools, and a prompt in one function — and returns a compiled LangGraph graph, batteries included: memory, structured output, middleware, and MCP.

Engineering
Read
Oct 26, 2025 6 min read

Building with LangChain Core: models, messages, tools, and the loop

langchain-core is the bottom layer, where you control everything. Here's how to actually use its building blocks — one model interface, typed messages, the @tool decorator, the hand-written loop, and structured output.

Engineering
Read
Oct 25, 2025 8 min read

LangGraph internals: state, reducers, super-steps, and time travel

Beneath LangGraph's nodes and edges is a precise runtime model — immutable state merged by reducers, executed in super-steps, checkpointed after each one. Understanding it is what makes LangGraph predictable.

Engineering
Read
Oct 19, 2025 7 min read

The LangChain ecosystem in four layers: core, langgraph, create_agent, deepagents

LangChain confuses people because it isn't one library — it's a stack of abstraction layers, from raw building blocks you control completely up to a harness you just point at a goal. Here's the map.

Engineering
Read
Oct 18, 2025 7 min read

The OpenAI Agents SDK: a small, production-ready way to build agents

OpenAI's Agents SDK is a lightweight, Python-first framework built on a handful of primitives — agents, tools, handoffs, guardrails, and sessions. Here's what each does and when to reach for it.

Engineering
Read
Oct 12, 2025 7 min read

MCP servers: the USB-C port for AI tools and data

The Model Context Protocol is an open standard for connecting AI apps to tools and data — build a server once, and any MCP-aware client can use it. Here's the architecture, the primitives, and how to build one.

EngineeringProduct
Read
Oct 11, 2025 7 min read

asyncio: asynchronous Python without the weight of threads or processes

For I/O-bound work — network calls, databases, LLM APIs — asyncio runs thousands of tasks concurrently on a single thread. Here's the mental model: coroutines, the event loop, cooperative multitasking, and gather.

Engineering
Read
Oct 5, 2025 6 min read

Agent loops: the reason–act–observe cycle at the heart of every agent

Under the branding, every agent is a loop: the model reasons, calls a tool, sees the result, and goes again until it's done. Here's how that loop works, the ways it fails, and the controls that keep it in line.

EngineeringProduct
Read
Oct 4, 2025 7 min read

Context engineering: the discipline that decides whether your agent works

The frame is shifting from prompt engineering to context engineering — from crafting one clever string to building the system that assembles everything the model sees. Here's what it is and why most agent failures live here.

EngineeringProductFounders & Business
Read
Sep 28, 2025 6 min read

The AI agent landscape: builders, products, runtimes, and frameworks

"Agent" now spans no-code canvases, finished products, cloud runtimes, and developer libraries. Two questions — who it's for, and what they do with it — sort the whole confusing space into four clean categories.

Founders & BusinessProductEngineering
Read
Sep 21, 2025 6 min read

Prompt injection and LLM security: assume your model can be socially engineered

Any text your model reads can try to hijack it. Treat retrieved content as data not instructions, give tools least privilege, and design so the blast radius stays small when the model gets fooled.

EngineeringProductFounders & Business
Read
Sep 20, 2025 6 min read

Running LLMs in production: monitoring, fallbacks, latency, and cost

Shipping the prompt is the easy part. Tracing every step, circuit breakers for provider outages, model tiering, semantic caching — and why cost-per-successful-task is the KPI that matters.

EngineeringFounders & Business
Read
Sep 14, 2025 5 min read

Testing the untestable: unit tests, mocked LLMs, and eval suites

An LLM app is deterministic plumbing wrapped around probabilistic intelligence. Test each with the right tool — pytest and mocks for the machinery, evals for the brain — and stop trying to unit-test creativity.

Engineering
Read
Sep 13, 2025 6 min read

Eval discipline: proving your non-deterministic AI is actually correct

You can't unit-test a probabilistic system into correctness, but you can prove it clears a bar. Golden datasets, LLM-as-judge (and its biases), shadow rollouts, and treating prompt changes like code changes.

EngineeringProduct
Read
Sep 7, 2025 5 min read

Most RAG failures are retrieval, not generation — evaluate them separately

Most 'the AI answered wrong' bugs are actually retrieval misses. Splitting the two — recall@k and MRR for retrieval, faithfulness for generation — turns a vague failure into a fixable one.

Engineering
Read
Sep 6, 2025 6 min read

RAG beyond the tutorial: chunking, hybrid search, and metadata filtering

Production RAG needs more than embed-and-retrieve — semantic chunking, BM25 + vector fusion for exact tokens, tenant-scoped retrieval, and the discipline to know when NOT to use RAG at all.

EngineeringProduct
Read
Aug 31, 2025 6 min read

Scaling NLQ to large schemas with schema retrieval and few-shot banks

Hundreds of tables won't fit in a prompt. Embedded schema cards, similarity-retrieved examples, and a few targeted accuracy levers are how NLQ scales past a toy database.

Engineering
Read
Aug 30, 2025 6 min read

Query safety: how to stop an LLM from destroying your database

An LLM writing queries against your database is a liability unless you build defence in depth — read-only credentials, static validation, execution limits, and tenant filters the model never touches.

Engineering
Read
Aug 24, 2025 6 min read

The silent failure problem: when your NLQ runs but returns the wrong answer

The scariest NLQ bug isn't a crash — it's a query that executes cleanly and returns a confidently wrong number. Here's how to make wrong answers detectable instead of invisible.

EngineeringProduct
Read
Aug 23, 2025 6 min read

Text-to-query: building natural-language querying over MongoDB and SQL

The full pipeline from a user's question to an executed query and a readable answer — schema cards, constrained generation, validation layers, and result summarisation.

EngineeringProduct
Read
Aug 17, 2025 6 min read

Multi-agent systems: when you actually need them (usually you don't)

Supervisor patterns and planner-retriever-synthesizer architectures are elegant on a slide. Here's an honest look at when multiple coordinating agents beat one well-tooled agent — and when they just add failure modes.

EngineeringProduct
Read
Aug 16, 2025 5 min read

Memory and state in multi-turn agents: context without blowing the window

A long conversation will overflow any context window. Sliding windows, rolling summaries, and structured long-term memory keep an agent coherent without paying to resend everything, every turn.

Engineering
Read
Aug 10, 2025 6 min read

Tool calling done right: typed functions, allowlists, and guardrails

Letting an LLM take actions is where the risk lives. Typed schemas, per-state tool allowlists, timeouts, and iteration caps are what keep a capable agent from becoming a runaway one.

Engineering
Read
Aug 3, 2025 6 min read

Building production agents with LangGraph: state, checkpoints, and control

A practical walkthrough of modelling an agent as an explicit graph — nodes, conditional edges, shared state, and checkpoints that make sessions resumable and debuggable.

Engineering
Read
Jul 27, 2025 6 min read

QLoRA: fine-tuning a big model on one GPU — and the knobs that matter

QLoRA fine-tunes a large model by freezing a 4-bit-quantized base and training a tiny adapter. It put fine-tuning within reach of a single GPU. Here's how it works and which hyperparameters to tune.

Engineering
Read
Jul 26, 2025 6 min read

How an LLM is trained, end to end: from raw data to a fine-tuned model

Behind a chat model is a long pipeline: curate data, pre-process it, pretrain a base model, scale to a frontier model, then fine-tune. Here's the journey, stage by stage.

EngineeringProduct
Read
Jul 20, 2025 7 min read

Making RAG better: a menu of techniques that actually move the needle

Basic RAG is easy; good RAG is a stack of techniques. From chunking and reranking to query rewriting, Graph RAG, and agentic retrieval — here's the toolkit, organised by where it helps.

EngineeringProduct
Read
Jul 19, 2025 6 min read

Vector databases: where your embeddings live

Once you're embedding text, you need somewhere to store and search millions of vectors fast. Here's what a vector database does, the popular options, and when a Postgres extension is enough.

Engineering
Read
Jul 13, 2025 5 min read

LangSmith: seeing what your LLM app is actually doing

LLM apps fail in ways ordinary logs don't capture. LangSmith gives you tracing, evals, and monitoring across the whole chain — and it works whether or not you build on LangChain.

EngineeringProduct
Read
Jul 12, 2025 6 min read

LangGraph: when your agent needs a real state machine

LangGraph models an agent as a graph of nodes and edges with shared state — giving you explicit, durable control over the multi-step and cyclic flows a linear chain can't express.

Engineering
Read
Jul 6, 2025 6 min read

Encoders and vector embeddings: turning meaning into numbers

Generative models get the attention, but encoder models quietly power search, RAG, and recommendations by turning text into vectors that capture meaning. Here's how embeddings work.

EngineeringProduct
Read
Jul 5, 2025 6 min read

Measuring a GenAI solution: technical metrics and business metrics

A model can ace every technical metric and still fail the business — or the reverse. You need both layers, and you need them connected. Here's the two-part scorecard.

Founders & BusinessProductEngineering
Read
Jun 29, 2025 6 min read

Three kinds of AI product: GPT wrappers, specialists, and agents

Not all AI apps are built — or defended — the same way. A look at the three archetypes: thin wrappers, domain specialists, and agentic products, and where the moat actually is.

Founders & BusinessProductEngineering
Read
Jun 28, 2025 6 min read

Reading LLM benchmarks: MMLU, GPQA, AIME, LiveCodeBench, MuSR, HLE

Every model launch quotes a wall of benchmark scores. Here's what the big ones actually measure, why some are saturating, and how to read them without being sold to.

EngineeringProductFounders & Business
Read
Jun 22, 2025 5 min read

The Chinchilla scaling law: the 20-tokens-per-parameter rule of thumb

DeepMind's Chinchilla showed most big models were undertrained — and gave a rule of thumb for balancing model size against data. Here's the idea and why it reshaped how models are built.

EngineeringProduct
Read
Jun 21, 2025 5 min read

Three levels of the Hugging Face API: pipelines, tokenizers, and models

The transformers library lets you work at whatever altitude you need — a one-line pipeline, or hands-on with the tokenizer and the model. Knowing the three levels tells you when to drop down.

Engineering
Read
Jun 15, 2025 5 min read

Google Colab + Hugging Face Datasets: a free lab for AI experiments

Colab gives you a free GPU-backed notebook in the browser; Hugging Face Datasets gives you thousands of ready datasets in one line. Together they're the fastest way to start experimenting.

Engineering
Read
Jun 14, 2025 5 min read

Text to speech with OpenAI: a natural voice in one API call

One call turns text into natural-sounding speech. Here's how OpenAI's text-to-speech API works — the models, voices, formats, and streaming — and where it fits in a product.

EngineeringProduct
Read
Jun 8, 2025 5 min read

Image generation with DALL·E 3: prompt in, picture out

DALL·E 3 turns a plain-language description into an image, and its real leap was prompt-following. Here's how it works, how to call it, and what to know before you build on it.

EngineeringProduct
Read
Jun 7, 2025 6 min read

Generative AI vs Agentic AI: what's the difference, and which does your business need?

Generative AI produces content when you ask; agentic AI pursues a goal and takes actions on its own. The distinction matters because they solve different problems — and carry different risks.

Founders & BusinessProductEngineering
Read
Jun 1, 2025 6 min read

RAG fundamentals: giving a model the knowledge it wasn't trained on

Retrieval-Augmented Generation lets a model answer from your data — docs, tickets, a knowledge base — without retraining it. Here's the whole pipeline, from indexing to a grounded answer.

EngineeringProduct
Read
May 31, 2025 7 min read

The anatomy of an agent framework: loop, tools, MCP, and structured output

Strip away the branding and every agent framework assembles the same few parts: an agent loop, tool calling, a way to connect tools (increasingly MCP), and structured outputs. Here's how they fit.

Engineering
Read
May 25, 2025 8 min read

The agent framework landscape: lightweight SDKs vs orchestration frameworks

OpenAI Agents SDK, Google ADK, AWS Strands, CrewAI, LangGraph — the space looks crowded, but it splits into two camps. Knowing which camp a tool belongs to, and how they compare on control, state, and lock-in, tells you what each is good at.

EngineeringProduct
Read
May 24, 2025 6 min read

Where your agent actually runs: choosing an agent runtime

Writing an agent is one thing; hosting it so it runs reliably, holds state, and scales is another. A tour of the runtimes — from general-purpose serverless to purpose-built agent platforms.

EngineeringFounders & Business
Read
May 18, 2025 6 min read

The OpenAI Responses API: should you use it, or stick with Chat Completions?

OpenAI's Responses API is its new default for building on the platform — stateful, with built-in tools and agentic features. Here's what it changes, and when Chat Completions is still the right call.

EngineeringProduct
Read
May 17, 2025 5 min read

CrewAI Studio: assembling a team of agents, visually

CrewAI models multi-agent systems as a crew — agents with roles, given tasks, working together. Studio puts that on a low-code interface so you can build and run a crew without wiring it in Python.

ProductFounders & BusinessEngineering
Read
May 11, 2025 5 min read

ElevenLabs: building voice agents you can actually talk to

ElevenLabs built its name on lifelike text-to-speech; its conversational AI platform assembles the whole voice-agent stack — speech-in, LLM, speech-out, turn-taking — into one builder.

ProductFounders & BusinessEngineering
Read
May 10, 2025 5 min read

OpenAI's Agent Builder: agents on a visual canvas

OpenAI's Agent Builder brings agent creation onto a drag-and-drop canvas — nodes, tools, and guardrails wired together visually. Here's what it is and who it's really for.

EngineeringProductFounders & Business
Read
May 4, 2025 5 min read

Gradio: a working UI for your model in a dozen lines of Python

Gradio turns a Python function into a shareable web app with inputs, outputs, and a link — no front-end code. It's how AI demos and internal tools get built in an afternoon.

EngineeringProduct
Read
May 3, 2025 6 min read

Prompt caching: stop paying full price for the same prefix

If every call resends the same long system prompt and context, you're paying to process identical tokens over and over. Prompt caching fixes that — here's how OpenAI, Anthropic, and Gemini each do it.

EngineeringFounders & Business
Read
Apr 27, 2025 5 min read

LiteLLM: one OpenAI-shaped interface for a hundred providers

LiteLLM lets you call OpenAI, Anthropic, Bedrock, Gemini, and local models through one consistent format — as a library or a proxy with keys, budgets, and logging. Here's where it fits.

Engineering
Read
Apr 26, 2025 6 min read

LangChain: the framework everyone starts with (for better and worse)

LangChain is the best-known toolkit for building LLM apps — chains, agents, memory, and integrations for everything. Here's what it gives you, and an honest take on when to use it.

EngineeringProduct
Read
Apr 20, 2025 5 min read

The reasoning-effort knob: paying for thinking only when it's worth it

Reasoning models let you dial how hard they think — low, medium, or high. Used well it's a precise lever over the quality-versus-cost trade. Used carelessly it's a silent bill.

EngineeringProductFounders & Business
Read
Apr 19, 2025 5 min read

Streaming LLM responses: why the tokens arrive one at a time

That typewriter effect isn't a gimmick — it's the model streaming tokens as it generates them. Here's how streaming works, why it transforms perceived speed, and where it gets tricky.

EngineeringProduct
Read
Apr 13, 2025 6 min read

Structured outputs: getting reliable JSON out of a text model

If your code has to parse the model's answer, 'please return JSON' isn't enough. Structured outputs and schema enforcement turn a probabilistic text generator into a dependable API.

Engineering
Read
Apr 12, 2025 5 min read

Zero-, one-, and few-shot prompting: teaching by example, in the prompt

One of the most reliable ways to steer a model is to show it examples. Here's the difference between zero-, one-, and few-shot prompting, and when each earns its tokens.

EngineeringProduct
Read
Apr 6, 2025 5 min read

The context window: the model's short-term memory, and its hard limit

Everything a model 'knows' in a request has to fit in its context window. Understanding that budget — what fills it, what it costs, where it fails — is core to building with LLMs.

EngineeringProduct
Read
Apr 5, 2025 5 min read

Tokens and tokenisers: the units your model actually reads (and bills)

Models don't see words or characters — they see tokens. Understanding the tokeniser explains your context limits, your bill, and why some languages cost more than others.

EngineeringProduct
Read
Mar 30, 2025 5 min read

Inference-time scaling: buying better answers with more thinking

The newest way to get a smarter answer isn't a bigger model — it's letting the model think longer at inference. Reasoning models made test-time compute a scaling axis of its own.

EngineeringProductFounders & Business
Read
Mar 29, 2025 5 min read

Training-time scaling: why bigger, longer, and more data kept working

For years the recipe for a better model was blunt: more parameters, more data, more compute. Scaling laws made that predictable — and eventually expensive. Here's the idea and its limits.

EngineeringProductFounders & Business
Read
Mar 23, 2025 5 min read

Model parameters: what they are, and why the count drives your bill

"7B", "70B", "quantized to 4-bit" — parameter counts and precision decide a model's memory, speed, and cost. Here's what a parameter really is, in operational terms.

EngineeringProduct
Read
Mar 22, 2025 7 min read

Self-attention and RLHF: how models focus, and how they're aligned

Two ideas do a lot of the heavy lifting in modern LLMs: self-attention, which lets a model decide what to focus on, and RLHF, which aligns it to what humans actually want.

Engineering
Read
Mar 16, 2025 7 min read

The Transformer: the architecture behind every modern LLM

The 2017 paper "Attention Is All You Need" replaced sequential processing with attention — and made today's LLMs possible. Here's the architecture, in plain terms.

Engineering
Read
Mar 15, 2025 6 min read

Base, Instruct, Reasoning, Hybrid: the LLM model types, explained

Not all LLMs are the same kind of thing. Knowing the difference between a base model, an instruct model, a reasoning model, and a hybrid tells you which one to reach for.

EngineeringProductFounders & Business
Read
Mar 9, 2025 6 min read

LLMs, neural networks, and parameters: what "7B" actually means

Under the branding, a language model is a big neural network, and its parameters are just numbers — weights and biases — learned from data. Here's the foundation in plain terms.

EngineeringProduct
Read
Mar 8, 2025 5 min read

Hugging Face: the GitHub of machine learning

Models, datasets, demos, and the libraries that tie them together — Hugging Face is where the open-source AI world lives. Here's a tour and why it matters.

EngineeringProduct
Read
Mar 2, 2025 5 min read

Amazon Bedrock: managed GenAI for teams already on AWS

Bedrock gives you many foundation models behind one API, without running any infrastructure — and keeps your data inside your AWS boundary. Here's what it is and who it fits.

EngineeringFounders & Business
Read
Mar 1, 2025 6 min read

Distillation: how DeepSeek packed a big model's smarts into small ones

Knowledge distillation trains a small "student" model to imitate a large "teacher". DeepSeek used it to hand its reasoning ability to models a fraction of the size — here's how it works.

EngineeringProductFounders & Business
Read
Feb 23, 2025 6 min read

Playwright: scraping and automating modern websites

Half the web is rendered by JavaScript, and a plain HTTP request sees none of it. Playwright drives a real browser — here's how to use it to scrape, and how to do it responsibly.

Engineering
Read
Feb 22, 2025 6 min read

The Chat Completions API: the interface that became a standard

OpenAI's Chat Completions API is the messages-based interface most of the industry now speaks. Here's how it works — roles, parameters, streaming, and tool calls.

Engineering
Read
Feb 16, 2025 6 min read

Prompt foundations: the anatomy of a prompt that works

A prompt isn't a magic incantation — it's an interface. Here are the fundamentals: the roles, the structure, and the handful of techniques that do most of the work.

EngineeringProduct
Read
Feb 15, 2025 5 min read

Jupyter Notebooks: the AI engineer's scratchpad

Before the code becomes a service, it lives in a notebook. Here's what Jupyter is, why it's the default workspace for AI work, and the traps that bite teams who forget it's a scratchpad.

Engineering
Read
Feb 9, 2025 5 min read

OpenRouter: one API for every model

OpenRouter is a single, OpenAI-compatible gateway to hundreds of models across providers — with fallback, routing, and unified billing. Here's when to route through it.

EngineeringProductFounders & Business
Read
Feb 8, 2025 5 min read

Groq: when latency is the feature

Groq runs open models on custom hardware at speeds that change what real-time AI can do. Here's what it is, and where that speed is worth designing around.

EngineeringProductFounders & Business
Read
Feb 2, 2025 5 min read

Ollama: run open models on your own machine — and when that's the right call

Ollama makes running open-weight LLMs locally as easy as one command. Here's what it is, when local inference is genuinely the right choice, and when it isn't.

EngineeringFounders & Business
Read
Feb 1, 2025 5 min read

Stop anthropomorphizing your agents (especially behind the scenes)

Giving your assistant a friendly name in the UI is fine. Designing your architecture as if the model thinks, remembers, or intends anything is a bug waiting to page you.

EngineeringProduct
Read
Jan 26, 2025 5 min read

Before you build the AI thing: what problem are we actually solving?

A snarky reminder from your friendly neighbourhood AI engineer: not everything needs a large language model. Some problems need a database index, a form, or someone to just make a decision.

Founders & BusinessProductEngineering
Read
Jan 25, 2025 8 min read

Guardrails: the line between an agent demo and one you can deploy

An agent with real tools and real users needs more than a clever prompt. Input and output guardrails are what keep it safe, on-topic, and out of trouble in production.

EngineeringProduct
Read
Jan 19, 2025 6 min read

Observability and evals: measure against the production goal, not the vibe

A dashboard full of green metrics means nothing if none of them predict the outcome you actually ship for. Tie every trace and eval back to the end goal — or you're measuring theatre.

EngineeringProduct
Read
Jan 18, 2025 Live dashboard

Independent AI model benchmarks: a live, always-current dashboard

Model quality, speed, and price move every week. Rather than quote a snapshot that's stale by tomorrow, here's the live, vendor-neutral dashboard from Artificial Analysis — embedded in full.

EngineeringProductFounders & Business
Read
Jan 12, 2025 7 min read

n8n explained: what it is, and which businesses it's right for

n8n has become the default way teams glue their tools together and add AI to everyday operations. Here's a plain-English intro — and an honest look at who it fits, and who should look elsewhere.

Founders & BusinessProductEngineering
Read
Jan 11, 2025 8 min read

Workflows vs AI agents: which one does your problem actually need?

"Agent" is the word everyone reaches for — and often the wrong tool. A clear-eyed comparison of workflows vs agentic frameworks: the pros, the cons, the risks nobody demos, and how to choose.

EngineeringProductFounders & Business
Read
Jan 4, 2025 10 min read

Building RAG that doesn't hallucinate in front of customers

A demo that answers your three test questions is easy. A retrieval system that stays grounded across thousands of real queries is a different discipline. Here's what actually matters.

EngineeringProduct
Read
Dec 29, 2024 6 min read

Designing AI agents that actually finish the job

Most agent demos loop, stall, or wander off. Shipping agents that complete real workflows is about state, tools, validation, and knowing when to ask a human.

EngineeringProduct
Read
Dec 28, 2024 5 min read

Evaluator–Optimizer: let the model critique its own work

Pair a generator with an evaluator in a feedback loop — generate, critique, refine — until the output clears a bar you define. Iteration as an AI workflow.

Engineering
Read
Dec 22, 2024 5 min read

Orchestrator–Workers: decompose the task at runtime

When you can't know the subtasks in advance, let an orchestrator LLM break the work down on the fly, delegate to workers, and synthesise their output.

Engineering
Read
Dec 21, 2024 4 min read

Parallelization: run LLM calls side by side, then combine

Fan a task out into independent LLM calls that run at once — either sectioning a job into parts, or voting for reliability — then aggregate the results.

Engineering
Read
Dec 15, 2024 4 min read

Routing: send each input to the model that should handle it

Classify the incoming request first, then dispatch it to a specialised prompt or model. Separation of concerns for AI workflows — and a quiet way to cut cost.

Engineering
Read
Dec 14, 2024 4 min read

Prompt Chaining: the workflow pattern to reach for first

The simplest AI workflow pattern — decompose a task into a fixed sequence of LLM calls, each handling one job, with gates to catch failures before they compound.

Engineering
Read