Cut LLM costs in a small SaaS with flex-tier fallback, cache-friendly prompt order, side-by-side model migrations and backend circuit breakers

A small SaaS that runs many background LLM calls (here, transcript extraction and analysis) can see its bill jump, its outputs break when a model changes, or a single button get abused into large costs. A bootstrapper has no budget for that.

1. Put every AI call in its own service that builds the prompt and calls the model, so the model or prompt version can be switched or rolled back. 2. When moving to a new model, run the old and new prompt and model in parallel for a while, log both outputs and diff the JSON to find missing keys; use a coding agent such as Claude Code or Codex to adjust the new prompt until the outputs match. The author hit exactly this when a prompt written for GPT-4.1's JSON mode started dropping keys on GPT-5. 3. For work that can wait, send jobs to OpenAI's Flex tier first (about half price, sometimes slower or unavailable) and, on a 429 rate-limit error, retry on the standard tier. 4. Order each prompt so the stable parts come first: system role, fixed instructions and output format, then the long repeated data (such as the full transcript) kept byte-identical, and the task-specific instruction last, so repeated calls hit the cache. 5. Route all AI calls through the backend, never from the client with your own key; rate-limit both user-triggered and background calls, add backend feature switches that can turn an AI feature off, and alert on unusual token use per account, plan or IP.

The author reports that moving suitable jobs to the Flex tier cut that cost by about half straight away, which let him double extraction volume for the same spend. He says cached input tokens cost about a tenth of uncached ones. No total bill, before-and-after figures or error rates are given.

Self-reported by the founder from one product (Podscan); no bills or measurements are shown. The tier advice is specific to OpenAI and its prices and tier availability change; Anthropic and Google are mentioned only in passing. Running two models in parallel doubles cost while it is on. Flex suits only delay-tolerant background work. Alert thresholds in the post are examples, not benchmarks. The page carries sponsor and own-product promotion.

A product that already makes LLM API calls from a backend, and enough code access to add a fallback, logging and feature switches.

• The Bootstrapped Founder (Arvid Kahl) ↗ Firsthand write-up, published November 28, 2025

Source published: November 28, 2025. Last reviewed here: October 23, 2026. Spot a mistake? Tell us.