How to reduce prompt tokens

Fewer input tokens means lower cost and more context headroom. These techniques cut token usage without sacrificing output quality. Measure each change with the token counter.

1. 1. Trim the system prompt

   Long, repetitive system prompts are billed on every call. Keep instructions terse and move rarely-needed detail into tools or retrieval.

2. 2. Cache the stable prefix

   If a large prefix repeats across calls, use prompt caching so it is billed at the cached rate. See the prompt caching calculator.

3. 3. Summarize history

   Instead of resending full conversation history, summarize older turns and keep only the recent ones verbatim.

4. 4. Retrieve less, rank better

   In RAG, fewer high-quality chunks usually beat many mediocre ones and cost far fewer input tokens.

5. 5. Minify structured data

   Strip whitespace from JSON, shorten key names, and drop fields the model does not need.

6. 6. Cap the output

   Set max output tokens. Output is priced higher than input, so a runaway response is expensive.

A dedicated in-browser optimizer that rewrites prompts is planned. For now, apply these techniques and verify the savings in the counter.