Last Updated: May 1, 2025
Token Counting & Pricing
| Model | Input $/1M tokens | Output $/1M tokens | Context Window |
|---|---|---|---|
| GPT-4o | $2.50 | $10.00 | 128K |
| GPT-4o-mini | $0.15 | $0.60 | 128K |
| GPT-4 Turbo | $10.00 | $30.00 | 128K |
| Claude 3 Opus | $15.00 | $75.00 | 200K |
| Claude 3 Sonnet | $3.00 | $15.00 | 200K |
| Claude 3 Haiku | $0.25 | $1.25 | 200K |
| Gemini 1.5 Pro | $1.25–$5.00 | $5.00–$20.00 | 1M (tiered) |
| Llama 3 70B (Together) | $0.90 | $0.90 | 8K |
| Mixtral 8x7B (Together) | $0.60 | $0.60 | 32K |
Prompt Caching Strategies
Anthropic Prompt CachingCache repeated content across API calls — 90% cost reduction on cache hits
Cache BreakpointsMark cache boundaries in your prompt — system instructions, tool definitions, few-shot examples
Minimum Cache SizeAnthropic: 1024 tokens min for cached blocks (Sonnet/Haiku); 2048 for Opus
Cache TTLCached content lives for ~5 minutes of inactivity — keep requests flowing within the window
Google Context CachingGemini: cache system instructions + large docs — pay reduced rate for cached tokens
OpenAI Prompt Caching (beta)Automatic caching — no API changes needed. 50% discount on cached prefixes
tiktoken for countingUse tiktoken to count tokens before sending — predict costs before API call
Streaming + CostStreaming doesn't change token count but you only see output cost after completion
Prompt Compression
| Item | Description |
|---|---|
LLMLingua | Compress prompts by 2-5x while preserving task performance — removes non-essential tokens |
Selective Context | Keep last N turns + summarize earlier conversation — reduces context bloat in long chats |
Few-Shot Pruning | Remove few-shot examples that are irrelevant to the current query — dynamic example selection |
Tool Description Trimming | Only include tools relevant to current task phase — don't send all tools every turn |
LongContextFusion | Fuse multiple retrieved documents into one condensed context before sending |
Extractive Summarization | Summarize retrieved chunks to 1-2 sentences each before passing to LLM |
Token Budget | Set hard token budget per request: if context > budget, summarize oldest content first |
Model Selection by Task
| Task Type | Recommended Model | Cost Level | Why |
|---|---|---|---|
| Simple classification | GPT-4o-mini / Haiku | $ | Fast, cheap, sufficient accuracy |
| Chat / customer support | GPT-4o / Claude Sonnet | $$ | Balance of quality and cost |
| Complex reasoning | Claude Opus / GPT-4o | $$$ | Deep reasoning, code generation |
| Summarization | GPT-4o-mini / Haiku | $ | Pattern matching, not deep reasoning |
| Code generation | Claude Sonnet / GPT-4o | $$ | Strong code capabilities |
| Translation | GPT-4o-mini / Haiku | $ | Specialized models often overkill |
| RAG synthesis | GPT-4o / Claude Sonnet | $$ | Needs to synthesize multiple sources |
| Safety/filtering | GPT-4o-mini / Haiku | $ | Lightweight guard model before main call |
Pro Tip: Prompt caching (Anthropic) and context caching (Google) can reduce costs by 90% for repeated system prompts. Design your application to reuse cached prefixes — it's the single biggest cost win.