11. Multi-Model Strategies

Why Use Multiple Models?

No single model is best at everything. GPT-4o-mini is great for classification but weak at complex reasoning. DeepSeek-R1 is great at reasoning but slow and expensive for simple tasks. Claude Sonnet handles long documents well but costs more than GPT-4o-mini or Haiku for trivial work.

The smart approach: use different models for different parts of your workflow. Route each task to the model that handles it best at the lowest cost.

The Router Pattern

The simplest multi-model strategy: a router that decides which model to use based on the task.

User request comes in
  → Router classifies the request
  → Simple question? → GPT-4o-mini (fast, cheap)
  → Complex reasoning? → o4-mini (thinking model)
  → Long document? → Gemini Pro (1M context)
  → Code generation? → Claude Sonnet (strong at code)

The router itself can be a cheap model (GPT-4o-mini) or even a simple rule-based system:

Rules-based router:
  - Input > 50,000 tokens → Gemini Pro
  - Contains "explain step by step" or "prove" → o4-mini
  - Contains code blocks → Claude Sonnet
  - Everything else → GPT-4o-mini

This is crude but effective. You can refine it over time based on what you observe in production.

LLM-based router

For more nuance, use a cheap model to classify the request before routing it:

System: "Classify this user request into one of:
  - simple (factual question, classification, formatting)
  - code (code generation, review, debugging)
  - reasoning (math, logic, multi-step analysis)
  - long-context (document analysis, large input)
  Respond with just the category name."

User: [the actual request]

Cost of routing: ~$0.0001 per request (GPT-4o-mini, ~50 tokens). Savings from routing correctly: often 10 to 50x on the requests that get sent to cheaper models instead of expensive ones.

The Cascade Pattern

Try the cheapest model first. If it fails or produces low-quality output, escalate to a better model.

Step 1: Send to GPT-4o-mini
Step 2: Check output quality (confidence score, format validation, etc.)
Step 3: If quality is below threshold → retry with Claude Sonnet
Step 4: If still below threshold → retry with o3

This works well for tasks where the cheap model handles 80% of cases correctly. You only pay for the expensive model on the 20% that actually need it.

Example: Code generation

Request: "Write a function to reverse a string in Go"
  → GPT-4o-mini generates it
  → Run the code (or syntax check it)
  → If it compiles and passes basic tests → done
  → If it fails → retry with Claude Sonnet

How to detect failure

• Format validation: Did the model return valid JSON? Did it follow the schema?
• Confidence signals: Some APIs return log probabilities. Low confidence means escalate.
• Automated testing: For code, run it. For classification, check if the output is one of the valid categories.
• Length check: If you asked for 3 bullet points and got a 500-word essay, that's a format failure.

The Ensemble Pattern

Run the same request through multiple models and combine their answers. This improves accuracy at the cost of latency and money.

Request: "Is this code thread-safe?"

  → GPT-4o says: "Yes, it uses a mutex correctly"
  → Claude Sonnet says: "No, there's a race condition on line 12"
  → DeepSeek-R1 says: "No, the mutex doesn't cover the read on line 15"

  → 2 out of 3 say no → final answer: not thread-safe

Majority voting works for binary or categorical outputs. For open-ended generation, you can use a "judge" model to pick the best response from the candidates.

When to use ensembles:

• High-stakes decisions where accuracy matters more than cost
• Tasks where models disagree frequently (ambiguous inputs)
• When you need confidence scores ("3/3 models agree" vs "2/3 agree")

When NOT to use ensembles:

• High-volume, low-stakes tasks (cost multiplies by the number of models)
• Tasks where one model is clearly dominant and others add nothing

The Pipeline Pattern

Different models handle different stages of a multi-step workflow.

Stage 1: Extract data from document
  → Gemini Pro (handles the long document)

Stage 2: Classify extracted items
  → GPT-4o-mini (cheap, fast classification)

Stage 3: Generate detailed analysis for flagged items
  → Claude Sonnet (strong at nuanced analysis)

Stage 4: Summarize findings
  → GPT-4o-mini (simple summarization)

Each stage uses the model best suited for that specific sub-task. The expensive models only touch the parts that actually need them.

Real example: PR review pipeline

1. Get the diff (tool call, no model needed)
2. Classify changes by risk level
   → GPT-4o-mini: "high-risk: auth changes, medium-risk: API changes, low-risk: docs"
3. Deep review of high-risk changes
   → Claude Sonnet: detailed security and correctness review
4. Quick review of medium-risk changes
   → GPT-4o-mini: basic sanity check
5. Skip low-risk changes (docs, comments)
   → No model needed, no cost

Result: thorough review where it matters, cheap review where it doesn't, no review where it's unnecessary.

Cost Comparison

Scenario: 1000 requests/day, mixed complexity.

Multi-model strategies typically save 60 to 80% compared to using a single expensive model for everything.

Implementation Tips

• Start simple. A rules-based router is fine to start. Add LLM-based routing later when you have data showing where the rules fail.
• Log everything. Track which model handled each request and the quality of the output. This data tells you where to optimize next.
• Set fallbacks. If your primary model is down or rate-limited, fall back to an alternative automatically.
• Measure before optimizing. Know your current cost and quality baseline before adding complexity.

Key Takeaways

• No single model is best at everything. Use different models for different tasks.
• Router pattern: classify requests and send them to the appropriate model.
• Cascade pattern: start cheap, escalate only when the cheap model fails.
• Ensemble pattern: run multiple models and combine answers for high-stakes decisions.
• Pipeline pattern: different models handle different stages of a workflow.
• Multi-model strategies save 60 to 80% compared to using one expensive model for everything.
• Start with a simple rules-based router. Add complexity only when the data shows you need it.