AI
Free-Tier Model Fallback Chains for an Autonomous Agent
August 202612 min read
Almost certainly because both are on the same provider account. Rate limits usually attach to the account or the API key, not to the model, so a fallback to a different model on the same account is a rename rather than a fallback. Make every stage cross the account boundary.
Two calls. A request to the models endpoint returning 200 proves the key is valid and the account is active. A completions request returning 429 on top of that proves the free allowance is spent rather than the key being revoked. Logs rarely separate these clearly, and the distinction changes what you should do next.
Yes, if your runtime honours the provider's Retry-After header. Raising the retry count well above the default turns a per-minute throttle into a pause rather than a failure, which is exactly the go-slow behaviour you want. Keep a wall-clock cap on the whole run so patience cannot become an infinite wait.
Check four things from the machine that will use it: whether the runtime has a first-class integration rather than a generic adapter, whether the context window meets the minimum, whether a real completion request is fast from that host, and whether the specific model actually answers twice in a row. A model list is a catalogue, not a promise.
Suspect the protocol before the provider. One endpoint in my chain stalled on HTTP/2 request bodies from a particular network path and worked perfectly over HTTP/1.1 — and the agent runtime, which never negotiated HTTP/2, could reach it the whole time. Also send a non-default user agent, since some gateways reject the default CLI string.
Key Takeaway
A fallback chain only helps if each stage draws on a different provider account, because a rate limit applies to the bucket and not the model. Pair separate buckets with a high retry count that honours Retry-After, and a free-tier agent survives throttling instead of dying at the first 429.
I run a coding agent that opens and merges pull requests every day, and for the whole first month it ran on free model tiers. That constraint taught me more about production model routing than any amount of paid usage would have, because free tiers fail constantly and in every possible way.
The lessons below are all things I got wrong before I got them right, and every one of them applies just as much to a paid setup — a paid account hits limits too, just less often and more expensively.
The first thing to internalise is that not all rate limits are the same, and the response you should build differs for each.
Distinguishing these matters because the first calls for patience, the second calls for a different bucket, and the third calls for accepting that you cannot know and designing for self-healing.
My first fallback chain was useless and I did not realise for weeks. Primary and fallback were two different models — from the same provider, on the same free account.
# The rule that took me three rewrites to learn:
# every stage of the chain must be a SEPARATE quota bucket.
model:
default: provider-a/coding-model-free # primary
fallback: provider-b/large-instruct-model # different vendor, own quota
last: provider-c/flash-model # different vendor again
agent:
max_turns: 80
api_max_retries: 10 # honours Retry-After, waits out per-minute windows
# The version that did NOT work:
# default: provider-a/model-one-free
# fallback: provider-a/model-two-free <- same account, same bucket
# When provider A rate-limited the account, both legs failed identically.When that account hit its allowance, both legs returned the same 429, so the chain provided exactly zero resilience while looking like a designed system. The rewrite made every stage a different vendor with its own key and its own quota, and the chain started earning its keep immediately: on the first day the primary was capped, a run completed through the second stage and I only found out by reading the log.
A fallback to a different model on the same account is not a fallback, it is a rename. Check what the quota is actually attached to — usually the account or the API key, not the model — and make each stage cross that boundary.
Not every provider is a viable stage, and the criteria are more practical than they are about model quality.
| Criterion | Why it matters | How to check before wiring it in |
|---|---|---|
| First-class integration | Generic OpenAI-compatible adapters can shape requests badly and blow up on tool-heavy turns | Prefer a provider the runtime ships a dedicated plugin for; test with your full toolset, not a bare prompt |
| Context length | An agent turn carries files, tool results and history; a small window fails mid-task | Reject anything below the runtime's minimum, and be honest that a documented number is not always a usable one |
| Reachability from your host | Some endpoints are fast to reach from one region and effectively unusable from another | Time an actual completion request from the VPS, not from your laptop |
| Model warmth | On free serverless tiers, some listed models are not kept warm and simply hang | Call each candidate model directly; keep only the ones that answer reliably twice in a row |
That last row cost me an afternoon. A provider's model list is a catalogue, not a promise, and the difference between a listed model and a served model is invisible until you call it.
Two curl calls tell you almost everything about a candidate provider, and they distinguish the failure modes that logs never separate clearly.
# Probe a candidate provider BEFORE wiring it in. Two calls, two answers.
# 1. Is the key valid and the account active?
curl -s -o /dev/null -w '%{http_code}\n' \
-H "Authorization: Bearer $KEY" -A "agent-probe/1.0" \
https://api.example.com/v1/models # 200 = key fine
# 2. Can it actually complete? This is the one that matters.
curl -s -X POST -H "Authorization: Bearer $KEY" \
-H 'content-type: application/json' -A "agent-probe/1.0" \
-d '{"model":"m","messages":[{"role":"user","content":"ping"}]}' \
https://api.example.com/v1/chat/completions | head -c 300
# A 200 on /models with a 429 on completions means the key is fine and
# the free allowance is spent — a distinction worth ten minutes of
# reading logs, and one no error message ever spells out.One more transport lesson worth carrying: if a request to a provider hangs rather than failing, suspect the protocol before the provider. I lost a day to an endpoint that stalled on HTTP/2 request bodies from one particular network path and worked perfectly over HTTP/1.1 — and the agent runtime, which did not negotiate HTTP/2, had been able to reach it the whole time.
Send a non-default user agent on every probe. Some gateways sit behind a web application firewall that rejects the default command line agent string, which produces a rejection that looks exactly like a bad key.
Once the buckets are right, the second lever is how patient each attempt is. Four settings interact and all of them need to be set together.
The most expensive lesson in this whole area was self-inflicted: two full validation runs plus a third job in one afternoon exhausted the day's free allowance, and the scheduled runs then failed on top of it. The rule I follow now is one real run per day per provider, and let the scheduled run be the test.
There is a point where free-tier engineering stops being frugal and starts being a hobby. Mine arrived when the config had three providers, a retry policy and a probe script, and the total spend that would have replaced all of it was roughly the price of a coffee per month.
The honest reason I kept going was that the constraint produced a better system: a chain with independent buckets, generous retries and a wall-clock cap is more robust on a paid account too. If you do this exercise, keep the architecture and then pay for the primary stage — you end up with something that survives an outage at your provider, which no amount of money buys on its own.
Free model tiers are an excellent teacher because they fail on schedule. Separate quota buckets per stage, retries that wait out per-minute windows, a wall-clock cap, and probes that distinguish a dead key from a spent allowance — those four ideas are what turned a daily agent from something that broke most mornings into something I stopped checking.