Table of Contents
How AWS Bedrock AgentCore Gateway's model fallback actually works
A throttled model used to mean a 503 landing straight in your agent, and a retry loop some engineer wired in at midnight. Starting October 22, 2026, AWS moved that retry loop into the gateway itself. Bedrock AgentCore Gateway's inference endpoint now speaks Bedrock's native InvokeModel and InvokeModelWithResponseStream APIs, and a gateway rule can define an ordered fallback chain with a circuit breaker, so a dead model gets routed around without the calling app knowing, per AWS's What's New post (Oct 22, 2026).
Before this release, AgentCore Gateway's Bedrock connector only forwarded to Bedrock Mantle, and a plain InvokeModel call had to go straight to Bedrock, skipping the gateway's IAM scoping and routing logs entirely, per DevelopersIO's walkthrough (Oct 25, 2026). That connector gap closed with this release, along with the billing details and two limits worth knowing before you wire it in.
Point your existing client at the gateway
The InvokeModel side is a one-line change. Create a bedrock-runtime client and set its endpoint to the gateway's /inference path; every existing invoke_model and invoke_model_with_response_stream call keeps working, now routed through whatever rules you've written:
import boto3
client = boto3.client(
"bedrock-runtime",
region_name="us-east-1",
endpoint_url="https://<GATEWAY_ID>.gateway.bedrock-agentcore.us-east-1.amazonaws.com/inference",
)
response = client.invoke_model(
modelId="us.anthropic.claude-haiku-4-5-20251001-v1:0",
body=claude_body,
contentType="application/json",
)That part is new as of this release, per AWS's What's New post. The Bedrock Runtime connector itself still needs the CLI or SDK to create, because as of October 25, 2026 it wasn't yet a selectable option in the gateway console, per DevelopersIO.
The rule: a chain, a trigger, a breaker
Fallback lives on a gateway rule, not in your client code. This is AWS's own example from the Gateway rules documentation: route gpt-5.5 to Bedrock Mantle first, fall back to an OpenAI target on failure, and only then degrade to the smaller gpt-5.4 on the same Mantle target.
aws bedrock-agentcore-control create-gateway-rule \
--gateway-identifier my-gateway-abc1234567 \
--priority 100 \
--description "Prefer gpt-5.5 on bedrock-mantle, fall back to OpenAI, then degrade to gpt-5.4" \
--conditions '[{"matchModels": {"anyOf": [{"value": "gpt-5.5"}]}}]' \
--actions '[{
"routeToTarget": {
"staticRoute": {
"targetName": "bedrock-mantle",
"model": "gpt-5.5",
"fallback": {
"circuitBreaker": {"mode": "ENABLED"},
"triggerOn": {
"statusCodes": ["5XX", "429"],
"errors": ["CONNECTION_ERROR", "FIRST_BYTE_TIMEOUT", "UNROUTABLE"]
},
"firstByteTimeoutMilliseconds": 5000,
"targetFallback": [
{"targetName": "openai"},
{"targetName": "bedrock-mantle", "model": "gpt-5.4"}
]
}
}
}
}]'One rule does three jobs a team usually codes by hand: triggerOn decides which errors count as a failure (5XX, 429, a dropped connection, or a first byte that never arrives within 5 seconds here), targetFallback names the backup chain in order, and circuitBreaker stops the gateway from retrying a target that's already down. DevelopersIO's own test confirms the behavior: with the primary forced to return 503, three calls that would have failed outright instead came back 200 from the fallback target in 1.57 to 1.93 seconds each, with zero client-side changes, per their walkthrough.
What the circuit breaker changes about your bill
The breaker isn't just a latency trick. It changes what AWS charges you for a failing primary, per the Gateway rules documentation:
| Breaker state | What happens to the request | What gets billed |
|---|---|---|
| Closed (healthy) | Primary gets every request; a failing response triggers the fallback chain | Every attempt, including the failed primary call, counts as a gateway invocation |
| Open (failing) | Primary is skipped outright; the request goes straight to the next destination | The skipped primary is not billed; only the destinations actually attempted are |
| Half-open (recovering) | A small slice of traffic probes the primary; the rest keep skipping it | Probed requests bill as attempts; the rest bill as in the open state |
Gateway API calls, InvokeModel among them, bill at $0.005 per 1,000 invocations, per AWS's AgentCore pricing page. A request that fails on the primary and succeeds on the first fallback is two invocations, not one, because each attempt bills separately; the model or inference cost of each attempt is billed by Bedrock or the model provider on top of that, per the Gateway docs. The breaker's real job is capping how many of those double-billed failures you pay for once it's obvious the primary is down.
Two caps that still mean hand-rolled code somewhere
A fallback chain holds the primary plus one to two alternates. Not an arbitrary list, two, per AWS's own spec on targetFallback. If your failure domain needs a third or fourth backup, that logic goes back in your application.
Streaming is the other wall. Once invoke_model_with_response_stream has sent its first byte, the gateway can't fall back anymore; a stream that dies mid-response returns a partial payload and an error, and the caller has to retry from the top, per AWS's docs. For anything long-running and streamed, the circuit breaker only protects the request that hasn't started yet.
One more operational gap: the caller never learns which target actually answered. That detail lives only in the gateway's routing logs, per AWS's docs, so tracing "why did this response sound different" means pulling logs, not reading a response header.
Where this sits next to LiteLLM, Portkey, and Google's answer
Ordered fallback across model providers isn't a new idea. LiteLLM runs the same pattern as a self-hosted open-source proxy, and managed gateways like Portkey and OpenRouter have done it for a while. What AWS shipped is that pattern living inside infrastructure you're already running, with the IAM scoping and audit trail AgentCore already has, instead of a side-car proxy you patch and monitor on your own.
Google took the opposite framing at Cloud Next '26: its Gemini Enterprise Agent Platform puts a Smart Routing layer in front of more than 200 models, including Anthropic's Claude, and optimizes for the cheapest model that clears a quality bar rather than just catching outages, per coverage of the announcement. AWS's gateway rule only catches failures; it doesn't pick a model by cost. That's a narrower job, and it's wired into a gateway you likely already pay for rather than a platform migration.
Turn it on if you're already inside AgentCore. Don't migrate for it alone
If your agents already call Bedrock through AgentCore Gateway, write the rule. It's a few lines of CLI against infrastructure you're already running, and it deletes the retry-and-switch code most teams hand-roll around Bedrock throttling. If you're multi-cloud behind LiteLLM or Portkey today, this isn't a reason to rip that out: those proxies cover providers outside AWS entirely, and you'd be trading a system you already operate for one capped at two fallback targets that only tells you what happened through a log you have to go fetch. The case for switching shows up when your fallback target is itself inside Bedrock, Mantle or Runtime, since AgentCore's IAM scoping and routing logs already cover it and a side-car proxy would just be duplicating what the gateway now does for the cost of one rule.