For the complete documentation index, see llms.txt. Markdown versions of all docs pages are available by appending .md to any docs URL.

What’s new, changed, and fixed in each agentgateway on Kubernetes release.

Review the release notes for agentgateway on Kubernetes.

✨ Highlights

Version 1.5 focuses on native provider API surfaces, outbound traffic, and tighter cluster permissions.

• Inline URLs for policy backends: Point a policy at an external service by URL, without creating an intermediate Kubernetes object.
• Native Gemini inbound API: Clients that are built on the Gemini and Vertex AI SDKs can call agentgateway in Gemini’s native wire format.
• Anthropic Messages to OpenAI Responses conversion: Send a client that speaks the Anthropic Messages API to a provider that advertises only the Responses format.
• Egress proxying, TCP backends, and CONNECT tunneling: Run agentgateway as an egress proxy for agent workloads.
• Namespace-scoped write permissions: Scope the controller’s write access to the namespaces that hold your gateways.

🔥 Breaking changes

LLM input and total token counts include cache tokens

LLM providers disagree about whether the input token count in a response includes the tokens that the provider read from or wrote to its prompt cache. Anthropic and Amazon Bedrock exclude cached tokens. OpenAI, Azure OpenAI, and Google Gemini include them. Agentgateway used to pass each provider’s number through unchanged, so the same prompt produced a different count depending on which provider served it. Agentgateway now normalizes the counts so that they mean the same thing for every provider.

• llm.inputTokens is the total input count, including cache-read and cache-creation tokens.
• llm.totalTokens is the normalized input count plus the output count.
• llm.providerInputTokens and llm.providerTotalTokens are new fields that report the counts exactly as the provider sent them.

The llm.cachedInputTokens and llm.cacheCreationInputTokens fields do not change, and both are now always a subset of llm.inputTokens. Only the providers that previously excluded cached tokens report different values. Those providers are Anthropic, Amazon Bedrock, Anthropic models served through Vertex AI or GitHub Copilot, and custom providers that use the messages or anthropicTokenCount format. Cost tracking does not change, because the model cost catalog already priced cached tokens separately.

Actions to take: The normalized counts reach every feature that reads a token count, including access logs, spans, metrics, token-based rate limits, and CEL expressions. To read the provider’s unmodified value instead, use llm.providerInputTokens or llm.providerTotalTokens. Review each token-based rate limit that you sized against a provider that excluded cached tokens, because requests now consume the limit sooner. Annotate the upgrade in any dashboard that trends input tokens.

To restore the previous behavior while you migrate, set the AGENTGATEWAY_LEGACY_LLM_USAGE_TOKEN_SEMANTICS environment variable to true in spec.env on an AgentgatewayParameters resource. Agentgateway plans to remove this variable after version 1.5, so treat it as a short-term migration aid.

For each token field and when to read it, see Token usage fields.

JWT validation requires the iss claim, and aud when you configure audiences

Agentgateway used to compare a JWT policy’s issuer and audiences settings against the token’s iss and aud claims only when the token contained them. A token that omitted a claim therefore passed validation. Agentgateway now requires the claim whenever the corresponding setting exists.

The requiredClaims field is unchanged and still defaults to ["exp"]. The iss and aud requirements are added on top of whatever you list there, so an empty requiredClaims list no longer means that no claims are required.

Actions to take: Confirm that your identity provider issues an iss claim in the tokens that reach agentgateway. Most providers do. If you accept tokens that have no aud claim, remove audiences from the policy or set it to an empty list, because a non-empty list now rejects those tokens. For the policy fields, see JWT authentication.

Cross-namespace route delegation requires a ReferenceGrant

In route delegation, a parent route attaches to a child HTTPRoute that has no parentRefs. When the parent and the child are in different namespaces, the attachment previously needed no authorization from the child. Agentgateway now requires a ReferenceGrant in the child’s namespace that allows HTTPRoute references from the parent’s namespace, matching how Gateway API governs every other cross-namespace reference.

Actions to take: For each cross-namespace delegation, create a ReferenceGrant in the child route’s namespace, as in the following example. Delegation within a single namespace is unaffected.

apiVersion: gateway.networking.k8s.io/v1beta1
kind: ReferenceGrant
metadata:
  name: allow-parent-delegation
  namespace: team-a
spec:
  from:
  - group: gateway.networking.k8s.io
    kind: HTTPRoute
    namespace: infra
  to:
  - group: gateway.networking.k8s.io
    kind: HTTPRoute

For more information, see Route delegation.

AgentgatewayModel model routers are scoped to a route

Agentgateway used to create a model router implicitly for every listener that an AgentgatewayModel bound to, which meant that every model with that listener as a parentRef shared one implicit route. Model router creation and route registration are now explicit.

A default model router and route are still created per listener when an AgentgatewayModel has a Gateway or ListenerSet as its parentRef, so a configuration that uses only listener parents keeps working. To run more than one model router on the same listener, for example on different paths, create an HTTPRoute for each router and point the models at it. Such a route must meet all of the following requirements.

• The kind must be HTTPRoute.
• The route must have at least one rule with a path prefix match. If it has more than one rule, the model’s parentRef must name the rule with a sectionName.
• The route must have exactly one backendRef, with no port, kind: AgentgatewayModel, and name: "*".
• That backendRef must have a weight of 1 and no filters.

Actions to take: Review any AgentgatewayModel whose parentRef names an HTTPRoute, and confirm that the route meets the requirements. A route that does not is reported in the model’s status. For more information, see Models.

AI policies on a backend merge with an attached policy

An AI policy set directly on a backend used to replace an attached AI policy in full. If the backend set even one field, every field of the attached policy was dropped, including prompt guards, prompt enrichment, defaults, transformations, model aliases, and prompt caching. The two policies now merge field by field, and the backend’s value wins for a field that both of them set.

Actions to take: Review each AgentgatewayBackend that sets spec.ai.groups[].providers[].policies.ai alongside an AgentgatewayPolicy that sets spec.backend.ai. A field that the AgentgatewayPolicy sets, and the backend does not, now takes effect where it was previously ignored. Remove any field from the AgentgatewayPolicy that you do not want the backend to inherit.

⚠️ Removed or deprecated

The MODEL_CATALOG_PATHS environment variable is removed

Agentgateway no longer reads the MODEL_CATALOG_PATHS environment variable, because it could not be reconciled with dynamic configuration reloading. Most deployments are unaffected. When you supply a model cost catalog through the AgentgatewayParameters resource, the controller now writes the mounted ConfigMap path into the generated configuration, and the resource keeps working unchanged.

Actions to take: If you set MODEL_CATALOG_PATHS yourself, in spec.env on an AgentgatewayParameters resource or in your Helm values, move the catalog to the AgentgatewayParameters model catalog field. Agentgateway silently ignores the environment variable, so a catalog that you load this way stops being applied after you upgrade. For the field, see the API reference.

The Istio identity TLV is no longer sent over HBONE

Agentgateway no longer sends the Istio-specific identity type-length-value (TLV) field on HBONE connections. The TLV existed so that agentgateway could be sandwiched with ztunnel and pass its peer identity along. That sandwich pattern is no longer recommended. Let agentgateway terminate mTLS directly instead.

Actions to take: If you sandwich agentgateway with ztunnel and rely on the forwarded identity in an authorization policy, move that policy to agentgateway. Agentgateway sees the peer identity through the source.tls.identity and source.spiffeId CEL attributes. A sandwich deployment still works, but without native identity propagation. For the recommended patterns, see Istio ambient mesh.

The agctl costs command is renamed to agctl catalog

The agctl command that manages model catalogs is renamed from agctl costs to agctl catalog, because a catalog entry now carries more than pricing data. The subcommand and its flags do not change, and the command produces the same catalog JSON. The agctl costs command still runs the same code, but it is deprecated and reports that you must use agctl catalog instead. Agentgateway plans to remove agctl costs in a future release.

Actions to take: Replace agctl costs with agctl catalog in any script or pipeline that generates a model catalog. For the flags and examples, see the agctl catalog import reference.

🌟 New features

LLM

Native Gemini inbound API

Clients that are built on the Gemini or Vertex AI SDKs can now call agentgateway in Gemini’s native wire format, rather than through an OpenAI-compatible endpoint. Two route types are added to the AI policy routes map, and by default agentgateway maps paths to them automatically.

The model comes from the models/{model} path segment, so any gemini-* model works without per-model configuration. Streaming requires alt=sse, because Gemini’s default JSON-array streaming mode is not supported. Guardrails apply to GenerateContent, and are skipped for GeminiCountTokens. A Gemini request to a non-Gemini provider returns an explicit unsupported-conversion error.

For the route types, see the API reference.

Anthropic Messages to OpenAI Responses conversion

Agentgateway can now translate an Anthropic Messages request into an OpenAI Responses request, and translate the buffered or streamed reply back into the Messages format. Use it when a client sends /v1/messages but the provider that you route to advertises only the Responses format. The existing Messages-to-Completions path still takes precedence for providers that advertise both formats, so dual-format OpenAI and Azure OpenAI providers are unchanged.

For the supported providers, see the LLM providers docs.

OpenAI inline moderation

An OpenAI provider can now carry a moderation block that agentgateway injects into chat completions and Responses requests, so that OpenAI moderates the request inline rather than in a separate call. The gateway sets the configuration, which means a client cannot turn moderation off or weaken it. You choose a moderation model and set block or score mode independently for input and output.

provider:
  openAI:
    model: gpt-5
    moderation:
      model: omni-moderation-latest
      policy:
        input:
          mode: block
        output:
          mode: score

This is separate from the existing OpenAI moderation guardrail, which calls the Moderation API from the gateway. For that approach, see OpenAI moderation.

Guardrails can scan tool input and output

A prompt guard now takes an optional scope list that selects which parts of an LLM request it inspects. The accepted values are SystemPrompt, Messages, ToolOutput, and ToolInput. Tool content is opt in, so a guard without a scope behaves as it did before. Scoping is currently supported by the regex guard.

For the field, the caveats on masking opaque tool arguments, and examples, see Regex guardrails.

Transformations after provider conversion

Transformations run before agentgateway converts a request into the provider’s format, so writing one meant understanding how that conversion works, and fields that the conversion adds could not be changed at all. A new finalTransformations field on AgentgatewayModel and AgentgatewayPolicy runs after the conversion instead. You only need to know the shape of the target API.

For more information, see Transformations.

Other LLM improvements

• Catalog tags: A catalog entry carries a freeform tags list alongside its pricing rates and tiers, which is why agctl costs became agctl catalog. Use tags to record capability or routing information about a model. The initial catalog refresh at startup is also fixed.
• Provider override for custom providers: A custom provider takes an optional providerOverride that sets the provider identity used for cost catalog lookup and telemetry. Without it, the existing custom fallback is used. The field is available on both AgentgatewayModel and AgentgatewayBackend.
• Bedrock: Amazon Nova multimodal embeddings and Cohere v4 embeddings are supported. This release also corrects top_k translation, handles image URLs consistently across input types, and mutates a guardrail payload in place so that the original structure is preserved.
• GitHub Copilot and DeepSeek: Grok models are routed through the Responses API, and the DeepSeek preset advertises the Responses format.
• Prompt caching across formats: OpenAI cache markers are translated into their Anthropic and Bedrock equivalents.
• Vertex AI embeddings: gemini-embedding-2 and later models are routed to the :embedContent endpoint, because Google no longer serves :predict for them. The gemini-embedding-001 and text-embedding-* models stay on :predict. Because :embedContent embeds one input per call, a multi-input array returns an explicit error instead of collapsing into a single vector.
• Token counting: The count-tokens endpoint is routed by default, and an Anthropic thinking budget is capped by the request’s maximum token count.
• Failover authorization: An AgentgatewayModel configures authorization correctly for its failover targets.
• Guardrail refactor: Guardrails are restructured internally, and prompt guard logs record which pattern matched.
• Error handling: Proxy errors are classified by the phase they occurred in, and the original upstream HTTP status code is preserved on an error response.

For the list of supported providers, see the LLM providers docs, and for cost tracking, see Cost tracking.

Security

Namespace-scoped write permissions for the controller

The controller Helm chart can now grant its write permissions in named namespaces instead of cluster-wide. Set rbac.gatewayNamespaces to the list of namespaces that hold your Gateway resources. The chart then creates namespaced roles for the objects that the controller provisions, and the cluster-wide role keeps only read access to them.

rbac:
  gatewayNamespaces:
  - gateway-system
  - team-a

The default is an empty list, which preserves the existing cluster-wide write access, so an upgrade does not change permissions on its own. Cluster-wide read permissions and writes to cluster-scoped resources, such as GatewayClass and status subresources, are unaffected. When you set the list, the namespaces must already exist, and only Gateway resources in those namespaces can be used.

For the provisioned objects and the chart values, see the Helm reference.

Signed JWT backend authentication

A new jwtSign backend authentication method signs a JSON Web Token per request with a private key that you supply, and attaches it to the backend request. Use it for upstreams that require a keypair-signed JWT rather than a static credential, such as the Snowflake SQL API. You reference the signing key through a Secret, and claim values accept CEL expressions, so you can derive a claim from the incoming request.

For the key ID, token lifetime, claims, placement, and supported algorithms, see Signed JWT backend authentication.

Secret references for backend CA certificates

The backend TLS configuration in an AgentgatewayPolicy can now read a CA bundle from a Kubernetes Secret as well as from a ConfigMap. Set kind: Secret on a caCertificateRefs entry, or set kind: ConfigMap explicitly. ConfigMap remains the default, so existing policies are unchanged. The controller watches the referenced Secret, so a CA rotation reaches dependent resources, and it does not fall back between a Secret and a ConfigMap that share a name.

Gateway API BackendTLSPolicy still accepts only ConfigMap references, because its upstream API constrains it. For more information, see Backend TLS.

Cross App Access and token exchange enhancements

• Separate scopes per leg: Cross App Access takes a new accessTokenScopes field, which sets the scopes for the access-token exchange independently of the scopes that request the OAuth Identity Assertion Authorization Grant (ID-JAG). Omit the field to inherit scopes. Set an empty list to omit the scope parameter entirely, which some authorization servers require, such as an Okta custom authorization server.
• Configurable subject token type: Cross App Access takes subjectToken.tokenType, so a workload identity that authenticates with client credentials can exchange an access token. The default is still id_token.
• Optional requested_token_type: The parameter is optional in OAuth token exchange, which matches RFC 8693.

For more information, see Cross App Access and OAuth token exchange.

Other security improvements

• SPIFFE Workload API identity: Add a spiffe block to the AgentgatewayParameters resource to source the mTLS identity and trust bundle from a local SPIFFE Workload API, such as a SPIRE agent. The identity then replaces a static certificate and key. The controller mounts the Workload API socket into the gateway pod, and agentgateway rotates the X.509 SPIFFE Verifiable Identity Document (SVID) without a restart. The socket comes from the SPIFFE Container Storage Interface (CSI) driver by default, and a hostPath source is also available. Prefer the CSI source, because hostPath mounts an arbitrary host directory into the gateway pod. Listeners and backends then opt in individually with the agentgateway.dev/tls-certificate-source: SPIFFE listener option and backend.tls.certificateSource: SPIFFE on an AgentgatewayPolicy, and the peer identity is available to policies as the source.spiffeId CEL attribute. For the fields, see the API reference and the CEL reference.
• Client endpoint headers are stripped for inference routing: The x-gateway-destination-endpoint header is an output of the endpoint picker, not an input that a client sets. Agentgateway now removes it from an incoming request before inference routing runs. No action is needed, because the header was already overwritten in most paths.

MCP and A2A

• Authorization server metadata: Agentgateway rewrites the issuer in the metadata that it serves, so a client that validates the issuer against the gateway address succeeds.
• Discovery failures are visible: A discovery failure is reported rather than masked when the backend is in failOpen mode.
• Server-initiated requests: A client’s JSON-RPC response to a server-initiated request is routed back to the server that asked.
• More targets per backend: An MCP backend accepts up to 128 targets, raised from 32.
• Trace context: An MCP call’s upstream trace context is derived from the gateway’s active span, and the rmcp library is updated to 3.1.0.

For more information, see the MCP docs.

Traffic management

Inline URLs for policy backends

A policy field that points at an external service now accepts a url as an alternative to a backendRef. Those fields include a JWKS endpoint, an OTLP collector, an external authorization or external processing server, a remote rate limit service, and a tunnel proxy. You no longer have to create an intermediate Kubernetes object just to describe an HTTPS endpoint.

• Use backendRef when you want Kubernetes service discovery, namespace scoping, a reusable backend, or backend policies attached to it.
• Use url when the target is naturally a direct HTTP or HTTPS endpoint. An HTTPS URL produces an inline backend TLS policy automatically, and the URL path is preserved where it is meaningful, such as for JWKS and OTLP. A tunnel URL is validated as origin-only, because a tunnel proxy is not an HTTP resource path.

For the fields, see the API reference.

Egress proxying, TCP backends, and CONNECT tunneling

This release fills in the pieces that agentgateway needs to serve as an egress proxy for agent workloads.

• Dynamic backends for TCP: A TCP route can use a dynamic backend, so the destination comes from the connection rather than from static configuration. The controller now translates TCP backends, which the proxy already supported.
• Tunnel mode: The backend tunnel policy takes a mode field. The default auto mode uses CONNECT for TLS and non-HTTP transports, and absolute-form requests for plaintext HTTP. The connect mode uses CONNECT for everything. You can also attach policies to the connection with the tunnel proxy itself, and CONNECT requests can be tunneled through a dynamic proxy backend.
• Forward proxy authentication: A client can authenticate with the Proxy-Authorization header instead of Authorization. Set the authentication policy’s location to that header. Agentgateway strips the header before the request goes upstream and marks it sensitive so that its value is not logged. A failed CONNECT authentication returns a 407 response with a Proxy-Authenticate header, as RFC 9110 requires.
• Backend connection timeouts: The controller now translates the backend.tcp section of a policy, so connectTimeout and the keepalive settings take effect on connections to a destination. A policy that set them in an earlier version was accepted but had no effect. The backend.http.requestTimeout field sets the deadline for a response.

For the tunnel proxy, see Backend tunnel proxy, and for the timeout fields, see Connection settings.

Rate limiting enhancements

• Multiple local limits: An AgentgatewayPolicy can define more than one local rate limit, which standalone mode already supported.
• Dynamic limit overrides: A remote rate limit descriptor takes an optional limitOverride, validated as CEL and forwarded to the rate limit service, so a limit can be computed per request.
• Consistent headers: The x-ratelimit-limit, x-ratelimit-remaining, and x-ratelimit-reset headers are returned on every rate-limited response, for both local and remote rate limiting, rather than only on some paths.

For more information, see HTTP rate limits and Global rate limits.

CEL enhancements

• Every request policy is registered for CEL: All request policies are available in the CEL context, not just a subset.
• Cost-class routing: A worked example derives a cost class from the request body with plain CEL and routes the same public model name to different upstream models.
• Internal improvements: A CEL error can be serialized to a string without leaking potentially private detail. Expression analysis can inspect call arity and function-versus-method usage. A parser bug is fixed, and has() takes a fast path for dynamic objects.

For the full CEL surface, see the CEL reference.

Operations

Observability enhancements

• Admin UI: A redesigned logs view, clearer multi-turn conversation rendering, and a trajectory view for agent activity with tool call and result details.
• Spans for every policy call: Tracing emits an outbound span for the upstream call and for each policy callout, such as external authorization or a guardrail webhook. MCP and gRPC spans are named with protocol-specific information, and request tracing with agctl follows the same outbound calls.
• Protobuf metrics: The /metrics and /stats/prometheus endpoints negotiate the response format from the Accept header, so a scraper that asks for application/vnd.google.protobuf gets protobuf instead of text.
• Native histograms: The proxy can collect classic histogram buckets, native buckets, or both. Native histograms are exposed only through the Prometheus protobuf format, and classic remains the default because native histograms add scrape overhead.
• LLM token timing in access logs: Access logs record time-to-first-token and related timing for LLM requests.
• CPU and heap profiles: A new agctl proxy profile command collects pprof CPU and heap profiles from the proxy admin endpoint.
• Generated metrics reference: The metrics documentation is generated from the schema, so it stays in step with the code.

For more information, see Observability and the agctl proxy profile reference.

Other operations improvements

• Custom xDS request headers: Set XDS_HEADER_* environment variables on the proxy to attach operator-defined headers, such as x-istio-revision or a tenant identifier, to outbound xDS requests. The headers are validated at startup.
• ListenerSet postrouting policies: The proxy applies postrouting policies that a ListenerSet attaches, which the controller already translated.
• Controller chart values: The controller Helm chart adds controller.revisionHistoryLimit and dnsConfig. For the chart values, see the Helm reference.
• Pluggable cryptography: A crypto module centralizes random number generation, authenticated encryption with associated data (AEAD), digest, JWT, and TLS provider selection behind crypto-* build flags. A SymCrypt provider is available for builds that need it. AWS-LC remains the default.
• Shared duration type: Duration fields across the CRDs use one shared type, which simplifies their validation rules.

🐛 Fixes

Security

• Valid JWKS targets are restricted, an invalid inline JWKS reports an error, and an invalid JWT produces a clearer message.
• Azure managed identity rejects multiple identity selectors and aligns its schema naming.

MCP and A2A

• A2A path rewriting is fixed, interface URL rewriting is correct when a path rewrite policy is active, and A2A v1.0 nested payloads record response telemetry and the context ID.

LLM

• Bedrock virtual models no longer bypass the transformed model on the upstream path, and Bedrock streaming indexes, invalid function inputs, and image URL handling are fixed.
• Gemini usage is extracted from the Cloud Code response envelope, and parallel tool calls are preserved across the Gemini and Completions conversions.
• Anthropic streaming sets the role on the first delta, honors the final input usage, and no longer fails the whole request when a server tool errors.
• A multi-turn request whose previous turn returned empty tool arguments no longer fails validation.
• An InferenceRouting policy resolves for AI provider backends.

Traffic management

• Listener port swaps are reconciled dynamically, and a bind that transitions to an internal bind is stopped.
• Invalid header modifications are rejected, route policy application is more consistent, and a :authority mutation is no longer a no-op for CONNECT requests.

Status and resource reporting

• AgentgatewayModel reports status, and the controller writes InferencePool status for pools that a model references. Redundant InferencePool status writes are suppressed.
• A failure to write a deployed object is reported in status rather than only logged, and a policy that targets a missing sectionName is surfaced in status.
• Conflicts on an internal Gateway or ListenerSet are handled correctly.
• AgentgatewayModel appears in the configuration dump, so agctl can inspect it.

Operations

• A negative duration is clamped to zero instead of being rejected, and upstream connect duration is recorded at full precision.
• The admin UI analytics summary no longer loops its request.
• The controller validates with CEL that a port is a number.

For the complete list of fixes, see the GitHub release notes.