2026/10/22 - Amazon Bedrock AgentCore Control - 7 updated api methods
Changes Adds the dimension name to evaluation results and batch evaluation summaries. For multi-dimensional LLM-as-a-Judge evaluators, each result and summary now identifies which dimension it corresponds to.
{'evaluatorConfig': {'llmAsAJudge': {'ratingScale': {'multiDimensional': [{'dimensionInstruction': 'string',
'dimensionName': 'string',
'dimensionRatingScale': {'categorical': [{'definition': 'string',
'label': 'string'}],
'numerical': [{'definition': 'string',
'label': 'string',
'value': 'double'}]}}]}}}}
Creates a custom evaluator for agent quality assessment. Custom evaluators can use either LLM-as-a-Judge configurations with user-defined prompts, rating scales, and model settings, or code-based configurations with customer-managed Lambda functions to evaluate agent performance at tool call, trace, or session levels.
See also: AWS API Documentation
Request Syntax
client.create_evaluator(
clientToken='string',
evaluatorName='string',
description='string',
evaluatorConfig={
'llmAsAJudge': {
'instructions': 'string',
'ratingScale': {
'numerical': [
{
'definition': 'string',
'value': 123.0,
'label': 'string'
},
],
'categorical': [
{
'definition': 'string',
'label': 'string'
},
],
'multiDimensional': [
{
'dimensionName': 'string',
'dimensionInstruction': 'string',
'dimensionRatingScale': {
'numerical': [
{
'definition': 'string',
'value': 123.0,
'label': 'string'
},
],
'categorical': [
{
'definition': 'string',
'label': 'string'
},
]
}
},
]
},
'modelConfig': {
'bedrockEvaluatorModelConfig': {
'modelId': 'string',
'inferenceConfig': {
'maxTokens': 123,
'temperature': ...,
'topP': ...,
'stopSequences': [
'string',
]
},
'additionalModelRequestFields': {...}|[...]|123|123.4|'string'|True|None
},
'responsesEvaluatorModelConfig': {
'modelId': 'string',
'maxOutputTokens': 123,
'temperature': ...,
'topP': ...,
'reasoning': {
'effort': 'string'
}
}
}
},
'codeBased': {
'lambdaConfig': {
'lambdaArn': 'string',
'lambdaTimeoutInSeconds': 123
}
},
'derived': {
'baseEvaluatorId': 'string',
'modelConfig': {
'bedrockEvaluatorModelConfig': {
'modelId': 'string',
'inferenceConfig': {
'maxTokens': 123,
'temperature': ...,
'topP': ...,
'stopSequences': [
'string',
]
},
'additionalModelRequestFields': {...}|[...]|123|123.4|'string'|True|None
},
'responsesEvaluatorModelConfig': {
'modelId': 'string',
'maxOutputTokens': 123,
'temperature': ...,
'topP': ...,
'reasoning': {
'effort': 'string'
}
}
}
}
},
level='TOOL_CALL'|'TRACE'|'SESSION',
kmsKeyArn='string',
tags={
'string': 'string'
}
)
string
A unique, case-sensitive identifier to ensure that the API request completes no more than one time. If you don't specify this field, a value is randomly generated for you. If this token matches a previous request, the service ignores the request, but doesn't return an error. For more information, see Ensuring idempotency.
This field is autopopulated if not provided.
string
[REQUIRED]
The name of the evaluator. Must be unique within your account.
string
The description of the evaluator that explains its purpose and evaluation criteria.
dict
[REQUIRED]
The configuration for the evaluator. Specify either LLM-as-a-Judge settings with instructions, rating scale, and model configuration, or code-based settings with a customer-managed Lambda function.
llmAsAJudge (dict) --
The LLM-as-a-Judge configuration that uses a language model to evaluate agent performance based on custom instructions and rating scales.
instructions (string) -- [REQUIRED]
The evaluation instructions that guide the language model in assessing agent performance, including criteria and evaluation guidelines.
ratingScale (dict) -- [REQUIRED]
The rating scale that defines how the evaluator should score agent performance, either numerical or categorical.
numerical (list) --
The numerical rating scale with defined score values and descriptions for quantitative evaluation.
(dict) --
The definition of a numerical rating scale option that provides a numeric value with its description for evaluation scoring.
definition (string) -- [REQUIRED]
The description that explains what this numerical rating represents and when it should be used.
value (float) -- [REQUIRED]
The numerical value for this rating scale option.
label (string) -- [REQUIRED]
The label or name that describes this numerical rating option.
categorical (list) --
The categorical rating scale with named categories and definitions for qualitative evaluation.
(dict) --
The definition of a categorical rating scale option that provides a named category with its description for evaluation scoring.
definition (string) -- [REQUIRED]
The description that explains what this categorical rating represents and when it should be used.
label (string) -- [REQUIRED]
The label or name of this categorical rating option.
multiDimensional (list) --
A list of dimensions that lets an LLM-as-a-judge evaluator score several dimensions of the same target in a single evaluation. Each dimension has its own instruction and rating scale, and produces its own result.
(dict) --
A single dimension of an LLM-as-a-judge evaluator, defining what the evaluator model should judge for that dimension and how it should be scored.
dimensionName (string) -- [REQUIRED]
The name of the dimension. This name is unique within the evaluator and identifies the dimension's results.
dimensionInstruction (string) -- [REQUIRED]
The instruction to the evaluator model for this dimension, describing what the model should judge when scoring the dimension.
dimensionRatingScale (dict) -- [REQUIRED]
The rating scale for this dimension, which can be numerical or categorical. A dimension's rating scale cannot itself be multi-dimensional.
numerical (list) --
The numerical rating scale with defined score values and descriptions for quantitative evaluation of the dimension.
(dict) --
The definition of a numerical rating scale option that provides a numeric value with its description for evaluation scoring.
definition (string) -- [REQUIRED]
The description that explains what this numerical rating represents and when it should be used.
value (float) -- [REQUIRED]
The numerical value for this rating scale option.
label (string) -- [REQUIRED]
The label or name that describes this numerical rating option.
categorical (list) --
The categorical rating scale with named categories and definitions for qualitative evaluation of the dimension.
(dict) --
The definition of a categorical rating scale option that provides a named category with its description for evaluation scoring.
definition (string) -- [REQUIRED]
The description that explains what this categorical rating represents and when it should be used.
label (string) -- [REQUIRED]
The label or name of this categorical rating option.
modelConfig (dict) -- [REQUIRED]
The model configuration that specifies which foundation model to use and how to configure it for evaluation.
bedrockEvaluatorModelConfig (dict) --
The Amazon Bedrock model configuration for evaluation.
modelId (string) -- [REQUIRED]
The identifier of the Amazon Bedrock model to use for evaluation. Must be a supported foundation model available in your region.
inferenceConfig (dict) --
The inference configuration parameters that control model behavior during evaluation, including temperature, token limits, and sampling settings.
maxTokens (integer) --
The maximum number of tokens to generate in the model response during evaluation.
temperature (float) --
The temperature value that controls randomness in the model's responses. Lower values produce more deterministic outputs.
topP (float) --
The top-p sampling parameter that controls the diversity of the model's responses by limiting the cumulative probability of token choices.
stopSequences (list) --
The list of sequences that will cause the model to stop generating tokens when encountered.
(string) --
additionalModelRequestFields (:ref:`document<document>`) --
Additional model-specific request fields to customize model behavior beyond the standard inference configuration.
responsesEvaluatorModelConfig (dict) --
The OpenResponses model configuration for evaluation.
modelId (string) -- [REQUIRED]
The identifier of the model to use for evaluation.
maxOutputTokens (integer) --
The maximum number of tokens to generate in the model response, including visible output and reasoning tokens.
temperature (float) --
The temperature value that controls randomness in the model's responses. Lower values produce more deterministic outputs.
topP (float) --
The top-p sampling parameter that controls the diversity of the model's responses by limiting the cumulative probability of token choices.
reasoning (dict) --
The reasoning configuration for reasoning models. Non-reasoning models ignore this configuration.
effort (string) --
The level of reasoning effort the model applies when generating a response. For supported values, see the model provider's documentation.
codeBased (dict) --
Configuration for a code-based evaluator that uses a customer-managed Lambda function to programmatically assess agent performance.
lambdaConfig (dict) --
The Lambda function configuration for code-based evaluation.
lambdaArn (string) -- [REQUIRED]
The Amazon Resource Name (ARN) of the Lambda function that implements the evaluation logic.
lambdaTimeoutInSeconds (integer) --
The timeout in seconds for the Lambda function invocation. Defaults to 60. Must be between 1 and 300.
derived (dict) --
The configuration for an evaluator derived from an existing base evaluator (a built-in or third-party evaluator), run on your own model. The base evaluator supplies the prompt and scoring.
baseEvaluatorId (string) -- [REQUIRED]
The identifier of the base evaluator whose logic to run (a Builtin.* or ThirdParty.* evaluator).
modelConfig (dict) -- [REQUIRED]
The configuration of the evaluator model that you supply.
bedrockEvaluatorModelConfig (dict) --
The Amazon Bedrock model configuration for evaluation.
modelId (string) -- [REQUIRED]
The identifier of the Amazon Bedrock model to use for evaluation. Must be a supported foundation model available in your region.
inferenceConfig (dict) --
The inference configuration parameters that control model behavior during evaluation, including temperature, token limits, and sampling settings.
maxTokens (integer) --
The maximum number of tokens to generate in the model response during evaluation.
temperature (float) --
The temperature value that controls randomness in the model's responses. Lower values produce more deterministic outputs.
topP (float) --
The top-p sampling parameter that controls the diversity of the model's responses by limiting the cumulative probability of token choices.
stopSequences (list) --
The list of sequences that will cause the model to stop generating tokens when encountered.
(string) --
additionalModelRequestFields (:ref:`document<document>`) --
Additional model-specific request fields to customize model behavior beyond the standard inference configuration.
responsesEvaluatorModelConfig (dict) --
The OpenResponses model configuration for evaluation.
modelId (string) -- [REQUIRED]
The identifier of the model to use for evaluation.
maxOutputTokens (integer) --
The maximum number of tokens to generate in the model response, including visible output and reasoning tokens.
temperature (float) --
The temperature value that controls randomness in the model's responses. Lower values produce more deterministic outputs.
topP (float) --
The top-p sampling parameter that controls the diversity of the model's responses by limiting the cumulative probability of token choices.
reasoning (dict) --
The reasoning configuration for reasoning models. Non-reasoning models ignore this configuration.
effort (string) --
The level of reasoning effort the model applies when generating a response. For supported values, see the model provider's documentation.
string
[REQUIRED]
The evaluation level that determines the scope of evaluation. Valid values are TOOL_CALL for individual tool invocations, TRACE for single request-response interactions, or SESSION for entire conversation sessions.
string
The Amazon Resource Name (ARN) of a customer managed KMS key to use for encrypting sensitive evaluator data, including instructions and rating scale. If you don't specify a KMS key, the evaluator data is encrypted with an Amazon Web Services owned key. Only symmetric encryption KMS keys are supported. For more information, see Encryption at rest for AgentCore Evaluations.
dict
A map of tag keys and values to assign to an AgentCore Evaluator. Tags enable you to categorize your resources in different ways, for example, by purpose, owner, or environment.
(string) --
(string) --
dict
Response Syntax
{
'evaluatorArn': 'string',
'evaluatorId': 'string',
'createdAt': datetime(2015, 1, 1),
'status': 'ACTIVE'|'CREATING'|'CREATE_FAILED'|'UPDATING'|'UPDATE_FAILED'|'DELETING'
}
Response Structure
(dict) --
evaluatorArn (string) --
The Amazon Resource Name (ARN) of the created evaluator.
evaluatorId (string) --
The unique identifier of the created evaluator.
createdAt (datetime) --
The timestamp when the evaluator was created.
status (string) --
The status of the evaluator creation operation.
{'actions': {'routeToTarget': {'staticRoute': {'fallback': {'circuitBreaker': {'mode': 'ENABLED '
'| '
'DISABLED'},
'firstByteTimeoutMilliseconds': 'integer',
'targetFallback': [{'firstByteTimeoutMilliseconds': 'integer',
'model': 'string',
'targetName': 'string',
'triggerOn': {'errors': ['CONNECTION_ERROR '
'| '
'FIRST_BYTE_TIMEOUT '
'| '
'UNROUTABLE'],
'statusCodes': ['string']}}],
'triggerOn': {'errors': ['CONNECTION_ERROR '
'| '
'FIRST_BYTE_TIMEOUT '
'| '
'UNROUTABLE'],
'statusCodes': ['string']}},
'model': 'string'},
'weightedRoute': {'trafficSplit': {'fallback': {'circuitBreaker': {'mode': 'ENABLED '
'| '
'DISABLED'},
'firstByteTimeoutMilliseconds': 'integer',
'targetFallback': [{'firstByteTimeoutMilliseconds': 'integer',
'model': 'string',
'targetName': 'string',
'triggerOn': {'errors': ['CONNECTION_ERROR '
'| '
'FIRST_BYTE_TIMEOUT '
'| '
'UNROUTABLE'],
'statusCodes': ['string']}}],
'triggerOn': {'errors': ['CONNECTION_ERROR '
'| '
'FIRST_BYTE_TIMEOUT '
'| '
'UNROUTABLE'],
'statusCodes': ['string']}},
'model': 'string'}}}},
'conditions': {'matchModels': {'anyOf': [{'operator': 'StringEquals | '
'StringLike',
'value': 'string'}]}}}
Creates a rule for a gateway. Rules define conditions and actions that control how requests are routed and processed through the gateway, including principal-based access control and path-based routing.
See also: AWS API Documentation
Request Syntax
client.create_gateway_rule(
gatewayIdentifier='string',
clientToken='string',
priority=123,
conditions=[
{
'matchPrincipals': {
'anyOf': [
{
'iamPrincipal': {
'arn': 'string',
'operator': 'StringEquals'|'StringLike'
}
},
]
},
'matchPaths': {
'anyOf': [
'string',
]
},
'matchModels': {
'anyOf': [
{
'value': 'string',
'operator': 'StringEquals'|'StringLike'
},
]
}
},
],
actions=[
{
'configurationBundle': {
'staticOverride': {
'bundleArn': 'string',
'bundleVersion': 'string'
},
'weightedOverride': {
'trafficSplit': [
{
'name': 'string',
'weight': 123,
'configurationBundle': {
'bundleArn': 'string',
'bundleVersion': 'string'
},
'description': 'string',
'metadata': {
'string': 'string'
}
},
]
}
},
'routeToTarget': {
'staticRoute': {
'targetName': 'string',
'model': 'string',
'fallback': {
'triggerOn': {
'statusCodes': [
'string',
],
'errors': [
'CONNECTION_ERROR'|'FIRST_BYTE_TIMEOUT'|'UNROUTABLE',
]
},
'firstByteTimeoutMilliseconds': 123,
'circuitBreaker': {
'mode': 'ENABLED'|'DISABLED'
},
'targetFallback': [
{
'targetName': 'string',
'model': 'string',
'triggerOn': {
'statusCodes': [
'string',
],
'errors': [
'CONNECTION_ERROR'|'FIRST_BYTE_TIMEOUT'|'UNROUTABLE',
]
},
'firstByteTimeoutMilliseconds': 123
},
]
}
},
'weightedRoute': {
'trafficSplit': [
{
'targetName': 'string',
'model': 'string',
'fallback': {
'triggerOn': {
'statusCodes': [
'string',
],
'errors': [
'CONNECTION_ERROR'|'FIRST_BYTE_TIMEOUT'|'UNROUTABLE',
]
},
'firstByteTimeoutMilliseconds': 123,
'circuitBreaker': {
'mode': 'ENABLED'|'DISABLED'
},
'targetFallback': [
{
'targetName': 'string',
'model': 'string',
'triggerOn': {
'statusCodes': [
'string',
],
'errors': [
'CONNECTION_ERROR'|'FIRST_BYTE_TIMEOUT'|'UNROUTABLE',
]
},
'firstByteTimeoutMilliseconds': 123
},
]
},
'name': 'string',
'weight': 123,
'description': 'string',
'metadata': {
'string': 'string'
}
},
]
}
}
},
],
description='string'
)
string
[REQUIRED]
The identifier of the gateway to create a rule for.
string
A unique, case-sensitive identifier to ensure that the API request completes no more than one time. If you don't specify this field, a value is randomly generated for you. If this token matches a previous request, the service ignores the request, but doesn't return an error. For more information, see Ensuring idempotency.
This field is autopopulated if not provided.
integer
[REQUIRED]
The priority of the rule. Rules are evaluated in order of priority, with lower numbers evaluated first. Must be between 1 and 1,000,000.
list
The conditions that must be met for the rule to apply. Conditions can match on principals (IAM ARNs) or request paths.
(dict) --
A condition that determines when a gateway rule applies. Conditions can match on principals, request paths, or the requested model.
matchPrincipals (dict) --
A condition that matches on the identity of the caller making the request.
anyOf (list) -- [REQUIRED]
A list of principal entries. The condition is met if any of the entries match the caller's identity.
(dict) --
Union for principal matching. Currently supports IAM principal ARN glob matching.
iamPrincipal (dict) --
An IAM principal to match against, specified by ARN.
arn (string) -- [REQUIRED]
The Amazon Resource Name (ARN) of the IAM principal. Supports user, role, and assumed-role ARNs. Wildcards can be used with the StringLike operator.
operator (string) --
The match operator. StringEquals requires an exact match. StringLike supports wildcard patterns using * and ?.
matchPaths (dict) --
A condition that matches on the request path.
anyOf (list) -- [REQUIRED]
A list of path patterns. The condition is met if the request path matches any of the patterns.
(string) --
matchModels (dict) --
A condition that matches on the model requested in an inference request.
anyOf (list) -- [REQUIRED]
A list of model entries. The condition is met if the requested model matches any of the entries.
(dict) --
A model ID or pattern to match against the requested model, and the operator to use for the comparison.
value (string) -- [REQUIRED]
The model ID or pattern to match. With the StringLike operator, you can use * and ? as wildcards, for example anthropic.claude-*.
operator (string) --
The operator to use when comparing the requested model to the value:
StringEquals – Requires an exact match.
StringLike – Supports wildcard patterns using * and ?. This is the default.
list
[REQUIRED]
The actions to take when the rule conditions are met. Actions can route to a specific target or apply a configuration bundle override.
(dict) --
An action to take when a gateway rule's conditions are met.
configurationBundle (dict) --
An action that applies a configuration bundle override to the request.
staticOverride (dict) --
A static configuration bundle override that applies a single bundle version to all matching requests.
bundleArn (string) -- [REQUIRED]
The Amazon Resource Name (ARN) of the configuration bundle to apply.
bundleVersion (string) -- [REQUIRED]
The version of the configuration bundle to apply.
weightedOverride (dict) --
A weighted configuration bundle override that splits traffic between multiple bundle versions based on configured weights.
trafficSplit (list) -- [REQUIRED]
The traffic split entries defining how traffic is distributed between configuration bundle versions.
(dict) --
An entry in a traffic split configuration, defining a named variant with a weight and configuration bundle reference.
name (string) -- [REQUIRED]
The name of this traffic split variant.
weight (integer) -- [REQUIRED]
The percentage of traffic to route to this variant. Weights across all entries must sum to 100.
configurationBundle (dict) -- [REQUIRED]
The configuration bundle reference for this variant.
bundleArn (string) -- [REQUIRED]
The Amazon Resource Name (ARN) of the configuration bundle.
bundleVersion (string) -- [REQUIRED]
The version of the configuration bundle.
description (string) --
The description of this traffic split variant.
metadata (dict) --
Key-value metadata associated with this traffic split variant.
(string) --
(string) --
routeToTarget (dict) --
An action that routes the request to a specific target.
staticRoute (dict) --
A static route that sends all matching requests to a single target.
targetName (string) -- [REQUIRED]
The name of the target to route requests to.
model (string) --
The model ID to send to the target. Supported only for inference targets. If you don't specify a model, the gateway sends the model from the original request.
fallback (dict) --
The fallback configuration for this route. Supported only for inference targets.
triggerOn (dict) --
The status codes and errors from the route's target that cause the gateway to send the request to the first fallback target. If you omit triggerOn, the gateway falls back on 5XX status codes and on the CONNECTION_ERROR, FIRST_BYTE_TIMEOUT, and UNROUTABLE errors. If you specify triggerOn, only the status codes and errors that you list trigger a fallback.
statusCodes (list) --
The HTTP status codes returned by the target that trigger a fallback. You can specify the following:
An exact status code – A three-digit code, such as 429.
A class of status codes – The first digit followed by XX, such as 5XX, which matches every code from 500 to 599.
Only 4XX and 5XX codes are supported. Throttling by the gateway itself doesn't trigger a fallback.
(string) --
errors (list) --
The errors that trigger a fallback. You can specify the following:
CONNECTION_ERROR – The gateway couldn't connect to the target because the connection was refused, reset, or timed out. The request didn't reach the target.
FIRST_BYTE_TIMEOUT – The target accepted the connection but didn't return the first byte of the response within firstByteTimeoutMilliseconds. The gateway abandons the request before any of the response reaches you.
UNROUTABLE – The gateway couldn't route the request to the target, for example because the target doesn't exist or doesn't support the request path. These errors don't count toward the circuit breaker.
(string) --
firstByteTimeoutMilliseconds (integer) --
The maximum time, in milliseconds, from sending the request to the route's target until the first byte of the response body arrives. If the time expires, the gateway abandons the request and treats it as a FIRST_BYTE_TIMEOUT error. If you don't specify a value, no timeout is enforced.
circuitBreaker (dict) --
Specifies whether the gateway temporarily stops sending your requests to a failing target. The circuit breaker tracks each combination of target and model separately. With the circuit breaker enabled, the gateway skips one or more targets and models that fail repeatedly, and your requests go to the next target in the chain. The gateway always tries the last remaining target in the chain. The gateway enables the circuit breaker by default.
mode (string) --
Specifies whether you want the circuit breaker turned on. To turn it off, set this value to DISABLED. The default is ENABLED.
targetFallback (list) -- [REQUIRED]
The ordered list of fallback targets. The gateway tries each target in order until a response doesn't match that target's triggers, and returns that response to the caller. If every target fails, the gateway returns the last response. You can specify up to two fallback targets. Each combination of target and model, including the route's target, can appear only once.
(dict) --
Specifies a target to send the request to when the previous target in the fallback chain fails. Includes the target name, the model to send, and the conditions that trigger a fallback from this target to the next one.
targetName (string) -- [REQUIRED]
The name of the gateway target to fall back to. The target must be an inference target.
model (string) --
The model ID to send to this fallback target. If you don't specify a model, the gateway sends the model from the original request.
triggerOn (dict) --
The status codes and errors from this target that cause the gateway to send the request to the next fallback target. This setting isn't inherited from the route. If you omit it, the gateway uses the default triggers: 5XX status codes and the CONNECTION_ERROR, FIRST_BYTE_TIMEOUT, and UNROUTABLE errors.
statusCodes (list) --
The HTTP status codes returned by the target that trigger a fallback. You can specify the following:
An exact status code – A three-digit code, such as 429.
A class of status codes – The first digit followed by XX, such as 5XX, which matches every code from 500 to 599.
Only 4XX and 5XX codes are supported. Throttling by the gateway itself doesn't trigger a fallback.
(string) --
errors (list) --
The errors that trigger a fallback. You can specify the following:
CONNECTION_ERROR – The gateway couldn't connect to the target because the connection was refused, reset, or timed out. The request didn't reach the target.
FIRST_BYTE_TIMEOUT – The target accepted the connection but didn't return the first byte of the response within firstByteTimeoutMilliseconds. The gateway abandons the request before any of the response reaches you.
UNROUTABLE – The gateway couldn't route the request to the target, for example because the target doesn't exist or doesn't support the request path. These errors don't count toward the circuit breaker.
(string) --
firstByteTimeoutMilliseconds (integer) --
The maximum time, in milliseconds, from sending the request to this target until the first byte of the response body arrives. This setting isn't inherited from the route. If you don't specify a value, no timeout is enforced.
weightedRoute (dict) --
A weighted route that splits traffic between multiple targets.
trafficSplit (list) -- [REQUIRED]
The traffic split entries defining how traffic is distributed between targets.
(dict) --
An entry in a target traffic split configuration.
targetName (string) -- [REQUIRED]
The name of the target to route traffic to.
model (string) --
The model ID to send to the target for this variant. Supported only for inference targets. If you don't specify a model, the gateway sends the model from the original request.
fallback (dict) --
The fallback configuration for this variant. Supported only for inference targets.
triggerOn (dict) --
The status codes and errors from the route's target that cause the gateway to send the request to the first fallback target. If you omit triggerOn, the gateway falls back on 5XX status codes and on the CONNECTION_ERROR, FIRST_BYTE_TIMEOUT, and UNROUTABLE errors. If you specify triggerOn, only the status codes and errors that you list trigger a fallback.
statusCodes (list) --
The HTTP status codes returned by the target that trigger a fallback. You can specify the following:
An exact status code – A three-digit code, such as 429.
A class of status codes – The first digit followed by XX, such as 5XX, which matches every code from 500 to 599.
Only 4XX and 5XX codes are supported. Throttling by the gateway itself doesn't trigger a fallback.
(string) --
errors (list) --
The errors that trigger a fallback. You can specify the following:
CONNECTION_ERROR – The gateway couldn't connect to the target because the connection was refused, reset, or timed out. The request didn't reach the target.
FIRST_BYTE_TIMEOUT – The target accepted the connection but didn't return the first byte of the response within firstByteTimeoutMilliseconds. The gateway abandons the request before any of the response reaches you.
UNROUTABLE – The gateway couldn't route the request to the target, for example because the target doesn't exist or doesn't support the request path. These errors don't count toward the circuit breaker.
(string) --
firstByteTimeoutMilliseconds (integer) --
The maximum time, in milliseconds, from sending the request to the route's target until the first byte of the response body arrives. If the time expires, the gateway abandons the request and treats it as a FIRST_BYTE_TIMEOUT error. If you don't specify a value, no timeout is enforced.
circuitBreaker (dict) --
Specifies whether the gateway temporarily stops sending your requests to a failing target. The circuit breaker tracks each combination of target and model separately. With the circuit breaker enabled, the gateway skips one or more targets and models that fail repeatedly, and your requests go to the next target in the chain. The gateway always tries the last remaining target in the chain. The gateway enables the circuit breaker by default.
mode (string) --
Specifies whether you want the circuit breaker turned on. To turn it off, set this value to DISABLED. The default is ENABLED.
targetFallback (list) -- [REQUIRED]
The ordered list of fallback targets. The gateway tries each target in order until a response doesn't match that target's triggers, and returns that response to the caller. If every target fails, the gateway returns the last response. You can specify up to two fallback targets. Each combination of target and model, including the route's target, can appear only once.
(dict) --
Specifies a target to send the request to when the previous target in the fallback chain fails. Includes the target name, the model to send, and the conditions that trigger a fallback from this target to the next one.
targetName (string) -- [REQUIRED]
The name of the gateway target to fall back to. The target must be an inference target.
model (string) --
The model ID to send to this fallback target. If you don't specify a model, the gateway sends the model from the original request.
triggerOn (dict) --
The status codes and errors from this target that cause the gateway to send the request to the next fallback target. This setting isn't inherited from the route. If you omit it, the gateway uses the default triggers: 5XX status codes and the CONNECTION_ERROR, FIRST_BYTE_TIMEOUT, and UNROUTABLE errors.
statusCodes (list) --
The HTTP status codes returned by the target that trigger a fallback. You can specify the following:
An exact status code – A three-digit code, such as 429.
A class of status codes – The first digit followed by XX, such as 5XX, which matches every code from 500 to 599.
Only 4XX and 5XX codes are supported. Throttling by the gateway itself doesn't trigger a fallback.
(string) --
errors (list) --
The errors that trigger a fallback. You can specify the following:
CONNECTION_ERROR – The gateway couldn't connect to the target because the connection was refused, reset, or timed out. The request didn't reach the target.
FIRST_BYTE_TIMEOUT – The target accepted the connection but didn't return the first byte of the response within firstByteTimeoutMilliseconds. The gateway abandons the request before any of the response reaches you.
UNROUTABLE – The gateway couldn't route the request to the target, for example because the target doesn't exist or doesn't support the request path. These errors don't count toward the circuit breaker.
(string) --
firstByteTimeoutMilliseconds (integer) --
The maximum time, in milliseconds, from sending the request to this target until the first byte of the response body arrives. This setting isn't inherited from the route. If you don't specify a value, no timeout is enforced.
name (string) -- [REQUIRED]
The name of this traffic split variant.
weight (integer) -- [REQUIRED]
The percentage of traffic to route to this variant.
description (string) --
The description of this traffic split variant.
metadata (dict) --
Key-value metadata associated with this traffic split variant.
(string) --
(string) --
string
The description of the gateway rule.
dict
Response Syntax
{
'ruleId': 'string',
'gatewayArn': 'string',
'priority': 123,
'conditions': [
{
'matchPrincipals': {
'anyOf': [
{
'iamPrincipal': {
'arn': 'string',
'operator': 'StringEquals'|'StringLike'
}
},
]
},
'matchPaths': {
'anyOf': [
'string',
]
},
'matchModels': {
'anyOf': [
{
'value': 'string',
'operator': 'StringEquals'|'StringLike'
},
]
}
},
],
'actions': [
{
'configurationBundle': {
'staticOverride': {
'bundleArn': 'string',
'bundleVersion': 'string'
},
'weightedOverride': {
'trafficSplit': [
{
'name': 'string',
'weight': 123,
'configurationBundle': {
'bundleArn': 'string',
'bundleVersion': 'string'
},
'description': 'string',
'metadata': {
'string': 'string'
}
},
]
}
},
'routeToTarget': {
'staticRoute': {
'targetName': 'string',
'model': 'string',
'fallback': {
'triggerOn': {
'statusCodes': [
'string',
],
'errors': [
'CONNECTION_ERROR'|'FIRST_BYTE_TIMEOUT'|'UNROUTABLE',
]
},
'firstByteTimeoutMilliseconds': 123,
'circuitBreaker': {
'mode': 'ENABLED'|'DISABLED'
},
'targetFallback': [
{
'targetName': 'string',
'model': 'string',
'triggerOn': {
'statusCodes': [
'string',
],
'errors': [
'CONNECTION_ERROR'|'FIRST_BYTE_TIMEOUT'|'UNROUTABLE',
]
},
'firstByteTimeoutMilliseconds': 123
},
]
}
},
'weightedRoute': {
'trafficSplit': [
{
'targetName': 'string',
'model': 'string',
'fallback': {
'triggerOn': {
'statusCodes': [
'string',
],
'errors': [
'CONNECTION_ERROR'|'FIRST_BYTE_TIMEOUT'|'UNROUTABLE',
]
},
'firstByteTimeoutMilliseconds': 123,
'circuitBreaker': {
'mode': 'ENABLED'|'DISABLED'
},
'targetFallback': [
{
'targetName': 'string',
'model': 'string',
'triggerOn': {
'statusCodes': [
'string',
],
'errors': [
'CONNECTION_ERROR'|'FIRST_BYTE_TIMEOUT'|'UNROUTABLE',
]
},
'firstByteTimeoutMilliseconds': 123
},
]
},
'name': 'string',
'weight': 123,
'description': 'string',
'metadata': {
'string': 'string'
}
},
]
}
}
},
],
'description': 'string',
'createdAt': datetime(2015, 1, 1),
'status': 'CREATING'|'ACTIVE'|'UPDATING'|'DELETING',
'system': {
'managedBy': 'string'
}
}
Response Structure
(dict) --
ruleId (string) --
The unique identifier of the gateway rule.
gatewayArn (string) --
The Amazon Resource Name (ARN) of the gateway that the rule belongs to.
priority (integer) --
The priority of the rule. Rules are evaluated in order of priority, with lower numbers evaluated first.
conditions (list) --
The conditions that must be met for the rule to apply.
(dict) --
A condition that determines when a gateway rule applies. Conditions can match on principals, request paths, or the requested model.
matchPrincipals (dict) --
A condition that matches on the identity of the caller making the request.
anyOf (list) --
A list of principal entries. The condition is met if any of the entries match the caller's identity.
(dict) --
Union for principal matching. Currently supports IAM principal ARN glob matching.
iamPrincipal (dict) --
An IAM principal to match against, specified by ARN.
arn (string) --
The Amazon Resource Name (ARN) of the IAM principal. Supports user, role, and assumed-role ARNs. Wildcards can be used with the StringLike operator.
operator (string) --
The match operator. StringEquals requires an exact match. StringLike supports wildcard patterns using * and ?.
matchPaths (dict) --
A condition that matches on the request path.
anyOf (list) --
A list of path patterns. The condition is met if the request path matches any of the patterns.
(string) --
matchModels (dict) --
A condition that matches on the model requested in an inference request.
anyOf (list) --
A list of model entries. The condition is met if the requested model matches any of the entries.
(dict) --
A model ID or pattern to match against the requested model, and the operator to use for the comparison.
value (string) --
The model ID or pattern to match. With the StringLike operator, you can use * and ? as wildcards, for example anthropic.claude-*.
operator (string) --
The operator to use when comparing the requested model to the value:
StringEquals – Requires an exact match.
StringLike – Supports wildcard patterns using * and ?. This is the default.
actions (list) --
The actions to take when the rule conditions are met.
(dict) --
An action to take when a gateway rule's conditions are met.
configurationBundle (dict) --
An action that applies a configuration bundle override to the request.
staticOverride (dict) --
A static configuration bundle override that applies a single bundle version to all matching requests.
bundleArn (string) --
The Amazon Resource Name (ARN) of the configuration bundle to apply.
bundleVersion (string) --
The version of the configuration bundle to apply.
weightedOverride (dict) --
A weighted configuration bundle override that splits traffic between multiple bundle versions based on configured weights.
trafficSplit (list) --
The traffic split entries defining how traffic is distributed between configuration bundle versions.
(dict) --
An entry in a traffic split configuration, defining a named variant with a weight and configuration bundle reference.
name (string) --
The name of this traffic split variant.
weight (integer) --
The percentage of traffic to route to this variant. Weights across all entries must sum to 100.
configurationBundle (dict) --
The configuration bundle reference for this variant.
bundleArn (string) --
The Amazon Resource Name (ARN) of the configuration bundle.
bundleVersion (string) --
The version of the configuration bundle.
description (string) --
The description of this traffic split variant.
metadata (dict) --
Key-value metadata associated with this traffic split variant.
(string) --
(string) --
routeToTarget (dict) --
An action that routes the request to a specific target.
staticRoute (dict) --
A static route that sends all matching requests to a single target.
targetName (string) --
The name of the target to route requests to.
model (string) --
The model ID to send to the target. Supported only for inference targets. If you don't specify a model, the gateway sends the model from the original request.
fallback (dict) --
The fallback configuration for this route. Supported only for inference targets.
triggerOn (dict) --
The status codes and errors from the route's target that cause the gateway to send the request to the first fallback target. If you omit triggerOn, the gateway falls back on 5XX status codes and on the CONNECTION_ERROR, FIRST_BYTE_TIMEOUT, and UNROUTABLE errors. If you specify triggerOn, only the status codes and errors that you list trigger a fallback.
statusCodes (list) --
The HTTP status codes returned by the target that trigger a fallback. You can specify the following:
An exact status code – A three-digit code, such as 429.
A class of status codes – The first digit followed by XX, such as 5XX, which matches every code from 500 to 599.
Only 4XX and 5XX codes are supported. Throttling by the gateway itself doesn't trigger a fallback.
(string) --
errors (list) --
The errors that trigger a fallback. You can specify the following:
CONNECTION_ERROR – The gateway couldn't connect to the target because the connection was refused, reset, or timed out. The request didn't reach the target.
FIRST_BYTE_TIMEOUT – The target accepted the connection but didn't return the first byte of the response within firstByteTimeoutMilliseconds. The gateway abandons the request before any of the response reaches you.
UNROUTABLE – The gateway couldn't route the request to the target, for example because the target doesn't exist or doesn't support the request path. These errors don't count toward the circuit breaker.
(string) --
firstByteTimeoutMilliseconds (integer) --
The maximum time, in milliseconds, from sending the request to the route's target until the first byte of the response body arrives. If the time expires, the gateway abandons the request and treats it as a FIRST_BYTE_TIMEOUT error. If you don't specify a value, no timeout is enforced.
circuitBreaker (dict) --
Specifies whether the gateway temporarily stops sending your requests to a failing target. The circuit breaker tracks each combination of target and model separately. With the circuit breaker enabled, the gateway skips one or more targets and models that fail repeatedly, and your requests go to the next target in the chain. The gateway always tries the last remaining target in the chain. The gateway enables the circuit breaker by default.
mode (string) --
Specifies whether you want the circuit breaker turned on. To turn it off, set this value to DISABLED. The default is ENABLED.
targetFallback (list) --
The ordered list of fallback targets. The gateway tries each target in order until a response doesn't match that target's triggers, and returns that response to the caller. If every target fails, the gateway returns the last response. You can specify up to two fallback targets. Each combination of target and model, including the route's target, can appear only once.
(dict) --
Specifies a target to send the request to when the previous target in the fallback chain fails. Includes the target name, the model to send, and the conditions that trigger a fallback from this target to the next one.
targetName (string) --
The name of the gateway target to fall back to. The target must be an inference target.
model (string) --
The model ID to send to this fallback target. If you don't specify a model, the gateway sends the model from the original request.
triggerOn (dict) --
The status codes and errors from this target that cause the gateway to send the request to the next fallback target. This setting isn't inherited from the route. If you omit it, the gateway uses the default triggers: 5XX status codes and the CONNECTION_ERROR, FIRST_BYTE_TIMEOUT, and UNROUTABLE errors.
statusCodes (list) --
The HTTP status codes returned by the target that trigger a fallback. You can specify the following:
An exact status code – A three-digit code, such as 429.
A class of status codes – The first digit followed by XX, such as 5XX, which matches every code from 500 to 599.
Only 4XX and 5XX codes are supported. Throttling by the gateway itself doesn't trigger a fallback.
(string) --
errors (list) --
The errors that trigger a fallback. You can specify the following:
CONNECTION_ERROR – The gateway couldn't connect to the target because the connection was refused, reset, or timed out. The request didn't reach the target.
FIRST_BYTE_TIMEOUT – The target accepted the connection but didn't return the first byte of the response within firstByteTimeoutMilliseconds. The gateway abandons the request before any of the response reaches you.
UNROUTABLE – The gateway couldn't route the request to the target, for example because the target doesn't exist or doesn't support the request path. These errors don't count toward the circuit breaker.
(string) --
firstByteTimeoutMilliseconds (integer) --
The maximum time, in milliseconds, from sending the request to this target until the first byte of the response body arrives. This setting isn't inherited from the route. If you don't specify a value, no timeout is enforced.
weightedRoute (dict) --
A weighted route that splits traffic between multiple targets.
trafficSplit (list) --
The traffic split entries defining how traffic is distributed between targets.
(dict) --
An entry in a target traffic split configuration.
targetName (string) --
The name of the target to route traffic to.
model (string) --
The model ID to send to the target for this variant. Supported only for inference targets. If you don't specify a model, the gateway sends the model from the original request.
fallback (dict) --
The fallback configuration for this variant. Supported only for inference targets.
triggerOn (dict) --
The status codes and errors from the route's target that cause the gateway to send the request to the first fallback target. If you omit triggerOn, the gateway falls back on 5XX status codes and on the CONNECTION_ERROR, FIRST_BYTE_TIMEOUT, and UNROUTABLE errors. If you specify triggerOn, only the status codes and errors that you list trigger a fallback.
statusCodes (list) --
The HTTP status codes returned by the target that trigger a fallback. You can specify the following:
An exact status code – A three-digit code, such as 429.
A class of status codes – The first digit followed by XX, such as 5XX, which matches every code from 500 to 599.
Only 4XX and 5XX codes are supported. Throttling by the gateway itself doesn't trigger a fallback.
(string) --
errors (list) --
The errors that trigger a fallback. You can specify the following:
CONNECTION_ERROR – The gateway couldn't connect to the target because the connection was refused, reset, or timed out. The request didn't reach the target.
FIRST_BYTE_TIMEOUT – The target accepted the connection but didn't return the first byte of the response within firstByteTimeoutMilliseconds. The gateway abandons the request before any of the response reaches you.
UNROUTABLE – The gateway couldn't route the request to the target, for example because the target doesn't exist or doesn't support the request path. These errors don't count toward the circuit breaker.
(string) --
firstByteTimeoutMilliseconds (integer) --
The maximum time, in milliseconds, from sending the request to the route's target until the first byte of the response body arrives. If the time expires, the gateway abandons the request and treats it as a FIRST_BYTE_TIMEOUT error. If you don't specify a value, no timeout is enforced.
circuitBreaker (dict) --
Specifies whether the gateway temporarily stops sending your requests to a failing target. The circuit breaker tracks each combination of target and model separately. With the circuit breaker enabled, the gateway skips one or more targets and models that fail repeatedly, and your requests go to the next target in the chain. The gateway always tries the last remaining target in the chain. The gateway enables the circuit breaker by default.
mode (string) --
Specifies whether you want the circuit breaker turned on. To turn it off, set this value to DISABLED. The default is ENABLED.
targetFallback (list) --
The ordered list of fallback targets. The gateway tries each target in order until a response doesn't match that target's triggers, and returns that response to the caller. If every target fails, the gateway returns the last response. You can specify up to two fallback targets. Each combination of target and model, including the route's target, can appear only once.
(dict) --
Specifies a target to send the request to when the previous target in the fallback chain fails. Includes the target name, the model to send, and the conditions that trigger a fallback from this target to the next one.
targetName (string) --
The name of the gateway target to fall back to. The target must be an inference target.
model (string) --
The model ID to send to this fallback target. If you don't specify a model, the gateway sends the model from the original request.
triggerOn (dict) --
The status codes and errors from this target that cause the gateway to send the request to the next fallback target. This setting isn't inherited from the route. If you omit it, the gateway uses the default triggers: 5XX status codes and the CONNECTION_ERROR, FIRST_BYTE_TIMEOUT, and UNROUTABLE errors.
statusCodes (list) --
The HTTP status codes returned by the target that trigger a fallback. You can specify the following:
An exact status code – A three-digit code, such as 429.
A class of status codes – The first digit followed by XX, such as 5XX, which matches every code from 500 to 599.
Only 4XX and 5XX codes are supported. Throttling by the gateway itself doesn't trigger a fallback.
(string) --
errors (list) --
The errors that trigger a fallback. You can specify the following:
CONNECTION_ERROR – The gateway couldn't connect to the target because the connection was refused, reset, or timed out. The request didn't reach the target.
FIRST_BYTE_TIMEOUT – The target accepted the connection but didn't return the first byte of the response within firstByteTimeoutMilliseconds. The gateway abandons the request before any of the response reaches you.
UNROUTABLE – The gateway couldn't route the request to the target, for example because the target doesn't exist or doesn't support the request path. These errors don't count toward the circuit breaker.
(string) --
firstByteTimeoutMilliseconds (integer) --
The maximum time, in milliseconds, from sending the request to this target until the first byte of the response body arrives. This setting isn't inherited from the route. If you don't specify a value, no timeout is enforced.
name (string) --
The name of this traffic split variant.
weight (integer) --
The percentage of traffic to route to this variant.
description (string) --
The description of this traffic split variant.
metadata (dict) --
Key-value metadata associated with this traffic split variant.
(string) --
(string) --
description (string) --
The description of the gateway rule.
createdAt (datetime) --
The timestamp when the rule was created.
status (string) --
The current status of the rule.
system (dict) --
System-managed metadata for rules created by automated processes.
managedBy (string) --
The identifier of the system or process that manages this rule.
{'evaluatorConfig': {'llmAsAJudge': {'ratingScale': {'multiDimensional': [{'dimensionInstruction': 'string',
'dimensionName': 'string',
'dimensionRatingScale': {'categorical': [{'definition': 'string',
'label': 'string'}],
'numerical': [{'definition': 'string',
'label': 'string',
'value': 'double'}]}}]}}}}
Retrieves detailed information about an evaluator, including its configuration, status, and metadata. Works with both built-in and custom evaluators.
See also: AWS API Documentation
Request Syntax
client.get_evaluator(
evaluatorId='string',
includedData='ALL_DATA'|'METADATA_ONLY'
)
string
[REQUIRED]
The unique identifier of the evaluator to retrieve. Can be a built-in evaluator ID (e.g., Builtin.Helpfulness) or a custom evaluator ID.
string
Controls which data is returned in the response. ALL_DATA (default) returns the full evaluator including decrypted instructions and rating scale. For evaluators encrypted with a customer managed KMS key, this requires kms:Decrypt permission on the key. METADATA_ONLY returns evaluator metadata and model configuration without instructions or rating scale, and does not require any KMS permissions.
dict
Response Syntax
{
'evaluatorArn': 'string',
'evaluatorId': 'string',
'evaluatorName': 'string',
'description': 'string',
'evaluatorConfig': {
'llmAsAJudge': {
'instructions': 'string',
'ratingScale': {
'numerical': [
{
'definition': 'string',
'value': 123.0,
'label': 'string'
},
],
'categorical': [
{
'definition': 'string',
'label': 'string'
},
],
'multiDimensional': [
{
'dimensionName': 'string',
'dimensionInstruction': 'string',
'dimensionRatingScale': {
'numerical': [
{
'definition': 'string',
'value': 123.0,
'label': 'string'
},
],
'categorical': [
{
'definition': 'string',
'label': 'string'
},
]
}
},
]
},
'modelConfig': {
'bedrockEvaluatorModelConfig': {
'modelId': 'string',
'inferenceConfig': {
'maxTokens': 123,
'temperature': ...,
'topP': ...,
'stopSequences': [
'string',
]
},
'additionalModelRequestFields': {...}|[...]|123|123.4|'string'|True|None
},
'responsesEvaluatorModelConfig': {
'modelId': 'string',
'maxOutputTokens': 123,
'temperature': ...,
'topP': ...,
'reasoning': {
'effort': 'string'
}
}
}
},
'codeBased': {
'lambdaConfig': {
'lambdaArn': 'string',
'lambdaTimeoutInSeconds': 123
}
},
'derived': {
'baseEvaluatorId': 'string',
'modelConfig': {
'bedrockEvaluatorModelConfig': {
'modelId': 'string',
'inferenceConfig': {
'maxTokens': 123,
'temperature': ...,
'topP': ...,
'stopSequences': [
'string',
]
},
'additionalModelRequestFields': {...}|[...]|123|123.4|'string'|True|None
},
'responsesEvaluatorModelConfig': {
'modelId': 'string',
'maxOutputTokens': 123,
'temperature': ...,
'topP': ...,
'reasoning': {
'effort': 'string'
}
}
}
}
},
'evaluatorType': 'Builtin'|'ThirdParty'|'Custom'|'CustomCode'|'CustomDerived',
'provider': 'AWS'|'DeepEval'|'AutoEval'|'Custom',
'level': 'TOOL_CALL'|'TRACE'|'SESSION',
'status': 'ACTIVE'|'CREATING'|'CREATE_FAILED'|'UPDATING'|'UPDATE_FAILED'|'DELETING',
'createdAt': datetime(2015, 1, 1),
'updatedAt': datetime(2015, 1, 1),
'lockedForModification': True|False,
'kmsKeyArn': 'string'
}
Response Structure
(dict) --
evaluatorArn (string) --
The Amazon Resource Name (ARN) of the evaluator.
evaluatorId (string) --
The unique identifier of the evaluator.
evaluatorName (string) --
The name of the evaluator.
description (string) --
The description of the evaluator.
evaluatorConfig (dict) --
The configuration of the evaluator, including LLM-as-a-Judge or code-based settings.
llmAsAJudge (dict) --
The LLM-as-a-Judge configuration that uses a language model to evaluate agent performance based on custom instructions and rating scales.
instructions (string) --
The evaluation instructions that guide the language model in assessing agent performance, including criteria and evaluation guidelines.
ratingScale (dict) --
The rating scale that defines how the evaluator should score agent performance, either numerical or categorical.
numerical (list) --
The numerical rating scale with defined score values and descriptions for quantitative evaluation.
(dict) --
The definition of a numerical rating scale option that provides a numeric value with its description for evaluation scoring.
definition (string) --
The description that explains what this numerical rating represents and when it should be used.
value (float) --
The numerical value for this rating scale option.
label (string) --
The label or name that describes this numerical rating option.
categorical (list) --
The categorical rating scale with named categories and definitions for qualitative evaluation.
(dict) --
The definition of a categorical rating scale option that provides a named category with its description for evaluation scoring.
definition (string) --
The description that explains what this categorical rating represents and when it should be used.
label (string) --
The label or name of this categorical rating option.
multiDimensional (list) --
A list of dimensions that lets an LLM-as-a-judge evaluator score several dimensions of the same target in a single evaluation. Each dimension has its own instruction and rating scale, and produces its own result.
(dict) --
A single dimension of an LLM-as-a-judge evaluator, defining what the evaluator model should judge for that dimension and how it should be scored.
dimensionName (string) --
The name of the dimension. This name is unique within the evaluator and identifies the dimension's results.
dimensionInstruction (string) --
The instruction to the evaluator model for this dimension, describing what the model should judge when scoring the dimension.
dimensionRatingScale (dict) --
The rating scale for this dimension, which can be numerical or categorical. A dimension's rating scale cannot itself be multi-dimensional.
numerical (list) --
The numerical rating scale with defined score values and descriptions for quantitative evaluation of the dimension.
(dict) --
The definition of a numerical rating scale option that provides a numeric value with its description for evaluation scoring.
definition (string) --
The description that explains what this numerical rating represents and when it should be used.
value (float) --
The numerical value for this rating scale option.
label (string) --
The label or name that describes this numerical rating option.
categorical (list) --
The categorical rating scale with named categories and definitions for qualitative evaluation of the dimension.
(dict) --
The definition of a categorical rating scale option that provides a named category with its description for evaluation scoring.
definition (string) --
The description that explains what this categorical rating represents and when it should be used.
label (string) --
The label or name of this categorical rating option.
modelConfig (dict) --
The model configuration that specifies which foundation model to use and how to configure it for evaluation.
bedrockEvaluatorModelConfig (dict) --
The Amazon Bedrock model configuration for evaluation.
modelId (string) --
The identifier of the Amazon Bedrock model to use for evaluation. Must be a supported foundation model available in your region.
inferenceConfig (dict) --
The inference configuration parameters that control model behavior during evaluation, including temperature, token limits, and sampling settings.
maxTokens (integer) --
The maximum number of tokens to generate in the model response during evaluation.
temperature (float) --
The temperature value that controls randomness in the model's responses. Lower values produce more deterministic outputs.
topP (float) --
The top-p sampling parameter that controls the diversity of the model's responses by limiting the cumulative probability of token choices.
stopSequences (list) --
The list of sequences that will cause the model to stop generating tokens when encountered.
(string) --
additionalModelRequestFields (:ref:`document<document>`) --
Additional model-specific request fields to customize model behavior beyond the standard inference configuration.
responsesEvaluatorModelConfig (dict) --
The OpenResponses model configuration for evaluation.
modelId (string) --
The identifier of the model to use for evaluation.
maxOutputTokens (integer) --
The maximum number of tokens to generate in the model response, including visible output and reasoning tokens.
temperature (float) --
The temperature value that controls randomness in the model's responses. Lower values produce more deterministic outputs.
topP (float) --
The top-p sampling parameter that controls the diversity of the model's responses by limiting the cumulative probability of token choices.
reasoning (dict) --
The reasoning configuration for reasoning models. Non-reasoning models ignore this configuration.
effort (string) --
The level of reasoning effort the model applies when generating a response. For supported values, see the model provider's documentation.
codeBased (dict) --
Configuration for a code-based evaluator that uses a customer-managed Lambda function to programmatically assess agent performance.
lambdaConfig (dict) --
The Lambda function configuration for code-based evaluation.
lambdaArn (string) --
The Amazon Resource Name (ARN) of the Lambda function that implements the evaluation logic.
lambdaTimeoutInSeconds (integer) --
The timeout in seconds for the Lambda function invocation. Defaults to 60. Must be between 1 and 300.
derived (dict) --
The configuration for an evaluator derived from an existing base evaluator (a built-in or third-party evaluator), run on your own model. The base evaluator supplies the prompt and scoring.
baseEvaluatorId (string) --
The identifier of the base evaluator whose logic to run (a Builtin.* or ThirdParty.* evaluator).
modelConfig (dict) --
The configuration of the evaluator model that you supply.
bedrockEvaluatorModelConfig (dict) --
The Amazon Bedrock model configuration for evaluation.
modelId (string) --
The identifier of the Amazon Bedrock model to use for evaluation. Must be a supported foundation model available in your region.
inferenceConfig (dict) --
The inference configuration parameters that control model behavior during evaluation, including temperature, token limits, and sampling settings.
maxTokens (integer) --
The maximum number of tokens to generate in the model response during evaluation.
temperature (float) --
The temperature value that controls randomness in the model's responses. Lower values produce more deterministic outputs.
topP (float) --
The top-p sampling parameter that controls the diversity of the model's responses by limiting the cumulative probability of token choices.
stopSequences (list) --
The list of sequences that will cause the model to stop generating tokens when encountered.
(string) --
additionalModelRequestFields (:ref:`document<document>`) --
Additional model-specific request fields to customize model behavior beyond the standard inference configuration.
responsesEvaluatorModelConfig (dict) --
The OpenResponses model configuration for evaluation.
modelId (string) --
The identifier of the model to use for evaluation.
maxOutputTokens (integer) --
The maximum number of tokens to generate in the model response, including visible output and reasoning tokens.
temperature (float) --
The temperature value that controls randomness in the model's responses. Lower values produce more deterministic outputs.
topP (float) --
The top-p sampling parameter that controls the diversity of the model's responses by limiting the cumulative probability of token choices.
reasoning (dict) --
The reasoning configuration for reasoning models. Non-reasoning models ignore this configuration.
effort (string) --
The level of reasoning effort the model applies when generating a response. For supported values, see the model provider's documentation.
evaluatorType (string) --
The kind of evaluator resource. Valid values:
Builtin – An Amazon Web Services-managed global evaluator.
ThirdParty – An Amazon Web Services-managed global evaluator from a third-party provider.
Custom – A customer-created evaluator.
CustomCode – A customer-created code-based evaluator.
CustomDerived – A customer-created evaluator derived from an existing base evaluator.
provider (string) --
The source of the evaluator's logic: Amazon Web Services, a third-party library, or you.
level (string) --
The evaluation level ( TOOL_CALL, TRACE, or SESSION) that determines the scope of evaluation.
status (string) --
The current status of the evaluator.
createdAt (datetime) --
The timestamp when the evaluator was created.
updatedAt (datetime) --
The timestamp when the evaluator was last updated.
lockedForModification (boolean) --
Whether the evaluator is locked for modification due to being referenced by active online evaluation configurations.
kmsKeyArn (string) --
The Amazon Resource Name (ARN) of the customer managed KMS key used to encrypt the evaluator's sensitive data. This field is only present for evaluators encrypted with a customer managed key.
{'actions': {'routeToTarget': {'staticRoute': {'fallback': {'circuitBreaker': {'mode': 'ENABLED '
'| '
'DISABLED'},
'firstByteTimeoutMilliseconds': 'integer',
'targetFallback': [{'firstByteTimeoutMilliseconds': 'integer',
'model': 'string',
'targetName': 'string',
'triggerOn': {'errors': ['CONNECTION_ERROR '
'| '
'FIRST_BYTE_TIMEOUT '
'| '
'UNROUTABLE'],
'statusCodes': ['string']}}],
'triggerOn': {'errors': ['CONNECTION_ERROR '
'| '
'FIRST_BYTE_TIMEOUT '
'| '
'UNROUTABLE'],
'statusCodes': ['string']}},
'model': 'string'},
'weightedRoute': {'trafficSplit': {'fallback': {'circuitBreaker': {'mode': 'ENABLED '
'| '
'DISABLED'},
'firstByteTimeoutMilliseconds': 'integer',
'targetFallback': [{'firstByteTimeoutMilliseconds': 'integer',
'model': 'string',
'targetName': 'string',
'triggerOn': {'errors': ['CONNECTION_ERROR '
'| '
'FIRST_BYTE_TIMEOUT '
'| '
'UNROUTABLE'],
'statusCodes': ['string']}}],
'triggerOn': {'errors': ['CONNECTION_ERROR '
'| '
'FIRST_BYTE_TIMEOUT '
'| '
'UNROUTABLE'],
'statusCodes': ['string']}},
'model': 'string'}}}},
'conditions': {'matchModels': {'anyOf': [{'operator': 'StringEquals | '
'StringLike',
'value': 'string'}]}}}
Retrieves detailed information about a specific gateway rule.
See also: AWS API Documentation
Request Syntax
client.get_gateway_rule(
gatewayIdentifier='string',
ruleId='string'
)
string
[REQUIRED]
The identifier of the gateway containing the rule.
string
[REQUIRED]
The unique identifier of the rule to retrieve.
dict
Response Syntax
{
'ruleId': 'string',
'gatewayArn': 'string',
'priority': 123,
'conditions': [
{
'matchPrincipals': {
'anyOf': [
{
'iamPrincipal': {
'arn': 'string',
'operator': 'StringEquals'|'StringLike'
}
},
]
},
'matchPaths': {
'anyOf': [
'string',
]
},
'matchModels': {
'anyOf': [
{
'value': 'string',
'operator': 'StringEquals'|'StringLike'
},
]
}
},
],
'actions': [
{
'configurationBundle': {
'staticOverride': {
'bundleArn': 'string',
'bundleVersion': 'string'
},
'weightedOverride': {
'trafficSplit': [
{
'name': 'string',
'weight': 123,
'configurationBundle': {
'bundleArn': 'string',
'bundleVersion': 'string'
},
'description': 'string',
'metadata': {
'string': 'string'
}
},
]
}
},
'routeToTarget': {
'staticRoute': {
'targetName': 'string',
'model': 'string',
'fallback': {
'triggerOn': {
'statusCodes': [
'string',
],
'errors': [
'CONNECTION_ERROR'|'FIRST_BYTE_TIMEOUT'|'UNROUTABLE',
]
},
'firstByteTimeoutMilliseconds': 123,
'circuitBreaker': {
'mode': 'ENABLED'|'DISABLED'
},
'targetFallback': [
{
'targetName': 'string',
'model': 'string',
'triggerOn': {
'statusCodes': [
'string',
],
'errors': [
'CONNECTION_ERROR'|'FIRST_BYTE_TIMEOUT'|'UNROUTABLE',
]
},
'firstByteTimeoutMilliseconds': 123
},
]
}
},
'weightedRoute': {
'trafficSplit': [
{
'targetName': 'string',
'model': 'string',
'fallback': {
'triggerOn': {
'statusCodes': [
'string',
],
'errors': [
'CONNECTION_ERROR'|'FIRST_BYTE_TIMEOUT'|'UNROUTABLE',
]
},
'firstByteTimeoutMilliseconds': 123,
'circuitBreaker': {
'mode': 'ENABLED'|'DISABLED'
},
'targetFallback': [
{
'targetName': 'string',
'model': 'string',
'triggerOn': {
'statusCodes': [
'string',
],
'errors': [
'CONNECTION_ERROR'|'FIRST_BYTE_TIMEOUT'|'UNROUTABLE',
]
},
'firstByteTimeoutMilliseconds': 123
},
]
},
'name': 'string',
'weight': 123,
'description': 'string',
'metadata': {
'string': 'string'
}
},
]
}
}
},
],
'description': 'string',
'createdAt': datetime(2015, 1, 1),
'status': 'CREATING'|'ACTIVE'|'UPDATING'|'DELETING',
'system': {
'managedBy': 'string'
},
'updatedAt': datetime(2015, 1, 1)
}
Response Structure
(dict) --
Create response excludes updatedAt (redundant on create). Get/Update responses include it via their own output structures.
ruleId (string) --
The unique identifier of the gateway rule.
gatewayArn (string) --
The Amazon Resource Name (ARN) of the gateway that the rule belongs to.
priority (integer) --
The priority of the rule. Rules are evaluated in order of priority, with lower numbers evaluated first.
conditions (list) --
The conditions that must be met for the rule to apply.
(dict) --
A condition that determines when a gateway rule applies. Conditions can match on principals, request paths, or the requested model.
matchPrincipals (dict) --
A condition that matches on the identity of the caller making the request.
anyOf (list) --
A list of principal entries. The condition is met if any of the entries match the caller's identity.
(dict) --
Union for principal matching. Currently supports IAM principal ARN glob matching.
iamPrincipal (dict) --
An IAM principal to match against, specified by ARN.
arn (string) --
The Amazon Resource Name (ARN) of the IAM principal. Supports user, role, and assumed-role ARNs. Wildcards can be used with the StringLike operator.
operator (string) --
The match operator. StringEquals requires an exact match. StringLike supports wildcard patterns using * and ?.
matchPaths (dict) --
A condition that matches on the request path.
anyOf (list) --
A list of path patterns. The condition is met if the request path matches any of the patterns.
(string) --
matchModels (dict) --
A condition that matches on the model requested in an inference request.
anyOf (list) --
A list of model entries. The condition is met if the requested model matches any of the entries.
(dict) --
A model ID or pattern to match against the requested model, and the operator to use for the comparison.
value (string) --
The model ID or pattern to match. With the StringLike operator, you can use * and ? as wildcards, for example anthropic.claude-*.
operator (string) --
The operator to use when comparing the requested model to the value:
StringEquals – Requires an exact match.
StringLike – Supports wildcard patterns using * and ?. This is the default.
actions (list) --
The actions to take when the rule conditions are met.
(dict) --
An action to take when a gateway rule's conditions are met.
configurationBundle (dict) --
An action that applies a configuration bundle override to the request.
staticOverride (dict) --
A static configuration bundle override that applies a single bundle version to all matching requests.
bundleArn (string) --
The Amazon Resource Name (ARN) of the configuration bundle to apply.
bundleVersion (string) --
The version of the configuration bundle to apply.
weightedOverride (dict) --
A weighted configuration bundle override that splits traffic between multiple bundle versions based on configured weights.
trafficSplit (list) --
The traffic split entries defining how traffic is distributed between configuration bundle versions.
(dict) --
An entry in a traffic split configuration, defining a named variant with a weight and configuration bundle reference.
name (string) --
The name of this traffic split variant.
weight (integer) --
The percentage of traffic to route to this variant. Weights across all entries must sum to 100.
configurationBundle (dict) --
The configuration bundle reference for this variant.
bundleArn (string) --
The Amazon Resource Name (ARN) of the configuration bundle.
bundleVersion (string) --
The version of the configuration bundle.
description (string) --
The description of this traffic split variant.
metadata (dict) --
Key-value metadata associated with this traffic split variant.
(string) --
(string) --
routeToTarget (dict) --
An action that routes the request to a specific target.
staticRoute (dict) --
A static route that sends all matching requests to a single target.
targetName (string) --
The name of the target to route requests to.
model (string) --
The model ID to send to the target. Supported only for inference targets. If you don't specify a model, the gateway sends the model from the original request.
fallback (dict) --
The fallback configuration for this route. Supported only for inference targets.
triggerOn (dict) --
The status codes and errors from the route's target that cause the gateway to send the request to the first fallback target. If you omit triggerOn, the gateway falls back on 5XX status codes and on the CONNECTION_ERROR, FIRST_BYTE_TIMEOUT, and UNROUTABLE errors. If you specify triggerOn, only the status codes and errors that you list trigger a fallback.
statusCodes (list) --
The HTTP status codes returned by the target that trigger a fallback. You can specify the following:
An exact status code – A three-digit code, such as 429.
A class of status codes – The first digit followed by XX, such as 5XX, which matches every code from 500 to 599.
Only 4XX and 5XX codes are supported. Throttling by the gateway itself doesn't trigger a fallback.
(string) --
errors (list) --
The errors that trigger a fallback. You can specify the following:
CONNECTION_ERROR – The gateway couldn't connect to the target because the connection was refused, reset, or timed out. The request didn't reach the target.
FIRST_BYTE_TIMEOUT – The target accepted the connection but didn't return the first byte of the response within firstByteTimeoutMilliseconds. The gateway abandons the request before any of the response reaches you.
UNROUTABLE – The gateway couldn't route the request to the target, for example because the target doesn't exist or doesn't support the request path. These errors don't count toward the circuit breaker.
(string) --
firstByteTimeoutMilliseconds (integer) --
The maximum time, in milliseconds, from sending the request to the route's target until the first byte of the response body arrives. If the time expires, the gateway abandons the request and treats it as a FIRST_BYTE_TIMEOUT error. If you don't specify a value, no timeout is enforced.
circuitBreaker (dict) --
Specifies whether the gateway temporarily stops sending your requests to a failing target. The circuit breaker tracks each combination of target and model separately. With the circuit breaker enabled, the gateway skips one or more targets and models that fail repeatedly, and your requests go to the next target in the chain. The gateway always tries the last remaining target in the chain. The gateway enables the circuit breaker by default.
mode (string) --
Specifies whether you want the circuit breaker turned on. To turn it off, set this value to DISABLED. The default is ENABLED.
targetFallback (list) --
The ordered list of fallback targets. The gateway tries each target in order until a response doesn't match that target's triggers, and returns that response to the caller. If every target fails, the gateway returns the last response. You can specify up to two fallback targets. Each combination of target and model, including the route's target, can appear only once.
(dict) --
Specifies a target to send the request to when the previous target in the fallback chain fails. Includes the target name, the model to send, and the conditions that trigger a fallback from this target to the next one.
targetName (string) --
The name of the gateway target to fall back to. The target must be an inference target.
model (string) --
The model ID to send to this fallback target. If you don't specify a model, the gateway sends the model from the original request.
triggerOn (dict) --
The status codes and errors from this target that cause the gateway to send the request to the next fallback target. This setting isn't inherited from the route. If you omit it, the gateway uses the default triggers: 5XX status codes and the CONNECTION_ERROR, FIRST_BYTE_TIMEOUT, and UNROUTABLE errors.
statusCodes (list) --
The HTTP status codes returned by the target that trigger a fallback. You can specify the following:
An exact status code – A three-digit code, such as 429.
A class of status codes – The first digit followed by XX, such as 5XX, which matches every code from 500 to 599.
Only 4XX and 5XX codes are supported. Throttling by the gateway itself doesn't trigger a fallback.
(string) --
errors (list) --
The errors that trigger a fallback. You can specify the following:
CONNECTION_ERROR – The gateway couldn't connect to the target because the connection was refused, reset, or timed out. The request didn't reach the target.
FIRST_BYTE_TIMEOUT – The target accepted the connection but didn't return the first byte of the response within firstByteTimeoutMilliseconds. The gateway abandons the request before any of the response reaches you.
UNROUTABLE – The gateway couldn't route the request to the target, for example because the target doesn't exist or doesn't support the request path. These errors don't count toward the circuit breaker.
(string) --
firstByteTimeoutMilliseconds (integer) --
The maximum time, in milliseconds, from sending the request to this target until the first byte of the response body arrives. This setting isn't inherited from the route. If you don't specify a value, no timeout is enforced.
weightedRoute (dict) --
A weighted route that splits traffic between multiple targets.
trafficSplit (list) --
The traffic split entries defining how traffic is distributed between targets.
(dict) --
An entry in a target traffic split configuration.
targetName (string) --
The name of the target to route traffic to.
model (string) --
The model ID to send to the target for this variant. Supported only for inference targets. If you don't specify a model, the gateway sends the model from the original request.
fallback (dict) --
The fallback configuration for this variant. Supported only for inference targets.
triggerOn (dict) --
The status codes and errors from the route's target that cause the gateway to send the request to the first fallback target. If you omit triggerOn, the gateway falls back on 5XX status codes and on the CONNECTION_ERROR, FIRST_BYTE_TIMEOUT, and UNROUTABLE errors. If you specify triggerOn, only the status codes and errors that you list trigger a fallback.
statusCodes (list) --
The HTTP status codes returned by the target that trigger a fallback. You can specify the following:
An exact status code – A three-digit code, such as 429.
A class of status codes – The first digit followed by XX, such as 5XX, which matches every code from 500 to 599.
Only 4XX and 5XX codes are supported. Throttling by the gateway itself doesn't trigger a fallback.
(string) --
errors (list) --
The errors that trigger a fallback. You can specify the following:
CONNECTION_ERROR – The gateway couldn't connect to the target because the connection was refused, reset, or timed out. The request didn't reach the target.
FIRST_BYTE_TIMEOUT – The target accepted the connection but didn't return the first byte of the response within firstByteTimeoutMilliseconds. The gateway abandons the request before any of the response reaches you.
UNROUTABLE – The gateway couldn't route the request to the target, for example because the target doesn't exist or doesn't support the request path. These errors don't count toward the circuit breaker.
(string) --
firstByteTimeoutMilliseconds (integer) --
The maximum time, in milliseconds, from sending the request to the route's target until the first byte of the response body arrives. If the time expires, the gateway abandons the request and treats it as a FIRST_BYTE_TIMEOUT error. If you don't specify a value, no timeout is enforced.
circuitBreaker (dict) --
Specifies whether the gateway temporarily stops sending your requests to a failing target. The circuit breaker tracks each combination of target and model separately. With the circuit breaker enabled, the gateway skips one or more targets and models that fail repeatedly, and your requests go to the next target in the chain. The gateway always tries the last remaining target in the chain. The gateway enables the circuit breaker by default.
mode (string) --
Specifies whether you want the circuit breaker turned on. To turn it off, set this value to DISABLED. The default is ENABLED.
targetFallback (list) --
The ordered list of fallback targets. The gateway tries each target in order until a response doesn't match that target's triggers, and returns that response to the caller. If every target fails, the gateway returns the last response. You can specify up to two fallback targets. Each combination of target and model, including the route's target, can appear only once.
(dict) --
Specifies a target to send the request to when the previous target in the fallback chain fails. Includes the target name, the model to send, and the conditions that trigger a fallback from this target to the next one.
targetName (string) --
The name of the gateway target to fall back to. The target must be an inference target.
model (string) --
The model ID to send to this fallback target. If you don't specify a model, the gateway sends the model from the original request.
triggerOn (dict) --
The status codes and errors from this target that cause the gateway to send the request to the next fallback target. This setting isn't inherited from the route. If you omit it, the gateway uses the default triggers: 5XX status codes and the CONNECTION_ERROR, FIRST_BYTE_TIMEOUT, and UNROUTABLE errors.
statusCodes (list) --
The HTTP status codes returned by the target that trigger a fallback. You can specify the following:
An exact status code – A three-digit code, such as 429.
A class of status codes – The first digit followed by XX, such as 5XX, which matches every code from 500 to 599.
Only 4XX and 5XX codes are supported. Throttling by the gateway itself doesn't trigger a fallback.
(string) --
errors (list) --
The errors that trigger a fallback. You can specify the following:
CONNECTION_ERROR – The gateway couldn't connect to the target because the connection was refused, reset, or timed out. The request didn't reach the target.
FIRST_BYTE_TIMEOUT – The target accepted the connection but didn't return the first byte of the response within firstByteTimeoutMilliseconds. The gateway abandons the request before any of the response reaches you.
UNROUTABLE – The gateway couldn't route the request to the target, for example because the target doesn't exist or doesn't support the request path. These errors don't count toward the circuit breaker.
(string) --
firstByteTimeoutMilliseconds (integer) --
The maximum time, in milliseconds, from sending the request to this target until the first byte of the response body arrives. This setting isn't inherited from the route. If you don't specify a value, no timeout is enforced.
name (string) --
The name of this traffic split variant.
weight (integer) --
The percentage of traffic to route to this variant.
description (string) --
The description of this traffic split variant.
metadata (dict) --
Key-value metadata associated with this traffic split variant.
(string) --
(string) --
description (string) --
The description of the gateway rule.
createdAt (datetime) --
The timestamp when the rule was created.
status (string) --
The current status of the rule.
system (dict) --
System-managed metadata for rules created by automated processes.
managedBy (string) --
The identifier of the system or process that manages this rule.
updatedAt (datetime) --
The timestamp when the rule was last updated.
{'gatewayRules': {'actions': {'routeToTarget': {'staticRoute': {'fallback': {'circuitBreaker': {'mode': 'ENABLED '
'| '
'DISABLED'},
'firstByteTimeoutMilliseconds': 'integer',
'targetFallback': [{'firstByteTimeoutMilliseconds': 'integer',
'model': 'string',
'targetName': 'string',
'triggerOn': {'errors': ['CONNECTION_ERROR '
'| '
'FIRST_BYTE_TIMEOUT '
'| '
'UNROUTABLE'],
'statusCodes': ['string']}}],
'triggerOn': {'errors': ['CONNECTION_ERROR '
'| '
'FIRST_BYTE_TIMEOUT '
'| '
'UNROUTABLE'],
'statusCodes': ['string']}},
'model': 'string'},
'weightedRoute': {'trafficSplit': {'fallback': {'circuitBreaker': {'mode': 'ENABLED '
'| '
'DISABLED'},
'firstByteTimeoutMilliseconds': 'integer',
'targetFallback': [{'firstByteTimeoutMilliseconds': 'integer',
'model': 'string',
'targetName': 'string',
'triggerOn': {'errors': ['CONNECTION_ERROR '
'| '
'FIRST_BYTE_TIMEOUT '
'| '
'UNROUTABLE'],
'statusCodes': ['string']}}],
'triggerOn': {'errors': ['CONNECTION_ERROR '
'| '
'FIRST_BYTE_TIMEOUT '
'| '
'UNROUTABLE'],
'statusCodes': ['string']}},
'model': 'string'}}}},
'conditions': {'matchModels': {'anyOf': [{'operator': 'StringEquals '
'| '
'StringLike',
'value': 'string'}]}}}}
Lists all rules for a gateway.
See also: AWS API Documentation
Request Syntax
client.list_gateway_rules(
gatewayIdentifier='string',
maxResults=123,
nextToken='string'
)
string
[REQUIRED]
The identifier of the gateway to list rules for.
integer
The maximum number of results to return in the response. If the total number of results is greater than this value, use the token returned in the response in the nextToken field when making another request to return the next batch of results.
string
The pagination token from a previous request.
dict
Response Syntax
{
'gatewayRules': [
{
'ruleId': 'string',
'gatewayArn': 'string',
'priority': 123,
'conditions': [
{
'matchPrincipals': {
'anyOf': [
{
'iamPrincipal': {
'arn': 'string',
'operator': 'StringEquals'|'StringLike'
}
},
]
},
'matchPaths': {
'anyOf': [
'string',
]
},
'matchModels': {
'anyOf': [
{
'value': 'string',
'operator': 'StringEquals'|'StringLike'
},
]
}
},
],
'actions': [
{
'configurationBundle': {
'staticOverride': {
'bundleArn': 'string',
'bundleVersion': 'string'
},
'weightedOverride': {
'trafficSplit': [
{
'name': 'string',
'weight': 123,
'configurationBundle': {
'bundleArn': 'string',
'bundleVersion': 'string'
},
'description': 'string',
'metadata': {
'string': 'string'
}
},
]
}
},
'routeToTarget': {
'staticRoute': {
'targetName': 'string',
'model': 'string',
'fallback': {
'triggerOn': {
'statusCodes': [
'string',
],
'errors': [
'CONNECTION_ERROR'|'FIRST_BYTE_TIMEOUT'|'UNROUTABLE',
]
},
'firstByteTimeoutMilliseconds': 123,
'circuitBreaker': {
'mode': 'ENABLED'|'DISABLED'
},
'targetFallback': [
{
'targetName': 'string',
'model': 'string',
'triggerOn': {
'statusCodes': [
'string',
],
'errors': [
'CONNECTION_ERROR'|'FIRST_BYTE_TIMEOUT'|'UNROUTABLE',
]
},
'firstByteTimeoutMilliseconds': 123
},
]
}
},
'weightedRoute': {
'trafficSplit': [
{
'targetName': 'string',
'model': 'string',
'fallback': {
'triggerOn': {
'statusCodes': [
'string',
],
'errors': [
'CONNECTION_ERROR'|'FIRST_BYTE_TIMEOUT'|'UNROUTABLE',
]
},
'firstByteTimeoutMilliseconds': 123,
'circuitBreaker': {
'mode': 'ENABLED'|'DISABLED'
},
'targetFallback': [
{
'targetName': 'string',
'model': 'string',
'triggerOn': {
'statusCodes': [
'string',
],
'errors': [
'CONNECTION_ERROR'|'FIRST_BYTE_TIMEOUT'|'UNROUTABLE',
]
},
'firstByteTimeoutMilliseconds': 123
},
]
},
'name': 'string',
'weight': 123,
'description': 'string',
'metadata': {
'string': 'string'
}
},
]
}
}
},
],
'description': 'string',
'createdAt': datetime(2015, 1, 1),
'status': 'CREATING'|'ACTIVE'|'UPDATING'|'DELETING',
'system': {
'managedBy': 'string'
},
'updatedAt': datetime(2015, 1, 1)
},
],
'nextToken': 'string'
}
Response Structure
(dict) --
gatewayRules (list) --
The list of gateway rules.
(dict) --
Detailed information about a gateway rule.
ruleId (string) --
The unique identifier of the gateway rule.
gatewayArn (string) --
The Amazon Resource Name (ARN) of the gateway that the rule belongs to.
priority (integer) --
The priority of the rule. Rules are evaluated in order of priority, with lower numbers evaluated first.
conditions (list) --
The conditions that must be met for the rule to apply.
(dict) --
A condition that determines when a gateway rule applies. Conditions can match on principals, request paths, or the requested model.
matchPrincipals (dict) --
A condition that matches on the identity of the caller making the request.
anyOf (list) --
A list of principal entries. The condition is met if any of the entries match the caller's identity.
(dict) --
Union for principal matching. Currently supports IAM principal ARN glob matching.
iamPrincipal (dict) --
An IAM principal to match against, specified by ARN.
arn (string) --
The Amazon Resource Name (ARN) of the IAM principal. Supports user, role, and assumed-role ARNs. Wildcards can be used with the StringLike operator.
operator (string) --
The match operator. StringEquals requires an exact match. StringLike supports wildcard patterns using * and ?.
matchPaths (dict) --
A condition that matches on the request path.
anyOf (list) --
A list of path patterns. The condition is met if the request path matches any of the patterns.
(string) --
matchModels (dict) --
A condition that matches on the model requested in an inference request.
anyOf (list) --
A list of model entries. The condition is met if the requested model matches any of the entries.
(dict) --
A model ID or pattern to match against the requested model, and the operator to use for the comparison.
value (string) --
The model ID or pattern to match. With the StringLike operator, you can use * and ? as wildcards, for example anthropic.claude-*.
operator (string) --
The operator to use when comparing the requested model to the value:
StringEquals – Requires an exact match.
StringLike – Supports wildcard patterns using * and ?. This is the default.
actions (list) --
The actions to take when the rule conditions are met.
(dict) --
An action to take when a gateway rule's conditions are met.
configurationBundle (dict) --
An action that applies a configuration bundle override to the request.
staticOverride (dict) --
A static configuration bundle override that applies a single bundle version to all matching requests.
bundleArn (string) --
The Amazon Resource Name (ARN) of the configuration bundle to apply.
bundleVersion (string) --
The version of the configuration bundle to apply.
weightedOverride (dict) --
A weighted configuration bundle override that splits traffic between multiple bundle versions based on configured weights.
trafficSplit (list) --
The traffic split entries defining how traffic is distributed between configuration bundle versions.
(dict) --
An entry in a traffic split configuration, defining a named variant with a weight and configuration bundle reference.
name (string) --
The name of this traffic split variant.
weight (integer) --
The percentage of traffic to route to this variant. Weights across all entries must sum to 100.
configurationBundle (dict) --
The configuration bundle reference for this variant.
bundleArn (string) --
The Amazon Resource Name (ARN) of the configuration bundle.
bundleVersion (string) --
The version of the configuration bundle.
description (string) --
The description of this traffic split variant.
metadata (dict) --
Key-value metadata associated with this traffic split variant.
(string) --
(string) --
routeToTarget (dict) --
An action that routes the request to a specific target.
staticRoute (dict) --
A static route that sends all matching requests to a single target.
targetName (string) --
The name of the target to route requests to.
model (string) --
The model ID to send to the target. Supported only for inference targets. If you don't specify a model, the gateway sends the model from the original request.
fallback (dict) --
The fallback configuration for this route. Supported only for inference targets.
triggerOn (dict) --
The status codes and errors from the route's target that cause the gateway to send the request to the first fallback target. If you omit triggerOn, the gateway falls back on 5XX status codes and on the CONNECTION_ERROR, FIRST_BYTE_TIMEOUT, and UNROUTABLE errors. If you specify triggerOn, only the status codes and errors that you list trigger a fallback.
statusCodes (list) --
The HTTP status codes returned by the target that trigger a fallback. You can specify the following:
An exact status code – A three-digit code, such as 429.
A class of status codes – The first digit followed by XX, such as 5XX, which matches every code from 500 to 599.
Only 4XX and 5XX codes are supported. Throttling by the gateway itself doesn't trigger a fallback.
(string) --
errors (list) --
The errors that trigger a fallback. You can specify the following:
CONNECTION_ERROR – The gateway couldn't connect to the target because the connection was refused, reset, or timed out. The request didn't reach the target.
FIRST_BYTE_TIMEOUT – The target accepted the connection but didn't return the first byte of the response within firstByteTimeoutMilliseconds. The gateway abandons the request before any of the response reaches you.
UNROUTABLE – The gateway couldn't route the request to the target, for example because the target doesn't exist or doesn't support the request path. These errors don't count toward the circuit breaker.
(string) --
firstByteTimeoutMilliseconds (integer) --
The maximum time, in milliseconds, from sending the request to the route's target until the first byte of the response body arrives. If the time expires, the gateway abandons the request and treats it as a FIRST_BYTE_TIMEOUT error. If you don't specify a value, no timeout is enforced.
circuitBreaker (dict) --
Specifies whether the gateway temporarily stops sending your requests to a failing target. The circuit breaker tracks each combination of target and model separately. With the circuit breaker enabled, the gateway skips one or more targets and models that fail repeatedly, and your requests go to the next target in the chain. The gateway always tries the last remaining target in the chain. The gateway enables the circuit breaker by default.
mode (string) --
Specifies whether you want the circuit breaker turned on. To turn it off, set this value to DISABLED. The default is ENABLED.
targetFallback (list) --
The ordered list of fallback targets. The gateway tries each target in order until a response doesn't match that target's triggers, and returns that response to the caller. If every target fails, the gateway returns the last response. You can specify up to two fallback targets. Each combination of target and model, including the route's target, can appear only once.
(dict) --
Specifies a target to send the request to when the previous target in the fallback chain fails. Includes the target name, the model to send, and the conditions that trigger a fallback from this target to the next one.
targetName (string) --
The name of the gateway target to fall back to. The target must be an inference target.
model (string) --
The model ID to send to this fallback target. If you don't specify a model, the gateway sends the model from the original request.
triggerOn (dict) --
The status codes and errors from this target that cause the gateway to send the request to the next fallback target. This setting isn't inherited from the route. If you omit it, the gateway uses the default triggers: 5XX status codes and the CONNECTION_ERROR, FIRST_BYTE_TIMEOUT, and UNROUTABLE errors.
statusCodes (list) --
The HTTP status codes returned by the target that trigger a fallback. You can specify the following:
An exact status code – A three-digit code, such as 429.
A class of status codes – The first digit followed by XX, such as 5XX, which matches every code from 500 to 599.
Only 4XX and 5XX codes are supported. Throttling by the gateway itself doesn't trigger a fallback.
(string) --
errors (list) --
The errors that trigger a fallback. You can specify the following:
CONNECTION_ERROR – The gateway couldn't connect to the target because the connection was refused, reset, or timed out. The request didn't reach the target.
FIRST_BYTE_TIMEOUT – The target accepted the connection but didn't return the first byte of the response within firstByteTimeoutMilliseconds. The gateway abandons the request before any of the response reaches you.
UNROUTABLE – The gateway couldn't route the request to the target, for example because the target doesn't exist or doesn't support the request path. These errors don't count toward the circuit breaker.
(string) --
firstByteTimeoutMilliseconds (integer) --
The maximum time, in milliseconds, from sending the request to this target until the first byte of the response body arrives. This setting isn't inherited from the route. If you don't specify a value, no timeout is enforced.
weightedRoute (dict) --
A weighted route that splits traffic between multiple targets.
trafficSplit (list) --
The traffic split entries defining how traffic is distributed between targets.
(dict) --
An entry in a target traffic split configuration.
targetName (string) --
The name of the target to route traffic to.
model (string) --
The model ID to send to the target for this variant. Supported only for inference targets. If you don't specify a model, the gateway sends the model from the original request.
fallback (dict) --
The fallback configuration for this variant. Supported only for inference targets.
triggerOn (dict) --
The status codes and errors from the route's target that cause the gateway to send the request to the first fallback target. If you omit triggerOn, the gateway falls back on 5XX status codes and on the CONNECTION_ERROR, FIRST_BYTE_TIMEOUT, and UNROUTABLE errors. If you specify triggerOn, only the status codes and errors that you list trigger a fallback.
statusCodes (list) --
The HTTP status codes returned by the target that trigger a fallback. You can specify the following:
An exact status code – A three-digit code, such as 429.
A class of status codes – The first digit followed by XX, such as 5XX, which matches every code from 500 to 599.
Only 4XX and 5XX codes are supported. Throttling by the gateway itself doesn't trigger a fallback.
(string) --
errors (list) --
The errors that trigger a fallback. You can specify the following:
CONNECTION_ERROR – The gateway couldn't connect to the target because the connection was refused, reset, or timed out. The request didn't reach the target.
FIRST_BYTE_TIMEOUT – The target accepted the connection but didn't return the first byte of the response within firstByteTimeoutMilliseconds. The gateway abandons the request before any of the response reaches you.
UNROUTABLE – The gateway couldn't route the request to the target, for example because the target doesn't exist or doesn't support the request path. These errors don't count toward the circuit breaker.
(string) --
firstByteTimeoutMilliseconds (integer) --
The maximum time, in milliseconds, from sending the request to the route's target until the first byte of the response body arrives. If the time expires, the gateway abandons the request and treats it as a FIRST_BYTE_TIMEOUT error. If you don't specify a value, no timeout is enforced.
circuitBreaker (dict) --
Specifies whether the gateway temporarily stops sending your requests to a failing target. The circuit breaker tracks each combination of target and model separately. With the circuit breaker enabled, the gateway skips one or more targets and models that fail repeatedly, and your requests go to the next target in the chain. The gateway always tries the last remaining target in the chain. The gateway enables the circuit breaker by default.
mode (string) --
Specifies whether you want the circuit breaker turned on. To turn it off, set this value to DISABLED. The default is ENABLED.
targetFallback (list) --
The ordered list of fallback targets. The gateway tries each target in order until a response doesn't match that target's triggers, and returns that response to the caller. If every target fails, the gateway returns the last response. You can specify up to two fallback targets. Each combination of target and model, including the route's target, can appear only once.
(dict) --
Specifies a target to send the request to when the previous target in the fallback chain fails. Includes the target name, the model to send, and the conditions that trigger a fallback from this target to the next one.
targetName (string) --
The name of the gateway target to fall back to. The target must be an inference target.
model (string) --
The model ID to send to this fallback target. If you don't specify a model, the gateway sends the model from the original request.
triggerOn (dict) --
The status codes and errors from this target that cause the gateway to send the request to the next fallback target. This setting isn't inherited from the route. If you omit it, the gateway uses the default triggers: 5XX status codes and the CONNECTION_ERROR, FIRST_BYTE_TIMEOUT, and UNROUTABLE errors.
statusCodes (list) --
The HTTP status codes returned by the target that trigger a fallback. You can specify the following:
An exact status code – A three-digit code, such as 429.
A class of status codes – The first digit followed by XX, such as 5XX, which matches every code from 500 to 599.
Only 4XX and 5XX codes are supported. Throttling by the gateway itself doesn't trigger a fallback.
(string) --
errors (list) --
The errors that trigger a fallback. You can specify the following:
CONNECTION_ERROR – The gateway couldn't connect to the target because the connection was refused, reset, or timed out. The request didn't reach the target.
FIRST_BYTE_TIMEOUT – The target accepted the connection but didn't return the first byte of the response within firstByteTimeoutMilliseconds. The gateway abandons the request before any of the response reaches you.
UNROUTABLE – The gateway couldn't route the request to the target, for example because the target doesn't exist or doesn't support the request path. These errors don't count toward the circuit breaker.
(string) --
firstByteTimeoutMilliseconds (integer) --
The maximum time, in milliseconds, from sending the request to this target until the first byte of the response body arrives. This setting isn't inherited from the route. If you don't specify a value, no timeout is enforced.
name (string) --
The name of this traffic split variant.
weight (integer) --
The percentage of traffic to route to this variant.
description (string) --
The description of this traffic split variant.
metadata (dict) --
Key-value metadata associated with this traffic split variant.
(string) --
(string) --
description (string) --
The description of the gateway rule.
createdAt (datetime) --
The timestamp when the rule was created.
status (string) --
The current status of the rule.
system (dict) --
System-managed metadata for rules created by automated processes.
managedBy (string) --
The identifier of the system or process that manages this rule.
updatedAt (datetime) --
The timestamp when the rule was last updated.
nextToken (string) --
The pagination token to use in a subsequent request.
{'evaluatorConfig': {'llmAsAJudge': {'ratingScale': {'multiDimensional': [{'dimensionInstruction': 'string',
'dimensionName': 'string',
'dimensionRatingScale': {'categorical': [{'definition': 'string',
'label': 'string'}],
'numerical': [{'definition': 'string',
'label': 'string',
'value': 'double'}]}}]}}}}
Updates a custom evaluator's configuration, description, or evaluation level. Built-in evaluators cannot be updated. The evaluator must not be locked for modification.
See also: AWS API Documentation
Request Syntax
client.update_evaluator(
clientToken='string',
evaluatorId='string',
description='string',
evaluatorConfig={
'llmAsAJudge': {
'instructions': 'string',
'ratingScale': {
'numerical': [
{
'definition': 'string',
'value': 123.0,
'label': 'string'
},
],
'categorical': [
{
'definition': 'string',
'label': 'string'
},
],
'multiDimensional': [
{
'dimensionName': 'string',
'dimensionInstruction': 'string',
'dimensionRatingScale': {
'numerical': [
{
'definition': 'string',
'value': 123.0,
'label': 'string'
},
],
'categorical': [
{
'definition': 'string',
'label': 'string'
},
]
}
},
]
},
'modelConfig': {
'bedrockEvaluatorModelConfig': {
'modelId': 'string',
'inferenceConfig': {
'maxTokens': 123,
'temperature': ...,
'topP': ...,
'stopSequences': [
'string',
]
},
'additionalModelRequestFields': {...}|[...]|123|123.4|'string'|True|None
},
'responsesEvaluatorModelConfig': {
'modelId': 'string',
'maxOutputTokens': 123,
'temperature': ...,
'topP': ...,
'reasoning': {
'effort': 'string'
}
}
}
},
'codeBased': {
'lambdaConfig': {
'lambdaArn': 'string',
'lambdaTimeoutInSeconds': 123
}
},
'derived': {
'baseEvaluatorId': 'string',
'modelConfig': {
'bedrockEvaluatorModelConfig': {
'modelId': 'string',
'inferenceConfig': {
'maxTokens': 123,
'temperature': ...,
'topP': ...,
'stopSequences': [
'string',
]
},
'additionalModelRequestFields': {...}|[...]|123|123.4|'string'|True|None
},
'responsesEvaluatorModelConfig': {
'modelId': 'string',
'maxOutputTokens': 123,
'temperature': ...,
'topP': ...,
'reasoning': {
'effort': 'string'
}
}
}
}
},
level='TOOL_CALL'|'TRACE'|'SESSION',
kmsKeyArn='string'
)
string
A unique, case-sensitive identifier to ensure that the API request completes no more than one time. If you don't specify this field, a value is randomly generated for you. If this token matches a previous request, the service ignores the request, but doesn't return an error. For more information, see Ensuring idempotency.
This field is autopopulated if not provided.
string
[REQUIRED]
The unique identifier of the evaluator to update.
string
The updated description of the evaluator.
dict
The updated configuration for the evaluator. Specify either LLM-as-a-Judge settings with instructions, rating scale, and model configuration, or code-based settings with a customer-managed Lambda function.
llmAsAJudge (dict) --
The LLM-as-a-Judge configuration that uses a language model to evaluate agent performance based on custom instructions and rating scales.
instructions (string) -- [REQUIRED]
The evaluation instructions that guide the language model in assessing agent performance, including criteria and evaluation guidelines.
ratingScale (dict) -- [REQUIRED]
The rating scale that defines how the evaluator should score agent performance, either numerical or categorical.
numerical (list) --
The numerical rating scale with defined score values and descriptions for quantitative evaluation.
(dict) --
The definition of a numerical rating scale option that provides a numeric value with its description for evaluation scoring.
definition (string) -- [REQUIRED]
The description that explains what this numerical rating represents and when it should be used.
value (float) -- [REQUIRED]
The numerical value for this rating scale option.
label (string) -- [REQUIRED]
The label or name that describes this numerical rating option.
categorical (list) --
The categorical rating scale with named categories and definitions for qualitative evaluation.
(dict) --
The definition of a categorical rating scale option that provides a named category with its description for evaluation scoring.
definition (string) -- [REQUIRED]
The description that explains what this categorical rating represents and when it should be used.
label (string) -- [REQUIRED]
The label or name of this categorical rating option.
multiDimensional (list) --
A list of dimensions that lets an LLM-as-a-judge evaluator score several dimensions of the same target in a single evaluation. Each dimension has its own instruction and rating scale, and produces its own result.
(dict) --
A single dimension of an LLM-as-a-judge evaluator, defining what the evaluator model should judge for that dimension and how it should be scored.
dimensionName (string) -- [REQUIRED]
The name of the dimension. This name is unique within the evaluator and identifies the dimension's results.
dimensionInstruction (string) -- [REQUIRED]
The instruction to the evaluator model for this dimension, describing what the model should judge when scoring the dimension.
dimensionRatingScale (dict) -- [REQUIRED]
The rating scale for this dimension, which can be numerical or categorical. A dimension's rating scale cannot itself be multi-dimensional.
numerical (list) --
The numerical rating scale with defined score values and descriptions for quantitative evaluation of the dimension.
(dict) --
The definition of a numerical rating scale option that provides a numeric value with its description for evaluation scoring.
definition (string) -- [REQUIRED]
The description that explains what this numerical rating represents and when it should be used.
value (float) -- [REQUIRED]
The numerical value for this rating scale option.
label (string) -- [REQUIRED]
The label or name that describes this numerical rating option.
categorical (list) --
The categorical rating scale with named categories and definitions for qualitative evaluation of the dimension.
(dict) --
The definition of a categorical rating scale option that provides a named category with its description for evaluation scoring.
definition (string) -- [REQUIRED]
The description that explains what this categorical rating represents and when it should be used.
label (string) -- [REQUIRED]
The label or name of this categorical rating option.
modelConfig (dict) -- [REQUIRED]
The model configuration that specifies which foundation model to use and how to configure it for evaluation.
bedrockEvaluatorModelConfig (dict) --
The Amazon Bedrock model configuration for evaluation.
modelId (string) -- [REQUIRED]
The identifier of the Amazon Bedrock model to use for evaluation. Must be a supported foundation model available in your region.
inferenceConfig (dict) --
The inference configuration parameters that control model behavior during evaluation, including temperature, token limits, and sampling settings.
maxTokens (integer) --
The maximum number of tokens to generate in the model response during evaluation.
temperature (float) --
The temperature value that controls randomness in the model's responses. Lower values produce more deterministic outputs.
topP (float) --
The top-p sampling parameter that controls the diversity of the model's responses by limiting the cumulative probability of token choices.
stopSequences (list) --
The list of sequences that will cause the model to stop generating tokens when encountered.
(string) --
additionalModelRequestFields (:ref:`document<document>`) --
Additional model-specific request fields to customize model behavior beyond the standard inference configuration.
responsesEvaluatorModelConfig (dict) --
The OpenResponses model configuration for evaluation.
modelId (string) -- [REQUIRED]
The identifier of the model to use for evaluation.
maxOutputTokens (integer) --
The maximum number of tokens to generate in the model response, including visible output and reasoning tokens.
temperature (float) --
The temperature value that controls randomness in the model's responses. Lower values produce more deterministic outputs.
topP (float) --
The top-p sampling parameter that controls the diversity of the model's responses by limiting the cumulative probability of token choices.
reasoning (dict) --
The reasoning configuration for reasoning models. Non-reasoning models ignore this configuration.
effort (string) --
The level of reasoning effort the model applies when generating a response. For supported values, see the model provider's documentation.
codeBased (dict) --
Configuration for a code-based evaluator that uses a customer-managed Lambda function to programmatically assess agent performance.
lambdaConfig (dict) --
The Lambda function configuration for code-based evaluation.
lambdaArn (string) -- [REQUIRED]
The Amazon Resource Name (ARN) of the Lambda function that implements the evaluation logic.
lambdaTimeoutInSeconds (integer) --
The timeout in seconds for the Lambda function invocation. Defaults to 60. Must be between 1 and 300.
derived (dict) --
The configuration for an evaluator derived from an existing base evaluator (a built-in or third-party evaluator), run on your own model. The base evaluator supplies the prompt and scoring.
baseEvaluatorId (string) -- [REQUIRED]
The identifier of the base evaluator whose logic to run (a Builtin.* or ThirdParty.* evaluator).
modelConfig (dict) -- [REQUIRED]
The configuration of the evaluator model that you supply.
bedrockEvaluatorModelConfig (dict) --
The Amazon Bedrock model configuration for evaluation.
modelId (string) -- [REQUIRED]
The identifier of the Amazon Bedrock model to use for evaluation. Must be a supported foundation model available in your region.
inferenceConfig (dict) --
The inference configuration parameters that control model behavior during evaluation, including temperature, token limits, and sampling settings.
maxTokens (integer) --
The maximum number of tokens to generate in the model response during evaluation.
temperature (float) --
The temperature value that controls randomness in the model's responses. Lower values produce more deterministic outputs.
topP (float) --
The top-p sampling parameter that controls the diversity of the model's responses by limiting the cumulative probability of token choices.
stopSequences (list) --
The list of sequences that will cause the model to stop generating tokens when encountered.
(string) --
additionalModelRequestFields (:ref:`document<document>`) --
Additional model-specific request fields to customize model behavior beyond the standard inference configuration.
responsesEvaluatorModelConfig (dict) --
The OpenResponses model configuration for evaluation.
modelId (string) -- [REQUIRED]
The identifier of the model to use for evaluation.
maxOutputTokens (integer) --
The maximum number of tokens to generate in the model response, including visible output and reasoning tokens.
temperature (float) --
The temperature value that controls randomness in the model's responses. Lower values produce more deterministic outputs.
topP (float) --
The top-p sampling parameter that controls the diversity of the model's responses by limiting the cumulative probability of token choices.
reasoning (dict) --
The reasoning configuration for reasoning models. Non-reasoning models ignore this configuration.
effort (string) --
The level of reasoning effort the model applies when generating a response. For supported values, see the model provider's documentation.
string
The updated evaluation level ( TOOL_CALL, TRACE, or SESSION) that determines the scope of evaluation.
string
The Amazon Resource Name (ARN) of a customer managed KMS key to use for encrypting sensitive evaluator data. Specify a new key ARN to rotate the encryption key, or specify a key ARN to add encryption to an evaluator that was previously created without one. When you rotate to a new key, the service decrypts the existing data with the old key and re-encrypts it with the new key. Only symmetric encryption KMS keys are supported. For more information, see Encryption at rest for AgentCore Evaluations.
dict
Response Syntax
{
'evaluatorArn': 'string',
'evaluatorId': 'string',
'updatedAt': datetime(2015, 1, 1),
'status': 'ACTIVE'|'CREATING'|'CREATE_FAILED'|'UPDATING'|'UPDATE_FAILED'|'DELETING'
}
Response Structure
(dict) --
evaluatorArn (string) --
The Amazon Resource Name (ARN) of the updated evaluator.
evaluatorId (string) --
The unique identifier of the updated evaluator.
updatedAt (datetime) --
The timestamp when the evaluator was last updated.
status (string) --
The status of the evaluator update operation.
{'actions': {'routeToTarget': {'staticRoute': {'fallback': {'circuitBreaker': {'mode': 'ENABLED '
'| '
'DISABLED'},
'firstByteTimeoutMilliseconds': 'integer',
'targetFallback': [{'firstByteTimeoutMilliseconds': 'integer',
'model': 'string',
'targetName': 'string',
'triggerOn': {'errors': ['CONNECTION_ERROR '
'| '
'FIRST_BYTE_TIMEOUT '
'| '
'UNROUTABLE'],
'statusCodes': ['string']}}],
'triggerOn': {'errors': ['CONNECTION_ERROR '
'| '
'FIRST_BYTE_TIMEOUT '
'| '
'UNROUTABLE'],
'statusCodes': ['string']}},
'model': 'string'},
'weightedRoute': {'trafficSplit': {'fallback': {'circuitBreaker': {'mode': 'ENABLED '
'| '
'DISABLED'},
'firstByteTimeoutMilliseconds': 'integer',
'targetFallback': [{'firstByteTimeoutMilliseconds': 'integer',
'model': 'string',
'targetName': 'string',
'triggerOn': {'errors': ['CONNECTION_ERROR '
'| '
'FIRST_BYTE_TIMEOUT '
'| '
'UNROUTABLE'],
'statusCodes': ['string']}}],
'triggerOn': {'errors': ['CONNECTION_ERROR '
'| '
'FIRST_BYTE_TIMEOUT '
'| '
'UNROUTABLE'],
'statusCodes': ['string']}},
'model': 'string'}}}},
'conditions': {'matchModels': {'anyOf': [{'operator': 'StringEquals | '
'StringLike',
'value': 'string'}]}}}
Updates a gateway rule's priority, conditions, actions, or description.
See also: AWS API Documentation
Request Syntax
client.update_gateway_rule(
gatewayIdentifier='string',
ruleId='string',
priority=123,
conditions=[
{
'matchPrincipals': {
'anyOf': [
{
'iamPrincipal': {
'arn': 'string',
'operator': 'StringEquals'|'StringLike'
}
},
]
},
'matchPaths': {
'anyOf': [
'string',
]
},
'matchModels': {
'anyOf': [
{
'value': 'string',
'operator': 'StringEquals'|'StringLike'
},
]
}
},
],
actions=[
{
'configurationBundle': {
'staticOverride': {
'bundleArn': 'string',
'bundleVersion': 'string'
},
'weightedOverride': {
'trafficSplit': [
{
'name': 'string',
'weight': 123,
'configurationBundle': {
'bundleArn': 'string',
'bundleVersion': 'string'
},
'description': 'string',
'metadata': {
'string': 'string'
}
},
]
}
},
'routeToTarget': {
'staticRoute': {
'targetName': 'string',
'model': 'string',
'fallback': {
'triggerOn': {
'statusCodes': [
'string',
],
'errors': [
'CONNECTION_ERROR'|'FIRST_BYTE_TIMEOUT'|'UNROUTABLE',
]
},
'firstByteTimeoutMilliseconds': 123,
'circuitBreaker': {
'mode': 'ENABLED'|'DISABLED'
},
'targetFallback': [
{
'targetName': 'string',
'model': 'string',
'triggerOn': {
'statusCodes': [
'string',
],
'errors': [
'CONNECTION_ERROR'|'FIRST_BYTE_TIMEOUT'|'UNROUTABLE',
]
},
'firstByteTimeoutMilliseconds': 123
},
]
}
},
'weightedRoute': {
'trafficSplit': [
{
'targetName': 'string',
'model': 'string',
'fallback': {
'triggerOn': {
'statusCodes': [
'string',
],
'errors': [
'CONNECTION_ERROR'|'FIRST_BYTE_TIMEOUT'|'UNROUTABLE',
]
},
'firstByteTimeoutMilliseconds': 123,
'circuitBreaker': {
'mode': 'ENABLED'|'DISABLED'
},
'targetFallback': [
{
'targetName': 'string',
'model': 'string',
'triggerOn': {
'statusCodes': [
'string',
],
'errors': [
'CONNECTION_ERROR'|'FIRST_BYTE_TIMEOUT'|'UNROUTABLE',
]
},
'firstByteTimeoutMilliseconds': 123
},
]
},
'name': 'string',
'weight': 123,
'description': 'string',
'metadata': {
'string': 'string'
}
},
]
}
}
},
],
description='string'
)
string
[REQUIRED]
The identifier of the gateway containing the rule.
string
[REQUIRED]
The unique identifier of the rule to update.
integer
The updated priority of the rule.
list
The updated conditions for the rule.
(dict) --
A condition that determines when a gateway rule applies. Conditions can match on principals, request paths, or the requested model.
matchPrincipals (dict) --
A condition that matches on the identity of the caller making the request.
anyOf (list) -- [REQUIRED]
A list of principal entries. The condition is met if any of the entries match the caller's identity.
(dict) --
Union for principal matching. Currently supports IAM principal ARN glob matching.
iamPrincipal (dict) --
An IAM principal to match against, specified by ARN.
arn (string) -- [REQUIRED]
The Amazon Resource Name (ARN) of the IAM principal. Supports user, role, and assumed-role ARNs. Wildcards can be used with the StringLike operator.
operator (string) --
The match operator. StringEquals requires an exact match. StringLike supports wildcard patterns using * and ?.
matchPaths (dict) --
A condition that matches on the request path.
anyOf (list) -- [REQUIRED]
A list of path patterns. The condition is met if the request path matches any of the patterns.
(string) --
matchModels (dict) --
A condition that matches on the model requested in an inference request.
anyOf (list) -- [REQUIRED]
A list of model entries. The condition is met if the requested model matches any of the entries.
(dict) --
A model ID or pattern to match against the requested model, and the operator to use for the comparison.
value (string) -- [REQUIRED]
The model ID or pattern to match. With the StringLike operator, you can use * and ? as wildcards, for example anthropic.claude-*.
operator (string) --
The operator to use when comparing the requested model to the value:
StringEquals – Requires an exact match.
StringLike – Supports wildcard patterns using * and ?. This is the default.
list
The updated actions for the rule.
(dict) --
An action to take when a gateway rule's conditions are met.
configurationBundle (dict) --
An action that applies a configuration bundle override to the request.
staticOverride (dict) --
A static configuration bundle override that applies a single bundle version to all matching requests.
bundleArn (string) -- [REQUIRED]
The Amazon Resource Name (ARN) of the configuration bundle to apply.
bundleVersion (string) -- [REQUIRED]
The version of the configuration bundle to apply.
weightedOverride (dict) --
A weighted configuration bundle override that splits traffic between multiple bundle versions based on configured weights.
trafficSplit (list) -- [REQUIRED]
The traffic split entries defining how traffic is distributed between configuration bundle versions.
(dict) --
An entry in a traffic split configuration, defining a named variant with a weight and configuration bundle reference.
name (string) -- [REQUIRED]
The name of this traffic split variant.
weight (integer) -- [REQUIRED]
The percentage of traffic to route to this variant. Weights across all entries must sum to 100.
configurationBundle (dict) -- [REQUIRED]
The configuration bundle reference for this variant.
bundleArn (string) -- [REQUIRED]
The Amazon Resource Name (ARN) of the configuration bundle.
bundleVersion (string) -- [REQUIRED]
The version of the configuration bundle.
description (string) --
The description of this traffic split variant.
metadata (dict) --
Key-value metadata associated with this traffic split variant.
(string) --
(string) --
routeToTarget (dict) --
An action that routes the request to a specific target.
staticRoute (dict) --
A static route that sends all matching requests to a single target.
targetName (string) -- [REQUIRED]
The name of the target to route requests to.
model (string) --
The model ID to send to the target. Supported only for inference targets. If you don't specify a model, the gateway sends the model from the original request.
fallback (dict) --
The fallback configuration for this route. Supported only for inference targets.
triggerOn (dict) --
The status codes and errors from the route's target that cause the gateway to send the request to the first fallback target. If you omit triggerOn, the gateway falls back on 5XX status codes and on the CONNECTION_ERROR, FIRST_BYTE_TIMEOUT, and UNROUTABLE errors. If you specify triggerOn, only the status codes and errors that you list trigger a fallback.
statusCodes (list) --
The HTTP status codes returned by the target that trigger a fallback. You can specify the following:
An exact status code – A three-digit code, such as 429.
A class of status codes – The first digit followed by XX, such as 5XX, which matches every code from 500 to 599.
Only 4XX and 5XX codes are supported. Throttling by the gateway itself doesn't trigger a fallback.
(string) --
errors (list) --
The errors that trigger a fallback. You can specify the following:
CONNECTION_ERROR – The gateway couldn't connect to the target because the connection was refused, reset, or timed out. The request didn't reach the target.
FIRST_BYTE_TIMEOUT – The target accepted the connection but didn't return the first byte of the response within firstByteTimeoutMilliseconds. The gateway abandons the request before any of the response reaches you.
UNROUTABLE – The gateway couldn't route the request to the target, for example because the target doesn't exist or doesn't support the request path. These errors don't count toward the circuit breaker.
(string) --
firstByteTimeoutMilliseconds (integer) --
The maximum time, in milliseconds, from sending the request to the route's target until the first byte of the response body arrives. If the time expires, the gateway abandons the request and treats it as a FIRST_BYTE_TIMEOUT error. If you don't specify a value, no timeout is enforced.
circuitBreaker (dict) --
Specifies whether the gateway temporarily stops sending your requests to a failing target. The circuit breaker tracks each combination of target and model separately. With the circuit breaker enabled, the gateway skips one or more targets and models that fail repeatedly, and your requests go to the next target in the chain. The gateway always tries the last remaining target in the chain. The gateway enables the circuit breaker by default.
mode (string) --
Specifies whether you want the circuit breaker turned on. To turn it off, set this value to DISABLED. The default is ENABLED.
targetFallback (list) -- [REQUIRED]
The ordered list of fallback targets. The gateway tries each target in order until a response doesn't match that target's triggers, and returns that response to the caller. If every target fails, the gateway returns the last response. You can specify up to two fallback targets. Each combination of target and model, including the route's target, can appear only once.
(dict) --
Specifies a target to send the request to when the previous target in the fallback chain fails. Includes the target name, the model to send, and the conditions that trigger a fallback from this target to the next one.
targetName (string) -- [REQUIRED]
The name of the gateway target to fall back to. The target must be an inference target.
model (string) --
The model ID to send to this fallback target. If you don't specify a model, the gateway sends the model from the original request.
triggerOn (dict) --
The status codes and errors from this target that cause the gateway to send the request to the next fallback target. This setting isn't inherited from the route. If you omit it, the gateway uses the default triggers: 5XX status codes and the CONNECTION_ERROR, FIRST_BYTE_TIMEOUT, and UNROUTABLE errors.
statusCodes (list) --
The HTTP status codes returned by the target that trigger a fallback. You can specify the following:
An exact status code – A three-digit code, such as 429.
A class of status codes – The first digit followed by XX, such as 5XX, which matches every code from 500 to 599.
Only 4XX and 5XX codes are supported. Throttling by the gateway itself doesn't trigger a fallback.
(string) --
errors (list) --
The errors that trigger a fallback. You can specify the following:
CONNECTION_ERROR – The gateway couldn't connect to the target because the connection was refused, reset, or timed out. The request didn't reach the target.
FIRST_BYTE_TIMEOUT – The target accepted the connection but didn't return the first byte of the response within firstByteTimeoutMilliseconds. The gateway abandons the request before any of the response reaches you.
UNROUTABLE – The gateway couldn't route the request to the target, for example because the target doesn't exist or doesn't support the request path. These errors don't count toward the circuit breaker.
(string) --
firstByteTimeoutMilliseconds (integer) --
The maximum time, in milliseconds, from sending the request to this target until the first byte of the response body arrives. This setting isn't inherited from the route. If you don't specify a value, no timeout is enforced.
weightedRoute (dict) --
A weighted route that splits traffic between multiple targets.
trafficSplit (list) -- [REQUIRED]
The traffic split entries defining how traffic is distributed between targets.
(dict) --
An entry in a target traffic split configuration.
targetName (string) -- [REQUIRED]
The name of the target to route traffic to.
model (string) --
The model ID to send to the target for this variant. Supported only for inference targets. If you don't specify a model, the gateway sends the model from the original request.
fallback (dict) --
The fallback configuration for this variant. Supported only for inference targets.
triggerOn (dict) --
The status codes and errors from the route's target that cause the gateway to send the request to the first fallback target. If you omit triggerOn, the gateway falls back on 5XX status codes and on the CONNECTION_ERROR, FIRST_BYTE_TIMEOUT, and UNROUTABLE errors. If you specify triggerOn, only the status codes and errors that you list trigger a fallback.
statusCodes (list) --
The HTTP status codes returned by the target that trigger a fallback. You can specify the following:
An exact status code – A three-digit code, such as 429.
A class of status codes – The first digit followed by XX, such as 5XX, which matches every code from 500 to 599.
Only 4XX and 5XX codes are supported. Throttling by the gateway itself doesn't trigger a fallback.
(string) --
errors (list) --
The errors that trigger a fallback. You can specify the following:
CONNECTION_ERROR – The gateway couldn't connect to the target because the connection was refused, reset, or timed out. The request didn't reach the target.
FIRST_BYTE_TIMEOUT – The target accepted the connection but didn't return the first byte of the response within firstByteTimeoutMilliseconds. The gateway abandons the request before any of the response reaches you.
UNROUTABLE – The gateway couldn't route the request to the target, for example because the target doesn't exist or doesn't support the request path. These errors don't count toward the circuit breaker.
(string) --
firstByteTimeoutMilliseconds (integer) --
The maximum time, in milliseconds, from sending the request to the route's target until the first byte of the response body arrives. If the time expires, the gateway abandons the request and treats it as a FIRST_BYTE_TIMEOUT error. If you don't specify a value, no timeout is enforced.
circuitBreaker (dict) --
Specifies whether the gateway temporarily stops sending your requests to a failing target. The circuit breaker tracks each combination of target and model separately. With the circuit breaker enabled, the gateway skips one or more targets and models that fail repeatedly, and your requests go to the next target in the chain. The gateway always tries the last remaining target in the chain. The gateway enables the circuit breaker by default.
mode (string) --
Specifies whether you want the circuit breaker turned on. To turn it off, set this value to DISABLED. The default is ENABLED.
targetFallback (list) -- [REQUIRED]
The ordered list of fallback targets. The gateway tries each target in order until a response doesn't match that target's triggers, and returns that response to the caller. If every target fails, the gateway returns the last response. You can specify up to two fallback targets. Each combination of target and model, including the route's target, can appear only once.
(dict) --
Specifies a target to send the request to when the previous target in the fallback chain fails. Includes the target name, the model to send, and the conditions that trigger a fallback from this target to the next one.
targetName (string) -- [REQUIRED]
The name of the gateway target to fall back to. The target must be an inference target.
model (string) --
The model ID to send to this fallback target. If you don't specify a model, the gateway sends the model from the original request.
triggerOn (dict) --
The status codes and errors from this target that cause the gateway to send the request to the next fallback target. This setting isn't inherited from the route. If you omit it, the gateway uses the default triggers: 5XX status codes and the CONNECTION_ERROR, FIRST_BYTE_TIMEOUT, and UNROUTABLE errors.
statusCodes (list) --
The HTTP status codes returned by the target that trigger a fallback. You can specify the following:
An exact status code – A three-digit code, such as 429.
A class of status codes – The first digit followed by XX, such as 5XX, which matches every code from 500 to 599.
Only 4XX and 5XX codes are supported. Throttling by the gateway itself doesn't trigger a fallback.
(string) --
errors (list) --
The errors that trigger a fallback. You can specify the following:
CONNECTION_ERROR – The gateway couldn't connect to the target because the connection was refused, reset, or timed out. The request didn't reach the target.
FIRST_BYTE_TIMEOUT – The target accepted the connection but didn't return the first byte of the response within firstByteTimeoutMilliseconds. The gateway abandons the request before any of the response reaches you.
UNROUTABLE – The gateway couldn't route the request to the target, for example because the target doesn't exist or doesn't support the request path. These errors don't count toward the circuit breaker.
(string) --
firstByteTimeoutMilliseconds (integer) --
The maximum time, in milliseconds, from sending the request to this target until the first byte of the response body arrives. This setting isn't inherited from the route. If you don't specify a value, no timeout is enforced.
name (string) -- [REQUIRED]
The name of this traffic split variant.
weight (integer) -- [REQUIRED]
The percentage of traffic to route to this variant.
description (string) --
The description of this traffic split variant.
metadata (dict) --
Key-value metadata associated with this traffic split variant.
(string) --
(string) --
string
The updated description of the rule.
dict
Response Syntax
{
'ruleId': 'string',
'gatewayArn': 'string',
'priority': 123,
'conditions': [
{
'matchPrincipals': {
'anyOf': [
{
'iamPrincipal': {
'arn': 'string',
'operator': 'StringEquals'|'StringLike'
}
},
]
},
'matchPaths': {
'anyOf': [
'string',
]
},
'matchModels': {
'anyOf': [
{
'value': 'string',
'operator': 'StringEquals'|'StringLike'
},
]
}
},
],
'actions': [
{
'configurationBundle': {
'staticOverride': {
'bundleArn': 'string',
'bundleVersion': 'string'
},
'weightedOverride': {
'trafficSplit': [
{
'name': 'string',
'weight': 123,
'configurationBundle': {
'bundleArn': 'string',
'bundleVersion': 'string'
},
'description': 'string',
'metadata': {
'string': 'string'
}
},
]
}
},
'routeToTarget': {
'staticRoute': {
'targetName': 'string',
'model': 'string',
'fallback': {
'triggerOn': {
'statusCodes': [
'string',
],
'errors': [
'CONNECTION_ERROR'|'FIRST_BYTE_TIMEOUT'|'UNROUTABLE',
]
},
'firstByteTimeoutMilliseconds': 123,
'circuitBreaker': {
'mode': 'ENABLED'|'DISABLED'
},
'targetFallback': [
{
'targetName': 'string',
'model': 'string',
'triggerOn': {
'statusCodes': [
'string',
],
'errors': [
'CONNECTION_ERROR'|'FIRST_BYTE_TIMEOUT'|'UNROUTABLE',
]
},
'firstByteTimeoutMilliseconds': 123
},
]
}
},
'weightedRoute': {
'trafficSplit': [
{
'targetName': 'string',
'model': 'string',
'fallback': {
'triggerOn': {
'statusCodes': [
'string',
],
'errors': [
'CONNECTION_ERROR'|'FIRST_BYTE_TIMEOUT'|'UNROUTABLE',
]
},
'firstByteTimeoutMilliseconds': 123,
'circuitBreaker': {
'mode': 'ENABLED'|'DISABLED'
},
'targetFallback': [
{
'targetName': 'string',
'model': 'string',
'triggerOn': {
'statusCodes': [
'string',
],
'errors': [
'CONNECTION_ERROR'|'FIRST_BYTE_TIMEOUT'|'UNROUTABLE',
]
},
'firstByteTimeoutMilliseconds': 123
},
]
},
'name': 'string',
'weight': 123,
'description': 'string',
'metadata': {
'string': 'string'
}
},
]
}
}
},
],
'description': 'string',
'createdAt': datetime(2015, 1, 1),
'status': 'CREATING'|'ACTIVE'|'UPDATING'|'DELETING',
'system': {
'managedBy': 'string'
},
'updatedAt': datetime(2015, 1, 1)
}
Response Structure
(dict) --
Create response excludes updatedAt (redundant on create). Get/Update responses include it via their own output structures.
ruleId (string) --
The unique identifier of the gateway rule.
gatewayArn (string) --
The Amazon Resource Name (ARN) of the gateway that the rule belongs to.
priority (integer) --
The priority of the rule. Rules are evaluated in order of priority, with lower numbers evaluated first.
conditions (list) --
The conditions that must be met for the rule to apply.
(dict) --
A condition that determines when a gateway rule applies. Conditions can match on principals, request paths, or the requested model.
matchPrincipals (dict) --
A condition that matches on the identity of the caller making the request.
anyOf (list) --
A list of principal entries. The condition is met if any of the entries match the caller's identity.
(dict) --
Union for principal matching. Currently supports IAM principal ARN glob matching.
iamPrincipal (dict) --
An IAM principal to match against, specified by ARN.
arn (string) --
The Amazon Resource Name (ARN) of the IAM principal. Supports user, role, and assumed-role ARNs. Wildcards can be used with the StringLike operator.
operator (string) --
The match operator. StringEquals requires an exact match. StringLike supports wildcard patterns using * and ?.
matchPaths (dict) --
A condition that matches on the request path.
anyOf (list) --
A list of path patterns. The condition is met if the request path matches any of the patterns.
(string) --
matchModels (dict) --
A condition that matches on the model requested in an inference request.
anyOf (list) --
A list of model entries. The condition is met if the requested model matches any of the entries.
(dict) --
A model ID or pattern to match against the requested model, and the operator to use for the comparison.
value (string) --
The model ID or pattern to match. With the StringLike operator, you can use * and ? as wildcards, for example anthropic.claude-*.
operator (string) --
The operator to use when comparing the requested model to the value:
StringEquals – Requires an exact match.
StringLike – Supports wildcard patterns using * and ?. This is the default.
actions (list) --
The actions to take when the rule conditions are met.
(dict) --
An action to take when a gateway rule's conditions are met.
configurationBundle (dict) --
An action that applies a configuration bundle override to the request.
staticOverride (dict) --
A static configuration bundle override that applies a single bundle version to all matching requests.
bundleArn (string) --
The Amazon Resource Name (ARN) of the configuration bundle to apply.
bundleVersion (string) --
The version of the configuration bundle to apply.
weightedOverride (dict) --
A weighted configuration bundle override that splits traffic between multiple bundle versions based on configured weights.
trafficSplit (list) --
The traffic split entries defining how traffic is distributed between configuration bundle versions.
(dict) --
An entry in a traffic split configuration, defining a named variant with a weight and configuration bundle reference.
name (string) --
The name of this traffic split variant.
weight (integer) --
The percentage of traffic to route to this variant. Weights across all entries must sum to 100.
configurationBundle (dict) --
The configuration bundle reference for this variant.
bundleArn (string) --
The Amazon Resource Name (ARN) of the configuration bundle.
bundleVersion (string) --
The version of the configuration bundle.
description (string) --
The description of this traffic split variant.
metadata (dict) --
Key-value metadata associated with this traffic split variant.
(string) --
(string) --
routeToTarget (dict) --
An action that routes the request to a specific target.
staticRoute (dict) --
A static route that sends all matching requests to a single target.
targetName (string) --
The name of the target to route requests to.
model (string) --
The model ID to send to the target. Supported only for inference targets. If you don't specify a model, the gateway sends the model from the original request.
fallback (dict) --
The fallback configuration for this route. Supported only for inference targets.
triggerOn (dict) --
The status codes and errors from the route's target that cause the gateway to send the request to the first fallback target. If you omit triggerOn, the gateway falls back on 5XX status codes and on the CONNECTION_ERROR, FIRST_BYTE_TIMEOUT, and UNROUTABLE errors. If you specify triggerOn, only the status codes and errors that you list trigger a fallback.
statusCodes (list) --
The HTTP status codes returned by the target that trigger a fallback. You can specify the following:
An exact status code – A three-digit code, such as 429.
A class of status codes – The first digit followed by XX, such as 5XX, which matches every code from 500 to 599.
Only 4XX and 5XX codes are supported. Throttling by the gateway itself doesn't trigger a fallback.
(string) --
errors (list) --
The errors that trigger a fallback. You can specify the following:
CONNECTION_ERROR – The gateway couldn't connect to the target because the connection was refused, reset, or timed out. The request didn't reach the target.
FIRST_BYTE_TIMEOUT – The target accepted the connection but didn't return the first byte of the response within firstByteTimeoutMilliseconds. The gateway abandons the request before any of the response reaches you.
UNROUTABLE – The gateway couldn't route the request to the target, for example because the target doesn't exist or doesn't support the request path. These errors don't count toward the circuit breaker.
(string) --
firstByteTimeoutMilliseconds (integer) --
The maximum time, in milliseconds, from sending the request to the route's target until the first byte of the response body arrives. If the time expires, the gateway abandons the request and treats it as a FIRST_BYTE_TIMEOUT error. If you don't specify a value, no timeout is enforced.
circuitBreaker (dict) --
Specifies whether the gateway temporarily stops sending your requests to a failing target. The circuit breaker tracks each combination of target and model separately. With the circuit breaker enabled, the gateway skips one or more targets and models that fail repeatedly, and your requests go to the next target in the chain. The gateway always tries the last remaining target in the chain. The gateway enables the circuit breaker by default.
mode (string) --
Specifies whether you want the circuit breaker turned on. To turn it off, set this value to DISABLED. The default is ENABLED.
targetFallback (list) --
The ordered list of fallback targets. The gateway tries each target in order until a response doesn't match that target's triggers, and returns that response to the caller. If every target fails, the gateway returns the last response. You can specify up to two fallback targets. Each combination of target and model, including the route's target, can appear only once.
(dict) --
Specifies a target to send the request to when the previous target in the fallback chain fails. Includes the target name, the model to send, and the conditions that trigger a fallback from this target to the next one.
targetName (string) --
The name of the gateway target to fall back to. The target must be an inference target.
model (string) --
The model ID to send to this fallback target. If you don't specify a model, the gateway sends the model from the original request.
triggerOn (dict) --
The status codes and errors from this target that cause the gateway to send the request to the next fallback target. This setting isn't inherited from the route. If you omit it, the gateway uses the default triggers: 5XX status codes and the CONNECTION_ERROR, FIRST_BYTE_TIMEOUT, and UNROUTABLE errors.
statusCodes (list) --
The HTTP status codes returned by the target that trigger a fallback. You can specify the following:
An exact status code – A three-digit code, such as 429.
A class of status codes – The first digit followed by XX, such as 5XX, which matches every code from 500 to 599.
Only 4XX and 5XX codes are supported. Throttling by the gateway itself doesn't trigger a fallback.
(string) --
errors (list) --
The errors that trigger a fallback. You can specify the following:
CONNECTION_ERROR – The gateway couldn't connect to the target because the connection was refused, reset, or timed out. The request didn't reach the target.
FIRST_BYTE_TIMEOUT – The target accepted the connection but didn't return the first byte of the response within firstByteTimeoutMilliseconds. The gateway abandons the request before any of the response reaches you.
UNROUTABLE – The gateway couldn't route the request to the target, for example because the target doesn't exist or doesn't support the request path. These errors don't count toward the circuit breaker.
(string) --
firstByteTimeoutMilliseconds (integer) --
The maximum time, in milliseconds, from sending the request to this target until the first byte of the response body arrives. This setting isn't inherited from the route. If you don't specify a value, no timeout is enforced.
weightedRoute (dict) --
A weighted route that splits traffic between multiple targets.
trafficSplit (list) --
The traffic split entries defining how traffic is distributed between targets.
(dict) --
An entry in a target traffic split configuration.
targetName (string) --
The name of the target to route traffic to.
model (string) --
The model ID to send to the target for this variant. Supported only for inference targets. If you don't specify a model, the gateway sends the model from the original request.
fallback (dict) --
The fallback configuration for this variant. Supported only for inference targets.
triggerOn (dict) --
The status codes and errors from the route's target that cause the gateway to send the request to the first fallback target. If you omit triggerOn, the gateway falls back on 5XX status codes and on the CONNECTION_ERROR, FIRST_BYTE_TIMEOUT, and UNROUTABLE errors. If you specify triggerOn, only the status codes and errors that you list trigger a fallback.
statusCodes (list) --
The HTTP status codes returned by the target that trigger a fallback. You can specify the following:
An exact status code – A three-digit code, such as 429.
A class of status codes – The first digit followed by XX, such as 5XX, which matches every code from 500 to 599.
Only 4XX and 5XX codes are supported. Throttling by the gateway itself doesn't trigger a fallback.
(string) --
errors (list) --
The errors that trigger a fallback. You can specify the following:
CONNECTION_ERROR – The gateway couldn't connect to the target because the connection was refused, reset, or timed out. The request didn't reach the target.
FIRST_BYTE_TIMEOUT – The target accepted the connection but didn't return the first byte of the response within firstByteTimeoutMilliseconds. The gateway abandons the request before any of the response reaches you.
UNROUTABLE – The gateway couldn't route the request to the target, for example because the target doesn't exist or doesn't support the request path. These errors don't count toward the circuit breaker.
(string) --
firstByteTimeoutMilliseconds (integer) --
The maximum time, in milliseconds, from sending the request to the route's target until the first byte of the response body arrives. If the time expires, the gateway abandons the request and treats it as a FIRST_BYTE_TIMEOUT error. If you don't specify a value, no timeout is enforced.
circuitBreaker (dict) --
Specifies whether the gateway temporarily stops sending your requests to a failing target. The circuit breaker tracks each combination of target and model separately. With the circuit breaker enabled, the gateway skips one or more targets and models that fail repeatedly, and your requests go to the next target in the chain. The gateway always tries the last remaining target in the chain. The gateway enables the circuit breaker by default.
mode (string) --
Specifies whether you want the circuit breaker turned on. To turn it off, set this value to DISABLED. The default is ENABLED.
targetFallback (list) --
The ordered list of fallback targets. The gateway tries each target in order until a response doesn't match that target's triggers, and returns that response to the caller. If every target fails, the gateway returns the last response. You can specify up to two fallback targets. Each combination of target and model, including the route's target, can appear only once.
(dict) --
Specifies a target to send the request to when the previous target in the fallback chain fails. Includes the target name, the model to send, and the conditions that trigger a fallback from this target to the next one.
targetName (string) --
The name of the gateway target to fall back to. The target must be an inference target.
model (string) --
The model ID to send to this fallback target. If you don't specify a model, the gateway sends the model from the original request.
triggerOn (dict) --
The status codes and errors from this target that cause the gateway to send the request to the next fallback target. This setting isn't inherited from the route. If you omit it, the gateway uses the default triggers: 5XX status codes and the CONNECTION_ERROR, FIRST_BYTE_TIMEOUT, and UNROUTABLE errors.
statusCodes (list) --
The HTTP status codes returned by the target that trigger a fallback. You can specify the following:
An exact status code – A three-digit code, such as 429.
A class of status codes – The first digit followed by XX, such as 5XX, which matches every code from 500 to 599.
Only 4XX and 5XX codes are supported. Throttling by the gateway itself doesn't trigger a fallback.
(string) --
errors (list) --
The errors that trigger a fallback. You can specify the following:
CONNECTION_ERROR – The gateway couldn't connect to the target because the connection was refused, reset, or timed out. The request didn't reach the target.
FIRST_BYTE_TIMEOUT – The target accepted the connection but didn't return the first byte of the response within firstByteTimeoutMilliseconds. The gateway abandons the request before any of the response reaches you.
UNROUTABLE – The gateway couldn't route the request to the target, for example because the target doesn't exist or doesn't support the request path. These errors don't count toward the circuit breaker.
(string) --
firstByteTimeoutMilliseconds (integer) --
The maximum time, in milliseconds, from sending the request to this target until the first byte of the response body arrives. This setting isn't inherited from the route. If you don't specify a value, no timeout is enforced.
name (string) --
The name of this traffic split variant.
weight (integer) --
The percentage of traffic to route to this variant.
description (string) --
The description of this traffic split variant.
metadata (dict) --
Key-value metadata associated with this traffic split variant.
(string) --
(string) --
description (string) --
The description of the gateway rule.
createdAt (datetime) --
The timestamp when the rule was created.
status (string) --
The current status of the rule.
system (dict) --
System-managed metadata for rules created by automated processes.
managedBy (string) --
The identifier of the system or process that manages this rule.
updatedAt (datetime) --
The timestamp when the rule was last updated.