2026/11/17 - Amazon Bedrock AgentCore - 7 new 2 updated api methods
Changes Adds support for continuous insights and automatic dataset generation
Starts a dataset generation, which creates examples for an existing dataset from your agent's traces. The dataset must be ACTIVE. Each generation produces one scenario type, set by scenarioType. When the generation finishes, AgentCore adds the examples to the dataset's draft. Examples that duplicate each other, or that duplicate examples already in the draft, are left out.
A dataset can have only one generation in progress at a time. While it runs, requests that change the dataset's draft or publish a version are rejected with a ConflictException.
AgentCore reads the traces with your credentials, so you need read access to the trace log groups, and to the dataset's KMS key if the dataset uses a customer managed key. The draft can hold at most 1,000 examples, including the examples this generation can add ( maxScenarios).
The operation returns immediately. To track progress, call GetDatasetGeneration.
See also: AWS API Documentation
Request Syntax
client.start_dataset_generation(
datasetId='string',
name='string',
description='string',
scenarioType='FIXED'|'DYNAMIC',
source={
'traceGeneration': {
'sourceConfig': {
'cloudWatchLogs': {
'logGroupNames': [
'string',
],
'logGroupNamePrefixes': [
'string',
],
'serviceNames': [
'string',
],
'startTime': datetime(2015, 1, 1),
'endTime': datetime(2015, 1, 1)
},
'onlineEvaluation': {
'onlineEvaluationConfigArn': 'string',
'startTime': datetime(2015, 1, 1),
'endTime': datetime(2015, 1, 1)
},
'batchEvaluation': {
'batchEvaluationArn': 'string'
}
},
'selection': {
'filters': {
'attributeFilters': [
{
'key': 'string',
'operator': 'Equals'|'NotEquals'|'GreaterThan'|'LessThan'|'GreaterThanOrEqual'|'LessThanOrEqual'|'Contains'|'NotContains',
'value': {
'stringValue': 'string',
'doubleValue': 123.0,
'booleanValue': True|False
}
},
],
'turnCount': {
'min': 123,
'max': 123
},
'evaluatorFilters': [
{
'evaluatorId': 'string',
'evaluationDimensionName': 'string',
'operator': 'Equals'|'NotEquals'|'GreaterThan'|'LessThan'|'GreaterThanOrEqual'|'LessThanOrEqual'|'Contains'|'NotContains',
'value': {
'stringValue': 'string',
'doubleValue': 123.0,
'booleanValue': True|False
},
'aggregation': 'Min'|'Max'|'Mean'|'AllMatch'|'AnyMatch'
},
]
},
'identifiers': {
'sessionIds': [
'string',
],
'traceIds': [
'string',
]
}
},
'sampling': {
'maxScenarios': 123
},
'groundTruth': {
'fields': [
'expected_response'|'expected_trajectory',
],
'qualityFilters': [
{
'evaluatorId': 'string',
'evaluationDimensionName': 'string',
'operator': 'Equals'|'NotEquals'|'GreaterThan'|'LessThan'|'GreaterThanOrEqual'|'LessThanOrEqual'|'Contains'|'NotContains',
'value': {
'stringValue': 'string',
'doubleValue': 123.0,
'booleanValue': True|False
},
'aggregation': 'Min'|'Max'|'Mean'|'AllMatch'|'AnyMatch'
},
]
},
'redactionConfiguration': {
'identifiers': [
'ADDRESS'|'AWS_SECRET_KEY'|'BANK_ACCOUNT_NUMBER_DE'|'BANK_ACCOUNT_NUMBER_ES'|'BANK_ACCOUNT_NUMBER_FR'|'BANK_ACCOUNT_NUMBER_GB'|'BANK_ACCOUNT_NUMBER_IT'|'BANK_ACCOUNT_NUMBER_US'|'CEP_CODE_BR'|'CNPJ_BR'|'CPF_CODE_BR'|'CREDIT_CARD_EXPIRATION'|'CREDIT_CARD_MAGNETIC_STRIPE'|'CREDIT_CARD_NUMBER'|'CREDIT_CARD_SECURITY_CODE'|'DATE_OF_BIRTH'|'DRIVERS_LICENSE_AT'|'DRIVERS_LICENSE_AU'|'DRIVERS_LICENSE_BE'|'DRIVERS_LICENSE_BG'|'DRIVERS_LICENSE_CA'|'DRIVERS_LICENSE_CY'|'DRIVERS_LICENSE_CZ'|'DRIVERS_LICENSE_DE'|'DRIVERS_LICENSE_DK'|'DRIVERS_LICENSE_EE'|'DRIVERS_LICENSE_ES'|'DRIVERS_LICENSE_FI'|'DRIVERS_LICENSE_FR'|'DRIVERS_LICENSE_GB'|'DRIVERS_LICENSE_GR'|'DRIVERS_LICENSE_HR'|'DRIVERS_LICENSE_HU'|'DRIVERS_LICENSE_IE'|'DRIVERS_LICENSE_IT'|'DRIVERS_LICENSE_LT'|'DRIVERS_LICENSE_LU'|'DRIVERS_LICENSE_LV'|'DRIVERS_LICENSE_MT'|'DRIVERS_LICENSE_NL'|'DRIVERS_LICENSE_PL'|'DRIVERS_LICENSE_PT'|'DRIVERS_LICENSE_RO'|'DRIVERS_LICENSE_SE'|'DRIVERS_LICENSE_SI'|'DRIVERS_LICENSE_SK'|'DRIVERS_LICENSE_US'|'DRUG_ENFORCEMENT_AGENCY_NUMBER_US'|'ELECTORAL_ROLL_NUMBER_GB'|'EMAIL_ADDRESS'|'HEALTH_INSURANCE_CARD_NUMBER_EU'|'HEALTH_INSURANCE_CLAIM_NUMBER_US'|'HEALTH_INSURANCE_NUMBER_FR'|'HEALTHCARE_PROCEDURE_CODE_US'|'INDIVIDUAL_TAX_IDENTIFICATION_NUMBER_US'|'INSEE_CODE_FR'|'IP_ADDRESS'|'LAT_LONG'|'MEDICARE_BENEFICIARY_NUMBER_US'|'NAME'|'NATIONAL_DRUG_CODE_US'|'NATIONAL_IDENTIFICATION_NUMBER_DE'|'NATIONAL_IDENTIFICATION_NUMBER_ES'|'NATIONAL_IDENTIFICATION_NUMBER_IT'|'NATIONAL_INSURANCE_NUMBER_GB'|'NATIONAL_PROVIDER_ID_US'|'NHS_NUMBER_GB'|'NIE_NUMBER_ES'|'NIF_NUMBER_ES'|'OPEN_SSH_PRIVATE_KEY'|'PASSPORT_NUMBER_CA'|'PASSPORT_NUMBER_DE'|'PASSPORT_NUMBER_ES'|'PASSPORT_NUMBER_FR'|'PASSPORT_NUMBER_GB'|'PASSPORT_NUMBER_IT'|'PASSPORT_NUMBER_US'|'PERMANENT_RESIDENCE_NUMBER_CA'|'PERSONAL_HEALTH_NUMBER_CA'|'PGP_PRIVATE_KEY'|'PHONE_NUMBER_BR'|'PHONE_NUMBER_DE'|'PHONE_NUMBER_ES'|'PHONE_NUMBER_FR'|'PHONE_NUMBER_GB'|'PHONE_NUMBER_IT'|'PHONE_NUMBER_US'|'PKCS_PRIVATE_KEY'|'POSTAL_CODE_CA'|'PUTTY_PRIVATE_KEY'|'RG_NUMBER_BR'|'SOCIAL_INSURANCE_NUMBER_CA'|'SSN_ES'|'SSN_US'|'TAX_ID_DE'|'TAX_ID_ES'|'TAX_ID_FR'|'TAX_ID_GB'|'VEHICLE_IDENTIFICATION_NUMBER'|'ZIP_CODE_US',
],
'customDataIdentifiers': [
{
'name': 'string',
'regex': 'string'
},
]
}
}
},
clientToken='string'
)
string
[REQUIRED]
The unique identifier of the dataset to add the generated examples to. The dataset must be ACTIVE.
string
[REQUIRED]
The name of the dataset generation. The generation ID starts with this name.
string
A description of the dataset generation.
string
The type of scenario to generate. If you don't specify a value, the generation produces DYNAMIC scenarios.
dict
[REQUIRED]
Where the generation reads sessions from, and how it selects, samples, labels and redacts them.
traceGeneration (dict) --
Generates examples from your agent's traces.
sourceConfig (dict) -- [REQUIRED]
Where to read the sessions from.
cloudWatchLogs (dict) --
Reads sessions from CloudWatch Logs log groups.
logGroupNames (list) --
The names of the log groups to read, from 1 to 5. You can't specify this together with logGroupNamePrefixes.
(string) --
logGroupNamePrefixes (list) --
Prefixes of the names of the log groups to read, from 1 to 5. You can't specify this together with logGroupNames.
(string) --
Prefix of a CloudWatch Logs log group name.
serviceNames (list) -- [REQUIRED]
The service name of the agent, which matches the service.name resource attribute in its traces. Specify exactly one.
(string) --
startTime (datetime) --
The start of the time window to read sessions from. The default is 7 days before endTime.
endTime (datetime) --
The end of the time window to read sessions from. The default is the time the generation starts.
onlineEvaluation (dict) --
Reads sessions, with their evaluation results, from an online evaluation configuration.
onlineEvaluationConfigArn (string) -- [REQUIRED]
The Amazon Resource Name (ARN) of the online evaluation configuration.
startTime (datetime) --
The start of the time window to read sessions from. The default is 7 days before endTime.
endTime (datetime) --
The end of the time window to read sessions from. The default is the time the generation starts.
batchEvaluation (dict) --
Reads sessions, with their evaluation results, from a batch evaluation.
batchEvaluationArn (string) -- [REQUIRED]
The Amazon Resource Name (ARN) of the batch evaluation.
selection (dict) --
How to choose sessions from the source. If you don't specify a selection, the generation uses every session in the source and time window that has at most 100 turns.
filters (dict) --
Conditions that a session must meet to be selected.
attributeFilters (list) --
Attribute conditions, up to 5. A session must meet all of them. An attribute on a span matches when any span in the session meets the condition.
(dict) --
A condition on a session attribute.
key (string) -- [REQUIRED]
The attribute key.
operator (string) -- [REQUIRED]
How to compare the attribute with value.
value (dict) -- [REQUIRED]
The value to compare the attribute with.
stringValue (string) --
A string value.
doubleValue (float) --
A numeric value.
booleanValue (boolean) --
A Boolean value.
turnCount (dict) --
The range of turns a session can have. For example, set max to 1 for single-turn sessions only, or min to 2 for multi-turn sessions only. Sessions with more than 100 turns are never selected.
min (integer) --
The minimum number of turns.
max (integer) --
The maximum number of turns. The default is 100.
evaluatorFilters (list) --
Conditions on evaluation results, up to 10. A session must meet all of them. You can use these only with an onlineEvaluation or batchEvaluation source. A cloudWatchLogs source has no evaluation results.
(dict) --
A condition on a session's evaluation result.
evaluatorId (string) -- [REQUIRED]
The unique identifier of the evaluator whose result to compare.
evaluationDimensionName (string) --
The dimension of the evaluator's result to compare. Required for an evaluator that scores more than one dimension. Don't specify it for any other evaluator.
operator (string) -- [REQUIRED]
How to compare the evaluation result with value.
value (dict) -- [REQUIRED]
The value to compare the evaluation result with. Use doubleValue for a score and stringValue for a label. booleanValue isn't supported.
stringValue (string) --
A string value.
doubleValue (float) --
A numeric value.
booleanValue (boolean) --
A Boolean value.
aggregation (string) --
How to combine the evaluator's per-trace or per-span results into one result for the session. Required for an evaluator that scores traces or spans. Don't specify it for an evaluator that scores sessions.
identifiers (dict) --
The sessions and traces to use.
sessionIds (list) --
The IDs of the sessions to use, up to 25.
(string) --
traceIds (list) --
The IDs of the traces to use, up to 25.
(string) --
sampling (dict) --
How many scenarios to generate from the selected sessions.
maxScenarios (integer) --
The maximum number of scenarios to generate, from 1 to 1,000. The default is 100. The generation produces at most one scenario for each selected session. The dataset's draft examples plus this value can't be more than 1,000.
groundTruth (dict) --
Reference fields to copy from the sessions into the examples. If you don't specify this, the examples have no reference fields.
fields (list) -- [REQUIRED]
The reference fields to copy. Values are copied from the session as observed, not generated.
(string) --
A reference field in a dataset example.
qualityFilters (list) --
Evaluator conditions that a selected session must also meet to have reference fields copied. If you don't specify these, the generation uses the evaluator filters from selection. If the selection names sessions and traces by ID, every selected session gets reference fields. Specify an empty list to give every selected session reference fields. You can't specify quality filters with a cloudWatchLogs source.
(dict) --
A condition on a session's evaluation result.
evaluatorId (string) -- [REQUIRED]
The unique identifier of the evaluator whose result to compare.
evaluationDimensionName (string) --
The dimension of the evaluator's result to compare. Required for an evaluator that scores more than one dimension. Don't specify it for any other evaluator.
operator (string) -- [REQUIRED]
How to compare the evaluation result with value.
value (dict) -- [REQUIRED]
The value to compare the evaluation result with. Use doubleValue for a score and stringValue for a label. booleanValue isn't supported.
stringValue (string) --
A string value.
doubleValue (float) --
A numeric value.
booleanValue (boolean) --
A Boolean value.
aggregation (string) --
How to combine the evaluator's per-trace or per-span results into one result for the session. Required for an evaluator that scores traces or spans. Don't specify it for an evaluator that scores sessions.
redactionConfiguration (dict) --
Sensitive data to mask in the examples. If you don't specify this, nothing is redacted.
identifiers (list) --
The managed data identifiers to apply. These match the CloudWatch Logs managed data identifiers.
(string) --
A managed data identifier. The values match the CloudWatch Logs managed data identifiers.
customDataIdentifiers (list) --
Custom data identifiers to apply, up to 10. You can use custom identifiers without any managed identifiers.
(dict) --
A custom data identifier, defined by a regular expression.
name (string) -- [REQUIRED]
The name of the custom data identifier. It can't be the same as the name of a managed data identifier.
regex (string) -- [REQUIRED]
The regular expression that matches the data to mask. Lookahead and lookbehind aren't supported.
string
A unique, case-sensitive identifier to ensure that the API request completes no more than one time. If this token matches a previous request, the service ignores the request, but does not return an error.
This field is autopopulated if not provided.
dict
Response Syntax
{
'generationId': 'string',
'name': 'string',
'datasetId': 'string',
'status': 'IN_PROGRESS'|'COMPLETED'|'COMPLETED_WITH_ERRORS'|'FAILED'|'DELETING',
'createdAt': datetime(2015, 1, 1)
}
Response Structure
(dict) --
generationId (string) --
The unique identifier of the dataset generation.
name (string) --
The name of the dataset generation.
datasetId (string) --
The unique identifier of the dataset.
status (string) --
The status of the dataset generation.
createdAt (datetime) --
The time at which the dataset generation started.
Gets a dataset generation, including its status, the request it was started with, its progress counts, its redaction summary and its errors. Call this operation to track a generation that you started with StartDatasetGeneration.
See also: AWS API Documentation
Request Syntax
client.get_dataset_generation(
datasetId='string',
generationId='string'
)
string
[REQUIRED]
The unique identifier of the dataset.
string
[REQUIRED]
The unique identifier of the dataset generation.
dict
Response Syntax
{
'generationId': 'string',
'name': 'string',
'description': 'string',
'datasetId': 'string',
'scenarioType': 'FIXED'|'DYNAMIC',
'status': 'IN_PROGRESS'|'COMPLETED'|'COMPLETED_WITH_ERRORS'|'FAILED'|'DELETING',
'failureReason': 'string',
'createdAt': datetime(2015, 1, 1),
'updatedAt': datetime(2015, 1, 1),
'completedAt': datetime(2015, 1, 1),
'source': {
'traceGeneration': {
'sourceConfig': {
'cloudWatchLogs': {
'logGroupNames': [
'string',
],
'logGroupNamePrefixes': [
'string',
],
'serviceNames': [
'string',
],
'startTime': datetime(2015, 1, 1),
'endTime': datetime(2015, 1, 1)
},
'onlineEvaluation': {
'onlineEvaluationConfigArn': 'string',
'startTime': datetime(2015, 1, 1),
'endTime': datetime(2015, 1, 1)
},
'batchEvaluation': {
'batchEvaluationArn': 'string'
}
},
'selection': {
'filters': {
'attributeFilters': [
{
'key': 'string',
'operator': 'Equals'|'NotEquals'|'GreaterThan'|'LessThan'|'GreaterThanOrEqual'|'LessThanOrEqual'|'Contains'|'NotContains',
'value': {
'stringValue': 'string',
'doubleValue': 123.0,
'booleanValue': True|False
}
},
],
'turnCount': {
'min': 123,
'max': 123
},
'evaluatorFilters': [
{
'evaluatorId': 'string',
'evaluationDimensionName': 'string',
'operator': 'Equals'|'NotEquals'|'GreaterThan'|'LessThan'|'GreaterThanOrEqual'|'LessThanOrEqual'|'Contains'|'NotContains',
'value': {
'stringValue': 'string',
'doubleValue': 123.0,
'booleanValue': True|False
},
'aggregation': 'Min'|'Max'|'Mean'|'AllMatch'|'AnyMatch'
},
]
},
'identifiers': {
'sessionIdCount': 123,
'traceIdCount': 123
}
},
'sampling': {
'maxScenarios': 123
},
'groundTruth': {
'fields': [
'expected_response'|'expected_trajectory',
],
'qualityFilters': [
{
'evaluatorId': 'string',
'evaluationDimensionName': 'string',
'operator': 'Equals'|'NotEquals'|'GreaterThan'|'LessThan'|'GreaterThanOrEqual'|'LessThanOrEqual'|'Contains'|'NotContains',
'value': {
'stringValue': 'string',
'doubleValue': 123.0,
'booleanValue': True|False
},
'aggregation': 'Min'|'Max'|'Mean'|'AllMatch'|'AnyMatch'
},
]
},
'redactionConfiguration': {
'identifiers': [
'ADDRESS'|'AWS_SECRET_KEY'|'BANK_ACCOUNT_NUMBER_DE'|'BANK_ACCOUNT_NUMBER_ES'|'BANK_ACCOUNT_NUMBER_FR'|'BANK_ACCOUNT_NUMBER_GB'|'BANK_ACCOUNT_NUMBER_IT'|'BANK_ACCOUNT_NUMBER_US'|'CEP_CODE_BR'|'CNPJ_BR'|'CPF_CODE_BR'|'CREDIT_CARD_EXPIRATION'|'CREDIT_CARD_MAGNETIC_STRIPE'|'CREDIT_CARD_NUMBER'|'CREDIT_CARD_SECURITY_CODE'|'DATE_OF_BIRTH'|'DRIVERS_LICENSE_AT'|'DRIVERS_LICENSE_AU'|'DRIVERS_LICENSE_BE'|'DRIVERS_LICENSE_BG'|'DRIVERS_LICENSE_CA'|'DRIVERS_LICENSE_CY'|'DRIVERS_LICENSE_CZ'|'DRIVERS_LICENSE_DE'|'DRIVERS_LICENSE_DK'|'DRIVERS_LICENSE_EE'|'DRIVERS_LICENSE_ES'|'DRIVERS_LICENSE_FI'|'DRIVERS_LICENSE_FR'|'DRIVERS_LICENSE_GB'|'DRIVERS_LICENSE_GR'|'DRIVERS_LICENSE_HR'|'DRIVERS_LICENSE_HU'|'DRIVERS_LICENSE_IE'|'DRIVERS_LICENSE_IT'|'DRIVERS_LICENSE_LT'|'DRIVERS_LICENSE_LU'|'DRIVERS_LICENSE_LV'|'DRIVERS_LICENSE_MT'|'DRIVERS_LICENSE_NL'|'DRIVERS_LICENSE_PL'|'DRIVERS_LICENSE_PT'|'DRIVERS_LICENSE_RO'|'DRIVERS_LICENSE_SE'|'DRIVERS_LICENSE_SI'|'DRIVERS_LICENSE_SK'|'DRIVERS_LICENSE_US'|'DRUG_ENFORCEMENT_AGENCY_NUMBER_US'|'ELECTORAL_ROLL_NUMBER_GB'|'EMAIL_ADDRESS'|'HEALTH_INSURANCE_CARD_NUMBER_EU'|'HEALTH_INSURANCE_CLAIM_NUMBER_US'|'HEALTH_INSURANCE_NUMBER_FR'|'HEALTHCARE_PROCEDURE_CODE_US'|'INDIVIDUAL_TAX_IDENTIFICATION_NUMBER_US'|'INSEE_CODE_FR'|'IP_ADDRESS'|'LAT_LONG'|'MEDICARE_BENEFICIARY_NUMBER_US'|'NAME'|'NATIONAL_DRUG_CODE_US'|'NATIONAL_IDENTIFICATION_NUMBER_DE'|'NATIONAL_IDENTIFICATION_NUMBER_ES'|'NATIONAL_IDENTIFICATION_NUMBER_IT'|'NATIONAL_INSURANCE_NUMBER_GB'|'NATIONAL_PROVIDER_ID_US'|'NHS_NUMBER_GB'|'NIE_NUMBER_ES'|'NIF_NUMBER_ES'|'OPEN_SSH_PRIVATE_KEY'|'PASSPORT_NUMBER_CA'|'PASSPORT_NUMBER_DE'|'PASSPORT_NUMBER_ES'|'PASSPORT_NUMBER_FR'|'PASSPORT_NUMBER_GB'|'PASSPORT_NUMBER_IT'|'PASSPORT_NUMBER_US'|'PERMANENT_RESIDENCE_NUMBER_CA'|'PERSONAL_HEALTH_NUMBER_CA'|'PGP_PRIVATE_KEY'|'PHONE_NUMBER_BR'|'PHONE_NUMBER_DE'|'PHONE_NUMBER_ES'|'PHONE_NUMBER_FR'|'PHONE_NUMBER_GB'|'PHONE_NUMBER_IT'|'PHONE_NUMBER_US'|'PKCS_PRIVATE_KEY'|'POSTAL_CODE_CA'|'PUTTY_PRIVATE_KEY'|'RG_NUMBER_BR'|'SOCIAL_INSURANCE_NUMBER_CA'|'SSN_ES'|'SSN_US'|'TAX_ID_DE'|'TAX_ID_ES'|'TAX_ID_FR'|'TAX_ID_GB'|'VEHICLE_IDENTIFICATION_NUMBER'|'ZIP_CODE_US',
],
'customDataIdentifiers': [
{
'name': 'string',
'regex': 'string'
},
]
}
}
},
'generationResults': {
'sessionsMatched': 123,
'sessionsSelected': 123,
'sessionsUnscored': 123,
'sessionsFailed': 123,
'scenariosGenerated': 123,
'scenariosDeduplicated': 123,
'examplesWritten': 123,
'examplesWithGroundTruth': 123
},
'redactionSummary': {
'scenariosScanned': 123,
'findingCounts': {
'string': 123
}
},
'errorDetails': [
{
'errorCode': 'string',
'errorMessage': 'string',
'count': 123,
'sessionIds': [
'string',
]
},
]
}
Response Structure
(dict) --
generationId (string) --
The unique identifier of the dataset generation.
name (string) --
The name of the dataset generation.
description (string) --
The description of the dataset generation.
datasetId (string) --
The unique identifier of the dataset.
scenarioType (string) --
The type of scenario the generation produces. This is the value from the request, or DYNAMIC if the request didn't specify one.
status (string) --
The status of the dataset generation.
failureReason (string) --
The reason the dataset generation failed. Present only when the status is FAILED.
createdAt (datetime) --
The time at which the dataset generation started.
updatedAt (datetime) --
The time at which the dataset generation last changed status.
completedAt (datetime) --
The time at which the dataset generation finished. Present only when the generation has finished.
source (dict) --
The source the generation was started with, including the resolved time window. If you selected sessions and traces by ID, the response gives the number of IDs instead of the IDs.
traceGeneration (dict) --
The trace source the generation was started with.
sourceConfig (dict) --
Where the generation reads sessions from, including the resolved time window.
cloudWatchLogs (dict) --
Reads sessions from CloudWatch Logs log groups.
logGroupNames (list) --
The names of the log groups to read, from 1 to 5. You can't specify this together with logGroupNamePrefixes.
(string) --
logGroupNamePrefixes (list) --
Prefixes of the names of the log groups to read, from 1 to 5. You can't specify this together with logGroupNames.
(string) --
Prefix of a CloudWatch Logs log group name.
serviceNames (list) --
The service name of the agent, which matches the service.name resource attribute in its traces. Specify exactly one.
(string) --
startTime (datetime) --
The start of the time window to read sessions from. The default is 7 days before endTime.
endTime (datetime) --
The end of the time window to read sessions from. The default is the time the generation starts.
onlineEvaluation (dict) --
Reads sessions, with their evaluation results, from an online evaluation configuration.
onlineEvaluationConfigArn (string) --
The Amazon Resource Name (ARN) of the online evaluation configuration.
startTime (datetime) --
The start of the time window to read sessions from. The default is 7 days before endTime.
endTime (datetime) --
The end of the time window to read sessions from. The default is the time the generation starts.
batchEvaluation (dict) --
Reads sessions, with their evaluation results, from a batch evaluation.
batchEvaluationArn (string) --
The Amazon Resource Name (ARN) of the batch evaluation.
selection (dict) --
How the generation chooses sessions. Filters are returned as submitted, with the resolved maximum turn count. Session and trace IDs are returned as counts. If this is absent, the request had no selection.
filters (dict) --
The filters the generation was started with.
attributeFilters (list) --
Attribute conditions, up to 5. A session must meet all of them. An attribute on a span matches when any span in the session meets the condition.
(dict) --
A condition on a session attribute.
key (string) --
The attribute key.
operator (string) --
How to compare the attribute with value.
value (dict) --
The value to compare the attribute with.
stringValue (string) --
A string value.
doubleValue (float) --
A numeric value.
booleanValue (boolean) --
A Boolean value.
turnCount (dict) --
The range of turns a session can have. For example, set max to 1 for single-turn sessions only, or min to 2 for multi-turn sessions only. Sessions with more than 100 turns are never selected.
min (integer) --
The minimum number of turns.
max (integer) --
The maximum number of turns. The default is 100.
evaluatorFilters (list) --
Conditions on evaluation results, up to 10. A session must meet all of them. You can use these only with an onlineEvaluation or batchEvaluation source. A cloudWatchLogs source has no evaluation results.
(dict) --
A condition on a session's evaluation result.
evaluatorId (string) --
The unique identifier of the evaluator whose result to compare.
evaluationDimensionName (string) --
The dimension of the evaluator's result to compare. Required for an evaluator that scores more than one dimension. Don't specify it for any other evaluator.
operator (string) --
How to compare the evaluation result with value.
value (dict) --
The value to compare the evaluation result with. Use doubleValue for a score and stringValue for a label. booleanValue isn't supported.
stringValue (string) --
A string value.
doubleValue (float) --
A numeric value.
booleanValue (boolean) --
A Boolean value.
aggregation (string) --
How to combine the evaluator's per-trace or per-span results into one result for the session. Required for an evaluator that scores traces or spans. Don't specify it for an evaluator that scores sessions.
identifiers (dict) --
The number of session and trace IDs the generation was started with.
sessionIdCount (integer) --
The number of session IDs in the request.
traceIdCount (integer) --
The number of trace IDs in the request.
sampling (dict) --
How many scenarios the generation produces.
maxScenarios (integer) --
The maximum number of scenarios to generate, from 1 to 1,000. The default is 100. The generation produces at most one scenario for each selected session. The dataset's draft examples plus this value can't be more than 1,000.
groundTruth (dict) --
The reference fields the generation copies into the examples. If the request didn't specify quality filters, the filters the generation applies are returned here.
fields (list) --
The reference fields to copy. Values are copied from the session as observed, not generated.
(string) --
A reference field in a dataset example.
qualityFilters (list) --
Evaluator conditions that a selected session must also meet to have reference fields copied. If you don't specify these, the generation uses the evaluator filters from selection. If the selection names sessions and traces by ID, every selected session gets reference fields. Specify an empty list to give every selected session reference fields. You can't specify quality filters with a cloudWatchLogs source.
(dict) --
A condition on a session's evaluation result.
evaluatorId (string) --
The unique identifier of the evaluator whose result to compare.
evaluationDimensionName (string) --
The dimension of the evaluator's result to compare. Required for an evaluator that scores more than one dimension. Don't specify it for any other evaluator.
operator (string) --
How to compare the evaluation result with value.
value (dict) --
The value to compare the evaluation result with. Use doubleValue for a score and stringValue for a label. booleanValue isn't supported.
stringValue (string) --
A string value.
doubleValue (float) --
A numeric value.
booleanValue (boolean) --
A Boolean value.
aggregation (string) --
How to combine the evaluator's per-trace or per-span results into one result for the session. Required for an evaluator that scores traces or spans. Don't specify it for an evaluator that scores sessions.
redactionConfiguration (dict) --
The sensitive data the generation masks in the examples.
identifiers (list) --
The managed data identifiers to apply. These match the CloudWatch Logs managed data identifiers.
(string) --
A managed data identifier. The values match the CloudWatch Logs managed data identifiers.
customDataIdentifiers (list) --
Custom data identifiers to apply, up to 10. You can use custom identifiers without any managed identifiers.
(dict) --
A custom data identifier, defined by a regular expression.
name (string) --
The name of the custom data identifier. It can't be the same as the name of a managed data identifier.
regex (string) --
The regular expression that matches the data to mask. Lookahead and lookbehind aren't supported.
generationResults (dict) --
Counts of the sessions, scenarios and examples the generation has processed.
sessionsMatched (integer) --
The number of sessions that matched the filters or were named in the request, before sampling. This includes sessions that later failed.
sessionsSelected (integer) --
The number of sessions left after sampling.
sessionsUnscored (integer) --
The number of sessions left out because an evaluator filter had no result to compare, either because the session wasn't evaluated yet or because the evaluation failed.
sessionsFailed (integer) --
The number of sessions dropped because of an error. See errorDetails for the errors.
scenariosGenerated (integer) --
The number of scenarios generated.
scenariosDeduplicated (integer) --
The number of scenarios left out as duplicates, of each other or of examples already in the draft.
examplesWritten (integer) --
The number of examples added to the dataset's draft.
examplesWithGroundTruth (integer) --
The number of added examples that have reference fields.
redactionSummary (dict) --
A summary of what redaction found. Present only when the generation was started with redaction.
scenariosScanned (integer) --
The number of scenarios scanned for sensitive data.
findingCounts (dict) --
The number of matches for each data identifier.
(string) --
(integer) --
errorDetails (list) --
The errors the generation hit, grouped by error code. The list holds the 10 most frequent error codes, most frequent first.
(dict) --
The failures in a dataset generation that share an error code.
errorCode (string) --
The error code, for example ACCESS_DENIED or EXPIRED_CREDENTIALS. New codes can be added.
errorMessage (string) --
A description of the error.
count (integer) --
The number of sessions and traces that failed with this error code. Absent for a generation that finished before failures were counted by code. In that case, sessionIds is a sample.
sessionIds (list) --
Up to 10 of the sessions that failed with this error code. Each entry is a session ID, or the ID of a trace named in the request.
(string) --
Amazon Bedrock AgentCore Agent Insights identifies patterns detected across your agent sessions. Use Agent Insights to identify recurring issues and prioritize fixes.
Retrieves detailed information about an Amazon Bedrock AgentCore Agent Insight, including its archive status, affected sessions, and, when applicable, the batch evaluation that produced it.
See also: AWS API Documentation
Request Syntax
client.get_agent_insight(
applicationName='string',
agentInsightId='string'
)
string
[REQUIRED]
The application that the Agent Insight belongs to.
string
[REQUIRED]
The unique identifier of the Agent Insight to retrieve.
dict
Response Syntax
{
'type': 'FAILURE',
'name': 'string',
'description': 'string',
'createdAt': datetime(2015, 1, 1),
'affectedSessionCount': 123,
'sampleSessions': [
{
'sessionId': 'string',
'lastSeenAt': datetime(2015, 1, 1),
'findings': [
{
'targetId': 'string',
'targetType': 'SPAN_ID'|'TRACE_ID'|'SPAN_NAME',
'description': 'string',
'category': 'string',
'confidence': 123.0
},
]
},
],
'rootCauses': [
{
'name': 'string',
'explanation': 'string',
'affectedSessionCount': 123,
'suggestedFix': 'string',
'sampleSessions': [
{
'sessionId': 'string',
'lastSeenAt': datetime(2015, 1, 1),
'findings': [
{
'targetId': 'string',
'targetType': 'SPAN_ID'|'TRACE_ID'|'SPAN_NAME',
'description': 'string',
'category': 'string',
'confidence': 123.0
},
]
},
]
},
],
'agentInsightId': 'string',
'applicationName': 'string',
'batchEvaluationId': 'string',
'customerMetadataStatus': 'AVAILABLE'|'CUSTOMER_KEY_UNAVAILABLE'|'UNAVAILABLE',
'updatedAt': datetime(2015, 1, 1),
'archiveStatus': 'ACTIVE'|'ARCHIVED',
'lastSeenAt': datetime(2015, 1, 1)
}
Response Structure
(dict) --
type (string) --
The type of insight.
name (string) --
A generated display label for the Agent Insight. This is a human-readable summary of the detected pattern, not an identifier.
description (string) --
A generated description of the Agent Insight that explains the detected pattern and its observed behavior across sessions.
createdAt (datetime) --
The date and time when the Agent Insight was first created.
affectedSessionCount (integer) --
The number of sessions affected by the insight pattern.
sampleSessions (list) --
A list of sample sessions that illustrate the Agent Insight, including per-session findings that point to the relevant spans or traces.
(dict) --
A session affected by an insight or root cause. Use the session ID to find the per-session evaluation results in CloudWatch Logs.
sessionId (string) --
The unique identifier of the session.
lastSeenAt (datetime) --
The latest timestamp observed for the session in the batch evaluation input.
findings (list) --
The failures found in the session that belong to the insight or root cause, up to 10.
(dict) --
A failure found in a session.
targetId (string) --
The identifier or name of the location in the session where the failure was found, interpreted according to targetType.
targetType (string) --
The type of location that targetId refers to.
description (string) --
A description of the failure.
category (string) --
The category of the failure.
confidence (float) --
The confidence that the finding is a failure, from 0 to 1.
rootCauses (list) --
The root causes identified for the Agent Insight, each with an explanation, an affected session count, an optional suggested fix, and sample sessions that illustrate the root cause.
(dict) --
A root cause that contributes to an insight.
name (string) --
A generated label for the root cause.
explanation (string) --
An explanation of how the root cause leads to the failures in the insight.
affectedSessionCount (integer) --
The number of distinct sessions affected by the root cause. A session can be affected by more than one root cause of the same insight.
suggestedFix (string) --
A suggested change to the agent that addresses the root cause.
sampleSessions (list) --
A sample of up to 10 sessions affected by the root cause.
(dict) --
A session affected by an insight or root cause. Use the session ID to find the per-session evaluation results in CloudWatch Logs.
sessionId (string) --
The unique identifier of the session.
lastSeenAt (datetime) --
The latest timestamp observed for the session in the batch evaluation input.
findings (list) --
The failures found in the session that belong to the insight or root cause, up to 10.
(dict) --
A failure found in a session.
targetId (string) --
The identifier or name of the location in the session where the failure was found, interpreted according to targetType.
targetType (string) --
The type of location that targetId refers to.
description (string) --
A description of the failure.
category (string) --
The category of the failure.
confidence (float) --
The confidence that the finding is a failure, from 0 to 1.
agentInsightId (string) --
The unique identifier of the Agent Insight.
applicationName (string) --
The application that the Agent Insight belongs to.
batchEvaluationId (string) --
The Batch Evaluation that produced this Insight, when applicable.
customerMetadataStatus (string) --
Specifies whether customer-encrypted metadata is available for the Agent Insight.
updatedAt (datetime) --
The date and time when the Agent Insight was last updated.
archiveStatus (string) --
Specifies whether the Agent Insight is active or archived. Valid values:
ACTIVE - The insight is currently active.
ARCHIVED - The insight has been archived.
lastSeenAt (datetime) --
The most recent date and time when the insight pattern was observed.
Lists Amazon Bedrock AgentCore Agent Insight summaries in your account and Region. Use this operation to review patterns across sessions before you investigate or archive an insight. The response paginates the results. Use the nextToken parameter to retrieve additional results.
See also: AWS API Documentation
Request Syntax
client.list_agent_insights(
applicationName='string',
batchEvaluationId='string',
archiveStatus='ACTIVE'|'ARCHIVED',
maxResults=123,
nextToken='string'
)
string
Filters the results to Agent Insights for an application.
string
Filters the results to Agent Insights produced by a batch evaluation.
string
Filters the results by archive status.
integer
The maximum number of Agent Insight summaries to return in one response. If you don't specify a value, the operation returns up to 50 summaries.
string
A token to specify where to start paginating the results. This token is returned in a previous response.
dict
Response Syntax
{
'agentInsights': [
{
'type': 'FAILURE',
'name': 'string',
'description': 'string',
'createdAt': datetime(2015, 1, 1),
'affectedSessionCount': 123,
'agentInsightId': 'string',
'applicationName': 'string',
'batchEvaluationId': 'string',
'customerMetadataStatus': 'AVAILABLE'|'CUSTOMER_KEY_UNAVAILABLE'|'UNAVAILABLE',
'updatedAt': datetime(2015, 1, 1),
'archiveStatus': 'ACTIVE'|'ARCHIVED',
'lastSeenAt': datetime(2015, 1, 1)
},
],
'nextToken': 'string'
}
Response Structure
(dict) --
agentInsights (list) --
The Agent Insight summaries.
(dict) --
Contains summary information about an Amazon Bedrock AgentCore Agent Insight, including its label, description, and creation time. This structure is returned by ListAgentInsights; use GetAgentInsight to retrieve the sample sessions and root causes for a specific insight.
type (string) --
The type of insight.
name (string) --
A generated display label for the Agent Insight. This is a human-readable summary of the detected pattern, not an identifier.
description (string) --
A generated description of the Agent Insight that explains the detected pattern and its observed behavior across sessions.
createdAt (datetime) --
The date and time when the Agent Insight was first created.
affectedSessionCount (integer) --
The number of sessions affected by the insight pattern.
agentInsightId (string) --
The unique identifier of the Agent Insight.
applicationName (string) --
The application that the Agent Insight belongs to.
batchEvaluationId (string) --
The Batch Evaluation that produced this Insight, when applicable.
customerMetadataStatus (string) --
Specifies whether customer-encrypted metadata is available for the Agent Insight.
updatedAt (datetime) --
The date and time when the Agent Insight was last updated.
archiveStatus (string) --
Specifies whether the Agent Insight is active or archived. Valid values:
ACTIVE - The insight is currently active.
ARCHIVED - The insight has been archived.
lastSeenAt (datetime) --
The most recent date and time when the insight pattern was observed.
nextToken (string) --
A token to use to retrieve the next page of results. This value is present only when additional results are available.
Deletes the record of a dataset generation. The examples the generation added stay in the dataset's draft. To remove them, find them with the generationIds filter of ListDatasetExamples, then call DeleteDatasetExamples.
See also: AWS API Documentation
Request Syntax
client.delete_dataset_generation(
datasetId='string',
generationId='string'
)
string
[REQUIRED]
The unique identifier of the dataset.
string
[REQUIRED]
The unique identifier of the dataset generation.
dict
Response Syntax
{
'generationId': 'string',
'status': 'IN_PROGRESS'|'COMPLETED'|'COMPLETED_WITH_ERRORS'|'FAILED'|'DELETING'
}
Response Structure
(dict) --
generationId (string) --
The unique identifier of the dataset generation.
status (string) --
The status of the dataset generation.
Lists the generations for a dataset. AgentCore keeps a generation until you delete it with DeleteDatasetGeneration or delete the dataset. To get a generation's counts and the request it was started with, call GetDatasetGeneration.
See also: AWS API Documentation
Request Syntax
client.list_dataset_generations(
datasetId='string',
maxResults=123,
nextToken='string'
)
string
[REQUIRED]
The unique identifier of the dataset.
integer
The maximum number of results to return in the response. The default is 20. If the total number of results is greater than this value, use the token returned in the response in the nextToken field when making another request to return the next batch of results.
string
If the total number of results is greater than the maxResults value provided in the request, enter the token returned in the nextToken field in the response in this field to return the next batch of results.
dict
Response Syntax
{
'datasetGenerations': [
{
'generationId': 'string',
'datasetId': 'string',
'name': 'string',
'description': 'string',
'scenarioType': 'FIXED'|'DYNAMIC',
'status': 'IN_PROGRESS'|'COMPLETED'|'COMPLETED_WITH_ERRORS'|'FAILED'|'DELETING',
'createdAt': datetime(2015, 1, 1),
'completedAt': datetime(2015, 1, 1)
},
],
'nextToken': 'string'
}
Response Structure
(dict) --
datasetGenerations (list) --
The dataset generations.
(dict) --
A summary of a dataset generation. To get its counts and the request it was started with, call GetDatasetGeneration.
generationId (string) --
The unique identifier of the dataset generation.
datasetId (string) --
The unique identifier of the dataset.
name (string) --
The name of the dataset generation.
description (string) --
The description of the dataset generation.
scenarioType (string) --
The type of scenario the generation produces.
status (string) --
The status of the dataset generation.
createdAt (datetime) --
The time at which the dataset generation started.
completedAt (datetime) --
The time at which the dataset generation finished. Present only when the generation has finished.
nextToken (string) --
If the total number of results is greater than the maxResults value provided in the request, use this token when making another request in the nextToken field to return the next batch of results.
Updates the archive status of an Amazon Bedrock AgentCore Agent Insight.
See also: AWS API Documentation
Request Syntax
client.update_agent_insight(
applicationName='string',
agentInsightId='string',
archiveStatus='ACTIVE'|'ARCHIVED'
)
string
[REQUIRED]
The application that the Agent Insight belongs to.
string
[REQUIRED]
The unique identifier of the Agent Insight to update.
string
[REQUIRED]
The archive status to apply to the Agent Insight. Valid values:
ACTIVE - Marks the insight as active.
ARCHIVED - Marks the insight as archived.
dict
Response Syntax
{
'agentInsightId': 'string',
'archiveStatus': 'ACTIVE'|'ARCHIVED',
'updatedAt': datetime(2015, 1, 1)
}
Response Structure
(dict) --
agentInsightId (string) --
The unique identifier of the updated Agent Insight.
archiveStatus (string) --
The archive status of the updated Agent Insight. Valid values:
ACTIVE - The insight is active.
ARCHIVED - The insight is archived.
updatedAt (datetime) --
The date and time when the Agent Insight was updated.
{'datasetExecutionDetails': {'agentInvocation': {'agentCoreRuntime': {'agentRuntimeEndpointArn': 'string',
'agentRuntimeVersion': 'string',
'requestTemplate': 'string'}},
'dataset': {'datasetId': 'string',
'datasetVersion': 'string'}}}
Retrieves detailed information about a batch evaluation, including its status, configuration, results, and any error details.
See also: AWS API Documentation
Request Syntax
client.get_batch_evaluation(
batchEvaluationId='string'
)
string
[REQUIRED]
The unique identifier of the batch evaluation to retrieve.
dict
Response Syntax
{
'batchEvaluationId': 'string',
'batchEvaluationArn': 'string',
'batchEvaluationName': 'string',
'status': 'PENDING'|'IN_PROGRESS'|'COMPLETED'|'COMPLETED_WITH_ERRORS'|'FAILED'|'STOPPING'|'STOPPED'|'DELETING',
'createdAt': datetime(2015, 1, 1),
'evaluators': [
{
'evaluatorId': 'string'
},
],
'insights': [
{
'insightId': 'string'
},
],
'dataSourceConfig': {
'cloudWatchLogs': {
'serviceNames': [
'string',
],
'logGroupNames': [
'string',
],
'logGroupNamePrefixes': [
'string',
],
'filterConfig': {
'sessionIds': [
'string',
],
'timeRange': {
'startTime': datetime(2015, 1, 1),
'endTime': datetime(2015, 1, 1)
},
'sessionTraceIds': [
{
'sessionId': 'string',
'traceIds': [
'string',
]
},
]
}
},
'onlineEvaluationConfigSource': {
'onlineEvaluationConfigArn': 'string',
'timeRange': {
'startTime': datetime(2015, 1, 1),
'endTime': datetime(2015, 1, 1)
}
}
},
'outputConfig': {
'cloudWatchConfig': {
'logGroupName': 'string',
'logStreamName': 'string',
'metricsNamespace': 'string',
'resultDestination': 'DEDICATED_LOG_GROUP'|'SOURCE_LOG_GROUP'
}
},
'evaluationResults': {
'numberOfSessionsCompleted': 123,
'numberOfSessionsInProgress': 123,
'numberOfSessionsFailed': 123,
'totalNumberOfSessions': 123,
'numberOfSessionsIgnored': 123,
'evaluatorSummaries': [
{
'evaluatorId': 'string',
'evaluationDimensionName': 'string',
'statistics': {
'averageScore': 123.0
},
'totalEvaluated': 123,
'totalFailed': 123
},
]
},
'insightResults': [
{
'type': 'FAILURE',
'name': 'string',
'description': 'string',
'createdAt': datetime(2015, 1, 1),
'affectedSessionCount': 123,
'sampleSessions': [
{
'sessionId': 'string',
'lastSeenAt': datetime(2015, 1, 1),
'findings': [
{
'targetId': 'string',
'targetType': 'SPAN_ID'|'TRACE_ID'|'SPAN_NAME',
'description': 'string',
'category': 'string',
'confidence': 123.0
},
]
},
],
'rootCauses': [
{
'name': 'string',
'explanation': 'string',
'affectedSessionCount': 123,
'suggestedFix': 'string',
'sampleSessions': [
{
'sessionId': 'string',
'lastSeenAt': datetime(2015, 1, 1),
'findings': [
{
'targetId': 'string',
'targetType': 'SPAN_ID'|'TRACE_ID'|'SPAN_NAME',
'description': 'string',
'category': 'string',
'confidence': 123.0
},
]
},
]
},
]
},
],
'failureAnalysisResult': {
'failures': [
{
'clusterId': 123,
'name': 'string',
'description': 'string',
'affectedSessionCount': 123,
'subCategories': [
{
'clusterId': 123,
'name': 'string',
'description': 'string',
'affectedSessionCount': 123,
'rootCauses': [
{
'clusterId': 123,
'name': 'string',
'rootCause': 'string',
'recommendation': 'string',
'affectedSessionCount': 123,
'affectedSessions': [
{
'sessionId': 'string',
'explanation': 'string',
'fixType': 'string',
'recommendation': 'string',
'failureSpans': [
{
'spanId': 'string',
'traceId': 'string',
'signals': [
{
'category': 'execution-error-category-authentication'|'execution-error-category-resource-not-found'|'execution-error-category-service-errors'|'execution-error-category-rate-limiting'|'execution-error-category-formatting'|'execution-error-category-timeout'|'execution-error-category-resource-exhaustion'|'execution-error-category-environment'|'execution-error-category-tool-schema'|'task-instruction-category-non-compliance'|'task-instruction-category-problem-id'|'incorrect-actions-category-tool-selection'|'incorrect-actions-category-poor-information-retrieval'|'incorrect-actions-category-clarification'|'incorrect-actions-category-inappropriate-info-request'|'context-handling-error-category-context-handling-failures'|'hallucination-category-hall-capabilities'|'hallucination-category-hall-misunderstand'|'hallucination-category-hall-usage'|'hallucination-category-hall-history'|'hallucination-category-hall-params'|'hallucination-category-fabricate-tool-outputs'|'repetitive-behavior-category-repetition-tool'|'repetitive-behavior-category-repetition-info'|'repetitive-behavior-category-step-repetition'|'orchestration-related-errors-category-reasoning-mismatch'|'orchestration-related-errors-category-goal-deviation'|'orchestration-related-errors-category-premature-termination'|'orchestration-related-errors-category-unaware-termination'|'llm-output-category-nonsensical'|'configuration-mismatch-category-tool-definition'|'coding-use-case-specific-failure-types-category-edge-case-oversights'|'coding-use-case-specific-failure-types-category-dependency-issues'|'other',
'evidence': 'string',
'confidence': 123.0
},
]
},
]
},
]
},
]
},
]
},
]
},
'userIntentResult': {
'userIntents': [
{
'clusterId': 123,
'name': 'string',
'description': 'string',
'affectedSessionCount': 123,
'affectedSessions': [
{
'sessionId': 'string',
'userMessages': [
'string',
]
},
]
},
]
},
'executionSummaryResult': {
'executionSummaries': [
{
'clusterId': 123,
'name': 'string',
'description': 'string',
'affectedSessionCount': 123,
'affectedSessions': [
{
'sessionId': 'string',
'approachTaken': 'string',
'finalOutcome': 'string'
},
]
},
]
},
'errorDetails': [
'string',
],
'description': 'string',
'updatedAt': datetime(2015, 1, 1),
'kmsKeyArn': 'string',
'datasetExecutionDetails': {
'dataset': {
'datasetId': 'string',
'datasetVersion': 'string'
},
'agentInvocation': {
'agentCoreRuntime': {
'agentRuntimeEndpointArn': 'string',
'agentRuntimeVersion': 'string',
'requestTemplate': 'string'
}
}
}
}
Response Structure
(dict) --
batchEvaluationId (string) --
The unique identifier of the batch evaluation.
batchEvaluationArn (string) --
The Amazon Resource Name (ARN) of the batch evaluation.
batchEvaluationName (string) --
The name of the batch evaluation.
status (string) --
The current status of the batch evaluation.
createdAt (datetime) --
The timestamp when the batch evaluation was created.
evaluators (list) --
The list of evaluators applied during the batch evaluation.
(dict) --
An evaluator to run against sessions during batch evaluation.
evaluatorId (string) --
The unique identifier of the evaluator. Can reference built-in evaluators (e.g., Builtin.Helpfulness) or custom evaluators.
insights (list) --
The list of insight analyses applied during the batch evaluation.
(dict) --
A reference to an insight analysis to run against sessions during batch evaluation. Insights provide deeper analysis beyond individual evaluator scores, including failure detection, user intent clustering, and execution summarization.
insightId (string) --
The unique identifier of the insight to run.
dataSourceConfig (dict) --
The data source configuration specifying where agent traces are pulled from.
cloudWatchLogs (dict) --
Configuration for pulling agent session traces from CloudWatch Logs.
serviceNames (list) --
The list of agent service names to filter traces within the specified log groups.
(string) --
logGroupNames (list) --
The list of CloudWatch log group names to read agent traces from. Maximum of 10 log groups.
(string) --
logGroupNamePrefixes (list) --
The list of CloudWatch log group name prefixes to read agent traces from. Specify this instead of logGroupNames to match log groups by prefix. Maximum of 5 prefixes. Specify either logGroupNames or logGroupNamePrefixes, not both. One of the two is required.
(string) --
Prefix of a CloudWatch Logs log group name.
filterConfig (dict) --
Optional filter configuration to narrow down which sessions to evaluate.
sessionIds (list) --
A list of specific session IDs to evaluate. If specified, only these sessions are included in the evaluation.
(string) --
timeRange (dict) --
The time range filter for selecting sessions to evaluate.
startTime (datetime) --
The start time of the time range. Only sessions with activity at or after this timestamp are included.
endTime (datetime) --
The end time of the time range. Only sessions with activity before this timestamp are included.
sessionTraceIds (list) --
A list of session and trace ID pairs that restrict evaluation to specific traces within a session. If specified, only the listed traces are evaluated instead of the entire session.
(dict) --
A pairing of a session with the specific trace IDs to evaluate within that session. Use this to evaluate individual traces rather than an entire session.
sessionId (string) --
The unique identifier of the session that contains the traces to evaluate.
traceIds (list) --
The list of trace IDs within the session to evaluate.
(string) --
onlineEvaluationConfigSource (dict) --
Reference an existing OnlineEvaluationConfig as session source
onlineEvaluationConfigArn (string) --
The Amazon Resource Name (ARN) of the online evaluation configuration to use as the session source.
timeRange (dict) --
Optional session filter configuration to narrow down which sessions from the online evaluation configuration to include.
startTime (datetime) --
The start time of the time range. Only sessions with activity at or after this timestamp are included.
endTime (datetime) --
The end time of the time range. Only sessions with activity before this timestamp are included.
outputConfig (dict) --
The output configuration specifying where evaluation results are written.
cloudWatchConfig (dict) --
The CloudWatch Logs configuration for writing evaluation results.
logGroupName (string) --
The name of the CloudWatch log group where evaluation results will be written. This value doesn't apply when resultDestination is SOURCE_LOG_GROUP, because results are written back to the trace source log group. The name can't be under the service-reserved /aws/bedrock-agentcore/evaluations/ namespace, apart from the service-managed default group.
logStreamName (string) --
The name of the CloudWatch log stream where evaluation results will be written.
metricsNamespace (string) --
The CloudWatch metrics namespace where evaluation result metrics are published. If you omit this value, the service publishes metrics to Bedrock-AgentCore/Evaluations. This value can't begin with AWS/.
resultDestination (string) --
The destination where evaluation results are written. Valid values:
DEDICATED_LOG_GROUP (default) – Writes results to a dedicated result log group.
SOURCE_LOG_GROUP – Writes results back to the log group that the agent traces were read from. If you use this value, don't specify logGroupName.
evaluationResults (dict) --
The aggregated evaluation results, including session completion counts and evaluator score summaries.
numberOfSessionsCompleted (integer) --
The number of sessions that have been successfully evaluated.
numberOfSessionsInProgress (integer) --
The number of sessions currently being evaluated.
numberOfSessionsFailed (integer) --
The number of sessions that failed evaluation.
totalNumberOfSessions (integer) --
The total number of sessions included in the batch evaluation.
numberOfSessionsIgnored (integer) --
The number of sessions that were ignored during evaluation.
evaluatorSummaries (list) --
A list of per-evaluator summary statistics.
(dict) --
Summary statistics for a single evaluator within a batch evaluation.
evaluatorId (string) --
The unique identifier of the evaluator.
evaluationDimensionName (string) --
The name of the dimension that this summary covers. This field is set only for evaluators that use a multi-dimensional rating scale, where each dimension is summarized separately. For single-scale evaluators, this field is not set.
statistics (dict) --
The aggregated statistics for this evaluator.
averageScore (float) --
The average score across all evaluated sessions for this evaluator.
totalEvaluated (integer) --
The total number of sessions evaluated by this evaluator.
totalFailed (integer) --
The total number of sessions that failed evaluation by this evaluator.
insightResults (list) --
The insights generated when the batch evaluation includes the Builtin.FailureAnalysis evaluator. Each insight groups related failures found across the evaluated sessions. An empty list means that no failures were found. If the analysis fails, this field is omitted and errorDetails describes the error.
(dict) --
An insight that groups related failures found across the sessions in a batch evaluation.
type (string) --
The type of insight. FAILURE indicates a group of related failures.
name (string) --
A generated label for the insight. The name is not an identifier and can differ between batch evaluations.
description (string) --
A description of the failure pattern that the insight represents.
createdAt (datetime) --
The timestamp when the insight was generated.
affectedSessionCount (integer) --
The number of distinct sessions affected by the insight.
sampleSessions (list) --
A sample of up to 10 sessions affected by the insight. Returned only when the insight has no root causes. Otherwise, sample sessions are returned for each root cause.
(dict) --
A session affected by an insight or root cause. Use the session ID to find the per-session evaluation results in CloudWatch Logs.
sessionId (string) --
The unique identifier of the session.
lastSeenAt (datetime) --
The latest timestamp observed for the session in the batch evaluation input.
findings (list) --
The failures found in the session that belong to the insight or root cause, up to 10.
(dict) --
A failure found in a session.
targetId (string) --
The identifier or name of the location in the session where the failure was found, interpreted according to targetType.
targetType (string) --
The type of location that targetId refers to.
description (string) --
A description of the failure.
category (string) --
The category of the failure.
confidence (float) --
The confidence that the finding is a failure, from 0 to 1.
rootCauses (list) --
The root causes identified for the insight.
(dict) --
A root cause that contributes to an insight.
name (string) --
A generated label for the root cause.
explanation (string) --
An explanation of how the root cause leads to the failures in the insight.
affectedSessionCount (integer) --
The number of distinct sessions affected by the root cause. A session can be affected by more than one root cause of the same insight.
suggestedFix (string) --
A suggested change to the agent that addresses the root cause.
sampleSessions (list) --
A sample of up to 10 sessions affected by the root cause.
(dict) --
A session affected by an insight or root cause. Use the session ID to find the per-session evaluation results in CloudWatch Logs.
sessionId (string) --
The unique identifier of the session.
lastSeenAt (datetime) --
The latest timestamp observed for the session in the batch evaluation input.
findings (list) --
The failures found in the session that belong to the insight or root cause, up to 10.
(dict) --
A failure found in a session.
targetId (string) --
The identifier or name of the location in the session where the failure was found, interpreted according to targetType.
targetType (string) --
The type of location that targetId refers to.
description (string) --
A description of the failure.
category (string) --
The category of the failure.
confidence (float) --
The confidence that the finding is a failure, from 0 to 1.
failureAnalysisResult (dict) --
The failure analysis results from insights, containing categorized failure clusters with root causes and recommendations.
failures (list) --
The list of failure category clusters identified across analyzed sessions.
(dict) --
A top-level failure category identified by clustering similar failure patterns across sessions.
clusterId (integer) --
The unique identifier of the failure category cluster.
name (string) --
The name of the failure category.
description (string) --
A description of the failure category pattern.
affectedSessionCount (integer) --
The number of sessions affected by this failure category.
subCategories (list) --
The list of failure subcategories within this category.
(dict) --
A subcategory of failures within a top-level failure category.
clusterId (integer) --
The unique identifier of the failure subcategory cluster.
name (string) --
The name of the failure subcategory.
description (string) --
A description of the failure subcategory pattern.
affectedSessionCount (integer) --
The number of sessions affected by this failure subcategory.
rootCauses (list) --
The list of root cause clusters identified within this subcategory.
(dict) --
A cluster of similar root causes identified within a failure subcategory.
clusterId (integer) --
The unique identifier of the root cause cluster.
name (string) --
The name of the root cause cluster.
rootCause (string) --
The root cause explanation for this cluster of failures.
recommendation (string) --
The recommended fix for this root cause.
affectedSessionCount (integer) --
The number of sessions affected by this root cause.
affectedSessions (list) --
The list of sessions affected by this root cause.
(dict) --
A session affected by a detected failure pattern, including root cause details.
sessionId (string) --
The unique identifier of the affected session.
explanation (string) --
An explanation of how the failure manifested in this session.
fixType (string) --
The type of fix recommended for this failure.
recommendation (string) --
The specific fix recommendation for this session.
failureSpans (list) --
The list of spans where failures were detected in this session.
(dict) --
Details about a specific span where a failure was detected.
spanId (string) --
The unique identifier of the span where the failure occurred.
traceId (string) --
The trace identifier associated with the failure span.
signals (list) --
The failure signals detected in this span.
(dict) --
A signal indicating a detected failure within a span.
category (string) --
The failure category classification for this signal.
evidence (string) --
The evidence supporting the failure detection.
confidence (float) --
The confidence score of the failure detection.
userIntentResult (dict) --
The user intent clustering results from insights, containing grouped user intents across evaluated sessions.
userIntents (list) --
The list of user intent clusters identified across analyzed sessions.
(dict) --
A cluster of similar user intents identified across sessions.
clusterId (integer) --
The unique identifier of the user intent cluster.
name (string) --
The name of the user intent cluster.
description (string) --
A description of the user intent pattern.
affectedSessionCount (integer) --
The number of sessions with this user intent.
affectedSessions (list) --
The list of sessions with this user intent.
(dict) --
A session associated with a user intent cluster.
sessionId (string) --
The unique identifier of the session.
userMessages (list) --
The user messages from this session that contributed to the intent cluster.
(string) --
executionSummaryResult (dict) --
The execution summary clustering results from insights, containing grouped execution patterns across evaluated sessions.
executionSummaries (list) --
The list of execution summary clusters identified across analyzed sessions.
(dict) --
A cluster of similar execution patterns identified across sessions.
clusterId (integer) --
The unique identifier of the execution summary cluster.
name (string) --
The name of the execution pattern cluster.
description (string) --
A description of the execution pattern.
affectedSessionCount (integer) --
The number of sessions with this execution pattern.
affectedSessions (list) --
The list of sessions with this execution pattern.
(dict) --
A session associated with an execution summary cluster.
sessionId (string) --
The unique identifier of the session.
approachTaken (string) --
The approach taken by the agent during this session.
finalOutcome (string) --
The final outcome of the session.
errorDetails (list) --
The error details if the batch evaluation encountered failures.
(string) --
description (string) --
The description of the batch evaluation.
updatedAt (datetime) --
The timestamp when the batch evaluation was last updated.
kmsKeyArn (string) --
The ARN of the KMS key used to encrypt evaluation data.
datasetExecutionDetails (dict) --
The details of the dataset and agent that the batch evaluation ran, including the dataset version and agent runtime version that the service resolved when the batch evaluation started. This field is present only if the batch evaluation was started with a datasetExecutionConfig.
dataset (dict) --
The dataset and the version of it that the batch evaluation ran.
datasetId (string) --
The unique identifier of the dataset.
datasetVersion (string) --
The version of the dataset that the batch evaluation ran. If the request didn't specify a version, this is the latest published version when the batch evaluation started.
agentInvocation (dict) --
The agent that the batch evaluation invoked.
agentCoreRuntime (dict) --
The details of the agent hosted on AgentCore Runtime that the batch evaluation invoked.
agentRuntimeEndpointArn (string) --
The Amazon Resource Name (ARN) of the agent runtime endpoint that the batch evaluation invoked.
agentRuntimeVersion (string) --
The version of the agent runtime that the endpoint pointed to when the batch evaluation started. If you update the endpoint after the batch evaluation starts, each later turn uses the version that the endpoint points to when the turn runs.
requestTemplate (string) --
The request payload template that the batch evaluation used for each turn. If the request didn't specify a template, this is the default template.
{'datasetExecutionConfig': {'agentInvocation': {'agentCoreRuntime': {'agentRuntimeEndpointArn': 'string',
'requestTemplate': 'string'}},
'dataset': {'datasetId': 'string',
'datasetVersion': 'string'}}}
Starts a batch evaluation job that evaluates agent performance across multiple sessions. Batch evaluations pull agent traces from CloudWatch Logs or an existing online evaluation configuration and run specified evaluators and insights against them.
See also: AWS API Documentation
Request Syntax
client.start_batch_evaluation(
batchEvaluationName='string',
evaluators=[
{
'evaluatorId': 'string'
},
],
insights=[
{
'insightId': 'string'
},
],
dataSourceConfig={
'cloudWatchLogs': {
'serviceNames': [
'string',
],
'logGroupNames': [
'string',
],
'logGroupNamePrefixes': [
'string',
],
'filterConfig': {
'sessionIds': [
'string',
],
'timeRange': {
'startTime': datetime(2015, 1, 1),
'endTime': datetime(2015, 1, 1)
},
'sessionTraceIds': [
{
'sessionId': 'string',
'traceIds': [
'string',
]
},
]
}
},
'onlineEvaluationConfigSource': {
'onlineEvaluationConfigArn': 'string',
'timeRange': {
'startTime': datetime(2015, 1, 1),
'endTime': datetime(2015, 1, 1)
}
}
},
clientToken='string',
evaluationMetadata={
'sessionMetadata': [
{
'sessionId': 'string',
'testScenarioId': 'string',
'groundTruth': {
'inline': {
'assertions': [
{
'text': 'string'
},
],
'expectedTrajectory': {
'toolNames': [
'string',
]
},
'turns': [
{
'input': {
'prompt': 'string'
},
'expectedResponse': {
'text': 'string'
}
},
]
}
},
'metadata': {
'string': 'string'
}
},
]
},
tags={
'string': 'string'
},
kmsKeyArn='string',
description='string',
outputConfig={
'cloudWatchConfig': {
'logGroupName': 'string',
'logStreamName': 'string',
'metricsNamespace': 'string',
'resultDestination': 'DEDICATED_LOG_GROUP'|'SOURCE_LOG_GROUP'
}
},
datasetExecutionConfig={
'dataset': {
'datasetId': 'string',
'datasetVersion': 'string'
},
'agentInvocation': {
'agentCoreRuntime': {
'agentRuntimeEndpointArn': 'string',
'requestTemplate': 'string'
}
}
}
)
string
[REQUIRED]
The name of the batch evaluation. Must be unique within your account.
list
The list of evaluators to apply during the batch evaluation. Can include both built-in evaluators and custom evaluators. Maximum of 10 evaluators.
(dict) --
An evaluator to run against sessions during batch evaluation.
evaluatorId (string) -- [REQUIRED]
The unique identifier of the evaluator. Can reference built-in evaluators (e.g., Builtin.Helpfulness) or custom evaluators.
list
The list of insight analyses to run against sessions during the batch evaluation. Maximum of 10 insights.
(dict) --
A reference to an insight analysis to run against sessions during batch evaluation. Insights provide deeper analysis beyond individual evaluator scores, including failure detection, user intent clustering, and execution summarization.
insightId (string) -- [REQUIRED]
The unique identifier of the insight to run.
dict
[REQUIRED]
The data source configuration that specifies where to pull agent session traces from for evaluation.
cloudWatchLogs (dict) --
Configuration for pulling agent session traces from CloudWatch Logs.
serviceNames (list) -- [REQUIRED]
The list of agent service names to filter traces within the specified log groups.
(string) --
logGroupNames (list) --
The list of CloudWatch log group names to read agent traces from. Maximum of 10 log groups.
(string) --
logGroupNamePrefixes (list) --
The list of CloudWatch log group name prefixes to read agent traces from. Specify this instead of logGroupNames to match log groups by prefix. Maximum of 5 prefixes. Specify either logGroupNames or logGroupNamePrefixes, not both. One of the two is required.
(string) --
Prefix of a CloudWatch Logs log group name.
filterConfig (dict) --
Optional filter configuration to narrow down which sessions to evaluate.
sessionIds (list) --
A list of specific session IDs to evaluate. If specified, only these sessions are included in the evaluation.
(string) --
timeRange (dict) --
The time range filter for selecting sessions to evaluate.
startTime (datetime) --
The start time of the time range. Only sessions with activity at or after this timestamp are included.
endTime (datetime) --
The end time of the time range. Only sessions with activity before this timestamp are included.
sessionTraceIds (list) --
A list of session and trace ID pairs that restrict evaluation to specific traces within a session. If specified, only the listed traces are evaluated instead of the entire session.
(dict) --
A pairing of a session with the specific trace IDs to evaluate within that session. Use this to evaluate individual traces rather than an entire session.
sessionId (string) -- [REQUIRED]
The unique identifier of the session that contains the traces to evaluate.
traceIds (list) -- [REQUIRED]
The list of trace IDs within the session to evaluate.
(string) --
onlineEvaluationConfigSource (dict) --
Reference an existing OnlineEvaluationConfig as session source
onlineEvaluationConfigArn (string) -- [REQUIRED]
The Amazon Resource Name (ARN) of the online evaluation configuration to use as the session source.
timeRange (dict) --
Optional session filter configuration to narrow down which sessions from the online evaluation configuration to include.
startTime (datetime) --
The start time of the time range. Only sessions with activity at or after this timestamp are included.
endTime (datetime) --
The end time of the time range. Only sessions with activity before this timestamp are included.
string
A unique, case-sensitive identifier to ensure that the API request completes no more than one time. If this token matches a previous request, the service ignores the request, but does not return an error.
This field is autopopulated if not provided.
dict
Optional metadata for the evaluation, including session-specific ground truth data and test scenario identifiers.
sessionMetadata (list) --
A list of session metadata entries containing ground truth data and test scenario identifiers for specific sessions.
(dict) --
Metadata for a specific session in a batch evaluation, including ground truth data and test scenario identifiers.
sessionId (string) -- [REQUIRED]
The unique identifier of the session this metadata applies to.
testScenarioId (string) --
An optional test scenario identifier for categorizing and tracking evaluation results.
groundTruth (dict) --
The ground truth data for this session, including expected responses and assertions.
inline (dict) --
Inline ground truth data provided directly in the request.
assertions (list) --
Assertions for evaluation, reuses common model EvaluationContentList.
(dict) --
A content block for ground truth data in evaluation reference inputs. Supports text content for expected responses and assertions.
text (string) --
The text content of the ground truth data. Used for expected response text and assertion statements.
expectedTrajectory (dict) --
The expected tool call sequence for trajectory evaluation.
toolNames (list) --
The list of tool names representing the expected tool call sequence.
(string) --
turns (list) --
A list of per-turn ground truth data, each containing an input prompt and expected response.
(dict) --
Ground truth data for a single conversation turn.
input (dict) --
The input for this conversation turn.
prompt (string) --
The text prompt for this conversation turn.
expectedResponse (dict) --
The expected response for this conversation turn.
text (string) --
The text content of the ground truth data. Used for expected response text and assertion statements.
metadata (dict) --
Additional key-value metadata associated with this session.
(string) --
(string) --
dict
A map of tag keys and values to associate with the batch evaluation.
(string) --
(string) --
string
The ARN of the KMS key used to encrypt evaluation data. If provided, customer data is encrypted at rest with the specified key.
string
The description of the batch evaluation.
dict
Output destination configuration.
cloudWatchConfig (dict) --
The CloudWatch Logs configuration for writing evaluation results.
logGroupName (string) --
The name of the CloudWatch log group where evaluation results will be written. This value doesn't apply when resultDestination is SOURCE_LOG_GROUP, because results are written back to the trace source log group. The name can't be under the service-reserved /aws/bedrock-agentcore/evaluations/ namespace, apart from the service-managed default group.
logStreamName (string) --
The name of the CloudWatch log stream where evaluation results will be written.
metricsNamespace (string) --
The CloudWatch metrics namespace where evaluation result metrics are published. If you omit this value, the service publishes metrics to Bedrock-AgentCore/Evaluations. This value can't begin with AWS/.
resultDestination (string) --
The destination where evaluation results are written. Valid values:
DEDICATED_LOG_GROUP (default) – Writes results to a dedicated result log group.
SOURCE_LOG_GROUP – Writes results back to the log group that the agent traces were read from. If you use this value, don't specify logGroupName.
dict
The configuration for running the scenarios in a dataset against an agent before evaluating the resulting sessions. When you specify this field, the batch evaluation runs each scenario in the dataset against the agent. It then evaluates the sessions that the agent produces. You must also specify evaluators and a dataSourceConfig that uses cloudWatchLogs. Configure cloudWatchLogs with the service name and log groups that the agent uses for its traces. You can't specify insights or an onlineEvaluationConfigSource data source with this field.
dataset (dict) -- [REQUIRED]
The dataset whose scenarios are sent to the agent. The dataset must use the predefined evaluation schema, and the version that runs must contain between 1 and 1,000 scenarios.
datasetId (string) -- [REQUIRED]
The unique identifier of the dataset.
datasetVersion (string) --
The number of a published version of the dataset. You can't run the DRAFT version. If you don't specify a version, the batch evaluation uses the latest published version.
agentInvocation (dict) -- [REQUIRED]
Specifies the agent runtime endpoint and the request template that the batch evaluation uses to invoke the agent for each scenario in the dataset.
agentCoreRuntime (dict) --
The configuration for invoking an agent hosted on AgentCore Runtime.
agentRuntimeEndpointArn (string) -- [REQUIRED]
The Amazon Resource Name (ARN) of the agent runtime endpoint to invoke. Specify an endpoint in the same account and Region as the batch evaluation. The service requires the endpoint and its agent runtime version to be ready. The agent runtime must use the HTTP protocol and IAM inbound authorization.
requestTemplate (string) --
A JSON object template for the request payload sent to the agent for each turn. The template must contain the placeholder {{turn.input}} exactly once, as the complete value of a JSON string and not as a field name. The service replaces the placeholder with the input of the turn. If the input isn't a string, the service inserts it as a JSON value. If you don't specify a template, the service uses {"prompt": "{{turn.input}}"}.
dict
Response Syntax
{
'batchEvaluationId': 'string',
'batchEvaluationArn': 'string',
'batchEvaluationName': 'string',
'evaluators': [
{
'evaluatorId': 'string'
},
],
'insights': [
{
'insightId': 'string'
},
],
'status': 'PENDING'|'IN_PROGRESS'|'COMPLETED'|'COMPLETED_WITH_ERRORS'|'FAILED'|'STOPPING'|'STOPPED'|'DELETING',
'createdAt': datetime(2015, 1, 1),
'outputConfig': {
'cloudWatchConfig': {
'logGroupName': 'string',
'logStreamName': 'string',
'metricsNamespace': 'string',
'resultDestination': 'DEDICATED_LOG_GROUP'|'SOURCE_LOG_GROUP'
}
},
'tags': {
'string': 'string'
},
'kmsKeyArn': 'string',
'description': 'string'
}
Response Structure
(dict) --
batchEvaluationId (string) --
The unique identifier of the created batch evaluation.
batchEvaluationArn (string) --
The Amazon Resource Name (ARN) of the created batch evaluation.
batchEvaluationName (string) --
The name of the batch evaluation.
evaluators (list) --
The list of evaluators applied during the batch evaluation.
(dict) --
An evaluator to run against sessions during batch evaluation.
evaluatorId (string) --
The unique identifier of the evaluator. Can reference built-in evaluators (e.g., Builtin.Helpfulness) or custom evaluators.
insights (list) --
The list of insight analyses applied during the batch evaluation.
(dict) --
A reference to an insight analysis to run against sessions during batch evaluation. Insights provide deeper analysis beyond individual evaluator scores, including failure detection, user intent clustering, and execution summarization.
insightId (string) --
The unique identifier of the insight to run.
status (string) --
The status of the batch evaluation.
createdAt (datetime) --
The timestamp when the batch evaluation was created.
outputConfig (dict) --
The output configuration specifying where evaluation results are written.
cloudWatchConfig (dict) --
The CloudWatch Logs configuration for writing evaluation results.
logGroupName (string) --
The name of the CloudWatch log group where evaluation results will be written. This value doesn't apply when resultDestination is SOURCE_LOG_GROUP, because results are written back to the trace source log group. The name can't be under the service-reserved /aws/bedrock-agentcore/evaluations/ namespace, apart from the service-managed default group.
logStreamName (string) --
The name of the CloudWatch log stream where evaluation results will be written.
metricsNamespace (string) --
The CloudWatch metrics namespace where evaluation result metrics are published. If you omit this value, the service publishes metrics to Bedrock-AgentCore/Evaluations. This value can't begin with AWS/.
resultDestination (string) --
The destination where evaluation results are written. Valid values:
DEDICATED_LOG_GROUP (default) – Writes results to a dedicated result log group.
SOURCE_LOG_GROUP – Writes results back to the log group that the agent traces were read from. If you use this value, don't specify logGroupName.
tags (dict) --
The tags associated with the batch evaluation.
(string) --
(string) --
kmsKeyArn (string) --
The ARN of the KMS key used to encrypt evaluation data.
description (string) --
The description of the batch evaluation.