2021/11/29 - Amazon CloudWatch Evidently - 32 new api methods
Changes Introducing Amazon CloudWatch Evidently. This is the first public release of Amazon CloudWatch Evidently.
Returns configuration details about all the projects in the current Region in your account.
See also: AWS API Documentation
Request Syntax
client.list_projects(
maxResults=123,
nextToken='string'
)
integer
The maximum number of results to include in the response.
string
The token to use when requesting the next set of results. You received this token from a previous ListProjects operation.
dict
Response Syntax
{
'nextToken': 'string',
'projects': [
{
'activeExperimentCount': 123,
'activeLaunchCount': 123,
'arn': 'string',
'createdTime': datetime(2015, 1, 1),
'description': 'string',
'experimentCount': 123,
'featureCount': 123,
'lastUpdatedTime': datetime(2015, 1, 1),
'launchCount': 123,
'name': 'string',
'status': 'AVAILABLE'|'UPDATING',
'tags': {
'string': 'string'
}
},
]
}
Response Structure
(dict) --
nextToken (string) --
The token to use in a subsequent ListProjects operation to return the next set of results.
projects (list) --
An array of structures that contain the configuration details of the projects in the Region.
(dict) --
A structure that contains configuration information about an Evidently project.
activeExperimentCount (integer) --
The number of experiments currently in the project.
activeLaunchCount (integer) --
The number of ongoing launches currently in the project.
arn (string) --
The name or ARN of the project.
createdTime (datetime) --
The date and time that the project is created.
description (string) --
The description of the project.
experimentCount (integer) --
The number of experiments currently in the project.
featureCount (integer) --
The number of features currently in the project.
lastUpdatedTime (datetime) --
The date and time that the project was most recently updated.
launchCount (integer) --
The number of launches currently in the project, including launches that are ongoing, completed, and not started yet.
name (string) --
The name of the project.
status (string) --
The current state of the project.
tags (dict) --
The list of tag keys and values associated with this project.
(string) --
(string) --
Deletes an Evidently launch. The feature used for the launch is not deleted.
To stop a launch without deleting it, use StopLaunch .
See also: AWS API Documentation
Request Syntax
client.delete_launch(
launch='string',
project='string'
)
string
[REQUIRED]
The name of the launch to delete.
string
[REQUIRED]
The name or ARN of the project that contains the launch to delete.
dict
Response Syntax
{}
Response Structure
(dict) --
Updates a launch of a given feature.
Don't use this operation to update the tags of an existing launch. Instead, use TagResource .
See also: AWS API Documentation
Request Syntax
client.update_launch(
description='string',
groups=[
{
'description': 'string',
'feature': 'string',
'name': 'string',
'variation': 'string'
},
],
launch='string',
metricMonitors=[
{
'metricDefinition': {
'entityIdKey': 'string',
'eventPattern': 'string',
'name': 'string',
'unitLabel': 'string',
'valueKey': 'string'
}
},
],
project='string',
randomizationSalt='string',
scheduledSplitsConfig={
'steps': [
{
'groupWeights': {
'string': 123
},
'startTime': datetime(2015, 1, 1)
},
]
}
)
string
An optional description for the launch.
list
An array of structures that contains the feature and variations that are to be used for the launch.
(dict) --
A structure that defines one launch group in a launch. A launch group is a variation of the feature that you are including in the launch.
description (string) --
A description of the launch group.
feature (string) -- [REQUIRED]
The feature that this launch is using.
name (string) -- [REQUIRED]
A name for this launch group.
variation (string) -- [REQUIRED]
The feature variation to use for this launch group.
string
[REQUIRED]
The name of the launch that is to be updated.
list
An array of structures that define the metrics that will be used to monitor the launch performance.
(dict) --
A structure that defines a metric to be used to monitor performance of the variations during a launch.
metricDefinition (dict) -- [REQUIRED]
A structure that defines the metric.
entityIdKey (string) --
The entity, such as a user or session, that does an action that causes a metric value to be recorded. An example is userDetails.userID .
eventPattern (string) --
The EventBridge event pattern that defines how the metric is recorded.
For more information about EventBridge event patterns, see Amazon EventBridge event patterns .
name (string) --
A name for the metric.
unitLabel (string) --
A label for the units that the metric is measuring.
valueKey (string) --
The value that is tracked to produce the metric.
string
[REQUIRED]
The name or ARN of the project that contains the launch that you want to update.
string
When Evidently assigns a particular user session to a launch, it must use a randomization ID to determine which variation the user session is served. This randomization ID is a combination of the entity ID and randomizationSalt . If you omit randomizationSalt , Evidently uses the launch name as the randomizationSalt .
dict
An array of structures that define the traffic allocation percentages among the feature variations during each step of the launch.
steps (list) -- [REQUIRED]
An array of structures that define the traffic allocation percentages among the feature variations during each step of the launch. This also defines the start time of each step.
(dict) --
This structure defines the traffic allocation percentages among the feature variations during one step of a launch, and the start time of that step.
groupWeights (dict) -- [REQUIRED]
The traffic allocation percentages among the feature variations during one step of a launch. This is a set of key-value pairs. The keys are variation names. The values represent the percentage of traffic to allocate to that variation during this step.
(string) --
(integer) --
startTime (datetime) -- [REQUIRED]
The date and time that this step of the launch starts.
dict
Response Syntax
{
'launch': {
'arn': 'string',
'createdTime': datetime(2015, 1, 1),
'description': 'string',
'execution': {
'endedTime': datetime(2015, 1, 1),
'startedTime': datetime(2015, 1, 1)
},
'groups': [
{
'description': 'string',
'featureVariations': {
'string': 'string'
},
'name': 'string'
},
],
'lastUpdatedTime': datetime(2015, 1, 1),
'metricMonitors': [
{
'metricDefinition': {
'entityIdKey': 'string',
'eventPattern': 'string',
'name': 'string',
'unitLabel': 'string',
'valueKey': 'string'
}
},
],
'name': 'string',
'project': 'string',
'randomizationSalt': 'string',
'scheduledSplitsDefinition': {
'steps': [
{
'groupWeights': {
'string': 123
},
'startTime': datetime(2015, 1, 1)
},
]
},
'status': 'CREATED'|'UPDATING'|'RUNNING'|'COMPLETED'|'CANCELLED',
'statusReason': 'string',
'tags': {
'string': 'string'
},
'type': 'aws.evidently.splits'
}
}
Response Structure
(dict) --
launch (dict) --
A structure that contains the new configuration of the launch that was updated.
arn (string) --
The ARN of the launch.
createdTime (datetime) --
The date and time that the launch is created.
description (string) --
The description of the launch.
execution (dict) --
A structure that contains information about the start and end times of the launch.
endedTime (datetime) --
The date and time that the launch ended.
startedTime (datetime) --
The date and time that the launch started.
groups (list) --
An array of structures that define the feature variations that are being used in the launch.
(dict) --
A structure that defines one launch group in a launch. A launch group is a variation of the feature that you are including in the launch.
description (string) --
A description of the launch group.
featureVariations (dict) --
The feature variation for this launch group. This is a key-value pair.
(string) --
(string) --
name (string) --
The name of the launch group.
lastUpdatedTime (datetime) --
The date and time that the launch was most recently updated.
metricMonitors (list) --
An array of structures that define the metrics that are being used to monitor the launch performance.
(dict) --
A structure that defines a metric to be used to monitor performance of the variations during a launch.
metricDefinition (dict) --
A structure that defines the metric.
entityIdKey (string) --
The entity, such as a user or session, that does an action that causes a metric value to be recorded.
eventPattern (string) --
The EventBridge event pattern that defines how the metric is recorded.
For more information about EventBridge event patterns, see Amazon EventBridge event patterns .
name (string) --
The name of the metric.
unitLabel (string) --
The label for the units that the metric is measuring.
valueKey (string) --
The value that is tracked to produce the metric.
name (string) --
The name of the launch.
project (string) --
The name or ARN of the project that contains the launch.
randomizationSalt (string) --
This value is used when Evidently assigns a particular user session to the launch, to help create a randomization ID to determine which variation the user session is served. This randomization ID is a combination of the entity ID and randomizationSalt .
scheduledSplitsDefinition (dict) --
An array of structures that define the traffic allocation percentages among the feature variations during each step of the launch.
steps (list) --
An array of structures that define the traffic allocation percentages among the feature variations during each step of the launch. This also defines the start time of each step.
(dict) --
This structure defines the traffic allocation percentages among the feature variations during one step of a launch, and the start time of that step.
groupWeights (dict) --
The traffic allocation percentages among the feature variations during one step of a launch. This is a set of key-value pairs. The keys are variation names. The values represent the percentage of traffic to allocate to that variation during this step.
(string) --
(integer) --
startTime (datetime) --
The date and time that this step of the launch starts.
status (string) --
The current state of the launch.
statusReason (string) --
If the launch was stopped, this is the string that was entered by the person who stopped the launch, to explain why it was stopped.
tags (dict) --
The list of tag keys and values associated with this launch.
(string) --
(string) --
type (string) --
The type of launch.
Updates an Evidently experiment.
Don't use this operation to update an experiment's tag. Instead, use TagResource .
See also: AWS API Documentation
Request Syntax
client.update_experiment(
description='string',
experiment='string',
metricGoals=[
{
'desiredChange': 'INCREASE'|'DECREASE',
'metricDefinition': {
'entityIdKey': 'string',
'eventPattern': 'string',
'name': 'string',
'unitLabel': 'string',
'valueKey': 'string'
}
},
],
onlineAbConfig={
'controlTreatmentName': 'string',
'treatmentWeights': {
'string': 123
}
},
project='string',
randomizationSalt='string',
samplingRate=123,
treatments=[
{
'description': 'string',
'feature': 'string',
'name': 'string',
'variation': 'string'
},
]
)
string
An optional description of the experiment.
string
[REQUIRED]
The name of the experiment to update.
list
An array of structures that defines the metrics used for the experiment, and whether a higher or lower value for each metric is the goal.
(dict) --
Use this structure to tell Evidently whether higher or lower values are desired for a metric that is used in an experiment.
desiredChange (string) --
INCREASE means that a variation with a higher number for this metric is performing better.
DECREASE means that a variation with a lower number for this metric is performing better.
metricDefinition (dict) -- [REQUIRED]
A structure that contains details about the metric.
entityIdKey (string) --
The entity, such as a user or session, that does an action that causes a metric value to be recorded. An example is userDetails.userID .
eventPattern (string) --
The EventBridge event pattern that defines how the metric is recorded.
For more information about EventBridge event patterns, see Amazon EventBridge event patterns .
name (string) --
A name for the metric.
unitLabel (string) --
A label for the units that the metric is measuring.
valueKey (string) --
The value that is tracked to produce the metric.
dict
A structure that contains the configuration of which variation o use as the "control" version. The "control" version is used for comparison with other variations. This structure also specifies how much experiment traffic is allocated to each variation.
controlTreatmentName (string) --
The name of the variation that is to be the default variation that the other variations are compared to.
treatmentWeights (dict) --
A set of key-value pairs. The keys are variation names, and the values are the portion of experiment traffic to be assigned to that variation. Specify the traffic portion in thousandths of a percent, so 20,000 for a variation would allocate 20% of the experiment traffic to that variation.
(string) --
(integer) --
string
[REQUIRED]
The name or ARN of the project that contains the experiment that you want to update.
string
When Evidently assigns a particular user session to an experiment, it must use a randomization ID to determine which variation the user session is served. This randomization ID is a combination of the entity ID and randomizationSalt . If you omit randomizationSalt , Evidently uses the experiment name as the randomizationSalt .
integer
The portion of the available audience that you want to allocate to this experiment, in thousandths of a percent. The available audience is the total audience minus the audience that you have allocated to overrides or current launches of this feature.
This is represented in thousandths of a percent. For example, specify 20,000 to allocate 20% of the available audience.
list
An array of structures that define the variations being tested in the experiment.
(dict) --
A structure that defines one treatment in an experiment. A treatment is a variation of the feature that you are including in the experiment.
description (string) --
A description for this treatment.
feature (string) -- [REQUIRED]
The feature that this experiment is testing.
name (string) -- [REQUIRED]
A name for this treatment.
variation (string) -- [REQUIRED]
The name of the variation to use as this treatment in the experiment.
dict
Response Syntax
{
'experiment': {
'arn': 'string',
'createdTime': datetime(2015, 1, 1),
'description': 'string',
'execution': {
'endedTime': datetime(2015, 1, 1),
'startedTime': datetime(2015, 1, 1)
},
'lastUpdatedTime': datetime(2015, 1, 1),
'metricGoals': [
{
'desiredChange': 'INCREASE'|'DECREASE',
'metricDefinition': {
'entityIdKey': 'string',
'eventPattern': 'string',
'name': 'string',
'unitLabel': 'string',
'valueKey': 'string'
}
},
],
'name': 'string',
'onlineAbDefinition': {
'controlTreatmentName': 'string',
'treatmentWeights': {
'string': 123
}
},
'project': 'string',
'randomizationSalt': 'string',
'samplingRate': 123,
'schedule': {
'analysisCompleteTime': datetime(2015, 1, 1)
},
'status': 'CREATED'|'UPDATING'|'RUNNING'|'COMPLETED'|'CANCELLED',
'statusReason': 'string',
'tags': {
'string': 'string'
},
'treatments': [
{
'description': 'string',
'featureVariations': {
'string': 'string'
},
'name': 'string'
},
],
'type': 'aws.evidently.onlineab'
}
}
Response Structure
(dict) --
experiment (dict) --
A structure containing the configuration details of the experiment that was updated.
arn (string) --
The ARN of the experiment.
createdTime (datetime) --
The date and time that the experiment is first created.
description (string) --
A description of the experiment.
execution (dict) --
A structure that contains the date and time that the experiment started and ended.
endedTime (datetime) --
The date and time that the experiment ended.
startedTime (datetime) --
The date and time that the experiment started.
lastUpdatedTime (datetime) --
The date and time that the experiment was most recently updated.
metricGoals (list) --
An array of structures that defines the metrics used for the experiment, and whether a higher or lower value for each metric is the goal.
(dict) --
A structure that tells Evidently whether higher or lower values are desired for a metric that is used in an experiment.
desiredChange (string) --
INCREASE means that a variation with a higher number for this metric is performing better.
DECREASE means that a variation with a lower number for this metric is performing better.
metricDefinition (dict) --
A structure that contains details about the metric.
entityIdKey (string) --
The entity, such as a user or session, that does an action that causes a metric value to be recorded.
eventPattern (string) --
The EventBridge event pattern that defines how the metric is recorded.
For more information about EventBridge event patterns, see Amazon EventBridge event patterns .
name (string) --
The name of the metric.
unitLabel (string) --
The label for the units that the metric is measuring.
valueKey (string) --
The value that is tracked to produce the metric.
name (string) --
The name of the experiment.
onlineAbDefinition (dict) --
A structure that contains the configuration of which variation to use as the "control" version. The "control" version is used for comparison with other variations. This structure also specifies how much experiment traffic is allocated to each variation.
controlTreatmentName (string) --
The name of the variation that is the default variation that the other variations are compared to.
treatmentWeights (dict) --
A set of key-value pairs. The keys are variation names, and the values are the portion of experiment traffic to be assigned to that variation. The traffic portion is specified in thousandths of a percent, so 20,000 for a variation would allocate 20% of the experiment traffic to that variation.
(string) --
(integer) --
project (string) --
The name or ARN of the project that contains this experiment.
randomizationSalt (string) --
This value is used when Evidently assigns a particular user session to the experiment. It helps create a randomization ID to determine which variation the user session is served. This randomization ID is a combination of the entity ID and randomizationSalt .
samplingRate (integer) --
In thousandths of a percent, the amount of the available audience that is allocated to this experiment. The available audience is the total audience minus the audience that you have allocated to overrides or current launches of this feature.
This is represented in thousandths of a percent, so a value of 10,000 is 10% of the available audience.
schedule (dict) --
A structure that contains the time and date that Evidently completed the analysis of the experiment.
analysisCompleteTime (datetime) --
The time and date that Evidently completed the analysis of the experiment.
status (string) --
The current state of the experiment.
statusReason (string) --
If the experiment was stopped, this is the string that was entered by the person who stopped the experiment, to explain why it was stopped.
tags (dict) --
The list of tag keys and values associated with this experiment.
(string) --
(string) --
treatments (list) --
An array of structures that describe the configuration of each feature variation used in the experiment.
(dict) --
A structure that defines one treatment in an experiment. A treatment is a variation of the feature that you are including in the experiment.
description (string) --
The description of the treatment.
featureVariations (dict) --
The feature variation used for this treatment. This is a key-value pair. The key is the feature name, and the value is the variation name.
(string) --
(string) --
name (string) --
The name of this treatment.
type (string) --
The type of this experiment. Currently, this value must be aws.experiment.onlineab .
Creates a launch of a given feature. Before you create a launch, you must create the feature to use for the launch.
You can use a launch to safely validate new features by serving them to a specified percentage of your users while you roll out the feature. You can monitor the performance of the new feature to help you decide when to ramp up traffic to more users. This helps you reduce risk and identify unintended consequences before you fully launch the feature.
Don't use this operation to update an existing launch. Instead, use UpdateLaunch .
See also: AWS API Documentation
Request Syntax
client.create_launch(
description='string',
groups=[
{
'description': 'string',
'feature': 'string',
'name': 'string',
'variation': 'string'
},
],
metricMonitors=[
{
'metricDefinition': {
'entityIdKey': 'string',
'eventPattern': 'string',
'name': 'string',
'unitLabel': 'string',
'valueKey': 'string'
}
},
],
name='string',
project='string',
randomizationSalt='string',
scheduledSplitsConfig={
'steps': [
{
'groupWeights': {
'string': 123
},
'startTime': datetime(2015, 1, 1)
},
]
},
tags={
'string': 'string'
}
)
string
An optional description for the launch.
list
[REQUIRED]
An array of structures that contains the feature and variations that are to be used for the launch.
(dict) --
A structure that defines one launch group in a launch. A launch group is a variation of the feature that you are including in the launch.
description (string) --
A description of the launch group.
feature (string) -- [REQUIRED]
The feature that this launch is using.
name (string) -- [REQUIRED]
A name for this launch group.
variation (string) -- [REQUIRED]
The feature variation to use for this launch group.
list
An array of structures that define the metrics that will be used to monitor the launch performance.
(dict) --
A structure that defines a metric to be used to monitor performance of the variations during a launch.
metricDefinition (dict) -- [REQUIRED]
A structure that defines the metric.
entityIdKey (string) --
The entity, such as a user or session, that does an action that causes a metric value to be recorded. An example is userDetails.userID .
eventPattern (string) --
The EventBridge event pattern that defines how the metric is recorded.
For more information about EventBridge event patterns, see Amazon EventBridge event patterns .
name (string) --
A name for the metric.
unitLabel (string) --
A label for the units that the metric is measuring.
valueKey (string) --
The value that is tracked to produce the metric.
string
[REQUIRED]
The name for the new launch.
string
[REQUIRED]
The name or ARN of the project that you want to create the launch in.
string
When Evidently assigns a particular user session to a launch, it must use a randomization ID to determine which variation the user session is served. This randomization ID is a combination of the entity ID and randomizationSalt . If you omit randomizationSalt , Evidently uses the launch name as the randomizationsSalt .
dict
An array of structures that define the traffic allocation percentages among the feature variations during each step of the launch.
steps (list) -- [REQUIRED]
An array of structures that define the traffic allocation percentages among the feature variations during each step of the launch. This also defines the start time of each step.
(dict) --
This structure defines the traffic allocation percentages among the feature variations during one step of a launch, and the start time of that step.
groupWeights (dict) -- [REQUIRED]
The traffic allocation percentages among the feature variations during one step of a launch. This is a set of key-value pairs. The keys are variation names. The values represent the percentage of traffic to allocate to that variation during this step.
(string) --
(integer) --
startTime (datetime) -- [REQUIRED]
The date and time that this step of the launch starts.
dict
Assigns one or more tags (key-value pairs) to the launch.
Tags can help you organize and categorize your resources. You can also use them to scope user permissions by granting a user permission to access or change only resources with certain tag values.
Tags don't have any semantic meaning to Amazon Web Services and are interpreted strictly as strings of characters.
<p>You can associate as many as 50 tags with a launch.</p> <p>For more information, see <a href="https://docs.aws.amazon.com/general/latest/gr/aws_tagging.html">Tagging Amazon Web Services resources</a>.</p>
(string) --
(string) --
dict
Response Syntax
{
'launch': {
'arn': 'string',
'createdTime': datetime(2015, 1, 1),
'description': 'string',
'execution': {
'endedTime': datetime(2015, 1, 1),
'startedTime': datetime(2015, 1, 1)
},
'groups': [
{
'description': 'string',
'featureVariations': {
'string': 'string'
},
'name': 'string'
},
],
'lastUpdatedTime': datetime(2015, 1, 1),
'metricMonitors': [
{
'metricDefinition': {
'entityIdKey': 'string',
'eventPattern': 'string',
'name': 'string',
'unitLabel': 'string',
'valueKey': 'string'
}
},
],
'name': 'string',
'project': 'string',
'randomizationSalt': 'string',
'scheduledSplitsDefinition': {
'steps': [
{
'groupWeights': {
'string': 123
},
'startTime': datetime(2015, 1, 1)
},
]
},
'status': 'CREATED'|'UPDATING'|'RUNNING'|'COMPLETED'|'CANCELLED',
'statusReason': 'string',
'tags': {
'string': 'string'
},
'type': 'aws.evidently.splits'
}
}
Response Structure
(dict) --
launch (dict) --
A structure that contains the configuration of the launch that was created.
arn (string) --
The ARN of the launch.
createdTime (datetime) --
The date and time that the launch is created.
description (string) --
The description of the launch.
execution (dict) --
A structure that contains information about the start and end times of the launch.
endedTime (datetime) --
The date and time that the launch ended.
startedTime (datetime) --
The date and time that the launch started.
groups (list) --
An array of structures that define the feature variations that are being used in the launch.
(dict) --
A structure that defines one launch group in a launch. A launch group is a variation of the feature that you are including in the launch.
description (string) --
A description of the launch group.
featureVariations (dict) --
The feature variation for this launch group. This is a key-value pair.
(string) --
(string) --
name (string) --
The name of the launch group.
lastUpdatedTime (datetime) --
The date and time that the launch was most recently updated.
metricMonitors (list) --
An array of structures that define the metrics that are being used to monitor the launch performance.
(dict) --
A structure that defines a metric to be used to monitor performance of the variations during a launch.
metricDefinition (dict) --
A structure that defines the metric.
entityIdKey (string) --
The entity, such as a user or session, that does an action that causes a metric value to be recorded.
eventPattern (string) --
The EventBridge event pattern that defines how the metric is recorded.
For more information about EventBridge event patterns, see Amazon EventBridge event patterns .
name (string) --
The name of the metric.
unitLabel (string) --
The label for the units that the metric is measuring.
valueKey (string) --
The value that is tracked to produce the metric.
name (string) --
The name of the launch.
project (string) --
The name or ARN of the project that contains the launch.
randomizationSalt (string) --
This value is used when Evidently assigns a particular user session to the launch, to help create a randomization ID to determine which variation the user session is served. This randomization ID is a combination of the entity ID and randomizationSalt .
scheduledSplitsDefinition (dict) --
An array of structures that define the traffic allocation percentages among the feature variations during each step of the launch.
steps (list) --
An array of structures that define the traffic allocation percentages among the feature variations during each step of the launch. This also defines the start time of each step.
(dict) --
This structure defines the traffic allocation percentages among the feature variations during one step of a launch, and the start time of that step.
groupWeights (dict) --
The traffic allocation percentages among the feature variations during one step of a launch. This is a set of key-value pairs. The keys are variation names. The values represent the percentage of traffic to allocate to that variation during this step.
(string) --
(integer) --
startTime (datetime) --
The date and time that this step of the launch starts.
status (string) --
The current state of the launch.
statusReason (string) --
If the launch was stopped, this is the string that was entered by the person who stopped the launch, to explain why it was stopped.
tags (dict) --
The list of tag keys and values associated with this launch.
(string) --
(string) --
type (string) --
The type of launch.
Updates the description of an existing project.
To create a new project, use CreateProject .
Don't use this operation to update the data storage options of a project. Instead, use UpdateProjectDataDelivery .
Don't use this operation to update the tags of a project. Instead, use TagResource .
See also: AWS API Documentation
Request Syntax
client.update_project(
description='string',
project='string'
)
string
An optional description of the project.
string
[REQUIRED]
The name or ARN of the project to update.
dict
Response Syntax
{
'project': {
'activeExperimentCount': 123,
'activeLaunchCount': 123,
'arn': 'string',
'createdTime': datetime(2015, 1, 1),
'dataDelivery': {
'cloudWatchLogs': {
'logGroup': 'string'
},
's3Destination': {
'bucket': 'string',
'prefix': 'string'
}
},
'description': 'string',
'experimentCount': 123,
'featureCount': 123,
'lastUpdatedTime': datetime(2015, 1, 1),
'launchCount': 123,
'name': 'string',
'status': 'AVAILABLE'|'UPDATING',
'tags': {
'string': 'string'
}
}
}
Response Structure
(dict) --
project (dict) --
A structure containing information about the updated project.
activeExperimentCount (integer) --
The number of ongoing experiments currently in the project.
activeLaunchCount (integer) --
The number of ongoing launches currently in the project.
arn (string) --
The name or ARN of the project.
createdTime (datetime) --
The date and time that the project is created.
dataDelivery (dict) --
A structure that contains information about where Evidently is to store evaluation events for longer term storage.
cloudWatchLogs (dict) --
If the project stores evaluation events in CloudWatch Logs, this structure stores the log group name.
logGroup (string) --
The name of the log group where the project stores evaluation events.
s3Destination (dict) --
If the project stores evaluation events in an Amazon S3 bucket, this structure stores the bucket name and bucket prefix.
bucket (string) --
The name of the bucket in which Evidently stores evaluation events.
prefix (string) --
The bucket prefix in which Evidently stores evaluation events.
description (string) --
The user-entered description of the project.
experimentCount (integer) --
The number of experiments currently in the project. This includes all experiments that have been created and not deleted, whether they are ongoing or not.
featureCount (integer) --
The number of features currently in the project.
lastUpdatedTime (datetime) --
The date and time that the project was most recently updated.
launchCount (integer) --
The number of launches currently in the project. This includes all launches that have been created and not deleted, whether they are ongoing or not.
name (string) --
The name of the project.
status (string) --
The current state of the project.
tags (dict) --
The list of tag keys and values associated with this project.
(string) --
(string) --
Deletes an Evidently experiment. The feature used for the experiment is not deleted.
To stop an experiment without deleting it, use StopExperiment .
See also: AWS API Documentation
Request Syntax
client.delete_experiment(
experiment='string',
project='string'
)
string
[REQUIRED]
The name of the experiment to delete.
string
[REQUIRED]
The name or ARN of the project that contains the experiment to delete.
dict
Response Syntax
{}
Response Structure
(dict) --
Returns configuration details about all the features in the specified project.
See also: AWS API Documentation
Request Syntax
client.list_features(
maxResults=123,
nextToken='string',
project='string'
)
integer
The maximum number of results to include in the response.
string
The token to use when requesting the next set of results. You received this token from a previous ListFeatures operation.
string
[REQUIRED]
The name or ARN of the project to return the feature list from.
dict
Response Syntax
{
'features': [
{
'arn': 'string',
'createdTime': datetime(2015, 1, 1),
'defaultVariation': 'string',
'evaluationRules': [
{
'name': 'string',
'type': 'string'
},
],
'evaluationStrategy': 'ALL_RULES'|'DEFAULT_VARIATION',
'lastUpdatedTime': datetime(2015, 1, 1),
'name': 'string',
'project': 'string',
'status': 'AVAILABLE'|'UPDATING',
'tags': {
'string': 'string'
}
},
],
'nextToken': 'string'
}
Response Structure
(dict) --
features (list) --
An array of structures that contain the configuration details of the features in the specified project.
(dict) --
This structure contains information about one Evidently feature in your account.
arn (string) --
The ARN of the feature.
createdTime (datetime) --
The date and time that the feature is created.
defaultVariation (string) --
The name of the variation that is used as the default variation. The default variation is served to users who are not allocated to any ongoing launches or experiments of this feature.
evaluationRules (list) --
An array of structures that define
(dict) --
A structure that contains the information about an evaluation rule for this feature, if it is used in a launch or experiment.
name (string) --
The name of the experiment or launch.
type (string) --
This value is aws.evidently.splits if this is an evaluation rule for a launch, and it is aws.evidently.onlineab if this is an evaluation rule for an experiment.
evaluationStrategy (string) --
If this value is ALL_RULES , the traffic allocation specified by any ongoing launches or experiments is being used. If this is DEFAULT_VARIATION , the default variation is being served to all users.
lastUpdatedTime (datetime) --
The date and time that the feature was most recently updated.
name (string) --
The name of the feature.
project (string) --
The name or ARN of the project that contains the feature.
status (string) --
The current state of the feature.
tags (dict) --
The list of tag keys and values associated with this feature.
(string) --
(string) --
nextToken (string) --
The token to use in a subsequent ListFeatures operation to return the next set of results.
This operation assigns feature variation to user sessions. For each user session, you pass in an entityID that represents the user. Evidently then checks the evaluation rules and assigns the variation.
The first rules that are evaluated are the override rules. If the user's entityID matches an override rule, the user is served the variation specified by that rule.
Next, if there is a launch of the feature, the user might be assigned to a variation in the launch. The chance of this depends on the percentage of users that are allocated to that launch. If the user is enrolled in the launch, the variation they are served depends on the allocation of the various feature variations used for the launch.
If the user is not assigned to a launch, and there is an ongoing experiment for this feature, the user might be assigned to a variation in the experiment. The chance of this depends on the percentage of users that are allocated to that experiment. If the user is enrolled in the experiment, the variation they are served depends on the allocation of the various feature variations used for the experiment.
If the user is not assigned to a launch or experiment, they are served the default variation.
See also: AWS API Documentation
Request Syntax
client.batch_evaluate_feature(
project='string',
requests=[
{
'entityId': 'string',
'evaluationContext': 'string',
'feature': 'string'
},
]
)
string
[REQUIRED]
The name or ARN of the project that contains the feature being evaluated.
list
[REQUIRED]
An array of structures, where each structure assigns a feature variation to one user session.
(dict) --
This structure assigns a feature variation to one user session.
entityId (string) -- [REQUIRED]
An internal ID that represents a unique user session of the application. This entityID is checked against any override rules assigned for this feature.
evaluationContext (string) --
A JSON block of attributes that you can optionally pass in. This JSON block is included in the evaluation events sent to Evidently from the user session.
feature (string) -- [REQUIRED]
The name of the feature being evaluated.
dict
Response Syntax
{
'results': [
{
'details': 'string',
'entityId': 'string',
'feature': 'string',
'project': 'string',
'reason': 'string',
'value': {
'boolValue': True|False,
'doubleValue': 123.0,
'longValue': 123,
'stringValue': 'string'
},
'variation': 'string'
},
]
}
Response Structure
(dict) --
results (list) --
An array of structures, where each structure displays the results of one feature evaluation assignment to one user session.
(dict) --
This structure displays the results of one feature evaluation assignment to one user session.
details (string) --
If this user was assigned to a launch or experiment, this field lists the launch or experiment name.
entityId (string) --
An internal ID that represents a unique user session of the application.
feature (string) --
The name of the feature being evaluated.
project (string) --
The name or ARN of the project that contains the feature being evaluated.
reason (string) --
Specifies the reason that the user session was assigned this variation. Possible values include DEFAULT , meaning the user was served the default variation; LAUNCH_RULE_MATCH , if the user session was enrolled in a launch; or EXPERIMENT_RULE_MATCH , if the user session was enrolled in an experiment.
value (dict) --
The value assigned to this variation to differentiate it from the other variations of this feature.
boolValue (boolean) --
If this feature uses the Boolean variation type, this field contains the Boolean value of this variation.
doubleValue (float) --
If this feature uses the double integer variation type, this field contains the double integer value of this variation.
longValue (integer) --
If this feature uses the long variation type, this field contains the long value of this variation.
stringValue (string) --
If this feature uses the string variation type, this field contains the string value of this variation.
variation (string) --
The name of the variation that was served to the user session.
Removes one or more tags from the specified resource.
See also: AWS API Documentation
Request Syntax
client.untag_resource(
resourceArn='string',
tagKeys=[
'string',
]
)
string
[REQUIRED]
The ARN of the CloudWatch Evidently resource that you're removing tags from.
list
[REQUIRED]
The list of tag keys to remove from the resource.
(string) --
dict
Response Syntax
{}
Response Structure
(dict) --
Deletes an Evidently feature.
See also: AWS API Documentation
Request Syntax
client.delete_feature(
feature='string',
project='string'
)
string
[REQUIRED]
The name of the feature to delete.
string
[REQUIRED]
The name or ARN of the project that contains the feature to delete.
dict
Response Syntax
{}
Response Structure
(dict) --
Starts an existing launch. To create a launch, use CreateLaunch .
See also: AWS API Documentation
Request Syntax
client.start_launch(
launch='string',
project='string'
)
string
[REQUIRED]
The name of the launch to start.
string
[REQUIRED]
The name or ARN of the project that contains the launch to start.
dict
Response Syntax
{
'launch': {
'arn': 'string',
'createdTime': datetime(2015, 1, 1),
'description': 'string',
'execution': {
'endedTime': datetime(2015, 1, 1),
'startedTime': datetime(2015, 1, 1)
},
'groups': [
{
'description': 'string',
'featureVariations': {
'string': 'string'
},
'name': 'string'
},
],
'lastUpdatedTime': datetime(2015, 1, 1),
'metricMonitors': [
{
'metricDefinition': {
'entityIdKey': 'string',
'eventPattern': 'string',
'name': 'string',
'unitLabel': 'string',
'valueKey': 'string'
}
},
],
'name': 'string',
'project': 'string',
'randomizationSalt': 'string',
'scheduledSplitsDefinition': {
'steps': [
{
'groupWeights': {
'string': 123
},
'startTime': datetime(2015, 1, 1)
},
]
},
'status': 'CREATED'|'UPDATING'|'RUNNING'|'COMPLETED'|'CANCELLED',
'statusReason': 'string',
'tags': {
'string': 'string'
},
'type': 'aws.evidently.splits'
}
}
Response Structure
(dict) --
launch (dict) --
A structure that contains information about the launch that was started.
arn (string) --
The ARN of the launch.
createdTime (datetime) --
The date and time that the launch is created.
description (string) --
The description of the launch.
execution (dict) --
A structure that contains information about the start and end times of the launch.
endedTime (datetime) --
The date and time that the launch ended.
startedTime (datetime) --
The date and time that the launch started.
groups (list) --
An array of structures that define the feature variations that are being used in the launch.
(dict) --
A structure that defines one launch group in a launch. A launch group is a variation of the feature that you are including in the launch.
description (string) --
A description of the launch group.
featureVariations (dict) --
The feature variation for this launch group. This is a key-value pair.
(string) --
(string) --
name (string) --
The name of the launch group.
lastUpdatedTime (datetime) --
The date and time that the launch was most recently updated.
metricMonitors (list) --
An array of structures that define the metrics that are being used to monitor the launch performance.
(dict) --
A structure that defines a metric to be used to monitor performance of the variations during a launch.
metricDefinition (dict) --
A structure that defines the metric.
entityIdKey (string) --
The entity, such as a user or session, that does an action that causes a metric value to be recorded.
eventPattern (string) --
The EventBridge event pattern that defines how the metric is recorded.
For more information about EventBridge event patterns, see Amazon EventBridge event patterns .
name (string) --
The name of the metric.
unitLabel (string) --
The label for the units that the metric is measuring.
valueKey (string) --
The value that is tracked to produce the metric.
name (string) --
The name of the launch.
project (string) --
The name or ARN of the project that contains the launch.
randomizationSalt (string) --
This value is used when Evidently assigns a particular user session to the launch, to help create a randomization ID to determine which variation the user session is served. This randomization ID is a combination of the entity ID and randomizationSalt .
scheduledSplitsDefinition (dict) --
An array of structures that define the traffic allocation percentages among the feature variations during each step of the launch.
steps (list) --
An array of structures that define the traffic allocation percentages among the feature variations during each step of the launch. This also defines the start time of each step.
(dict) --
This structure defines the traffic allocation percentages among the feature variations during one step of a launch, and the start time of that step.
groupWeights (dict) --
The traffic allocation percentages among the feature variations during one step of a launch. This is a set of key-value pairs. The keys are variation names. The values represent the percentage of traffic to allocate to that variation during this step.
(string) --
(integer) --
startTime (datetime) --
The date and time that this step of the launch starts.
status (string) --
The current state of the launch.
statusReason (string) --
If the launch was stopped, this is the string that was entered by the person who stopped the launch, to explain why it was stopped.
tags (dict) --
The list of tag keys and values associated with this launch.
(string) --
(string) --
type (string) --
The type of launch.
Returns the details about one launch. You must already know the launch name. To retrieve a list of launches in your account, use ListLaunches .
See also: AWS API Documentation
Request Syntax
client.get_launch(
launch='string',
project='string'
)
string
[REQUIRED]
The name of the launch that you want to see the details of.
string
[REQUIRED]
The name or ARN of the project that contains the launch.
dict
Response Syntax
{
'launch': {
'arn': 'string',
'createdTime': datetime(2015, 1, 1),
'description': 'string',
'execution': {
'endedTime': datetime(2015, 1, 1),
'startedTime': datetime(2015, 1, 1)
},
'groups': [
{
'description': 'string',
'featureVariations': {
'string': 'string'
},
'name': 'string'
},
],
'lastUpdatedTime': datetime(2015, 1, 1),
'metricMonitors': [
{
'metricDefinition': {
'entityIdKey': 'string',
'eventPattern': 'string',
'name': 'string',
'unitLabel': 'string',
'valueKey': 'string'
}
},
],
'name': 'string',
'project': 'string',
'randomizationSalt': 'string',
'scheduledSplitsDefinition': {
'steps': [
{
'groupWeights': {
'string': 123
},
'startTime': datetime(2015, 1, 1)
},
]
},
'status': 'CREATED'|'UPDATING'|'RUNNING'|'COMPLETED'|'CANCELLED',
'statusReason': 'string',
'tags': {
'string': 'string'
},
'type': 'aws.evidently.splits'
}
}
Response Structure
(dict) --
launch (dict) --
A structure containing the configuration details of the launch.
arn (string) --
The ARN of the launch.
createdTime (datetime) --
The date and time that the launch is created.
description (string) --
The description of the launch.
execution (dict) --
A structure that contains information about the start and end times of the launch.
endedTime (datetime) --
The date and time that the launch ended.
startedTime (datetime) --
The date and time that the launch started.
groups (list) --
An array of structures that define the feature variations that are being used in the launch.
(dict) --
A structure that defines one launch group in a launch. A launch group is a variation of the feature that you are including in the launch.
description (string) --
A description of the launch group.
featureVariations (dict) --
The feature variation for this launch group. This is a key-value pair.
(string) --
(string) --
name (string) --
The name of the launch group.
lastUpdatedTime (datetime) --
The date and time that the launch was most recently updated.
metricMonitors (list) --
An array of structures that define the metrics that are being used to monitor the launch performance.
(dict) --
A structure that defines a metric to be used to monitor performance of the variations during a launch.
metricDefinition (dict) --
A structure that defines the metric.
entityIdKey (string) --
The entity, such as a user or session, that does an action that causes a metric value to be recorded.
eventPattern (string) --
The EventBridge event pattern that defines how the metric is recorded.
For more information about EventBridge event patterns, see Amazon EventBridge event patterns .
name (string) --
The name of the metric.
unitLabel (string) --
The label for the units that the metric is measuring.
valueKey (string) --
The value that is tracked to produce the metric.
name (string) --
The name of the launch.
project (string) --
The name or ARN of the project that contains the launch.
randomizationSalt (string) --
This value is used when Evidently assigns a particular user session to the launch, to help create a randomization ID to determine which variation the user session is served. This randomization ID is a combination of the entity ID and randomizationSalt .
scheduledSplitsDefinition (dict) --
An array of structures that define the traffic allocation percentages among the feature variations during each step of the launch.
steps (list) --
An array of structures that define the traffic allocation percentages among the feature variations during each step of the launch. This also defines the start time of each step.
(dict) --
This structure defines the traffic allocation percentages among the feature variations during one step of a launch, and the start time of that step.
groupWeights (dict) --
The traffic allocation percentages among the feature variations during one step of a launch. This is a set of key-value pairs. The keys are variation names. The values represent the percentage of traffic to allocate to that variation during this step.
(string) --
(integer) --
startTime (datetime) --
The date and time that this step of the launch starts.
status (string) --
The current state of the launch.
statusReason (string) --
If the launch was stopped, this is the string that was entered by the person who stopped the launch, to explain why it was stopped.
tags (dict) --
The list of tag keys and values associated with this launch.
(string) --
(string) --
type (string) --
The type of launch.
Returns configuration details about all the experiments in the specified project.
See also: AWS API Documentation
Request Syntax
client.list_experiments(
maxResults=123,
nextToken='string',
project='string'
)
integer
The maximum number of results to include in the response.
string
The token to use when requesting the next set of results. You received this token from a previous ListExperiments operation.
string
[REQUIRED]
The name or ARN of the project to return the experiment list from.
dict
Response Syntax
{
'experiments': [
{
'arn': 'string',
'createdTime': datetime(2015, 1, 1),
'description': 'string',
'execution': {
'endedTime': datetime(2015, 1, 1),
'startedTime': datetime(2015, 1, 1)
},
'lastUpdatedTime': datetime(2015, 1, 1),
'metricGoals': [
{
'desiredChange': 'INCREASE'|'DECREASE',
'metricDefinition': {
'entityIdKey': 'string',
'eventPattern': 'string',
'name': 'string',
'unitLabel': 'string',
'valueKey': 'string'
}
},
],
'name': 'string',
'onlineAbDefinition': {
'controlTreatmentName': 'string',
'treatmentWeights': {
'string': 123
}
},
'project': 'string',
'randomizationSalt': 'string',
'samplingRate': 123,
'schedule': {
'analysisCompleteTime': datetime(2015, 1, 1)
},
'status': 'CREATED'|'UPDATING'|'RUNNING'|'COMPLETED'|'CANCELLED',
'statusReason': 'string',
'tags': {
'string': 'string'
},
'treatments': [
{
'description': 'string',
'featureVariations': {
'string': 'string'
},
'name': 'string'
},
],
'type': 'aws.evidently.onlineab'
},
],
'nextToken': 'string'
}
Response Structure
(dict) --
experiments (list) --
An array of structures that contain the configuration details of the experiments in the specified project.
(dict) --
A structure containing the configuration details of an experiment.
arn (string) --
The ARN of the experiment.
createdTime (datetime) --
The date and time that the experiment is first created.
description (string) --
A description of the experiment.
execution (dict) --
A structure that contains the date and time that the experiment started and ended.
endedTime (datetime) --
The date and time that the experiment ended.
startedTime (datetime) --
The date and time that the experiment started.
lastUpdatedTime (datetime) --
The date and time that the experiment was most recently updated.
metricGoals (list) --
An array of structures that defines the metrics used for the experiment, and whether a higher or lower value for each metric is the goal.
(dict) --
A structure that tells Evidently whether higher or lower values are desired for a metric that is used in an experiment.
desiredChange (string) --
INCREASE means that a variation with a higher number for this metric is performing better.
DECREASE means that a variation with a lower number for this metric is performing better.
metricDefinition (dict) --
A structure that contains details about the metric.
entityIdKey (string) --
The entity, such as a user or session, that does an action that causes a metric value to be recorded.
eventPattern (string) --
The EventBridge event pattern that defines how the metric is recorded.
For more information about EventBridge event patterns, see Amazon EventBridge event patterns .
name (string) --
The name of the metric.
unitLabel (string) --
The label for the units that the metric is measuring.
valueKey (string) --
The value that is tracked to produce the metric.
name (string) --
The name of the experiment.
onlineAbDefinition (dict) --
A structure that contains the configuration of which variation to use as the "control" version. The "control" version is used for comparison with other variations. This structure also specifies how much experiment traffic is allocated to each variation.
controlTreatmentName (string) --
The name of the variation that is the default variation that the other variations are compared to.
treatmentWeights (dict) --
A set of key-value pairs. The keys are variation names, and the values are the portion of experiment traffic to be assigned to that variation. The traffic portion is specified in thousandths of a percent, so 20,000 for a variation would allocate 20% of the experiment traffic to that variation.
(string) --
(integer) --
project (string) --
The name or ARN of the project that contains this experiment.
randomizationSalt (string) --
This value is used when Evidently assigns a particular user session to the experiment. It helps create a randomization ID to determine which variation the user session is served. This randomization ID is a combination of the entity ID and randomizationSalt .
samplingRate (integer) --
In thousandths of a percent, the amount of the available audience that is allocated to this experiment. The available audience is the total audience minus the audience that you have allocated to overrides or current launches of this feature.
This is represented in thousandths of a percent, so a value of 10,000 is 10% of the available audience.
schedule (dict) --
A structure that contains the time and date that Evidently completed the analysis of the experiment.
analysisCompleteTime (datetime) --
The time and date that Evidently completed the analysis of the experiment.
status (string) --
The current state of the experiment.
statusReason (string) --
If the experiment was stopped, this is the string that was entered by the person who stopped the experiment, to explain why it was stopped.
tags (dict) --
The list of tag keys and values associated with this experiment.
(string) --
(string) --
treatments (list) --
An array of structures that describe the configuration of each feature variation used in the experiment.
(dict) --
A structure that defines one treatment in an experiment. A treatment is a variation of the feature that you are including in the experiment.
description (string) --
The description of the treatment.
featureVariations (dict) --
The feature variation used for this treatment. This is a key-value pair. The key is the feature name, and the value is the variation name.
(string) --
(string) --
name (string) --
The name of this treatment.
type (string) --
The type of this experiment. Currently, this value must be aws.experiment.onlineab .
nextToken (string) --
The token to use in a subsequent ListExperiments operation to return the next set of results.
Assigns one or more tags (key-value pairs) to the specified CloudWatch Evidently resource. Projects, features, launches, and experiments can be tagged.
Tags can help you organize and categorize your resources. You can also use them to scope user permissions by granting a user permission to access or change only resources with certain tag values.
Tags don't have any semantic meaning to Amazon Web Services and are interpreted strictly as strings of characters.
You can use the TagResource action with a resource that already has tags. If you specify a new tag key for the resource, this tag is appended to the list of tags associated with the alarm. If you specify a tag key that is already associated with the resource, the new tag value that you specify replaces the previous value for that tag.
You can associate as many as 50 tags with a resource.
For more information, see Tagging Amazon Web Services resources .
See also: AWS API Documentation
Request Syntax
client.tag_resource(
resourceArn='string',
tags={
'string': 'string'
}
)
string
[REQUIRED]
The ARN of the CloudWatch Evidently resource that you're adding tags to.
dict
[REQUIRED]
The list of key-value pairs to associate with the resource.
(string) --
(string) --
dict
Response Syntax
{}
Response Structure
(dict) --
Sends performance events to Evidently. These events can be used to evaluate a launch or an experiment.
See also: AWS API Documentation
Request Syntax
client.put_project_events(
events=[
{
'data': 'string',
'timestamp': datetime(2015, 1, 1),
'type': 'aws.evidently.evaluation'|'aws.evidently.custom'
},
],
project='string'
)
list
[REQUIRED]
An array of event structures that contain the performance data that is being sent to Evidently.
(dict) --
A structure that contains the information about one evaluation event or custom event sent to Evidently. This is a JSON payload. If this event specifies a pre-defined event type, the payload must follow the defined event schema.
data (string) -- [REQUIRED]
The event data.
timestamp (datetime) -- [REQUIRED]
The timestamp of the event.
type (string) -- [REQUIRED]
aws.evidently.evaluation specifies an evaluation event, which determines which feature variation that a user sees. aws.evidently.custom specifies a custom event, which generates metrics from user actions such as clicks and checkouts.
string
[REQUIRED]
The name or ARN of the project to write the events to.
dict
Response Syntax
{
'eventResults': [
{
'errorCode': 'string',
'errorMessage': 'string',
'eventId': 'string'
},
],
'failedEventCount': 123
}
Response Structure
(dict) --
eventResults (list) --
A structure that contains Evidently's response to the sent events, including an event ID and error codes, if any.
(dict) --
A structure that contains Evidently's response to the sent events, including an event ID and error codes, if any.
errorCode (string) --
If the PutProjectEvents operation has an error, the error code is returned here.
errorMessage (string) --
If the PutProjectEvents operation has an error, the error message is returned here.
eventId (string) --
A unique ID assigned to this PutProjectEvents operation.
failedEventCount (integer) --
The number of events in the operation that could not be used by Evidently.
Creates an Evidently feature that you want to launch or test. You can define up to five variations of a feature, and use these variations in your launches and experiments. A feature must be created in a project. For information about creating a project, see CreateProject .
Don't use this operation to update an existing feature. Instead, use UpdateFeature .
See also: AWS API Documentation
Request Syntax
client.create_feature(
defaultVariation='string',
description='string',
entityOverrides={
'string': 'string'
},
evaluationStrategy='ALL_RULES'|'DEFAULT_VARIATION',
name='string',
project='string',
tags={
'string': 'string'
},
variations=[
{
'name': 'string',
'value': {
'boolValue': True|False,
'doubleValue': 123.0,
'longValue': 123,
'stringValue': 'string'
}
},
]
)
string
The name of the variation to use as the default variation. The default variation is served to users who are not allocated to any ongoing launches or experiments of this feature.
This variation must also be listed in the variations structure.
If you omit defaultVariation , the first variation listed in the variations structure is used as the default variation.
string
An optional description of the feature.
dict
Specify users that should always be served a specific variation of a feature. Each user is specified by a key-value pair . For each key, specify a user by entering their user ID, account ID, or some other identifier. For the value, specify the name of the variation that they are to be served.
(string) --
(string) --
string
Specify ALL_RULES to activate the traffic allocation specified by any ongoing launches or experiments. Specify DEFAULT_VARIATION to serve the default variation to all users instead.
string
[REQUIRED]
The name for the new feature.
string
[REQUIRED]
The name or ARN of the project that is to contain the new feature.
dict
Assigns one or more tags (key-value pairs) to the feature.
Tags can help you organize and categorize your resources. You can also use them to scope user permissions by granting a user permission to access or change only resources with certain tag values.
Tags don't have any semantic meaning to Amazon Web Services and are interpreted strictly as strings of characters.
<p>You can associate as many as 50 tags with a feature.</p> <p>For more information, see <a href="https://docs.aws.amazon.com/general/latest/gr/aws_tagging.html">Tagging Amazon Web Services resources</a>.</p>
(string) --
(string) --
list
[REQUIRED]
An array of structures that contain the configuration of the feature's different variations.
(dict) --
This structure contains the name and variation value of one variation of a feature.
name (string) -- [REQUIRED]
The name of the variation.
value (dict) -- [REQUIRED]
The value assigned to this variation.
boolValue (boolean) --
If this feature uses the Boolean variation type, this field contains the Boolean value of this variation.
doubleValue (float) --
If this feature uses the double integer variation type, this field contains the double integer value of this variation.
longValue (integer) --
If this feature uses the long variation type, this field contains the long value of this variation.
stringValue (string) --
If this feature uses the string variation type, this field contains the string value of this variation.
dict
Response Syntax
{
'feature': {
'arn': 'string',
'createdTime': datetime(2015, 1, 1),
'defaultVariation': 'string',
'description': 'string',
'entityOverrides': {
'string': 'string'
},
'evaluationRules': [
{
'name': 'string',
'type': 'string'
},
],
'evaluationStrategy': 'ALL_RULES'|'DEFAULT_VARIATION',
'lastUpdatedTime': datetime(2015, 1, 1),
'name': 'string',
'project': 'string',
'status': 'AVAILABLE'|'UPDATING',
'tags': {
'string': 'string'
},
'valueType': 'STRING'|'LONG'|'DOUBLE'|'BOOLEAN',
'variations': [
{
'name': 'string',
'value': {
'boolValue': True|False,
'doubleValue': 123.0,
'longValue': 123,
'stringValue': 'string'
}
},
]
}
}
Response Structure
(dict) --
feature (dict) --
A structure that contains information about the new feature.
arn (string) --
The ARN of the feature.
createdTime (datetime) --
The date and time that the feature is created.
defaultVariation (string) --
The name of the variation that is used as the default variation. The default variation is served to users who are not allocated to any ongoing launches or experiments of this feature.
This variation must also be listed in the variations structure.
If you omit defaultVariation , the first variation listed in the variations structure is used as the default variation.
description (string) --
The description of the feature.
entityOverrides (dict) --
A set of key-value pairs that specify users who should always be served a specific variation of a feature. Each key specifies a user using their user ID, account ID, or some other identifier. The value specifies the name of the variation that the user is to be served.
For the override to be successful, the value of the key must match the entityId used in the EvaluateFeature operation.
(string) --
(string) --
evaluationRules (list) --
An array of structures that define the evaluation rules for the feature.
(dict) --
A structure that contains the information about an evaluation rule for this feature, if it is used in a launch or experiment.
name (string) --
The name of the experiment or launch.
type (string) --
This value is aws.evidently.splits if this is an evaluation rule for a launch, and it is aws.evidently.onlineab if this is an evaluation rule for an experiment.
evaluationStrategy (string) --
If this value is ALL_RULES , the traffic allocation specified by any ongoing launches or experiments is being used. If this is DEFAULT_VARIATION , the default variation is being served to all users.
lastUpdatedTime (datetime) --
The date and time that the feature was most recently updated.
name (string) --
The name of the feature.
project (string) --
The name or ARN of the project that contains the feature.
status (string) --
The current state of the feature.
tags (dict) --
The list of tag keys and values associated with this feature.
(string) --
(string) --
valueType (string) --
Defines the type of value used to define the different feature variations. For more information, see Variation types
variations (list) --
An array of structures that contain the configuration of the feature's different variations.
(dict) --
This structure contains the name and variation value of one variation of a feature.
name (string) --
The name of the variation.
value (dict) --
The value assigned to this variation.
boolValue (boolean) --
If this feature uses the Boolean variation type, this field contains the Boolean value of this variation.
doubleValue (float) --
If this feature uses the double integer variation type, this field contains the double integer value of this variation.
longValue (integer) --
If this feature uses the long variation type, this field contains the long value of this variation.
stringValue (string) --
If this feature uses the string variation type, this field contains the string value of this variation.
Returns the details about one experiment. You must already know the experiment name. To retrieve a list of experiments in your account, use ListExperiments .
See also: AWS API Documentation
Request Syntax
client.get_experiment(
experiment='string',
project='string'
)
string
[REQUIRED]
The name of the experiment that you want to see the details of.
string
[REQUIRED]
The name or ARN of the project that contains the experiment.
dict
Response Syntax
{
'experiment': {
'arn': 'string',
'createdTime': datetime(2015, 1, 1),
'description': 'string',
'execution': {
'endedTime': datetime(2015, 1, 1),
'startedTime': datetime(2015, 1, 1)
},
'lastUpdatedTime': datetime(2015, 1, 1),
'metricGoals': [
{
'desiredChange': 'INCREASE'|'DECREASE',
'metricDefinition': {
'entityIdKey': 'string',
'eventPattern': 'string',
'name': 'string',
'unitLabel': 'string',
'valueKey': 'string'
}
},
],
'name': 'string',
'onlineAbDefinition': {
'controlTreatmentName': 'string',
'treatmentWeights': {
'string': 123
}
},
'project': 'string',
'randomizationSalt': 'string',
'samplingRate': 123,
'schedule': {
'analysisCompleteTime': datetime(2015, 1, 1)
},
'status': 'CREATED'|'UPDATING'|'RUNNING'|'COMPLETED'|'CANCELLED',
'statusReason': 'string',
'tags': {
'string': 'string'
},
'treatments': [
{
'description': 'string',
'featureVariations': {
'string': 'string'
},
'name': 'string'
},
],
'type': 'aws.evidently.onlineab'
}
}
Response Structure
(dict) --
experiment (dict) --
A structure containing the configuration details of the experiment.
arn (string) --
The ARN of the experiment.
createdTime (datetime) --
The date and time that the experiment is first created.
description (string) --
A description of the experiment.
execution (dict) --
A structure that contains the date and time that the experiment started and ended.
endedTime (datetime) --
The date and time that the experiment ended.
startedTime (datetime) --
The date and time that the experiment started.
lastUpdatedTime (datetime) --
The date and time that the experiment was most recently updated.
metricGoals (list) --
An array of structures that defines the metrics used for the experiment, and whether a higher or lower value for each metric is the goal.
(dict) --
A structure that tells Evidently whether higher or lower values are desired for a metric that is used in an experiment.
desiredChange (string) --
INCREASE means that a variation with a higher number for this metric is performing better.
DECREASE means that a variation with a lower number for this metric is performing better.
metricDefinition (dict) --
A structure that contains details about the metric.
entityIdKey (string) --
The entity, such as a user or session, that does an action that causes a metric value to be recorded.
eventPattern (string) --
The EventBridge event pattern that defines how the metric is recorded.
For more information about EventBridge event patterns, see Amazon EventBridge event patterns .
name (string) --
The name of the metric.
unitLabel (string) --
The label for the units that the metric is measuring.
valueKey (string) --
The value that is tracked to produce the metric.
name (string) --
The name of the experiment.
onlineAbDefinition (dict) --
A structure that contains the configuration of which variation to use as the "control" version. The "control" version is used for comparison with other variations. This structure also specifies how much experiment traffic is allocated to each variation.
controlTreatmentName (string) --
The name of the variation that is the default variation that the other variations are compared to.
treatmentWeights (dict) --
A set of key-value pairs. The keys are variation names, and the values are the portion of experiment traffic to be assigned to that variation. The traffic portion is specified in thousandths of a percent, so 20,000 for a variation would allocate 20% of the experiment traffic to that variation.
(string) --
(integer) --
project (string) --
The name or ARN of the project that contains this experiment.
randomizationSalt (string) --
This value is used when Evidently assigns a particular user session to the experiment. It helps create a randomization ID to determine which variation the user session is served. This randomization ID is a combination of the entity ID and randomizationSalt .
samplingRate (integer) --
In thousandths of a percent, the amount of the available audience that is allocated to this experiment. The available audience is the total audience minus the audience that you have allocated to overrides or current launches of this feature.
This is represented in thousandths of a percent, so a value of 10,000 is 10% of the available audience.
schedule (dict) --
A structure that contains the time and date that Evidently completed the analysis of the experiment.
analysisCompleteTime (datetime) --
The time and date that Evidently completed the analysis of the experiment.
status (string) --
The current state of the experiment.
statusReason (string) --
If the experiment was stopped, this is the string that was entered by the person who stopped the experiment, to explain why it was stopped.
tags (dict) --
The list of tag keys and values associated with this experiment.
(string) --
(string) --
treatments (list) --
An array of structures that describe the configuration of each feature variation used in the experiment.
(dict) --
A structure that defines one treatment in an experiment. A treatment is a variation of the feature that you are including in the experiment.
description (string) --
The description of the treatment.
featureVariations (dict) --
The feature variation used for this treatment. This is a key-value pair. The key is the feature name, and the value is the variation name.
(string) --
(string) --
name (string) --
The name of this treatment.
type (string) --
The type of this experiment. Currently, this value must be aws.experiment.onlineab .
Starts an existing experiment. To create an experiment, use CreateExperiment .
See also: AWS API Documentation
Request Syntax
client.start_experiment(
analysisCompleteTime=datetime(2015, 1, 1),
experiment='string',
project='string'
)
datetime
[REQUIRED]
The date and time to end the experiment.
string
[REQUIRED]
The name of the experiment to start.
string
[REQUIRED]
The name or ARN of the project that contains the experiment to start.
dict
Response Syntax
{
'startedTime': datetime(2015, 1, 1)
}
Response Structure
(dict) --
startedTime (datetime) --
A timestamp that indicates when the experiment started.
Returns configuration details about all the launches in the specified project.
See also: AWS API Documentation
Request Syntax
client.list_launches(
maxResults=123,
nextToken='string',
project='string'
)
integer
The maximum number of results to include in the response.
string
The token to use when requesting the next set of results. You received this token from a previous ListLaunches operation.
string
[REQUIRED]
The name or ARN of the project to return the launch list from.
dict
Response Syntax
{
'launches': [
{
'arn': 'string',
'createdTime': datetime(2015, 1, 1),
'description': 'string',
'execution': {
'endedTime': datetime(2015, 1, 1),
'startedTime': datetime(2015, 1, 1)
},
'groups': [
{
'description': 'string',
'featureVariations': {
'string': 'string'
},
'name': 'string'
},
],
'lastUpdatedTime': datetime(2015, 1, 1),
'metricMonitors': [
{
'metricDefinition': {
'entityIdKey': 'string',
'eventPattern': 'string',
'name': 'string',
'unitLabel': 'string',
'valueKey': 'string'
}
},
],
'name': 'string',
'project': 'string',
'randomizationSalt': 'string',
'scheduledSplitsDefinition': {
'steps': [
{
'groupWeights': {
'string': 123
},
'startTime': datetime(2015, 1, 1)
},
]
},
'status': 'CREATED'|'UPDATING'|'RUNNING'|'COMPLETED'|'CANCELLED',
'statusReason': 'string',
'tags': {
'string': 'string'
},
'type': 'aws.evidently.splits'
},
],
'nextToken': 'string'
}
Response Structure
(dict) --
launches (list) --
An array of structures that contain the configuration details of the launches in the specified project.
(dict) --
This structure contains the configuration details of one Evidently launch.
arn (string) --
The ARN of the launch.
createdTime (datetime) --
The date and time that the launch is created.
description (string) --
The description of the launch.
execution (dict) --
A structure that contains information about the start and end times of the launch.
endedTime (datetime) --
The date and time that the launch ended.
startedTime (datetime) --
The date and time that the launch started.
groups (list) --
An array of structures that define the feature variations that are being used in the launch.
(dict) --
A structure that defines one launch group in a launch. A launch group is a variation of the feature that you are including in the launch.
description (string) --
A description of the launch group.
featureVariations (dict) --
The feature variation for this launch group. This is a key-value pair.
(string) --
(string) --
name (string) --
The name of the launch group.
lastUpdatedTime (datetime) --
The date and time that the launch was most recently updated.
metricMonitors (list) --
An array of structures that define the metrics that are being used to monitor the launch performance.
(dict) --
A structure that defines a metric to be used to monitor performance of the variations during a launch.
metricDefinition (dict) --
A structure that defines the metric.
entityIdKey (string) --
The entity, such as a user or session, that does an action that causes a metric value to be recorded.
eventPattern (string) --
The EventBridge event pattern that defines how the metric is recorded.
For more information about EventBridge event patterns, see Amazon EventBridge event patterns .
name (string) --
The name of the metric.
unitLabel (string) --
The label for the units that the metric is measuring.
valueKey (string) --
The value that is tracked to produce the metric.
name (string) --
The name of the launch.
project (string) --
The name or ARN of the project that contains the launch.
randomizationSalt (string) --
This value is used when Evidently assigns a particular user session to the launch, to help create a randomization ID to determine which variation the user session is served. This randomization ID is a combination of the entity ID and randomizationSalt .
scheduledSplitsDefinition (dict) --
An array of structures that define the traffic allocation percentages among the feature variations during each step of the launch.
steps (list) --
An array of structures that define the traffic allocation percentages among the feature variations during each step of the launch. This also defines the start time of each step.
(dict) --
This structure defines the traffic allocation percentages among the feature variations during one step of a launch, and the start time of that step.
groupWeights (dict) --
The traffic allocation percentages among the feature variations during one step of a launch. This is a set of key-value pairs. The keys are variation names. The values represent the percentage of traffic to allocate to that variation during this step.
(string) --
(integer) --
startTime (datetime) --
The date and time that this step of the launch starts.
status (string) --
The current state of the launch.
statusReason (string) --
If the launch was stopped, this is the string that was entered by the person who stopped the launch, to explain why it was stopped.
tags (dict) --
The list of tag keys and values associated with this launch.
(string) --
(string) --
type (string) --
The type of launch.
nextToken (string) --
The token to use in a subsequent ListLaunches operation to return the next set of results.
Returns the details about one launch. You must already know the project name. To retrieve a list of projects in your account, use ListProjects .
See also: AWS API Documentation
Request Syntax
client.get_project(
project='string'
)
string
[REQUIRED]
The name or ARN of the project that you want to see the details of.
dict
Response Syntax
{
'project': {
'activeExperimentCount': 123,
'activeLaunchCount': 123,
'arn': 'string',
'createdTime': datetime(2015, 1, 1),
'dataDelivery': {
'cloudWatchLogs': {
'logGroup': 'string'
},
's3Destination': {
'bucket': 'string',
'prefix': 'string'
}
},
'description': 'string',
'experimentCount': 123,
'featureCount': 123,
'lastUpdatedTime': datetime(2015, 1, 1),
'launchCount': 123,
'name': 'string',
'status': 'AVAILABLE'|'UPDATING',
'tags': {
'string': 'string'
}
}
}
Response Structure
(dict) --
project (dict) --
A structure containing the configuration details of the project.
activeExperimentCount (integer) --
The number of ongoing experiments currently in the project.
activeLaunchCount (integer) --
The number of ongoing launches currently in the project.
arn (string) --
The name or ARN of the project.
createdTime (datetime) --
The date and time that the project is created.
dataDelivery (dict) --
A structure that contains information about where Evidently is to store evaluation events for longer term storage.
cloudWatchLogs (dict) --
If the project stores evaluation events in CloudWatch Logs, this structure stores the log group name.
logGroup (string) --
The name of the log group where the project stores evaluation events.
s3Destination (dict) --
If the project stores evaluation events in an Amazon S3 bucket, this structure stores the bucket name and bucket prefix.
bucket (string) --
The name of the bucket in which Evidently stores evaluation events.
prefix (string) --
The bucket prefix in which Evidently stores evaluation events.
description (string) --
The user-entered description of the project.
experimentCount (integer) --
The number of experiments currently in the project. This includes all experiments that have been created and not deleted, whether they are ongoing or not.
featureCount (integer) --
The number of features currently in the project.
lastUpdatedTime (datetime) --
The date and time that the project was most recently updated.
launchCount (integer) --
The number of launches currently in the project. This includes all launches that have been created and not deleted, whether they are ongoing or not.
name (string) --
The name of the project.
status (string) --
The current state of the project.
tags (dict) --
The list of tag keys and values associated with this project.
(string) --
(string) --
Returns the details about one feature. You must already know the feature name. To retrieve a list of features in your account, use ListFeatures .
See also: AWS API Documentation
Request Syntax
client.get_feature(
feature='string',
project='string'
)
string
[REQUIRED]
The name of the feature that you want to retrieve information for.
string
[REQUIRED]
The name or ARN of the project that contains the feature.
dict
Response Syntax
{
'feature': {
'arn': 'string',
'createdTime': datetime(2015, 1, 1),
'defaultVariation': 'string',
'description': 'string',
'entityOverrides': {
'string': 'string'
},
'evaluationRules': [
{
'name': 'string',
'type': 'string'
},
],
'evaluationStrategy': 'ALL_RULES'|'DEFAULT_VARIATION',
'lastUpdatedTime': datetime(2015, 1, 1),
'name': 'string',
'project': 'string',
'status': 'AVAILABLE'|'UPDATING',
'tags': {
'string': 'string'
},
'valueType': 'STRING'|'LONG'|'DOUBLE'|'BOOLEAN',
'variations': [
{
'name': 'string',
'value': {
'boolValue': True|False,
'doubleValue': 123.0,
'longValue': 123,
'stringValue': 'string'
}
},
]
}
}
Response Structure
(dict) --
feature (dict) --
A structure containing the configuration details of the feature.
arn (string) --
The ARN of the feature.
createdTime (datetime) --
The date and time that the feature is created.
defaultVariation (string) --
The name of the variation that is used as the default variation. The default variation is served to users who are not allocated to any ongoing launches or experiments of this feature.
This variation must also be listed in the variations structure.
If you omit defaultVariation , the first variation listed in the variations structure is used as the default variation.
description (string) --
The description of the feature.
entityOverrides (dict) --
A set of key-value pairs that specify users who should always be served a specific variation of a feature. Each key specifies a user using their user ID, account ID, or some other identifier. The value specifies the name of the variation that the user is to be served.
For the override to be successful, the value of the key must match the entityId used in the EvaluateFeature operation.
(string) --
(string) --
evaluationRules (list) --
An array of structures that define the evaluation rules for the feature.
(dict) --
A structure that contains the information about an evaluation rule for this feature, if it is used in a launch or experiment.
name (string) --
The name of the experiment or launch.
type (string) --
This value is aws.evidently.splits if this is an evaluation rule for a launch, and it is aws.evidently.onlineab if this is an evaluation rule for an experiment.
evaluationStrategy (string) --
If this value is ALL_RULES , the traffic allocation specified by any ongoing launches or experiments is being used. If this is DEFAULT_VARIATION , the default variation is being served to all users.
lastUpdatedTime (datetime) --
The date and time that the feature was most recently updated.
name (string) --
The name of the feature.
project (string) --
The name or ARN of the project that contains the feature.
status (string) --
The current state of the feature.
tags (dict) --
The list of tag keys and values associated with this feature.
(string) --
(string) --
valueType (string) --
Defines the type of value used to define the different feature variations. For more information, see Variation types
variations (list) --
An array of structures that contain the configuration of the feature's different variations.
(dict) --
This structure contains the name and variation value of one variation of a feature.
name (string) --
The name of the variation.
value (dict) --
The value assigned to this variation.
boolValue (boolean) --
If this feature uses the Boolean variation type, this field contains the Boolean value of this variation.
doubleValue (float) --
If this feature uses the double integer variation type, this field contains the double integer value of this variation.
longValue (integer) --
If this feature uses the long variation type, this field contains the long value of this variation.
stringValue (string) --
If this feature uses the string variation type, this field contains the string value of this variation.
Updates an existing feature.
You can't use this operation to update the tags of an existing feature. Instead, use TagResource .
See also: AWS API Documentation
Request Syntax
client.update_feature(
addOrUpdateVariations=[
{
'name': 'string',
'value': {
'boolValue': True|False,
'doubleValue': 123.0,
'longValue': 123,
'stringValue': 'string'
}
},
],
defaultVariation='string',
description='string',
entityOverrides={
'string': 'string'
},
evaluationStrategy='ALL_RULES'|'DEFAULT_VARIATION',
feature='string',
project='string',
removeVariations=[
'string',
]
)
list
To update variation configurations for this feature, or add new ones, specify this structure. In this array, include any variations that you want to add or update. If the array includes a variation name that already exists for this feature, it is updated. If it includes a new variation name, it is added as a new variation.
(dict) --
This structure contains the name and variation value of one variation of a feature.
name (string) -- [REQUIRED]
The name of the variation.
value (dict) -- [REQUIRED]
The value assigned to this variation.
boolValue (boolean) --
If this feature uses the Boolean variation type, this field contains the Boolean value of this variation.
doubleValue (float) --
If this feature uses the double integer variation type, this field contains the double integer value of this variation.
longValue (integer) --
If this feature uses the long variation type, this field contains the long value of this variation.
stringValue (string) --
If this feature uses the string variation type, this field contains the string value of this variation.
string
The name of the variation to use as the default variation. The default variation is served to users who are not allocated to any ongoing launches or experiments of this feature.
string
An optional description of the feature.
dict
Specified users that should always be served a specific variation of a feature. Each user is specified by a key-value pair . For each key, specify a user by entering their user ID, account ID, or some other identifier. For the value, specify the name of the variation that they are to be served.
(string) --
(string) --
string
Specify ALL_RULES to activate the traffic allocation specified by any ongoing launches or experiments. Specify DEFAULT_VARIATION to serve the default variation to all users instead.
string
[REQUIRED]
The name of the feature to be updated.
string
[REQUIRED]
The name or ARN of the project that contains the feature to be updated.
list
Removes a variation from the feature. If the variation you specify doesn't exist, then this makes no change and does not report an error.
This operation fails if you try to remove a variation that is part of an ongoing launch or experiment.
(string) --
dict
Response Syntax
{
'feature': {
'arn': 'string',
'createdTime': datetime(2015, 1, 1),
'defaultVariation': 'string',
'description': 'string',
'entityOverrides': {
'string': 'string'
},
'evaluationRules': [
{
'name': 'string',
'type': 'string'
},
],
'evaluationStrategy': 'ALL_RULES'|'DEFAULT_VARIATION',
'lastUpdatedTime': datetime(2015, 1, 1),
'name': 'string',
'project': 'string',
'status': 'AVAILABLE'|'UPDATING',
'tags': {
'string': 'string'
},
'valueType': 'STRING'|'LONG'|'DOUBLE'|'BOOLEAN',
'variations': [
{
'name': 'string',
'value': {
'boolValue': True|False,
'doubleValue': 123.0,
'longValue': 123,
'stringValue': 'string'
}
},
]
}
}
Response Structure
(dict) --
feature (dict) --
A structure that contains information about the updated feature.
arn (string) --
The ARN of the feature.
createdTime (datetime) --
The date and time that the feature is created.
defaultVariation (string) --
The name of the variation that is used as the default variation. The default variation is served to users who are not allocated to any ongoing launches or experiments of this feature.
This variation must also be listed in the variations structure.
If you omit defaultVariation , the first variation listed in the variations structure is used as the default variation.
description (string) --
The description of the feature.
entityOverrides (dict) --
A set of key-value pairs that specify users who should always be served a specific variation of a feature. Each key specifies a user using their user ID, account ID, or some other identifier. The value specifies the name of the variation that the user is to be served.
For the override to be successful, the value of the key must match the entityId used in the EvaluateFeature operation.
(string) --
(string) --
evaluationRules (list) --
An array of structures that define the evaluation rules for the feature.
(dict) --
A structure that contains the information about an evaluation rule for this feature, if it is used in a launch or experiment.
name (string) --
The name of the experiment or launch.
type (string) --
This value is aws.evidently.splits if this is an evaluation rule for a launch, and it is aws.evidently.onlineab if this is an evaluation rule for an experiment.
evaluationStrategy (string) --
If this value is ALL_RULES , the traffic allocation specified by any ongoing launches or experiments is being used. If this is DEFAULT_VARIATION , the default variation is being served to all users.
lastUpdatedTime (datetime) --
The date and time that the feature was most recently updated.
name (string) --
The name of the feature.
project (string) --
The name or ARN of the project that contains the feature.
status (string) --
The current state of the feature.
tags (dict) --
The list of tag keys and values associated with this feature.
(string) --
(string) --
valueType (string) --
Defines the type of value used to define the different feature variations. For more information, see Variation types
variations (list) --
An array of structures that contain the configuration of the feature's different variations.
(dict) --
This structure contains the name and variation value of one variation of a feature.
name (string) --
The name of the variation.
value (dict) --
The value assigned to this variation.
boolValue (boolean) --
If this feature uses the Boolean variation type, this field contains the Boolean value of this variation.
doubleValue (float) --
If this feature uses the double integer variation type, this field contains the double integer value of this variation.
longValue (integer) --
If this feature uses the long variation type, this field contains the long value of this variation.
stringValue (string) --
If this feature uses the string variation type, this field contains the string value of this variation.
Creates an Evidently experiment . Before you create an experiment, you must create the feature to use for the experiment.
An experiment helps you make feature design decisions based on evidence and data. An experiment can test as many as five variations at once. Evidently collects experiment data and analyzes it by statistical methods, and provides clear recommendations about which variations perform better.
Don't use this operation to update an existing experiment. Instead, use UpdateExperiment .
See also: AWS API Documentation
Request Syntax
client.create_experiment(
description='string',
metricGoals=[
{
'desiredChange': 'INCREASE'|'DECREASE',
'metricDefinition': {
'entityIdKey': 'string',
'eventPattern': 'string',
'name': 'string',
'unitLabel': 'string',
'valueKey': 'string'
}
},
],
name='string',
onlineAbConfig={
'controlTreatmentName': 'string',
'treatmentWeights': {
'string': 123
}
},
project='string',
randomizationSalt='string',
samplingRate=123,
tags={
'string': 'string'
},
treatments=[
{
'description': 'string',
'feature': 'string',
'name': 'string',
'variation': 'string'
},
]
)
string
An optional description of the experiment.
list
[REQUIRED]
An array of structures that defines the metrics used for the experiment, and whether a higher or lower value for each metric is the goal.
(dict) --
Use this structure to tell Evidently whether higher or lower values are desired for a metric that is used in an experiment.
desiredChange (string) --
INCREASE means that a variation with a higher number for this metric is performing better.
DECREASE means that a variation with a lower number for this metric is performing better.
metricDefinition (dict) -- [REQUIRED]
A structure that contains details about the metric.
entityIdKey (string) --
The entity, such as a user or session, that does an action that causes a metric value to be recorded. An example is userDetails.userID .
eventPattern (string) --
The EventBridge event pattern that defines how the metric is recorded.
For more information about EventBridge event patterns, see Amazon EventBridge event patterns .
name (string) --
A name for the metric.
unitLabel (string) --
A label for the units that the metric is measuring.
valueKey (string) --
The value that is tracked to produce the metric.
string
[REQUIRED]
A name for the new experiment.
dict
A structure that contains the configuration of which variation to use as the "control" version. tThe "control" version is used for comparison with other variations. This structure also specifies how much experiment traffic is allocated to each variation.
controlTreatmentName (string) --
The name of the variation that is to be the default variation that the other variations are compared to.
treatmentWeights (dict) --
A set of key-value pairs. The keys are variation names, and the values are the portion of experiment traffic to be assigned to that variation. Specify the traffic portion in thousandths of a percent, so 20,000 for a variation would allocate 20% of the experiment traffic to that variation.
(string) --
(integer) --
string
[REQUIRED]
The name or ARN of the project that you want to create the new experiment in.
string
When Evidently assigns a particular user session to an experiment, it must use a randomization ID to determine which variation the user session is served. This randomization ID is a combination of the entity ID and randomizationSalt . If you omit randomizationSalt , Evidently uses the experiment name as the randomizationSalt .
integer
The portion of the available audience that you want to allocate to this experiment, in thousandths of a percent. The available audience is the total audience minus the audience that you have allocated to overrides or current launches of this feature.
This is represented in thousandths of a percent. For example, specify 10,000 to allocate 10% of the available audience.
dict
Assigns one or more tags (key-value pairs) to the experiment.
Tags can help you organize and categorize your resources. You can also use them to scope user permissions by granting a user permission to access or change only resources with certain tag values.
Tags don't have any semantic meaning to Amazon Web Services and are interpreted strictly as strings of characters.
<p>You can associate as many as 50 tags with an experiment.</p> <p>For more information, see <a href="https://docs.aws.amazon.com/general/latest/gr/aws_tagging.html">Tagging Amazon Web Services resources</a>.</p>
(string) --
(string) --
list
[REQUIRED]
An array of structures that describe the configuration of each feature variation used in the experiment.
(dict) --
A structure that defines one treatment in an experiment. A treatment is a variation of the feature that you are including in the experiment.
description (string) --
A description for this treatment.
feature (string) -- [REQUIRED]
The feature that this experiment is testing.
name (string) -- [REQUIRED]
A name for this treatment.
variation (string) -- [REQUIRED]
The name of the variation to use as this treatment in the experiment.
dict
Response Syntax
{
'experiment': {
'arn': 'string',
'createdTime': datetime(2015, 1, 1),
'description': 'string',
'execution': {
'endedTime': datetime(2015, 1, 1),
'startedTime': datetime(2015, 1, 1)
},
'lastUpdatedTime': datetime(2015, 1, 1),
'metricGoals': [
{
'desiredChange': 'INCREASE'|'DECREASE',
'metricDefinition': {
'entityIdKey': 'string',
'eventPattern': 'string',
'name': 'string',
'unitLabel': 'string',
'valueKey': 'string'
}
},
],
'name': 'string',
'onlineAbDefinition': {
'controlTreatmentName': 'string',
'treatmentWeights': {
'string': 123
}
},
'project': 'string',
'randomizationSalt': 'string',
'samplingRate': 123,
'schedule': {
'analysisCompleteTime': datetime(2015, 1, 1)
},
'status': 'CREATED'|'UPDATING'|'RUNNING'|'COMPLETED'|'CANCELLED',
'statusReason': 'string',
'tags': {
'string': 'string'
},
'treatments': [
{
'description': 'string',
'featureVariations': {
'string': 'string'
},
'name': 'string'
},
],
'type': 'aws.evidently.onlineab'
}
}
Response Structure
(dict) --
experiment (dict) --
A structure containing the configuration details of the experiment that you created.
arn (string) --
The ARN of the experiment.
createdTime (datetime) --
The date and time that the experiment is first created.
description (string) --
A description of the experiment.
execution (dict) --
A structure that contains the date and time that the experiment started and ended.
endedTime (datetime) --
The date and time that the experiment ended.
startedTime (datetime) --
The date and time that the experiment started.
lastUpdatedTime (datetime) --
The date and time that the experiment was most recently updated.
metricGoals (list) --
An array of structures that defines the metrics used for the experiment, and whether a higher or lower value for each metric is the goal.
(dict) --
A structure that tells Evidently whether higher or lower values are desired for a metric that is used in an experiment.
desiredChange (string) --
INCREASE means that a variation with a higher number for this metric is performing better.
DECREASE means that a variation with a lower number for this metric is performing better.
metricDefinition (dict) --
A structure that contains details about the metric.
entityIdKey (string) --
The entity, such as a user or session, that does an action that causes a metric value to be recorded.
eventPattern (string) --
The EventBridge event pattern that defines how the metric is recorded.
For more information about EventBridge event patterns, see Amazon EventBridge event patterns .
name (string) --
The name of the metric.
unitLabel (string) --
The label for the units that the metric is measuring.
valueKey (string) --
The value that is tracked to produce the metric.
name (string) --
The name of the experiment.
onlineAbDefinition (dict) --
A structure that contains the configuration of which variation to use as the "control" version. The "control" version is used for comparison with other variations. This structure also specifies how much experiment traffic is allocated to each variation.
controlTreatmentName (string) --
The name of the variation that is the default variation that the other variations are compared to.
treatmentWeights (dict) --
A set of key-value pairs. The keys are variation names, and the values are the portion of experiment traffic to be assigned to that variation. The traffic portion is specified in thousandths of a percent, so 20,000 for a variation would allocate 20% of the experiment traffic to that variation.
(string) --
(integer) --
project (string) --
The name or ARN of the project that contains this experiment.
randomizationSalt (string) --
This value is used when Evidently assigns a particular user session to the experiment. It helps create a randomization ID to determine which variation the user session is served. This randomization ID is a combination of the entity ID and randomizationSalt .
samplingRate (integer) --
In thousandths of a percent, the amount of the available audience that is allocated to this experiment. The available audience is the total audience minus the audience that you have allocated to overrides or current launches of this feature.
This is represented in thousandths of a percent, so a value of 10,000 is 10% of the available audience.
schedule (dict) --
A structure that contains the time and date that Evidently completed the analysis of the experiment.
analysisCompleteTime (datetime) --
The time and date that Evidently completed the analysis of the experiment.
status (string) --
The current state of the experiment.
statusReason (string) --
If the experiment was stopped, this is the string that was entered by the person who stopped the experiment, to explain why it was stopped.
tags (dict) --
The list of tag keys and values associated with this experiment.
(string) --
(string) --
treatments (list) --
An array of structures that describe the configuration of each feature variation used in the experiment.
(dict) --
A structure that defines one treatment in an experiment. A treatment is a variation of the feature that you are including in the experiment.
description (string) --
The description of the treatment.
featureVariations (dict) --
The feature variation used for this treatment. This is a key-value pair. The key is the feature name, and the value is the variation name.
(string) --
(string) --
name (string) --
The name of this treatment.
type (string) --
The type of this experiment. Currently, this value must be aws.experiment.onlineab .
Retrieves the results of a running or completed experiment.
See also: AWS API Documentation
Request Syntax
client.get_experiment_results(
baseStat='Mean',
endTime=datetime(2015, 1, 1),
experiment='string',
metricNames=[
'string',
],
period=123,
project='string',
reportNames=[
'BayesianInference',
],
resultStats=[
'BaseStat'|'TreatmentEffect'|'ConfidenceInterval'|'PValue',
],
startTime=datetime(2015, 1, 1),
treatmentNames=[
'string',
]
)
string
The statistic used to calculate experiment results. Currently the only valid value is mean , which uses the mean of the collected values as the statistic.
datetime
The date and time that the experiment ended, if it is completed.
string
[REQUIRED]
The name of the experiment to retrieve the results of.
list
[REQUIRED]
The names of the experiment metrics that you want to see the results of.
(string) --
integer
In seconds, the amount of time to aggregate results together.
string
[REQUIRED]
The name or ARN of the project that contains the experiment that you want to see the results of.
list
The names of the report types that you want to see. Currently, BayesianInference is the only valid value.
(string) --
list
The statistics that you want to see in the returned results.
PValue specifies to use p-values for the results. A p-value is used in hypothesis testing to measure how often you are willing to make a mistake in rejecting the null hypothesis. A general practice is to reject the null hypothesis and declare that the results are statistically significant when the p-value is less than 0.05.
ConfidenceInterval specifies a confidence interval for the results. The confidence interval represents the range of values for the chosen metric that is likely to contain the true difference between the baseStat of a variation and the baseline. Evidently returns the 95% confidence interval.
TreatmentEffect is the difference in the statistic specified by the baseStat parameter between each variation and the default variation.
BaseStat returns the statistical values collected for the metric for each variation. The statistic uses the same statistic specified in the baseStat parameter. Therefore, if baseStat is mean , this returns the mean of the values collected for each variation.
(string) --
datetime
The date and time that the experiment started.
list
[REQUIRED]
The names of the experiment treatments that you want to see the results for.
(string) --
dict
Response Syntax
{
'reports': [
{
'content': 'string',
'metricName': 'string',
'reportName': 'BayesianInference',
'treatmentName': 'string'
},
],
'resultsData': [
{
'metricName': 'string',
'resultStat': 'Mean'|'TreatmentEffect'|'ConfidenceIntervalUpperBound'|'ConfidenceIntervalLowerBound'|'PValue',
'treatmentName': 'string',
'values': [
123.0,
]
},
],
'timestamps': [
datetime(2015, 1, 1),
]
}
Response Structure
(dict) --
reports (list) --
An array of structures that include the reports that you requested.
(dict) --
A structure that contains results of an experiment.
content (string) --
The content of the report.
metricName (string) --
The name of the metric that is analyzed in this experiment report.
reportName (string) --
The type of analysis used for this report.
treatmentName (string) --
The name of the variation that this report pertains to.
resultsData (list) --
An array of structures that include experiment results including metric names and values.
(dict) --
A structure that contains experiment results for one metric that is monitored in the experiment.
metricName (string) --
The name of the metric.
resultStat (string) --
The experiment statistic that these results pertain to.
treatmentName (string) --
The treatment, or variation, that returned the values in this structure.
values (list) --
The values for the metricName that were recorded in the experiment.
(float) --
timestamps (list) --
The timestamps of each result returned.
(datetime) --
Stops a launch that is currently running. After you stop a launch, you will not be able to resume it or restart it. Also, it will not be evaluated as a rule for traffic allocation, and the traffic that was allocated to the launch will instead be available to the feature's experiment, if there is one. Otherwise, all traffic will be served the default variation after the launch is stopped.
See also: AWS API Documentation
Request Syntax
client.stop_launch(
desiredState='COMPLETED'|'CANCELLED',
launch='string',
project='string',
reason='string'
)
string
Specify whether to consider the launch as COMPLETED or CANCELLED after it stops.
string
[REQUIRED]
The name of the launch to stop.
string
[REQUIRED]
The name or ARN of the project that contains the launch that you want to stop.
string
A string that describes why you are stopping the launch.
dict
Response Syntax
{
'endedTime': datetime(2015, 1, 1)
}
Response Structure
(dict) --
endedTime (datetime) --
The date and time that the launch stopped.
Creates a project, which is the logical object in Evidently that can contain features, launches, and experiments. Use projects to group similar features together.
To update an existing project, use UpdateProject .
See also: AWS API Documentation
Request Syntax
client.create_project(
dataDelivery={
'cloudWatchLogs': {
'logGroup': 'string'
},
's3Destination': {
'bucket': 'string',
'prefix': 'string'
}
},
description='string',
name='string',
tags={
'string': 'string'
}
)
dict
A structure that contains information about where Evidently is to store evaluation events for longer term storage, if you choose to do so. If you choose not to store these events, Evidently deletes them after using them to produce metrics and other experiment results that you can view.
cloudWatchLogs (dict) --
If the project stores evaluation events in CloudWatch Logs, this structure stores the log group name.
logGroup (string) --
The name of the log group where the project stores evaluation events.
s3Destination (dict) --
If the project stores evaluation events in an Amazon S3 bucket, this structure stores the bucket name and bucket prefix.
bucket (string) --
The name of the bucket in which Evidently stores evaluation events.
prefix (string) --
The bucket prefix in which Evidently stores evaluation events.
string
An optional description of the project.
string
[REQUIRED]
The name for the project.
dict
Assigns one or more tags (key-value pairs) to the project.
Tags can help you organize and categorize your resources. You can also use them to scope user permissions by granting a user permission to access or change only resources with certain tag values.
Tags don't have any semantic meaning to Amazon Web Services and are interpreted strictly as strings of characters.
<p>You can associate as many as 50 tags with a project.</p> <p>For more information, see <a href="https://docs.aws.amazon.com/general/latest/gr/aws_tagging.html">Tagging Amazon Web Services resources</a>.</p>
(string) --
(string) --
dict
Response Syntax
{
'project': {
'activeExperimentCount': 123,
'activeLaunchCount': 123,
'arn': 'string',
'createdTime': datetime(2015, 1, 1),
'dataDelivery': {
'cloudWatchLogs': {
'logGroup': 'string'
},
's3Destination': {
'bucket': 'string',
'prefix': 'string'
}
},
'description': 'string',
'experimentCount': 123,
'featureCount': 123,
'lastUpdatedTime': datetime(2015, 1, 1),
'launchCount': 123,
'name': 'string',
'status': 'AVAILABLE'|'UPDATING',
'tags': {
'string': 'string'
}
}
}
Response Structure
(dict) --
project (dict) --
A structure that contains information about the created project.
activeExperimentCount (integer) --
The number of ongoing experiments currently in the project.
activeLaunchCount (integer) --
The number of ongoing launches currently in the project.
arn (string) --
The name or ARN of the project.
createdTime (datetime) --
The date and time that the project is created.
dataDelivery (dict) --
A structure that contains information about where Evidently is to store evaluation events for longer term storage.
cloudWatchLogs (dict) --
If the project stores evaluation events in CloudWatch Logs, this structure stores the log group name.
logGroup (string) --
The name of the log group where the project stores evaluation events.
s3Destination (dict) --
If the project stores evaluation events in an Amazon S3 bucket, this structure stores the bucket name and bucket prefix.
bucket (string) --
The name of the bucket in which Evidently stores evaluation events.
prefix (string) --
The bucket prefix in which Evidently stores evaluation events.
description (string) --
The user-entered description of the project.
experimentCount (integer) --
The number of experiments currently in the project. This includes all experiments that have been created and not deleted, whether they are ongoing or not.
featureCount (integer) --
The number of features currently in the project.
lastUpdatedTime (datetime) --
The date and time that the project was most recently updated.
launchCount (integer) --
The number of launches currently in the project. This includes all launches that have been created and not deleted, whether they are ongoing or not.
name (string) --
The name of the project.
status (string) --
The current state of the project.
tags (dict) --
The list of tag keys and values associated with this project.
(string) --
(string) --
Stops an experiment that is currently running. If you stop an experiment, you can't resume it or restart it.
See also: AWS API Documentation
Request Syntax
client.stop_experiment(
desiredState='COMPLETED'|'CANCELLED',
experiment='string',
project='string',
reason='string'
)
string
Specify whether the experiment is to be considered COMPLETED or CANCELLED after it stops.
string
[REQUIRED]
The name of the experiment to stop.
string
[REQUIRED]
The name or ARN of the project that contains the experiment to stop.
string
A string that describes why you are stopping the experiment.
dict
Response Syntax
{
'endedTime': datetime(2015, 1, 1)
}
Response Structure
(dict) --
endedTime (datetime) --
The date and time that the experiment stopped.
This operation assigns a feature variation to one given user session. You pass in an entityID that represents the user. Evidently then checks the evaluation rules and assigns the variation.
The first rules that are evaluated are the override rules. If the user's entityID matches an override rule, the user is served the variation specified by that rule.
Next, if there is a launch of the feature, the user might be assigned to a variation in the launch. The chance of this depends on the percentage of users that are allocated to that launch. If the user is enrolled in the launch, the variation they are served depends on the allocation of the various feature variations used for the launch.
If the user is not assigned to a launch, and there is an ongoing experiment for this feature, the user might be assigned to a variation in the experiment. The chance of this depends on the percentage of users that are allocated to that experiment. If the user is enrolled in the experiment, the variation they are served depends on the allocation of the various feature variations used for the experiment.
If the user is not assigned to a launch or experiment, they are served the default variation.
See also: AWS API Documentation
Request Syntax
client.evaluate_feature(
entityId='string',
evaluationContext='string',
feature='string',
project='string'
)
string
[REQUIRED]
An internal ID that represents a unique user of the application. This entityID is checked against any override rules assigned for this feature.
string
A JSON block of attributes that you can optionally pass in. This JSON block is included in the evaluation events sent to Evidently from the user session.
string
[REQUIRED]
The name of the feature being evaluated.
string
[REQUIRED]
The name or ARN of the project that contains this feature.
dict
Response Syntax
{
'details': 'string',
'reason': 'string',
'value': {
'boolValue': True|False,
'doubleValue': 123.0,
'longValue': 123,
'stringValue': 'string'
},
'variation': 'string'
}
Response Structure
(dict) --
details (string) --
If this user was assigned to a launch or experiment, this field lists the launch or experiment name.
reason (string) --
Specifies the reason that the user session was assigned this variation. Possible values include DEFAULT , meaning the user was served the default variation; LAUNCH_RULE_MATCH , if the user session was enrolled in a launch; EXPERIMENT_RULE_MATCH , if the user session was enrolled in an experiment; or ENTITY_OVERRIDES_MATCH , if the user's entityId matches an override rule.
value (dict) --
The value assigned to this variation to differentiate it from the other variations of this feature.
boolValue (boolean) --
If this feature uses the Boolean variation type, this field contains the Boolean value of this variation.
doubleValue (float) --
If this feature uses the double integer variation type, this field contains the double integer value of this variation.
longValue (integer) --
If this feature uses the long variation type, this field contains the long value of this variation.
stringValue (string) --
If this feature uses the string variation type, this field contains the string value of this variation.
variation (string) --
The name of the variation that was served to the user session.
Deletes an Evidently project. Before you can delete a project, you must delete all the features that the project contains. To delete a feature, use DeleteFeature .
See also: AWS API Documentation
Request Syntax
client.delete_project(
project='string'
)
string
[REQUIRED]
The name or ARN of the project to delete.
dict
Response Syntax
{}
Response Structure
(dict) --
Updates the data storage options for this project. If you store evaluation events, you an keep them and analyze them on your own. If you choose not to store evaluation events, Evidently deletes them after using them to produce metrics and other experiment results that you can view.
You can't specify both cloudWatchLogs and s3Destination in the same operation.
See also: AWS API Documentation
Request Syntax
client.update_project_data_delivery(
cloudWatchLogs={
'logGroup': 'string'
},
project='string',
s3Destination={
'bucket': 'string',
'prefix': 'string'
}
)
dict
A structure containing the CloudWatch Logs log group where you want to store evaluation events.
logGroup (string) --
The name of the log group where the project stores evaluation events.
string
[REQUIRED]
The name or ARN of the project that you want to modify the data storage options for.
dict
A structure containing the S3 bucket name and bucket prefix where you want to store evaluation events.
bucket (string) --
The name of the bucket in which Evidently stores evaluation events.
prefix (string) --
The bucket prefix in which Evidently stores evaluation events.
dict
Response Syntax
{
'project': {
'activeExperimentCount': 123,
'activeLaunchCount': 123,
'arn': 'string',
'createdTime': datetime(2015, 1, 1),
'dataDelivery': {
'cloudWatchLogs': {
'logGroup': 'string'
},
's3Destination': {
'bucket': 'string',
'prefix': 'string'
}
},
'description': 'string',
'experimentCount': 123,
'featureCount': 123,
'lastUpdatedTime': datetime(2015, 1, 1),
'launchCount': 123,
'name': 'string',
'status': 'AVAILABLE'|'UPDATING',
'tags': {
'string': 'string'
}
}
}
Response Structure
(dict) --
project (dict) --
A structure containing details about the project that you updated.
activeExperimentCount (integer) --
The number of ongoing experiments currently in the project.
activeLaunchCount (integer) --
The number of ongoing launches currently in the project.
arn (string) --
The name or ARN of the project.
createdTime (datetime) --
The date and time that the project is created.
dataDelivery (dict) --
A structure that contains information about where Evidently is to store evaluation events for longer term storage.
cloudWatchLogs (dict) --
If the project stores evaluation events in CloudWatch Logs, this structure stores the log group name.
logGroup (string) --
The name of the log group where the project stores evaluation events.
s3Destination (dict) --
If the project stores evaluation events in an Amazon S3 bucket, this structure stores the bucket name and bucket prefix.
bucket (string) --
The name of the bucket in which Evidently stores evaluation events.
prefix (string) --
The bucket prefix in which Evidently stores evaluation events.
description (string) --
The user-entered description of the project.
experimentCount (integer) --
The number of experiments currently in the project. This includes all experiments that have been created and not deleted, whether they are ongoing or not.
featureCount (integer) --
The number of features currently in the project.
lastUpdatedTime (datetime) --
The date and time that the project was most recently updated.
launchCount (integer) --
The number of launches currently in the project. This includes all launches that have been created and not deleted, whether they are ongoing or not.
name (string) --
The name of the project.
status (string) --
The current state of the project.
tags (dict) --
The list of tag keys and values associated with this project.
(string) --
(string) --
Displays the tags associated with an Evidently resource.
See also: AWS API Documentation
Request Syntax
client.list_tags_for_resource(
resourceArn='string'
)
string
[REQUIRED]
The ARN of the resource that you want to see the tags of.
dict
Response Syntax
{
'tags': {
'string': 'string'
}
}
Response Structure
(dict) --
tags (dict) --
The list of tag keys and values associated with the resource you specified.
(string) --
(string) --