2021/09/07 - Amazon Forecast Service - 3 updated api methods
Changes Predictor creation now supports selecting an accuracy metric to optimize in AutoML and hyperparameter optimization. This release adds additional accuracy metrics for predictors - AverageWeightedQuantileLoss, MAPE and MASE.
{'OptimizationMetric': 'WAPE | RMSE | AverageWeightedQuantileLoss | MASE | '
'MAPE'}
Creates an Amazon Forecast predictor.
In the request, provide a dataset group and either specify an algorithm or let Amazon Forecast choose an algorithm for you using AutoML. If you specify an algorithm, you also can override algorithm-specific hyperparameters.
Amazon Forecast uses the algorithm to train a predictor using the latest version of the datasets in the specified dataset group. You can then generate a forecast using the CreateForecast operation.
To see the evaluation metrics, use the GetAccuracyMetrics operation.
You can specify a featurization configuration to fill and aggregate the data fields in the TARGET_TIME_SERIES dataset to improve model training. For more information, see FeaturizationConfig .
For RELATED_TIME_SERIES datasets, CreatePredictor verifies that the DataFrequency specified when the dataset was created matches the ForecastFrequency . TARGET_TIME_SERIES datasets don't have this restriction. Amazon Forecast also verifies the delimiter and timestamp format. For more information, see howitworks-datasets-groups .
By default, predictors are trained and evaluated at the 0.1 (P10), 0.5 (P50), and 0.9 (P90) quantiles. You can choose custom forecast types to train and evaluate your predictor by setting the ForecastTypes .
AutoML
If you want Amazon Forecast to evaluate each algorithm and choose the one that minimizes the objective function , set PerformAutoML to true . The objective function is defined as the mean of the weighted losses over the forecast types. By default, these are the p10, p50, and p90 quantile losses. For more information, see EvaluationResult .
When AutoML is enabled, the following properties are disallowed:
AlgorithmArn
HPOConfig
PerformHPO
TrainingParameters
To get a list of all of your predictors, use the ListPredictors operation.
Note
Before you can use the predictor to create a forecast, the Status of the predictor must be ACTIVE , signifying that training has completed. To get the status, use the DescribePredictor operation.
See also: AWS API Documentation
Request Syntax
client.create_predictor(
PredictorName='string',
AlgorithmArn='string',
ForecastHorizon=123,
ForecastTypes=[
'string',
],
PerformAutoML=True|False,
AutoMLOverrideStrategy='LatencyOptimized',
PerformHPO=True|False,
TrainingParameters={
'string': 'string'
},
EvaluationParameters={
'NumberOfBacktestWindows': 123,
'BackTestWindowOffset': 123
},
HPOConfig={
'ParameterRanges': {
'CategoricalParameterRanges': [
{
'Name': 'string',
'Values': [
'string',
]
},
],
'ContinuousParameterRanges': [
{
'Name': 'string',
'MaxValue': 123.0,
'MinValue': 123.0,
'ScalingType': 'Auto'|'Linear'|'Logarithmic'|'ReverseLogarithmic'
},
],
'IntegerParameterRanges': [
{
'Name': 'string',
'MaxValue': 123,
'MinValue': 123,
'ScalingType': 'Auto'|'Linear'|'Logarithmic'|'ReverseLogarithmic'
},
]
}
},
InputDataConfig={
'DatasetGroupArn': 'string',
'SupplementaryFeatures': [
{
'Name': 'string',
'Value': 'string'
},
]
},
FeaturizationConfig={
'ForecastFrequency': 'string',
'ForecastDimensions': [
'string',
],
'Featurizations': [
{
'AttributeName': 'string',
'FeaturizationPipeline': [
{
'FeaturizationMethodName': 'filling',
'FeaturizationMethodParameters': {
'string': 'string'
}
},
]
},
]
},
EncryptionConfig={
'RoleArn': 'string',
'KMSKeyArn': 'string'
},
Tags=[
{
'Key': 'string',
'Value': 'string'
},
],
OptimizationMetric='WAPE'|'RMSE'|'AverageWeightedQuantileLoss'|'MASE'|'MAPE'
)
string
[REQUIRED]
A name for the predictor.
string
The Amazon Resource Name (ARN) of the algorithm to use for model training. Required if PerformAutoML is not set to true .
Supported algorithms:
arn:aws:forecast:::algorithm/ARIMA
arn:aws:forecast:::algorithm/CNN-QR
arn:aws:forecast:::algorithm/Deep_AR_Plus
arn:aws:forecast:::algorithm/ETS
arn:aws:forecast:::algorithm/NPTS
arn:aws:forecast:::algorithm/Prophet
integer
[REQUIRED]
Specifies the number of time-steps that the model is trained to predict. The forecast horizon is also called the prediction length.
For example, if you configure a dataset for daily data collection (using the DataFrequency parameter of the CreateDataset operation) and set the forecast horizon to 10, the model returns predictions for 10 days.
The maximum forecast horizon is the lesser of 500 time-steps or 1/3 of the TARGET_TIME_SERIES dataset length.
list
Specifies the forecast types used to train a predictor. You can specify up to five forecast types. Forecast types can be quantiles from 0.01 to 0.99, by increments of 0.01 or higher. You can also specify the mean forecast with mean .
The default value is ["0.10", "0.50", "0.9"] .
(string) --
boolean
Whether to perform AutoML. When Amazon Forecast performs AutoML, it evaluates the algorithms it provides and chooses the best algorithm and configuration for your training dataset.
The default value is false . In this case, you are required to specify an algorithm.
Set PerformAutoML to true to have Amazon Forecast perform AutoML. This is a good option if you aren't sure which algorithm is suitable for your training data. In this case, PerformHPO must be false.
string
Note
The LatencyOptimized AutoML override strategy is only available in private beta. Contact AWS Support or your account manager to learn more about access privileges.
Used to overide the default AutoML strategy, which is to optimize predictor accuracy. To apply an AutoML strategy that minimizes training time, use LatencyOptimized .
This parameter is only valid for predictors trained using AutoML.
boolean
Whether to perform hyperparameter optimization (HPO). HPO finds optimal hyperparameter values for your training data. The process of performing HPO is known as running a hyperparameter tuning job.
The default value is false . In this case, Amazon Forecast uses default hyperparameter values from the chosen algorithm.
To override the default values, set PerformHPO to true and, optionally, supply the HyperParameterTuningJobConfig object. The tuning job specifies a metric to optimize, which hyperparameters participate in tuning, and the valid range for each tunable hyperparameter. In this case, you are required to specify an algorithm and PerformAutoML must be false.
The following algorithms support HPO:
DeepAR+
CNN-QR
dict
The hyperparameters to override for model training. The hyperparameters that you can override are listed in the individual algorithms. For the list of supported algorithms, see aws-forecast-choosing-recipes .
(string) --
(string) --
dict
Used to override the default evaluation parameters of the specified algorithm. Amazon Forecast evaluates a predictor by splitting a dataset into training data and testing data. The evaluation parameters define how to perform the split and the number of iterations.
NumberOfBacktestWindows (integer) --
The number of times to split the input data. The default is 1. Valid values are 1 through 5.
BackTestWindowOffset (integer) --
The point from the end of the dataset where you want to split the data for model training and testing (evaluation). Specify the value as the number of data points. The default is the value of the forecast horizon. BackTestWindowOffset can be used to mimic a past virtual forecast start date. This value must be greater than or equal to the forecast horizon and less than half of the TARGET_TIME_SERIES dataset length.
ForecastHorizon <= BackTestWindowOffset < 1/2 * TARGET_TIME_SERIES dataset length
dict
Provides hyperparameter override values for the algorithm. If you don't provide this parameter, Amazon Forecast uses default values. The individual algorithms specify which hyperparameters support hyperparameter optimization (HPO). For more information, see aws-forecast-choosing-recipes .
If you included the HPOConfig object, you must set PerformHPO to true.
ParameterRanges (dict) --
Specifies the ranges of valid values for the hyperparameters.
CategoricalParameterRanges (list) --
Specifies the tunable range for each categorical hyperparameter.
(dict) --
Specifies a categorical hyperparameter and it's range of tunable values. This object is part of the ParameterRanges object.
Name (string) -- [REQUIRED]
The name of the categorical hyperparameter to tune.
Values (list) -- [REQUIRED]
A list of the tunable categories for the hyperparameter.
(string) --
ContinuousParameterRanges (list) --
Specifies the tunable range for each continuous hyperparameter.
(dict) --
Specifies a continuous hyperparameter and it's range of tunable values. This object is part of the ParameterRanges object.
Name (string) -- [REQUIRED]
The name of the hyperparameter to tune.
MaxValue (float) -- [REQUIRED]
The maximum tunable value of the hyperparameter.
MinValue (float) -- [REQUIRED]
The minimum tunable value of the hyperparameter.
ScalingType (string) --
The scale that hyperparameter tuning uses to search the hyperparameter range. Valid values:
Auto
Amazon Forecast hyperparameter tuning chooses the best scale for the hyperparameter.
Linear
Hyperparameter tuning searches the values in the hyperparameter range by using a linear scale.
Logarithmic
Hyperparameter tuning searches the values in the hyperparameter range by using a logarithmic scale.
Logarithmic scaling works only for ranges that have values greater than 0.
ReverseLogarithmic
hyperparameter tuning searches the values in the hyperparameter range by using a reverse logarithmic scale.
Reverse logarithmic scaling works only for ranges that are entirely within the range 0 <= x < 1.0.
For information about choosing a hyperparameter scale, see Hyperparameter Scaling . One of the following values:
IntegerParameterRanges (list) --
Specifies the tunable range for each integer hyperparameter.
(dict) --
Specifies an integer hyperparameter and it's range of tunable values. This object is part of the ParameterRanges object.
Name (string) -- [REQUIRED]
The name of the hyperparameter to tune.
MaxValue (integer) -- [REQUIRED]
The maximum tunable value of the hyperparameter.
MinValue (integer) -- [REQUIRED]
The minimum tunable value of the hyperparameter.
ScalingType (string) --
The scale that hyperparameter tuning uses to search the hyperparameter range. Valid values:
Auto
Amazon Forecast hyperparameter tuning chooses the best scale for the hyperparameter.
Linear
Hyperparameter tuning searches the values in the hyperparameter range by using a linear scale.
Logarithmic
Hyperparameter tuning searches the values in the hyperparameter range by using a logarithmic scale.
Logarithmic scaling works only for ranges that have values greater than 0.
ReverseLogarithmic
Not supported for IntegerParameterRange .
Reverse logarithmic scaling works only for ranges that are entirely within the range 0 <= x < 1.0.
For information about choosing a hyperparameter scale, see Hyperparameter Scaling . One of the following values:
dict
[REQUIRED]
Describes the dataset group that contains the data to use to train the predictor.
DatasetGroupArn (string) -- [REQUIRED]
The Amazon Resource Name (ARN) of the dataset group.
SupplementaryFeatures (list) --
An array of supplementary features. The only supported feature is a holiday calendar.
(dict) --
Describes a supplementary feature of a dataset group. This object is part of the InputDataConfig object. Forecast supports the Weather Index and Holidays built-in featurizations.
Weather Index
The Amazon Forecast Weather Index is a built-in featurization that incorporates historical and projected weather information into your model. The Weather Index supplements your datasets with over two years of historical weather data and up to 14 days of projected weather data. For more information, see Amazon Forecast Weather Index .
Holidays
Holidays is a built-in featurization that incorporates a feature-engineered dataset of national holiday information into your model. It provides native support for the holiday calendars of 66 countries. To view the holiday calendars, refer to the Jollyday library. For more information, see Holidays Featurization .
Name (string) -- [REQUIRED]
The name of the feature. Valid values: "holiday" and "weather" .
Value (string) -- [REQUIRED]
Weather Index
To enable the Weather Index, set the value to "true"
Holidays
To enable Holidays, specify a country with one of the following two-letter country codes:
"AL" - ALBANIA
"AR" - ARGENTINA
"AT" - AUSTRIA
"AU" - AUSTRALIA
"BA" - BOSNIA HERZEGOVINA
"BE" - BELGIUM
"BG" - BULGARIA
"BO" - BOLIVIA
"BR" - BRAZIL
"BY" - BELARUS
"CA" - CANADA
"CL" - CHILE
"CO" - COLOMBIA
"CR" - COSTA RICA
"HR" - CROATIA
"CZ" - CZECH REPUBLIC
"DK" - DENMARK
"EC" - ECUADOR
"EE" - ESTONIA
"ET" - ETHIOPIA
"FI" - FINLAND
"FR" - FRANCE
"DE" - GERMANY
"GR" - GREECE
"HU" - HUNGARY
"IS" - ICELAND
"IN" - INDIA
"IE" - IRELAND
"IT" - ITALY
"JP" - JAPAN
"KZ" - KAZAKHSTAN
"KR" - KOREA
"LV" - LATVIA
"LI" - LIECHTENSTEIN
"LT" - LITHUANIA
"LU" - LUXEMBOURG
"MK" - MACEDONIA
"MT" - MALTA
"MX" - MEXICO
"MD" - MOLDOVA
"ME" - MONTENEGRO
"NL" - NETHERLANDS
"NZ" - NEW ZEALAND
"NI" - NICARAGUA
"NG" - NIGERIA
"NO" - NORWAY
"PA" - PANAMA
"PY" - PARAGUAY
"PE" - PERU
"PL" - POLAND
"PT" - PORTUGAL
"RO" - ROMANIA
"RU" - RUSSIA
"RS" - SERBIA
"SK" - SLOVAKIA
"SI" - SLOVENIA
"ZA" - SOUTH AFRICA
"ES" - SPAIN
"SE" - SWEDEN
"CH" - SWITZERLAND
"UA" - UKRAINE
"AE" - UNITED ARAB EMIRATES
"US" - UNITED STATES
"UK" - UNITED KINGDOM
"UY" - URUGUAY
"VE" - VENEZUELA
dict
[REQUIRED]
The featurization configuration.
ForecastFrequency (string) -- [REQUIRED]
The frequency of predictions in a forecast.
Valid intervals are Y (Year), M (Month), W (Week), D (Day), H (Hour), 30min (30 minutes), 15min (15 minutes), 10min (10 minutes), 5min (5 minutes), and 1min (1 minute). For example, "Y" indicates every year and "5min" indicates every five minutes.
The frequency must be greater than or equal to the TARGET_TIME_SERIES dataset frequency.
When a RELATED_TIME_SERIES dataset is provided, the frequency must be equal to the RELATED_TIME_SERIES dataset frequency.
ForecastDimensions (list) --
An array of dimension (field) names that specify how to group the generated forecast.
For example, suppose that you are generating a forecast for item sales across all of your stores, and your dataset contains a store_id field. If you want the sales forecast for each item by store, you would specify store_id as the dimension.
All forecast dimensions specified in the TARGET_TIME_SERIES dataset don't need to be specified in the CreatePredictor request. All forecast dimensions specified in the RELATED_TIME_SERIES dataset must be specified in the CreatePredictor request.
(string) --
Featurizations (list) --
An array of featurization (transformation) information for the fields of a dataset.
(dict) --
Provides featurization (transformation) information for a dataset field. This object is part of the FeaturizationConfig object.
For example:
{
"AttributeName": "demand",
FeaturizationPipeline [ {
"FeaturizationMethodName": "filling",
"FeaturizationMethodParameters": {"aggregation": "avg", "backfill": "nan"}
} ]
}
AttributeName (string) -- [REQUIRED]
The name of the schema attribute that specifies the data field to be featurized. Amazon Forecast supports the target field of the TARGET_TIME_SERIES and the RELATED_TIME_SERIES datasets. For example, for the RETAIL domain, the target is demand , and for the CUSTOM domain, the target is target_value . For more information, see howitworks-missing-values .
FeaturizationPipeline (list) --
An array of one FeaturizationMethod object that specifies the feature transformation method.
(dict) --
Provides information about the method that featurizes (transforms) a dataset field. The method is part of the FeaturizationPipeline of the Featurization object.
The following is an example of how you specify a FeaturizationMethod object.
{
"FeaturizationMethodName": "filling",
"FeaturizationMethodParameters": {"aggregation": "sum", "middlefill": "zero", "backfill": "zero"}
}
FeaturizationMethodName (string) -- [REQUIRED]
The name of the method. The "filling" method is the only supported method.
FeaturizationMethodParameters (dict) --
The method parameters (key-value pairs), which are a map of override parameters. Specify these parameters to override the default values. Related Time Series attributes do not accept aggregation parameters.
The following list shows the parameters and their valid values for the "filling" featurization method for a Target Time Series dataset. Bold signifies the default value.
aggregation : sum , avg , first , min , max
frontfill : none
middlefill : zero , nan (not a number), value , median , mean , min , max
backfill : zero , nan , value , median , mean , min , max
The following list shows the parameters and their valid values for a Related Time Series featurization method (there are no defaults):
middlefill : zero , value , median , mean , min , max
backfill : zero , value , median , mean , min , max
futurefill : zero , value , median , mean , min , max
To set a filling method to a specific value, set the fill parameter to value and define the value in a corresponding _value parameter. For example, to set backfilling to a value of 2, include the following: "backfill": "value" and "backfill_value":"2" .
(string) --
(string) --
dict
An AWS Key Management Service (KMS) key and the AWS Identity and Access Management (IAM) role that Amazon Forecast can assume to access the key.
RoleArn (string) -- [REQUIRED]
The ARN of the IAM role that Amazon Forecast can assume to access the AWS KMS key.
Passing a role across AWS accounts is not allowed. If you pass a role that isn't in your account, you get an InvalidInputException error.
KMSKeyArn (string) -- [REQUIRED]
The Amazon Resource Name (ARN) of the KMS key.
list
The optional metadata that you apply to the predictor to help you categorize and organize them. Each tag consists of a key and an optional value, both of which you define.
The following basic restrictions apply to tags:
Maximum number of tags per resource - 50.
For each resource, each tag key must be unique, and each tag key can have only one value.
Maximum key length - 128 Unicode characters in UTF-8.
Maximum value length - 256 Unicode characters in UTF-8.
If your tagging schema is used across multiple services and resources, remember that other services may have restrictions on allowed characters. Generally allowed characters are: letters, numbers, and spaces representable in UTF-8, and the following characters: + - = . _ : / @.
Tag keys and values are case sensitive.
Do not use aws: , AWS: , or any upper or lowercase combination of such as a prefix for keys as it is reserved for AWS use. You cannot edit or delete tag keys with this prefix. Values can have this prefix. If a tag value has aws as its prefix but the key does not, then Forecast considers it to be a user tag and will count against the limit of 50 tags. Tags with only the key prefix of aws do not count against your tags per resource limit.
(dict) --
The optional metadata that you apply to a resource to help you categorize and organize them. Each tag consists of a key and an optional value, both of which you define.
The following basic restrictions apply to tags:
Maximum number of tags per resource - 50.
For each resource, each tag key must be unique, and each tag key can have only one value.
Maximum key length - 128 Unicode characters in UTF-8.
Maximum value length - 256 Unicode characters in UTF-8.
If your tagging schema is used across multiple services and resources, remember that other services may have restrictions on allowed characters. Generally allowed characters are: letters, numbers, and spaces representable in UTF-8, and the following characters: + - = . _ : / @.
Tag keys and values are case sensitive.
Do not use aws: , AWS: , or any upper or lowercase combination of such as a prefix for keys as it is reserved for AWS use. You cannot edit or delete tag keys with this prefix. Values can have this prefix. If a tag value has aws as its prefix but the key does not, then Forecast considers it to be a user tag and will count against the limit of 50 tags. Tags with only the key prefix of aws do not count against your tags per resource limit.
Key (string) -- [REQUIRED]
One part of a key-value pair that makes up a tag. A key is a general label that acts like a category for more specific tag values.
Value (string) -- [REQUIRED]
The optional part of a key-value pair that makes up a tag. A value acts as a descriptor within a tag category (key).
string
The accuracy metric used to optimize the predictor.
dict
Response Syntax
{
'PredictorArn': 'string'
}
Response Structure
(dict) --
PredictorArn (string) --
The Amazon Resource Name (ARN) of the predictor.
{'OptimizationMetric': 'WAPE | RMSE | AverageWeightedQuantileLoss | MASE | '
'MAPE'}
Describes a predictor created using the CreatePredictor operation.
In addition to listing the properties provided in the CreatePredictor request, this operation lists the following properties:
DatasetImportJobArns - The dataset import jobs used to import training data.
AutoMLAlgorithmArns - If AutoML is performed, the algorithms that were evaluated.
CreationTime
LastModificationTime
Status
Message - If an error occurred, information about the error.
See also: AWS API Documentation
Request Syntax
client.describe_predictor(
PredictorArn='string'
)
string
[REQUIRED]
The Amazon Resource Name (ARN) of the predictor that you want information about.
dict
Response Syntax
{
'PredictorArn': 'string',
'PredictorName': 'string',
'AlgorithmArn': 'string',
'ForecastHorizon': 123,
'ForecastTypes': [
'string',
],
'PerformAutoML': True|False,
'AutoMLOverrideStrategy': 'LatencyOptimized',
'PerformHPO': True|False,
'TrainingParameters': {
'string': 'string'
},
'EvaluationParameters': {
'NumberOfBacktestWindows': 123,
'BackTestWindowOffset': 123
},
'HPOConfig': {
'ParameterRanges': {
'CategoricalParameterRanges': [
{
'Name': 'string',
'Values': [
'string',
]
},
],
'ContinuousParameterRanges': [
{
'Name': 'string',
'MaxValue': 123.0,
'MinValue': 123.0,
'ScalingType': 'Auto'|'Linear'|'Logarithmic'|'ReverseLogarithmic'
},
],
'IntegerParameterRanges': [
{
'Name': 'string',
'MaxValue': 123,
'MinValue': 123,
'ScalingType': 'Auto'|'Linear'|'Logarithmic'|'ReverseLogarithmic'
},
]
}
},
'InputDataConfig': {
'DatasetGroupArn': 'string',
'SupplementaryFeatures': [
{
'Name': 'string',
'Value': 'string'
},
]
},
'FeaturizationConfig': {
'ForecastFrequency': 'string',
'ForecastDimensions': [
'string',
],
'Featurizations': [
{
'AttributeName': 'string',
'FeaturizationPipeline': [
{
'FeaturizationMethodName': 'filling',
'FeaturizationMethodParameters': {
'string': 'string'
}
},
]
},
]
},
'EncryptionConfig': {
'RoleArn': 'string',
'KMSKeyArn': 'string'
},
'PredictorExecutionDetails': {
'PredictorExecutions': [
{
'AlgorithmArn': 'string',
'TestWindows': [
{
'TestWindowStart': datetime(2015, 1, 1),
'TestWindowEnd': datetime(2015, 1, 1),
'Status': 'string',
'Message': 'string'
},
]
},
]
},
'EstimatedTimeRemainingInMinutes': 123,
'DatasetImportJobArns': [
'string',
],
'AutoMLAlgorithmArns': [
'string',
],
'Status': 'string',
'Message': 'string',
'CreationTime': datetime(2015, 1, 1),
'LastModificationTime': datetime(2015, 1, 1),
'OptimizationMetric': 'WAPE'|'RMSE'|'AverageWeightedQuantileLoss'|'MASE'|'MAPE'
}
Response Structure
(dict) --
PredictorArn (string) --
The ARN of the predictor.
PredictorName (string) --
The name of the predictor.
AlgorithmArn (string) --
The Amazon Resource Name (ARN) of the algorithm used for model training.
ForecastHorizon (integer) --
The number of time-steps of the forecast. The forecast horizon is also called the prediction length.
ForecastTypes (list) --
The forecast types used during predictor training. Default value is ["0.1","0.5","0.9"]
(string) --
PerformAutoML (boolean) --
Whether the predictor is set to perform AutoML.
AutoMLOverrideStrategy (string) --
Note
The LatencyOptimized AutoML override strategy is only available in private beta. Contact AWS Support or your account manager to learn more about access privileges.
The AutoML strategy used to train the predictor. Unless LatencyOptimized is specified, the AutoML strategy optimizes predictor accuracy.
This parameter is only valid for predictors trained using AutoML.
PerformHPO (boolean) --
Whether the predictor is set to perform hyperparameter optimization (HPO).
TrainingParameters (dict) --
The default training parameters or overrides selected during model training. When running AutoML or choosing HPO with CNN-QR or DeepAR+, the optimized values for the chosen hyperparameters are returned. For more information, see aws-forecast-choosing-recipes .
(string) --
(string) --
EvaluationParameters (dict) --
Used to override the default evaluation parameters of the specified algorithm. Amazon Forecast evaluates a predictor by splitting a dataset into training data and testing data. The evaluation parameters define how to perform the split and the number of iterations.
NumberOfBacktestWindows (integer) --
The number of times to split the input data. The default is 1. Valid values are 1 through 5.
BackTestWindowOffset (integer) --
The point from the end of the dataset where you want to split the data for model training and testing (evaluation). Specify the value as the number of data points. The default is the value of the forecast horizon. BackTestWindowOffset can be used to mimic a past virtual forecast start date. This value must be greater than or equal to the forecast horizon and less than half of the TARGET_TIME_SERIES dataset length.
ForecastHorizon <= BackTestWindowOffset < 1/2 * TARGET_TIME_SERIES dataset length
HPOConfig (dict) --
The hyperparameter override values for the algorithm.
ParameterRanges (dict) --
Specifies the ranges of valid values for the hyperparameters.
CategoricalParameterRanges (list) --
Specifies the tunable range for each categorical hyperparameter.
(dict) --
Specifies a categorical hyperparameter and it's range of tunable values. This object is part of the ParameterRanges object.
Name (string) --
The name of the categorical hyperparameter to tune.
Values (list) --
A list of the tunable categories for the hyperparameter.
(string) --
ContinuousParameterRanges (list) --
Specifies the tunable range for each continuous hyperparameter.
(dict) --
Specifies a continuous hyperparameter and it's range of tunable values. This object is part of the ParameterRanges object.
Name (string) --
The name of the hyperparameter to tune.
MaxValue (float) --
The maximum tunable value of the hyperparameter.
MinValue (float) --
The minimum tunable value of the hyperparameter.
ScalingType (string) --
The scale that hyperparameter tuning uses to search the hyperparameter range. Valid values:
Auto
Amazon Forecast hyperparameter tuning chooses the best scale for the hyperparameter.
Linear
Hyperparameter tuning searches the values in the hyperparameter range by using a linear scale.
Logarithmic
Hyperparameter tuning searches the values in the hyperparameter range by using a logarithmic scale.
Logarithmic scaling works only for ranges that have values greater than 0.
ReverseLogarithmic
hyperparameter tuning searches the values in the hyperparameter range by using a reverse logarithmic scale.
Reverse logarithmic scaling works only for ranges that are entirely within the range 0 <= x < 1.0.
For information about choosing a hyperparameter scale, see Hyperparameter Scaling . One of the following values:
IntegerParameterRanges (list) --
Specifies the tunable range for each integer hyperparameter.
(dict) --
Specifies an integer hyperparameter and it's range of tunable values. This object is part of the ParameterRanges object.
Name (string) --
The name of the hyperparameter to tune.
MaxValue (integer) --
The maximum tunable value of the hyperparameter.
MinValue (integer) --
The minimum tunable value of the hyperparameter.
ScalingType (string) --
The scale that hyperparameter tuning uses to search the hyperparameter range. Valid values:
Auto
Amazon Forecast hyperparameter tuning chooses the best scale for the hyperparameter.
Linear
Hyperparameter tuning searches the values in the hyperparameter range by using a linear scale.
Logarithmic
Hyperparameter tuning searches the values in the hyperparameter range by using a logarithmic scale.
Logarithmic scaling works only for ranges that have values greater than 0.
ReverseLogarithmic
Not supported for IntegerParameterRange .
Reverse logarithmic scaling works only for ranges that are entirely within the range 0 <= x < 1.0.
For information about choosing a hyperparameter scale, see Hyperparameter Scaling . One of the following values:
InputDataConfig (dict) --
Describes the dataset group that contains the data to use to train the predictor.
DatasetGroupArn (string) --
The Amazon Resource Name (ARN) of the dataset group.
SupplementaryFeatures (list) --
An array of supplementary features. The only supported feature is a holiday calendar.
(dict) --
Describes a supplementary feature of a dataset group. This object is part of the InputDataConfig object. Forecast supports the Weather Index and Holidays built-in featurizations.
Weather Index
The Amazon Forecast Weather Index is a built-in featurization that incorporates historical and projected weather information into your model. The Weather Index supplements your datasets with over two years of historical weather data and up to 14 days of projected weather data. For more information, see Amazon Forecast Weather Index .
Holidays
Holidays is a built-in featurization that incorporates a feature-engineered dataset of national holiday information into your model. It provides native support for the holiday calendars of 66 countries. To view the holiday calendars, refer to the Jollyday library. For more information, see Holidays Featurization .
Name (string) --
The name of the feature. Valid values: "holiday" and "weather" .
Value (string) --
Weather Index
To enable the Weather Index, set the value to "true"
Holidays
To enable Holidays, specify a country with one of the following two-letter country codes:
"AL" - ALBANIA
"AR" - ARGENTINA
"AT" - AUSTRIA
"AU" - AUSTRALIA
"BA" - BOSNIA HERZEGOVINA
"BE" - BELGIUM
"BG" - BULGARIA
"BO" - BOLIVIA
"BR" - BRAZIL
"BY" - BELARUS
"CA" - CANADA
"CL" - CHILE
"CO" - COLOMBIA
"CR" - COSTA RICA
"HR" - CROATIA
"CZ" - CZECH REPUBLIC
"DK" - DENMARK
"EC" - ECUADOR
"EE" - ESTONIA
"ET" - ETHIOPIA
"FI" - FINLAND
"FR" - FRANCE
"DE" - GERMANY
"GR" - GREECE
"HU" - HUNGARY
"IS" - ICELAND
"IN" - INDIA
"IE" - IRELAND
"IT" - ITALY
"JP" - JAPAN
"KZ" - KAZAKHSTAN
"KR" - KOREA
"LV" - LATVIA
"LI" - LIECHTENSTEIN
"LT" - LITHUANIA
"LU" - LUXEMBOURG
"MK" - MACEDONIA
"MT" - MALTA
"MX" - MEXICO
"MD" - MOLDOVA
"ME" - MONTENEGRO
"NL" - NETHERLANDS
"NZ" - NEW ZEALAND
"NI" - NICARAGUA
"NG" - NIGERIA
"NO" - NORWAY
"PA" - PANAMA
"PY" - PARAGUAY
"PE" - PERU
"PL" - POLAND
"PT" - PORTUGAL
"RO" - ROMANIA
"RU" - RUSSIA
"RS" - SERBIA
"SK" - SLOVAKIA
"SI" - SLOVENIA
"ZA" - SOUTH AFRICA
"ES" - SPAIN
"SE" - SWEDEN
"CH" - SWITZERLAND
"UA" - UKRAINE
"AE" - UNITED ARAB EMIRATES
"US" - UNITED STATES
"UK" - UNITED KINGDOM
"UY" - URUGUAY
"VE" - VENEZUELA
FeaturizationConfig (dict) --
The featurization configuration.
ForecastFrequency (string) --
The frequency of predictions in a forecast.
Valid intervals are Y (Year), M (Month), W (Week), D (Day), H (Hour), 30min (30 minutes), 15min (15 minutes), 10min (10 minutes), 5min (5 minutes), and 1min (1 minute). For example, "Y" indicates every year and "5min" indicates every five minutes.
The frequency must be greater than or equal to the TARGET_TIME_SERIES dataset frequency.
When a RELATED_TIME_SERIES dataset is provided, the frequency must be equal to the RELATED_TIME_SERIES dataset frequency.
ForecastDimensions (list) --
An array of dimension (field) names that specify how to group the generated forecast.
For example, suppose that you are generating a forecast for item sales across all of your stores, and your dataset contains a store_id field. If you want the sales forecast for each item by store, you would specify store_id as the dimension.
All forecast dimensions specified in the TARGET_TIME_SERIES dataset don't need to be specified in the CreatePredictor request. All forecast dimensions specified in the RELATED_TIME_SERIES dataset must be specified in the CreatePredictor request.
(string) --
Featurizations (list) --
An array of featurization (transformation) information for the fields of a dataset.
(dict) --
Provides featurization (transformation) information for a dataset field. This object is part of the FeaturizationConfig object.
For example:
{
"AttributeName": "demand",
FeaturizationPipeline [ {
"FeaturizationMethodName": "filling",
"FeaturizationMethodParameters": {"aggregation": "avg", "backfill": "nan"}
} ]
}
AttributeName (string) --
The name of the schema attribute that specifies the data field to be featurized. Amazon Forecast supports the target field of the TARGET_TIME_SERIES and the RELATED_TIME_SERIES datasets. For example, for the RETAIL domain, the target is demand , and for the CUSTOM domain, the target is target_value . For more information, see howitworks-missing-values .
FeaturizationPipeline (list) --
An array of one FeaturizationMethod object that specifies the feature transformation method.
(dict) --
Provides information about the method that featurizes (transforms) a dataset field. The method is part of the FeaturizationPipeline of the Featurization object.
The following is an example of how you specify a FeaturizationMethod object.
{
"FeaturizationMethodName": "filling",
"FeaturizationMethodParameters": {"aggregation": "sum", "middlefill": "zero", "backfill": "zero"}
}
FeaturizationMethodName (string) --
The name of the method. The "filling" method is the only supported method.
FeaturizationMethodParameters (dict) --
The method parameters (key-value pairs), which are a map of override parameters. Specify these parameters to override the default values. Related Time Series attributes do not accept aggregation parameters.
The following list shows the parameters and their valid values for the "filling" featurization method for a Target Time Series dataset. Bold signifies the default value.
aggregation : sum , avg , first , min , max
frontfill : none
middlefill : zero , nan (not a number), value , median , mean , min , max
backfill : zero , nan , value , median , mean , min , max
The following list shows the parameters and their valid values for a Related Time Series featurization method (there are no defaults):
middlefill : zero , value , median , mean , min , max
backfill : zero , value , median , mean , min , max
futurefill : zero , value , median , mean , min , max
To set a filling method to a specific value, set the fill parameter to value and define the value in a corresponding _value parameter. For example, to set backfilling to a value of 2, include the following: "backfill": "value" and "backfill_value":"2" .
(string) --
(string) --
EncryptionConfig (dict) --
An AWS Key Management Service (KMS) key and the AWS Identity and Access Management (IAM) role that Amazon Forecast can assume to access the key.
RoleArn (string) --
The ARN of the IAM role that Amazon Forecast can assume to access the AWS KMS key.
Passing a role across AWS accounts is not allowed. If you pass a role that isn't in your account, you get an InvalidInputException error.
KMSKeyArn (string) --
The Amazon Resource Name (ARN) of the KMS key.
PredictorExecutionDetails (dict) --
Details on the the status and results of the backtests performed to evaluate the accuracy of the predictor. You specify the number of backtests to perform when you call the operation.
PredictorExecutions (list) --
An array of the backtests performed to evaluate the accuracy of the predictor against a particular algorithm. The NumberOfBacktestWindows from the object determines the number of windows in the array.
(dict) --
The algorithm used to perform a backtest and the status of those tests.
AlgorithmArn (string) --
The ARN of the algorithm used to test the predictor.
TestWindows (list) --
An array of test windows used to evaluate the algorithm. The NumberOfBacktestWindows from the object determines the number of windows in the array.
(dict) --
The status, start time, and end time of a backtest, as well as a failure reason if applicable.
TestWindowStart (datetime) --
The time at which the test began.
TestWindowEnd (datetime) --
The time at which the test ended.
Status (string) --
The status of the test. Possible status values are:
ACTIVE
CREATE_IN_PROGRESS
CREATE_FAILED
Message (string) --
If the test failed, the reason why it failed.
EstimatedTimeRemainingInMinutes (integer) --
The estimated time remaining in minutes for the predictor training job to complete.
DatasetImportJobArns (list) --
An array of the ARNs of the dataset import jobs used to import training data for the predictor.
(string) --
AutoMLAlgorithmArns (list) --
When PerformAutoML is specified, the ARN of the chosen algorithm.
(string) --
Status (string) --
The status of the predictor. States include:
ACTIVE
CREATE_PENDING , CREATE_IN_PROGRESS , CREATE_FAILED
DELETE_PENDING , DELETE_IN_PROGRESS , DELETE_FAILED
CREATE_STOPPING , CREATE_STOPPED
Note
The Status of the predictor must be ACTIVE before you can use the predictor to create a forecast.
Message (string) --
If an error occurred, an informational message about the error.
CreationTime (datetime) --
When the model training task was created.
LastModificationTime (datetime) --
The last time the resource was modified. The timestamp depends on the status of the job:
CREATE_PENDING - The CreationTime .
CREATE_IN_PROGRESS - The current timestamp.
CREATE_STOPPING - The current timestamp.
CREATE_STOPPED - When the job stopped.
ACTIVE or CREATE_FAILED - When the job finished or failed.
OptimizationMetric (string) --
The accuracy metric used to optimize the predictor.
{'OptimizationMetric': 'WAPE | RMSE | AverageWeightedQuantileLoss | MASE | '
'MAPE',
'PredictorEvaluationResults': {'TestWindows': {'Metrics': {'AverageWeightedQuantileLoss': 'double',
'ErrorMetrics': {'MAPE': 'double',
'MASE': 'double'}}}}}
Provides metrics on the accuracy of the models that were trained by the CreatePredictor operation. Use metrics to see how well the model performed and to decide whether to use the predictor to generate a forecast. For more information, see Predictor Metrics .
This operation generates metrics for each backtest window that was evaluated. The number of backtest windows (NumberOfBacktestWindows ) is specified using the EvaluationParameters object, which is optionally included in the CreatePredictor request. If NumberOfBacktestWindows isn't specified, the number defaults to one.
The parameters of the filling method determine which items contribute to the metrics. If you want all items to contribute, specify zero . If you want only those items that have complete data in the range being evaluated to contribute, specify nan . For more information, see FeaturizationMethod .
Note
Before you can get accuracy metrics, the Status of the predictor must be ACTIVE , signifying that training has completed. To get the status, use the DescribePredictor operation.
See also: AWS API Documentation
Request Syntax
client.get_accuracy_metrics(
PredictorArn='string'
)
string
[REQUIRED]
The Amazon Resource Name (ARN) of the predictor to get metrics for.
dict
Response Syntax
{
'PredictorEvaluationResults': [
{
'AlgorithmArn': 'string',
'TestWindows': [
{
'TestWindowStart': datetime(2015, 1, 1),
'TestWindowEnd': datetime(2015, 1, 1),
'ItemCount': 123,
'EvaluationType': 'SUMMARY'|'COMPUTED',
'Metrics': {
'RMSE': 123.0,
'WeightedQuantileLosses': [
{
'Quantile': 123.0,
'LossValue': 123.0
},
],
'ErrorMetrics': [
{
'ForecastType': 'string',
'WAPE': 123.0,
'RMSE': 123.0,
'MASE': 123.0,
'MAPE': 123.0
},
],
'AverageWeightedQuantileLoss': 123.0
}
},
]
},
],
'AutoMLOverrideStrategy': 'LatencyOptimized',
'OptimizationMetric': 'WAPE'|'RMSE'|'AverageWeightedQuantileLoss'|'MASE'|'MAPE'
}
Response Structure
(dict) --
PredictorEvaluationResults (list) --
An array of results from evaluating the predictor.
(dict) --
The results of evaluating an algorithm. Returned as part of the GetAccuracyMetrics response.
AlgorithmArn (string) --
The Amazon Resource Name (ARN) of the algorithm that was evaluated.
TestWindows (list) --
The array of test windows used for evaluating the algorithm. The NumberOfBacktestWindows from the EvaluationParameters object determines the number of windows in the array.
(dict) --
The metrics for a time range within the evaluation portion of a dataset. This object is part of the EvaluationResult object.
The TestWindowStart and TestWindowEnd parameters are determined by the BackTestWindowOffset parameter of the EvaluationParameters object.
TestWindowStart (datetime) --
The timestamp that defines the start of the window.
TestWindowEnd (datetime) --
The timestamp that defines the end of the window.
ItemCount (integer) --
The number of data points within the window.
EvaluationType (string) --
The type of evaluation.
SUMMARY - The average metrics across all windows.
COMPUTED - The metrics for the specified window.
Metrics (dict) --
Provides metrics used to evaluate the performance of a predictor.
RMSE (float) --
The root-mean-square error (RMSE).
WeightedQuantileLosses (list) --
An array of weighted quantile losses. Quantiles divide a probability distribution into regions of equal probability. The distribution in this case is the loss function.
(dict) --
The weighted loss value for a quantile. This object is part of the Metrics object.
Quantile (float) --
The quantile. Quantiles divide a probability distribution into regions of equal probability. For example, if the distribution was divided into 5 regions of equal probability, the quantiles would be 0.2, 0.4, 0.6, and 0.8.
LossValue (float) --
The difference between the predicted value and the actual value over the quantile, weighted (normalized) by dividing by the sum over all quantiles.
ErrorMetrics (list) --
Provides detailed error metrics for each forecast type. Metrics include root-mean square-error (RMSE), mean absolute percentage error (MAPE), mean absolute scaled error (MASE), and weighted average percentage error (WAPE).
(dict) --
Provides detailed error metrics to evaluate the performance of a predictor. This object is part of the Metrics object.
ForecastType (string) --
The Forecast type used to compute WAPE, MAPE, MASE, and RMSE.
WAPE (float) --
The weighted absolute percentage error (WAPE).
RMSE (float) --
The root-mean-square error (RMSE).
MASE (float) --
The Mean Absolute Scaled Error (MASE)
MAPE (float) --
The Mean Absolute Percentage Error (MAPE)
AverageWeightedQuantileLoss (float) --
The average value of all weighted quantile losses.
AutoMLOverrideStrategy (string) --
Note
The LatencyOptimized AutoML override strategy is only available in private beta. Contact AWS Support or your account manager to learn more about access privileges.
The AutoML strategy used to train the predictor. Unless LatencyOptimized is specified, the AutoML strategy optimizes predictor accuracy.
This parameter is only valid for predictors trained using AutoML.
OptimizationMetric (string) --
The accuracy metric used to optimize the predictor.